Automation of CAVS Testing: Bringing CAVP and Vendor Together (C21c)
The automation of CAVS Testing with the ACVP framework was introduced more than a year ago by the CAVP supported by NIST. Test definitions and a supporting library were developed and published under an Open Source license.
Amid the development by the CAVP and NIST in collaboration with several selected vendors, the current test approach would require some modifications to comply with constraints faced by many different vendors.
The speaker will present the current ACVP approach and compare it with the CAVS testing applied by different vendors. One of the aspects the discussion
covers is the constraint that vendor test environments commonly do not have access to the Internet. Another aspect is the issue of combinatorial explosion
in testing since vendors strive to provide different cipher implementations which are most effective for different use cases. CAVS testing is also used
during regression testing by vendors to validate changes to cipher implementations. Such regression testing requires test vectors with pre-defined
answers, excluding tests that imply the the use of random numbers.
The goal of the discussion is to show where the vendor and the CAVP align and where they do not align. For the non-aligned parts, suggestions are made on
how to bridge the gap to reach a test framework that is most effective for all parties involved.