Misc fixes & improvements to driver testing doc
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md
index 3c0e068..e9ac02e 100644
--- a/docs/architecture/testing/driver-interface-test-strategy.md
+++ b/docs/architecture/testing/driver-interface-test-strategy.md
@@ -194,7 +194,10 @@
test-driver support, most of the time, it automatically works for all
algorithms and key types supported by the library. (The exception being when
the driver needs to call a different function for different key types, as is
-the case with some asymmetric key management operations.)
+the case with some asymmetric key management operations.) (Note: it's still
+useful to test drivers in configurations with partial algorithm support, and
+that can still be done by configuring libtestdriver1 and the main library as
+desired.)
The renaming process for `libtestdriver1` is implemented as a few Perl regexes
applied to a copy of the library code, see the `libtestdriver1.a` target in
@@ -437,7 +440,7 @@
instrumentation in the test drivers described in the previous section, in
order to check that drivers are called when they're supposed to, and that the
core behaves as expected when they return errors (in particular, that we fall
-back to the built-in implementation when the driver returns `NOT_SUPPORTED).
+back to the built-in implementation when the driver returns `NOT_SUPPORTED`).
This is `test_suite_psa_crypto_driver_wrappers`, which is maintained manually
(that is, the test cases in the `.data` files are not auto-generated). The
@@ -445,6 +448,12 @@
(`PSA_CRYPTO_DRIVER_TEST`), which is not the case in the default or full
config.
+The test suite is focused on driver usage (mostly by checking the expected
+number of hits) but also does some validation of the results: for
+deterministic algorithms, known-answers tests are used, and for the rest, some
+consistency checks are done (more or less detailled depending on the algorithm
+and build configuration).
+
#### Configurations coverage
The driver wrappers test suite has cases that expect both the driver and the
@@ -519,12 +528,13 @@
#### Test case coverage
Since `test_suite_psa_crypto_driver_wrappers.data` is maintained manually,
-we need to make sure it exercises all the cases that need to be tested.
+we need to make sure it exercises all the cases that need to be tested. In the
+future, this file should be generated in order to ensure exhaustiveness.
-One way to evaluate this is to look at line coverage in test driver
-implementaitons - this doesn't reveal all gaps, but it does reveal cases where
-we thought about something when writing the test driver, but not when writing
-test functions/data.
+In the meantime, one way to observe (lack of) completeness is to look at line
+coverage in test driver implementaitons - this doesn't reveal all gaps, but it
+does reveal cases where we thought about something when writing the test
+driver, but not when writing test functions/data.
Key management:
- `mbedtls_test_transparent_generate_key()` is not tested with RSA keys.