Clarifications & improved documentation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
diff --git a/docs/architecture/psa-migration/outcome-analysis.sh b/docs/architecture/psa-migration/outcome-analysis.sh
index 2ed3ca9..1b23acf 100755
--- a/docs/architecture/psa-migration/outcome-analysis.sh
+++ b/docs/architecture/psa-migration/outcome-analysis.sh
@@ -1,13 +1,35 @@
#!/bin/sh
-# This is only an example/template script, you should make a copy and edit it
-# to suit your needs. The part that needs editing is at the top.
+# This script runs tests in various revisions and configurations and analyses
+# the results in order to highlight any difference in the set of tests skipped
+# in the test suites of interest.
+#
+# It can be used to ensure the testing criteria mentioned in strategy.md,
+# end of section "Supporting builds with drivers without the software
+# implementation" are met, namely:
+#
+# - the sets of tests skipped in the default config and the full config must be
+# the same before and after the PR that implements step 3;
+# - the set of tests skipped in the driver-only build is the same as in an
+# equivalent software-based configuration, or the difference is small enough,
+# justified, and a github issue is created to track it.
+#
+# WARNING: this script checks out a commit other than the head of the current
+# branch; it checks out the current branch again when running sucessfully,
+# but while the script is running, or if it terminates early in error, you
+# should be aware that yu might be at a different commit than expected.
+#
+# NOTE: This is only an example/template script, you should make a copy and
+# edit it to suit your needs. The part that needs editing is at the top.
#
# Also, you can comment out parts that don't need to be re-done when
# re-running this script (for example "get numbers before this PR").
# ----- BEGIN edit this -----
+# The component in all.sh that builds and tests with drivers.
DRIVER_COMPONENT=test_psa_crypto_config_accel_hash_use_psa
+# A similar configuration to that of the component, except without drivers,
+# for comparison.
reference_config () {
scripts/config.py set MBEDTLS_USE_PSA_CRYPTO
scripts/config.py unset MBEDTLS_PKCS1_V21
@@ -16,6 +38,7 @@
scripts/config.py unset MBEDTLS_PKCS12_C
scripts/config.py unset MBEDTLS_ECDSA_DETERMINISTIC
}
+# Space-separated list of test suites of interest.
SUITES="rsa pkcs1_v15 pk pkparse pkwrite"
# ----- END edit this -----
diff --git a/docs/architecture/psa-migration/strategy.md b/docs/architecture/psa-migration/strategy.md
index 6c3378b..d434a0a 100644
--- a/docs/architecture/psa-migration/strategy.md
+++ b/docs/architecture/psa-migration/strategy.md
@@ -270,12 +270,12 @@
docs/use-psa-crypto.md).
- Step 1 is not achieved for a lot of the crypto library including the PSA
core. For example, `entropy.c` calls the legacy API
-`mbedtls_sha256` (or `mbedtls_sha512` optionally); `hmac_drbg.c` calls the
-legacy API `mbedtls_md` and `ctr_drbg.c` calls the legacy API `mbedtls_aes`;
-the PSA core depends on the entropy module and at least one of the DRBG
-modules (unless `MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG` is used). Further, several
-crypto modules have similar issues, for example RSA PKCS#1 v2.1 calls
-`mbedtls_md` directly.
+ `mbedtls_sha256` (or `mbedtls_sha512` optionally); `hmac_drbg.c` calls the
+ legacy API `mbedtls_md` and `ctr_drbg.c` calls the legacy API `mbedtls_aes`;
+ the PSA core depends on the entropy module and at least one of the DRBG
+ modules (unless `MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG` is used). Further, several
+ crypto modules have similar issues, for example RSA PKCS#1 v2.1 calls
+ `mbedtls_md` directly.
- Step 2 is achieved for most of X.509 and TLS (same gaps as step 1) when
`MBEDTLS_USE_PSA_CRYPTO` is enabled - this was tasks like #5795, #5796,
#5797. It is being done in PK and RSA PKCS#1 v1.5 by PR #6065.
@@ -363,12 +363,13 @@
**Note on testing**
-Since supporting driver-only builds is not about adding features, but also
-supporting existing features in new types of builds, the existing tests should
-be enough to cover everything, as long as they're run in the newly supported
-configurations. An all.sh component needs to be present each main type of
-configutation we support (for example, with this (family of) algorithm
-driver-only, with or without `USE_PSA`).
+Since supporting driver-only builds is not about adding features, but about
+supporting existing features in new types of builds, testing will not involve
+adding cases to the test suites, but instead adding new components in `all.sh`
+that build and run tests in newly-supported configurations. For example, if
+we're making some part of the library work with hashes provided only by
+drivers when `MBEDTLS_USE_PSA_CRYPTO` is defined, there should be a place in
+`all.sh` that builds and run tests in such a configuration.
There is however a risk, especially in step 3 where we change how dependencies
are expressed (sometimes in bulk), to get things wrong in a way that would
@@ -379,7 +380,7 @@
the same before and after the PR that implements step 3;
- the set of tests skipped in the driver-only build is the same as in an
equivalent software-based configuration, or the difference is small enough,
-justified, and a github issue is created to track it.
+ justified, and a github issue is created to track it.
Note that the favourable case is when the number of tests skipped is 0 in the
driver-only build. In other cases, analysis of the outcome files is needed,