tf_fuzz: refactor expected value storage and simulation flow
Refactor the storage of expected values (expected_info), and add a
simulate function to psa_call, allowing expected value modelling and
call simulation code to be eventually combined into a single method.
* expected_info currently uses a set of mutually-exclusive boolean flags
to model the different possible return codes. As the flags are checked
in different orders across the code-base, failing to unset the right
flags could cause different areas of the code to use different
expected values.
In this commit, an enum is used instead. This allows expected result
checking to be performed by a single switch statement. This also makes
checking to see if the value needs to be simulated much clearer --
currently, this is denoted by both pf_nothing and pf_pass being false.
* Remove expected_info fields with no use.
* There is no one place where call simulation and expected error code
modelling occurs. Expected return code modelling currently occur in
fill_in_command and calc_result_code, and call simulation in
copy_call_to_asset.
The new psa_call::simulate() method is intended as the singular home
for all call simulation code. Combining the simulation of a call's
actions and its expected result should make modelling easier. This
also makes expected value modelling happen in the simulation step
only instead of in both the simulation and code-generation steps
(fill_in_command and calc_result_code are code generation methods).
Due to upcoming changes to the crypto simulation, moving call
simulation code to simulate() has been left for future commits.
Change-Id: I1173164f07a3615c157845ed3abfebefde675c89
Signed-off-by: Nik Dewally <Nik.Dewally@arm.com>
diff --git a/tf_fuzz/tfz-cpp/utility/data_blocks.hpp b/tf_fuzz/tfz-cpp/utility/data_blocks.hpp
index 1e56836..9f6cfc2 100644
--- a/tf_fuzz/tfz-cpp/utility/data_blocks.hpp
+++ b/tf_fuzz/tfz-cpp/utility/data_blocks.hpp
@@ -8,7 +8,6 @@
#include <stdint.h>
#include <string>
#include <vector>
-#include <iosfwd>
class psa_asset;
@@ -26,62 +25,99 @@
using namespace std;
+/// Possible values of return codes.
+///
+/// Intended to be used primarily through [expect_info]
+enum class expected_return_code_t {
+ /// This call should pass.
+ Pass,
+ /// This call should fail.
+ Fail,
+ /// This call should return a specific error code.
+ SpecificFail,
-/**********************************************************************************
- Class expect_info is all about expected data and expected pass/fail information.
- The members are therefore broken down with prefixes pf_ (for pass/fail) or
- data_. Pass/fail, is broadly:
- * "Pass" == the test passes
- * "Specified" == some specified failure (e.g., no such asset)
- * "Nothing" == no expectation
- Expected data refers to psa-asset data values, generally after reading them.
- Currently, they are limited to character strings, but that will probably be
- generalized in the future.
-**********************************************************************************/
+ /// The outcome of this call is unknown and should be simulated.
+ DontKnow,
+ /// the return code does not matter, and checks for this should not take
+ /// place.
+ DontCare,
+};
+
+/**
+ * expect_info is all about expected data and expected pass / fail information.
+ *
+ * The possible types of expected values returned from a PSA call are modelled
+ * in [expected_return_codes].
+ *
+ * Expected data refers to psa-asset data values, generally after reading
+ * them. Currently, they are limited to character strings, but that will
+ * probably be generalized in the future.
+ */
class expect_info
{
public:
- // Data members:
- // Expected-result info:
- bool pf_nothing; // true to not generate results-check(s)
- bool pf_pass; // if !expect.pf_nothing, then pass is expected
- bool pf_fail; // if "expect fail" was specified
- bool pf_specified; // if "expect <ERROR_CODE>" was specified
- /* if !pf_nothing && !pf_pass, then
- true == expected result was specified
- false == tf_fuzz must model expected result, and
- pf_result_string is the expected result */
- string pf_result_string;
- bool data_specified; // (literal expected data specified)
- string data; // what test template expects data from reading an asset to be
- int n_exp_vars; // how many check-value variables have been created
- bool data_var_specified; // check against a variable
- string data_var; // name of variable containing expected data
- bool pf_info_incomplete;
- /* In parsing the template, the expect information comes later than the
- rest of the call info. This flag tells us to fill in the pass/fail
- expect info when it comes available. */
- bool expected_results_saved;
- /* This indicates whether expected results have or have not already been
- copied to this call. It's a "one-shot," so to speak, to copy only
- once when results are known good. Since calls can be inserted into
- earlier points in the call sequence (not always appended), the call
- sequence has to be gone over for this process multiple times. */
- // Methods:
- expect_info (void); // (default constructor)
- ~expect_info (void); // (destructor)
- void set_pf_pass (void);
- void set_pf_fail (void);
- void set_pf_nothing (void);
- void set_pf_error (string error);
- void copy_expect_to_call (psa_call *the_call);
-protected:
- // Data members:
- bool data_matches_asset;
- /* true if template specifies expected data, and that expected data
- agrees with that in the asset */
+ // (literal expected data specified)
+ bool data_specified;
+
+ string data; // what test template expects data from reading an asset to be
+ string data_var; // name of variable containing expected data
+ int n_exp_vars; // how many check-value variables have been created
+ bool data_var_specified; // check against a variable
+
+ /* This indicates whether expected results have or have not already been
+ copied to this call. It's a "one-shot," so to speak, to copy only
+ once when results are known good. Since calls can be inserted into
+ earlier points in the call sequence (not always appended), the call
+ sequence has to be gone over for this process multiple times. */
+ bool expected_results_saved;
+
+ expect_info (void); // (default constructor)
+ ~expect_info (void); // (destructor)
+
+ /// Is simulation of the return code needed?
+ bool simulation_needed(void);
+
+ /// Sets the expected return code of the call to a pass.
+ void expect_pass (void);
+
+ /// Sets the expected return code of the call to a failure.
+ void expect_failure (void);
+
+ /// Sets the expected return code of the call to a specific error code.
+ void expect_error_code (string error);
+
+ /// Sets the expected return code of the call to "don't know".
+ ///
+ /// This will cause TF_Fuzz to later simulate this value.
+ void clear_expected_code(void);
+
+ /// Gets the expected return code.
+ expected_return_code_t get_expected_return_code(void);
+
+ /// When a specified error code is given, get that error code as a string.
+ /// When any other return code is expected, std::logic_error is thrown.
+ string get_expected_return_code_string(void);
+
+ void disable_result_code_checking (void);
+ void enable_result_code_checking (void);
+ bool result_code_checking_enabled(void);
+
+ void copy_expect_to_call (psa_call *the_call);
+
+private:
+ /* true if template specifies expected data, and that expected data
+ agrees with that in the asset */
+ bool data_matches_asset;
+ string return_code_result_string;
+
+ // The type of return code expected.
+ expected_return_code_t expected_return_code;
+
+ // if false, the return code does not matter, and checks for this should not
+ // take place.
+ bool result_code_checking_enabled_f;
};
@@ -96,31 +132,31 @@
{
public:
// Data members:
- bool string_specified;
- // true if a string of data is specified in template file
- bool random_data; // true to generate random data for the asset
- bool file_specified; // true if a file of expected data was specified
- bool literal_data_not_file;
- // true to use data strings rather than files as data source
- int n_set_vars; // how many implicit set variables have been created
- string file_path; // path to file, if specified
- string flags_string;
- // creation flags, nominally for SST but have to be in a vector of base-class
- uint32_t data_offset; // offset into asset data
+ bool string_specified;
+ // true if a string of data is specified in template file
+ bool random_data; // true to generate random data for the asset
+ bool file_specified; // true if a file of expected data was specified
+ bool literal_data_not_file;
+ // true to use data strings rather than files as data source
+ int n_set_vars; // how many implicit set variables have been created
+ string file_path; // path to file, if specified
+ string flags_string;
+ // creation flags, nominally for SST but have to be in a vector of base-class
+ uint32_t data_offset; // offset into asset data
// Methods:
- set_data_info (void); // (default constructor)
- ~set_data_info (void); // (destructor)
- void set (string set_val);
- void set_calculated (string set_val);
- void randomize (void);
- string get (void);
- bool set_file (string file_name);
+ set_data_info (void); // (default constructor)
+ ~set_data_info (void); // (destructor)
+ void set (string set_val);
+ void set_calculated (string set_val);
+ void randomize (void);
+ string get (void);
+ bool set_file (string file_name);
protected:
// Data members:
- string data; // String describing asset data.
+ string data; // String describing asset data.
// Methods:
- string rand_creation_flags (void);
+ string rand_creation_flags (void);
};
@@ -133,37 +169,37 @@
{
public:
// Data members (not much value in "hiding" these behind getters)
- psa_asset *the_asset;
- psa_asset_type asset_type; // SST vs. key vs. policy (etc.)
- bool id_n_not_name; // true to create a PSA asset by ID
- bool name_specified; // true iff template supplied human name
- bool id_n_specified; // true iff template supplied ID #
- vector<string> asset_name_vector;
- vector<int> asset_id_n_vector;
- long asset_ser_no; // unique ID for psa asset needed to find data string
- /* Note: The original theory is that we can't save away iterators to
+ psa_asset *the_asset;
+ psa_asset_type asset_type; // SST vs. key vs. policy (etc.)
+ bool id_n_not_name; // true to create a PSA asset by ID
+ bool name_specified; // true iff template supplied human name
+ bool id_n_specified; // true iff template supplied ID #
+ vector<string> asset_name_vector;
+ vector<int> asset_id_n_vector;
+ long asset_ser_no; // unique ID for psa asset needed to find data string
+ /* Note: The original theory is that we can't save away iterators to
assets, because STL vectors could get relocated. However,
we've switched over to lists, which don't get moved around, so
we should be safe now. */
- asset_search how_asset_found;
- uint64_t id_n; // asset ID# (e.g., SST UID).
- /* Note: This is just a holder to pass ID from template-line to call. The
+ asset_search how_asset_found;
+ uint64_t id_n; // asset ID# (e.g., SST UID).
+ /* Note: This is just a holder to pass ID from template-line to call. The
IDs for a given template line are in asset_info.asset_id_n_vector. */
// Methods:
- asset_name_id_info (void); // (default constructor)
- ~asset_name_id_info (void); // (destructor)
- void set_name (string set_val);
- void set_calc_name (string set_val);
- void set_just_name (string set_val);
- string get_name (void);
- void set_id_n (string set_val);
- void set_id_n (uint64_t set_val);
- string make_id_n_based_name (uint64_t id_n);
- // create UID-based asset name
+ asset_name_id_info (void); // (default constructor)
+ ~asset_name_id_info (void); // (destructor)
+ void set_name (string set_val);
+ void set_calc_name (string set_val);
+ void set_just_name (string set_val);
+ string get_name (void);
+ void set_id_n (string set_val);
+ void set_id_n (uint64_t set_val);
+ string make_id_n_based_name (uint64_t id_n);
+ // create UID-based asset name
protected:
// Data members:
- string asset_name; // parsed from template, assigned to psa_asset object
+ string asset_name; // parsed from template, assigned to psa_asset object
};
@@ -176,56 +212,56 @@
{
public:
// Data members:
- // Digested info:
- bool get_policy_from_key;
- /* if true, then we must get policy info from a stated key; the asset
+ // Digested info:
+ bool get_policy_from_key;
+ /* if true, then we must get policy info from a stated key; the asset
here is a key that uses the policy, and not the policy itself. */
- bool implicit_policy;
- /* if true, then the key was defined with policy specifications, but not
+ bool implicit_policy;
+ /* if true, then the key was defined with policy specifications, but not
a named policy, meaning that we have to create an implicit policy. */
- bool copy_key; // true to indicate copying one key to another
- bool exportable; // key data can be exported (viewed - fail exports if not).
- bool copyable; // can be copied (fail key-copies if not).
- bool can_encrypt; // OK for encryption (fail other uses).
- bool can_decrypt; // OK for decryption (fail other uses).
- bool can_sign; // OK for signing (fail other operations).
- bool can_verify; // OK for verifying a message signature (fail other uses).
- bool derivable; // OK for derive other keys (fail other uses).
- bool persistent; // must be deleted at the end of test.
- string usage_string;
- /* This string is set to a PSA_KEY_USAGE_* value in the template
+ bool copy_key; // true to indicate copying one key to another
+ bool exportable; // key data can be exported (viewed - fail exports if not).
+ bool copyable; // can be copied (fail key-copies if not).
+ bool can_encrypt; // OK for encryption (fail other uses).
+ bool can_decrypt; // OK for decryption (fail other uses).
+ bool can_sign; // OK for signing (fail other operations).
+ bool can_verify; // OK for verifying a message signature (fail other uses).
+ bool derivable; // OK for derive other keys (fail other uses).
+ bool persistent; // must be deleted at the end of test.
+ string usage_string;
+ /* This string is set to a PSA_KEY_USAGE_* value in the template
immediately prior to making define_call<add_policy_usage_call>.
The copy_template_to_call() therein sets the corresponding string
in the call, and that is copied into the code in the fill_in_command()
invocation. */
- string print_usage_true_string;
- /* For printing out policy usage, this states how to describe the usage
+ string print_usage_true_string;
+ /* For printing out policy usage, this states how to describe the usage
if it can be used this way. This is managed similarly with, and used
in conjunction with usage_string above. NOTE: THIS ALSO SERVES AS AN
INDICATOR WHETHER OR NOT TO PRINT ON A GET-USAGE CALL. "" means not
to print. */
- string print_usage_false_string;
- /* Also for printing out policy usage, this is how to describe usage if
+ string print_usage_false_string;
+ /* Also for printing out policy usage, this is how to describe usage if
it cannot be used this way. */
- string key_type; // AES, DES, RSA pair, DS public, etc.
- string key_algorithm;
- int n_bits;
- // for get_key_info call (possibly others) exected key size in bits
- string handle_str; // the text name of the key's "handle"
- string key_data; // the key data as best we can know it.
- string asset_2_name;
- // if there's a 2nd asset, such as policy on key call, this is its name
- string asset_3_name; // if there's a 3rd asset, then this is its name
+ string key_type; // AES, DES, RSA pair, DS public, etc.
+ string key_algorithm;
+ int n_bits;
+ // for get_key_info call (possibly others) exected key size in bits
+ string handle_str; // the text name of the key's "handle"
+ string key_data; // the key data as best we can know it.
+ string asset_2_name;
+ // if there's a 2nd asset, such as policy on key call, this is its name
+ string asset_3_name; // if there's a 3rd asset, then this is its name
// Methods:
- key_policy_info (void); // (default constructor)
- ~key_policy_info (void); // (destructor)
+ key_policy_info (void); // (default constructor)
+ ~key_policy_info (void); // (destructor)
protected:
// Data members:
- bool data_matches_asset;
- /* true if template specifies expected data, and that expected data
+ bool data_matches_asset;
+ /* true if template specifies expected data, and that expected data
agrees with that in the asset */
};