vizualisation: Add tf-main vizualization script
Add `tf-a-main-results.{bash,plot}` to fetch test results from a Jenkins
build of tf-main and vizualise them. Update README.rst with usage info
for the new script and plot description.
Change-Id: I837a2dcb4cd3247b24ba9eec6557699e9a75e9c2
Signed-off-by: Nathan Dunne <Nathan.Dunne@arm.com>
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
diff --git a/script/graphs/README.rst b/script/graphs/README.rst
index 0860cb2..b057943 100644
--- a/script/graphs/README.rst
+++ b/script/graphs/README.rst
@@ -41,6 +41,66 @@
.. code-block::
- bash ../<this-repo>/script/graph/sloc-viz.bash > sloc.png 2> sloc.tsv
+ bash ../<this-repo>/script/graphs/sloc-viz.bash > sloc.png 2> sloc.tsv
+
+Test Results
+------------
+
+The script `tf-main-results.bash` uses curl to retrieve test results for a
+tf-a-main Jenkins job, and generates a CSV and stacked histogram PNG of the
+combined data.
+
+Usage:
+======
+
+ bash tf-main-results.bash <jenkins-url> [ci_gateway] [filter]
+
+The Jenkins URL is the URL for the target build job.
+
+ https://ci.trustedfirmware.org/job/tf-a-main/1/
+
+The sub-builds for this job will all be queried to find the ones that contain
+tests, ignoring child builds that only build and don't run any platform tests.
+
+`tf-a-ci-gateway` is the default gateway, although, different gateways may be
+specified with the optional "ci_gateway" argument. This option will be combined
+with the build numbers and the base Jenkins URL to retrieve the results of
+sub-builds.
+
+This can be filtered further using the optional "filter" argument, which will
+select only test groups that match the provided regex.
+
+Example Useful Queries
+======================
+
+Only show tests running the test framework:
+
+ bash tf-main-results.bash <jenkins-url> [ci_gateway] "tftf"
+
+Only show tests for N1SDP & Juno platforms:
+
+ bash tf-main-results.bash <jenkins-url> [ci_gateway] "n1sdp|juno"
+
+Only show boot tests on FVP platforms:
+
+ bash tf-main-results.bash <jenkins-url> [ci_gateway] 'fvp.*boot'
+
+Note: for filters that return a small number of test groups, the graph is not
+ideal as it is sized for a large number. A CSV file of the data is also produced,
+however, so that you can use it to create your own graph, if required.
+
+Additional Config
+=================
+
+The script also allows the three output files to be configured via ENV variables:
+
+ PNGFILE=out.png CSVFILE=out.csv bash tf-main-results.bash
+
+If they are not set then default values based on the build number will be generated:
+
+ PNGFILE_DEFAULT="tf-main_${build_number}.png"
+ CSVFILE_DEFAULT="tf-main_${build_number}.csv"
+
+If any of these files already exist then they will be overwritten.
*Copyright (c) 2021-2022, Arm Limited. All rights reserved.*