irq_test: Add IRQ testing tool
Add python scripts for debuggers to test IRQ handling in TF-M.
Signed-off-by: Mate Toth-Pal <mate.toth-pal@arm.com>
Change-Id: I6c5c0b920e3a0c38b3a0c867c93dd5851c66ff8b
diff --git a/irq_test_tool/README.rst b/irq_test_tool/README.rst
new file mode 100644
index 0000000..f79fee8
--- /dev/null
+++ b/irq_test_tool/README.rst
@@ -0,0 +1,377 @@
+#############
+IRQ test tool
+#############
+
+************
+Introduction
+************
+
+This tool is to test interrupt handling in TF-M. Testing different interrupt
+scenarios is important as the ARMv8-M architecture does complex operations when
+interrupt happens, especially when security boundary crossing happens. These
+operations need to be considered by the TF-M implementation, as in a typical use
+case there is a separate scheduler on the Non-Secure and the secure side as
+well, and the SPM needs to maintain a consistent state, which might not be
+trivial.
+
+The aim of the tool is to be able to test scenarios, that are identified to be
+problematic, in a reproducible way, and do this in an automated way, so regular
+regression testing can have a low cost.
+
+******************
+How the tool works
+******************
+
+The tool is a set of Python scripts which need to be run **inside** a debugger.
+Currently Arm Development Studio and GDB are supported. During the test run, the
+script interacts with the debugger, sets breakpoints, triggers interrupts by
+writing into system registers, starts the target, and when the target is
+stopped, it examines the target's state.
+
+A typical execution scenario looks like this:
+
+.. uml::
+
+ @startuml
+
+ participant CPU
+ participant Debugger
+
+ CPU -> CPU: start_from_reset_handler
+ activate CPU
+
+ Debugger -> CPU: Attach & pause
+ deactivate CPU
+ Debugger-> Debugger: start_script
+ Activate Debugger
+
+ note right
+ Read config files ...
+
+ execute step 1
+ end note
+
+ Debugger -> CPU: set_breakpoint
+
+ Debugger -> CPU: Continue
+ deactivate Debugger
+ activate CPU
+
+
+ ... executing ...
+
+ loop for all the remaining steps
+
+ CPU->Debugger: bkpt hit
+ deactivate CPU
+ activate Debugger
+
+ note right
+ Sanity check on the triggered breakpoint
+ (is this the breakpoint expected)
+ If so, continue the sequence
+ end note
+
+ Debugger -> CPU: set_breakpoint
+
+ alt if required by step
+ Debugger -> CPU: set interrupt pending
+ end alt
+
+ Debugger -> CPU: Continue
+ deactivate Debugger
+ activate CPU
+
+ ... executing ...
+
+ end loop
+
+ CPU->Debugger: bkpt hit
+ deactivate CPU
+ activate Debugger
+
+ Debugger->Debugger: End of script
+ Deactivate Debugger
+
+
+ @enduml
+
+Once started inside the debugger, the script automatically deduces the debugger
+it is running in, by trying to import the support libraries for a specific
+debugger. The order the debuggers are tried in the following order:
+
+#. Arm Development studio
+#. GDB
+
+If both check fails, the script falls back to 'dummy' mode which means that the
+calls to the debugger log the call, and returns successfully.
+
+.. note::
+
+ This 'dummy' mode can be used out of a debugger environment as well.
+
+.. important::
+
+ The script assumes that the symbols for the software being debugged/tested
+ are loaded in the debugger.
+
+The available parameters are:
+
++----------------------+---------------------------------+--------------------------------------------------+
+| short option | long option | meaning |
++======================+=================================+==================================================+
+| ``-w`` | ``--sw-break`` | Use sw breakpoint (the default is HW breakpoint) |
++----------------------+---------------------------------+--------------------------------------------------+
+| ``-q <IRQS>`` | ``--irqs <IRQS>`` | The name of the IRQs json |
++----------------------+---------------------------------+--------------------------------------------------+
+| ``-t <TESTCASE>`` | ``--testcase <TESTCASE>`` | The name of the file containing the testcase |
++----------------------+---------------------------------+--------------------------------------------------+
+| ``-b <BREAKPOINTS>`` | ``--breakpoints <BREAKPOINTS>`` | The name of the breakpoints json file |
++----------------------+---------------------------------+--------------------------------------------------+
+
+***********
+Input files
+***********
+
+Breakpoints
+===========
+
+below is a sample file for breakpoints:
+
+.. code:: json
+
+ {
+ "breakpoints": {
+ "irq_test_iteration_before_service_calls": {
+ "file": "core_ns_positive_testsuite.c",
+ "line": 692
+ },
+ "irq_test_service1_high_handler": {
+ "symbol": "SPM_CORE_IRQ_TEST_1_SIGNAL_HIGH_isr"
+ },
+ "irq_test_service2_prepare_veneer": {
+ "offset": "4",
+ "symbol": "tfm_spm_irq_test_2_prepare_test_scenario_veneer"
+ }
+ }
+ }
+
+Each point where a breakpoint is to be set by the tool should be enumerated in
+this file, in the "breakpoints" object. For each breakpoint an object needs to
+be created. The name of the object can be used in the testcase description. The
+possible fields for a breakpoint object can be seen in the example above.
+
+tools/generate_breakpoints.py
+-----------------------------
+
+This script helps to automate the generation of the breakpoints from source files.
+Each code location that is to be used in a testcase, should be annotated with
+one of the following macro in the source files:
+
+.. code:: c
+
+ /* Put breakpoint on the address of the symbol */
+ #define IRQ_TEST_TOOL_SYMBOL(name, symbol)
+
+ /* Put a breakpoint on the address symbol + offset */
+ #define IRQ_TEST_TOOL_SYMBOL_OFFSET(name, symbol, offset)
+
+ /* Put a breakpoint at the specific location in the code where the macro is
+ * called. This creates a file + line type breakpoint
+ */
+ #define IRQ_TEST_TOOL_CODE_LOCATION(name)
+
+Usage of the script:
+
+.. code::
+
+ $ python3 generate_breakpoints.py --help
+ usage: generate_breakpoints.py [-h] tfm_source outfile
+
+ positional arguments:
+ tfm_source path to the TF-M source code
+ outfile The output json file with the breakpoints
+
+ optional arguments:
+ -h, --help show this help message and exit
+
+
+
+IRQs
+====
+
+.. code:: json
+
+ {
+ "irqs": {
+ "test_service1_low": {
+ "line_num" : 51
+ },
+ "ns_irq_low": {
+ "line_num" : 40
+ }
+ }
+ }
+
+Each IRQ that is to be triggered should have an object created inside the "irqs"
+object. The name of these objects is the name that could be used in a testcase
+description. The only valid field of the IRQ objects is "line_num" which refers
+to the number of the interrupt line.
+
+Testcase
+========
+
+.. code:: json
+
+ {
+ "description" : ["Trigger Non-Secure interrupt during SPM execution in",
+ "privileged mode"],
+ "steps": [
+ {
+ "wait_for" : "irq_test_iteration_start"
+ },
+ {
+ "wait_for" : "spm_partition_start"
+ },
+ {
+ "description" : ["Trigger the interrupt, but expect the operation",
+ "to be finished before the handler is called"],
+ "expect" : "spm_partition_start_ret_success",
+ "trigger" : "ns_irq_low"
+ },
+ {
+ "wait_for" : "ns_irq_low_handler"
+ },
+ {
+ "wait_for" : "irq_test_service2_prepare"
+ }
+ ]
+ }
+
+The test is executed by the script on a step by step basis. When the script is
+started, it processes the first step, then starts the target. After a breakpoint
+is hit, it processes the next target, and continues. This iteration is repeated
+until all the steps are processed
+
+For each step, the following activities are executed:
+
+#. All the breakpoints are cleared in the debugger
+#. If there is a 'wait_for' field, a breakpoint is set for the location
+ specified.
+#. If there is a 'trigger' field, an IRQ is pended by writing to NVIC
+ registers.
+#. If there is an 'expect' field, a breakpoint is set for the location
+ specified. Then the testcase file is scanned starting with the next step,
+ and a breakpoint is set at the first location specified with a 'wait_for'
+ field. Next time, when the execution is stopped, the breakpoint that was hit
+ is compared to the expected breakpoint.
+
+Each object can have a description field to add comments.
+
+**********************
+How to call the script
+**********************
+
+Arm Development Studio
+======================
+
+The script can be called directly from the debugger's command window:
+
+.. code:: shell
+
+ source irq_test.py -q irqs.json -b breakpoints_gen.json -t test_01.json
+
+GDB
+===
+
+The script should be sourced inside GDB, without passing any arguments to
+it.
+
+.. code:: shell
+
+ (gdb) source irq_test.py
+
+
+That registers a custom command ``test_irq``. ``test_irq`` should be called
+with three parameters: breakpoints, irqs, and the test file. This command will
+actually execute the tests.
+
+.. note::
+
+ This indirection in case of GDB is necessary because it is not possible to
+ pass parameters to the script when it is sourced.
+
+.. important::
+
+ The script needs to be run from the <TF-M root>/tools/irq_test directory
+ as the 'current working dir' is added as module search path.
+
+A typical execution of the script in GDB would look like the following:
+
+.. code::
+
+ (gdb) target remote localhost: 3333
+ (gdb) add-symbol-file /path/to/binaries/tfm_s.axf 0x1A020400
+ (gdb) add-symbol-file /path/to/binaries/tfm_ns.axf 0x0A070400
+ (gdb) add-symbol-file /path/to/binaries/mcuboot.axf 0x1A000000
+ (gdb) source /path/to/script/irq_test.py
+ (gdb) test_irq -q /path/to/data/irqs.json -b /path/to/data/breakpoints.json -t /path/to/data/test_03.json
+
+.. note::
+ ``add-symbol-file`` command is used above as other commands like ``file``
+ and ``symbol-file`` seem to be dropping the previously loaded symbols. The
+ addresses the axf files are loaded at are depending on the platform they
+ are built to. The address needs to be specified is the start of the code
+ section
+
+**********************
+Implementation details
+**********************
+
+Class hierarchy:
+
+.. uml::
+
+ @startuml
+
+ class gdb.Command
+ note right: Library provided by GDB
+
+ class TestIRQsCommand
+ note right: Only used in case debugger is GDB
+
+ gdb.Command <|.. TestIRQsCommand : implements
+
+ TestIRQsCommand o-- TestExecutor : Creates >
+
+ "<Main>" o-- TestExecutor : Creates >
+ note right on link
+ Only if running in Arm DS
+ end note
+
+ TestExecutor o-- AbstractDebugger : has a concrete >
+
+ AbstractDebugger <|.. GDBDebugger : implements
+ AbstractDebugger <|.. DummyDebugger : implements
+ AbstractDebugger <|.. ArmDSDebugger : implements
+
+ GDBDebugger o-- Breakpoint : has multiple >
+
+ GDBDebugger o-- Location : has multiple >
+ DummyDebugger o-- Location : has multiple >
+ ArmDSDebugger o-- Location : has multiple >
+
+ @enduml
+
+
+*****************************
+Possible further improvements
+*****************************
+
+- Add priority property to the IRQs data file
+- Add possibility to run randomized scenarios, to realise stress testing.
+
+
+--------------
+
+*Copyright (c) 2020, Arm Limited. All rights reserved.*
diff --git a/irq_test_tool/generate_breakpoints.py b/irq_test_tool/generate_breakpoints.py
new file mode 100644
index 0000000..c7b2fe3
--- /dev/null
+++ b/irq_test_tool/generate_breakpoints.py
@@ -0,0 +1,103 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+"""Script for scanning the source after breakpoints
+
+This script scans a source code, and finds patterns that sign a location to be
+used as a breakpoint.
+TODO: Describe format
+"""
+
+import os
+import sys
+import json
+import re
+import argparse
+
+MACRO_PATTERN = re.compile(r'^\s*IRQ_TEST_TOOL_([0-9a-zA-Z_]*)\s*\(([^\)]*)\)\s*')
+
+def process_line(filename, line, line_num, breakpoints):
+ m = MACRO_PATTERN.match(line)
+ if m:
+
+ macro_type = m.group(1)
+ parameters = m.group(2).split(',')
+
+ print ("Macro " + macro_type + ", parameters: " + str(parameters))
+
+ if (macro_type == 'SYMBOL'):
+ if (len(parameters) != 2):
+ print ("Invalid macro at " + filename + ":" + str(line_num))
+ print ("Expected number of parameters for *SYMBOL is 2")
+ exit(1)
+ bp = {}
+ bp["symbol"] = parameters[1].strip()
+ breakpoints[parameters[0].strip()] = bp
+ return
+
+ if (macro_type == 'CODE_LOCATION'):
+ if (len(parameters) != 1):
+ print ("Invalid macro at " + filename + ":" + str(line_num))
+ print ("Expected number of parameters for *CODE_LOCATION is 1")
+ exit(1)
+ bp = {}
+ bp["file"] = filename
+ bp["line"] = line_num
+ breakpoints[parameters[0].strip()] = bp
+ return
+
+ if (macro_type == 'SYMBOL_OFFSET'):
+ if (len(parameters) != 3):
+ print ("Invalid macro at " + filename + ":" + str(line_num))
+ print ("Expected number of parameters for *SYMBOL_OFFSET is 3")
+ exit(1)
+ bp = {}
+ bp["symbol"] = parameters[1].strip()
+ bp["offset"] = parameters[2].strip()
+ breakpoints[parameters[0].strip()] = bp
+ return
+
+ print ("invalid macro *" + macro_type + "at " + filename + ":" + str(line_num))
+ exit(1)
+
+
+def process_file(file_path, filename, breakpoints):
+ with open(file_path, 'r', encoding='latin_1') as f:
+ content = f.readlines()
+ for num, line in enumerate(content):
+ line_num = num+1
+ process_line(filename, line, line_num, breakpoints)
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("tfm_source", help="path to the TF-M source code")
+ parser.add_argument("outfile", help="The output json file with the breakpoints")
+ args = parser.parse_args()
+
+ tfm_source_dir = args.tfm_source
+
+ breakpoints = {}
+
+ for root, subdirs, files in os.walk(tfm_source_dir):
+ for filename in files:
+ # Scan other files as well?
+ if not filename.endswith('.c'):
+ continue
+ file_path = os.path.join(root, filename)
+ process_file(file_path, filename, breakpoints)
+
+
+ breakpoints = {"breakpoints":breakpoints}
+
+ with open(args.outfile, 'w') as outfile:
+ json.dump(breakpoints, outfile,
+ sort_keys=True, indent=4, separators=(',', ': '))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/irq_test_tool/irq_test.py b/irq_test_tool/irq_test.py
new file mode 100644
index 0000000..b9b2fd3
--- /dev/null
+++ b/irq_test_tool/irq_test.py
@@ -0,0 +1,107 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+""" This module is the entry point of the IRQ testing tool.
+"""
+
+import argparse
+import json
+import logging
+import os
+import sys
+
+# Workaround for GDB: Add current directory to the module search path
+sys.path.insert(0, os.getcwd())
+from irq_test_abstract_debugger import Location
+from irq_test_dummy_debugger import DummyDebugger
+from irq_test_executor import TestExecutor
+
+def create_argparser():
+ """ Create an argument parser for the script
+
+ This parser enumerates the arguments that are necessary for all the
+ debuggers. Debugger implementations might add other arguments to this
+ parser.
+ """
+ parser = argparse.ArgumentParser()
+ parser.add_argument("-w", "--sw-break",
+ help="use sw breakpoint (the default is HW breakpoint)",
+ action="store_true")
+ parser.add_argument("-q", "--irqs",
+ type=str,
+ help="the name of the irqs json",
+ required=True)
+ parser.add_argument("-b", "--breakpoints",
+ type=str,
+ help="the name of the breakpoints json",
+ required=True)
+ parser.add_argument("-t", "--testcase",
+ type=str,
+ help="The testcase to execute",
+ required=True)
+ return parser
+
+def main():
+ """ The main function of the script
+
+ Detects the debugger that it is started in, creates the debugger
+ implementation instance, and either executes the test, or registers a
+ command in the debugger. For details see the README.rst
+ """
+ try:
+ # TODO: evironment checking should be refactored to the debugger
+ # implementations
+ from arm_ds.debugger_v1 import Debugger
+ debugger_type = 'Arm-DS'
+ except ImportError:
+ logging.debug('Failed to import Arm-DS scripting env, try GDB')
+ try:
+ # TODO: evironment checking should be refactored to the debugger
+ # implementations
+ import gdb
+ debugger_type = 'GDB'
+ except ImportError:
+ logging.debug("Failed to import GDB scripting env, fall back do "
+ "dummy")
+ debugger_type = 'dummy'
+
+ logging.info("The debugger type selected is: %s", debugger_type)
+
+ # create a debugger controller instance
+ if debugger_type == 'Arm-DS':
+ from irq_test_Arm_DS_debugger import ArmDSDebugger
+ logging.debug("initialising debugger object...")
+ arg_parser = create_argparser()
+ try:
+ args = arg_parser.parse_args()
+ except:
+ logging.error("Failed to parse command line parameters")
+ return
+ # TODO: Fail gracefully in case of an argparse error
+ debugger = ArmDSDebugger(args.sw_break)
+ executor = TestExecutor(debugger)
+ executor.execute(args.irqs, args.breakpoints, args.testcase)
+ elif debugger_type == 'GDB':
+ from irq_test_gdb_debugger import GDBDebugger
+ from irq_test_gdb_debugger import TestIRQsCommand
+ logging.debug("initialising debugger object...")
+ arg_parser = create_argparser()
+
+ # register the 'test_irqs' custom command
+ TestIRQsCommand(arg_parser)
+ logging.info("Command 'test_irqs' is successfully registered")
+ elif debugger_type == 'dummy':
+ arg_parser = create_argparser()
+ args = arg_parser.parse_args()
+ debugger = DummyDebugger(args.sw_break)
+ executor = TestExecutor(debugger)
+ executor.execute(args.irqs, args.breakpoints, args.testcase)
+
+if __name__ == "__main__":
+ logging.basicConfig(format='===== %(levelname)s: %(message)s',
+ level=logging.DEBUG, stream=sys.stdout)
+ main()
diff --git a/irq_test_tool/irq_test_Arm_DS_debugger.py b/irq_test_tool/irq_test_Arm_DS_debugger.py
new file mode 100644
index 0000000..2bac20c
--- /dev/null
+++ b/irq_test_tool/irq_test_Arm_DS_debugger.py
@@ -0,0 +1,155 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+""" This module implements the debugger control interface for the Arm
+ Developement Studio
+"""
+
+import logging
+import re
+# pylint: disable=import-error
+from arm_ds.debugger_v1 import Debugger
+from arm_ds.debugger_v1 import DebugException
+# pylint: enable=import-error
+from irq_test_abstract_debugger import AbstractDebugger
+
+class ArmDSDebugger(AbstractDebugger):
+ """ This class is the implementation of the control interface for the Arm
+ Developement Studio
+ """
+
+ def __init__(self, use_sw_breakpoints):
+ super(ArmDSDebugger, self).__init__()
+ debugger = Debugger()
+ self.debugger = debugger
+ self.breakpoints = {} # map breakpoint IDs to names
+ self.use_sw_breakpoints = use_sw_breakpoints
+
+ if debugger.isRunning():
+ logging.info("debugger is running, stop it")
+ debugger.stop()
+ try:
+ timeout = 180*1000 # TODO: configureble timeout value?
+ debugger.waitForStop(timeout)
+ except DebugException as debug_exception:
+ logging.error("debugger wait timed out: %s", str(debug_exception))
+
+ def set_breakpoint(self, name, location):
+ logging.info("Add breakpoint for location %s:'%s'", name, str(location))
+
+ ec = self.debugger.getCurrentExecutionContext()
+ bps = ec.getBreakpointService()
+
+ try:
+ if location.symbol:
+ if location.offset != 0:
+ spec = '(((unsigned char*)' + location.symbol + ') + ' + location.offset + ')'
+ bps.setBreakpoint(spec, hw=(not self.use_sw_breakpoints))
+ else:
+ bps.setBreakpoint(location.symbol, hw=(not self.use_sw_breakpoints))
+ else:
+ bps.setBreakpoint(location.filename, location.line, hw=(not self.use_sw_breakpoints))
+ except DebugException as ex:
+ logging.error("Failed to set breakpoint for %s", str(location))
+ logging.error(str(ex))
+ # TODO: Remove exit (from all over the script), and drop custom
+ # exception that is handled in main.
+ exit(2)
+
+ # Add the new breakpoint to the list.
+ # Assume that the last breakpoint is the newly added one
+ breakpoint = bps.getBreakpoint(bps.getBreakpointCount()-1)
+
+ self.breakpoints[breakpoint.getId()] = name
+
+ def __triger_interrupt_using_STIR_address(self, line):
+ logging.debug("writing to STIR address %s", hex(line))
+ ec = self.debugger.getCurrentExecutionContext()
+ memory_service = ec.getMemoryService()
+ mem_params = {'width': 8, 'verify': 0, 'use_image': 0}
+ memory_service.writeMemory32(hex(0xE000EF00), line, mem_params)
+
+ def __triger_interrupt_using_STIR_register(self, line):
+ logging.debug("writing to STIR register %s", hex(line))
+ register_name = "STIR"
+ ec = self.debugger.getCurrentExecutionContext()
+ ec.getRegisterService().setValue(register_name, line)
+
+ def __triger_interrupt_using_NVIC_ISPR_register(self, line):
+ # write ISPR register directly
+ register_id = line//32
+ register_offset = line%32
+ register_name = "NVIC_ISPR" + str(register_id)
+
+ ec = self.debugger.getCurrentExecutionContext()
+ value = ec.getRegisterService().getValue(register_name)
+ value |= 1 << register_offset
+
+ logging.debug("Writing to {:s} register 0x{:08x}".
+ format(register_name, value))
+
+ ec.getRegisterService().setValue(register_name, hex(value))
+
+ def __triger_interrupt_using_NVIC_ISPR_address(self, line):
+ # write ISPR register directly
+ register_id = line//32
+ register_offset = line%32
+ # TODO: remove magic numbers
+ NVIC_ISPR_address = 0xE000E200
+ NVIC_ISPR_n_address = NVIC_ISPR_address + register_id * 4
+
+ ec = self.debugger.getCurrentExecutionContext()
+ memory_service = ec.getMemoryService()
+ mem_params = {'width': 8, 'verify': 0, 'use_image': 0}
+
+ value = 1 << register_offset # 0 bits are ignored on write
+
+ logging.debug("Writing to address 0x{:08x} register 0x{:08x}".
+ format(NVIC_ISPR_n_address, value))
+
+ memory_service.writeMemory32(NVIC_ISPR_n_address, value, mem_params)
+
+ def trigger_interrupt(self, interrupt_line):
+ logging.info("triggering interrupt for line %s", str(interrupt_line))
+
+ line = int(interrupt_line)
+
+ if line >= 0:
+ #self.__triger_interrupt_using_STIR_address(line)
+ #self.__triger_interrupt_using_STIR_register(line)
+ #self.__triger_interrupt_using_NVIC_ISPR_register(line) # seems to have bugs?
+ self.__triger_interrupt_using_NVIC_ISPR_address(line)
+ else:
+ logging.error("Invalid interrupt line value {:d}".format(line))
+ exit(0)
+
+ def continue_execution(self):
+ logging.info("Continuing execution ")
+ ec = self.debugger.getCurrentExecutionContext()
+ ec.executeDSCommand("info breakpoints")
+ self.debugger.run()
+
+ try:
+ timeout = 180*1000 # TODO: configureble timeout value?
+ self.debugger.waitForStop(timeout)
+ except DebugException as debug_exception:
+ logging.error("debugger wait timed out %s", str(debug_exception))
+ exit(0)
+
+
+ def clear_breakpoints(self):
+ logging.info("Remove all breakpoints")
+ self.debugger.removeAllBreakpoints()
+ self.breakpoints = {}
+
+ def get_triggered_breakpoint(self):
+ ec = self.debugger.getCurrentExecutionContext()
+ bps = ec.getBreakpointService()
+ breakpoint = bps.getHitBreakpoint()
+ id = breakpoint.getId()
+ logging.info("getting the triggered breakpoints, ID = {:d}".format(id))
+ return self.breakpoints[id]
diff --git a/irq_test_tool/irq_test_abstract_debugger.py b/irq_test_tool/irq_test_abstract_debugger.py
new file mode 100644
index 0000000..38600ce
--- /dev/null
+++ b/irq_test_tool/irq_test_abstract_debugger.py
@@ -0,0 +1,76 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+"""Defines the interface that a debugger control class have to implement
+"""
+
+class Location(object):
+ """A helper class to store the properties of a location where breakpoint
+ can be put
+ """
+ def __init__(self, symbol=None, offset=0, filename=None, line=None):
+ self.symbol = symbol
+ self.offset = offset
+ self.filename = filename
+ self.line = line
+
+ def __str__(self):
+ ret = ""
+ if self.symbol:
+ ret += str(self.symbol)
+
+ if self.offset:
+ ret += "+" + str(self.offset)
+
+ if self.filename:
+ if self.symbol:
+ ret += " @ "
+ ret += str(self.filename) + ":" + str(self.line)
+
+ return ret
+
+ def __unicode__(self):
+ return str(self), "utf-8"
+
+class AbstractDebugger(object):
+ """The interface that a debugger control class have to implement
+ """
+ def __init__(self):
+ pass
+
+ def set_breakpoint(self, name, location):
+ """Put a breakpoint at a location
+
+ Args:
+ name: The name of the location. This name is returned by
+ get_triggered_breakpoint
+ location: An instance of a Location class
+ """
+ raise NotImplementedError('subclasses must override set_breakpoint()!')
+
+ def trigger_interrupt(self, interrupt_line):
+ """trigger an interrupt on the interrupt line specified in the parameter
+
+ Args:
+ interrupt_line: The number of the interrupt line
+ """
+ raise NotImplementedError('subclasses must override trigger_interrupt()!')
+
+ def continue_execution(self):
+ """Continue the execution
+ """
+ raise NotImplementedError('subclasses must override continue_execution()!')
+
+ def clear_breakpoints(self):
+ """Clear all breakpoints
+ """
+ raise NotImplementedError('subclasses must override clear_breakpoints()!')
+
+ def get_triggered_breakpoint(self):
+ """Get the name of the last triggered breakpoint
+ """
+ raise NotImplementedError('subclasses must override get_triggered_breakpoint()!')
diff --git a/irq_test_tool/irq_test_dummy_debugger.py b/irq_test_tool/irq_test_dummy_debugger.py
new file mode 100644
index 0000000..c03077c
--- /dev/null
+++ b/irq_test_tool/irq_test_dummy_debugger.py
@@ -0,0 +1,50 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+""" This module contains a dummy implementation of the debugger control interface
+"""
+
+import logging
+from irq_test_abstract_debugger import AbstractDebugger
+
+class DummyDebugger(AbstractDebugger):
+ """A dummy implementation of the debugger control interface
+
+ This class can be used for rapidly testing the testcase execution algorithm.
+
+ Breakpoint names are put in a list to keep track of them. Interrupts are not
+ emulated in any way, the 'trigger_interrupt' function returns without doing
+ anything. 'continue_execution' returns immediately as well, and
+ 'get_triggered_breakpoint' returns the breakpoint added the earliest.
+ """
+ def __init__(self, use_sw_breakpoints):
+ super(DummyDebugger, self).__init__()
+ self.breakpoints = []
+ self.use_sw_breakpoints = use_sw_breakpoints
+
+ def set_breakpoint(self, name, location):
+ if (self.use_sw_breakpoints):
+ breakpoint_type = "sw"
+ else:
+ breakpoint_type = "hw"
+ logging.info("debugger: set %s breakpoint %s", breakpoint_type, name)
+ self.breakpoints.append(name)
+
+ def trigger_interrupt(self, interrupt_line):
+ logging.info("debugger: triggering interrupt line for %s", str(interrupt_line))
+
+ def continue_execution(self):
+ logging.info("debugger: continue")
+
+ def clear_breakpoints(self):
+ logging.info("debugger: clearing breakpoints")
+ self.breakpoints = []
+
+ def get_triggered_breakpoint(self):
+ if self.breakpoints:
+ return self.breakpoints[0]
+ return None
diff --git a/irq_test_tool/irq_test_executor.py b/irq_test_tool/irq_test_executor.py
new file mode 100644
index 0000000..d98096f
--- /dev/null
+++ b/irq_test_tool/irq_test_executor.py
@@ -0,0 +1,180 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+from irq_test_abstract_debugger import Location
+import logging
+import json
+
+def create_locations_from_file(breakpoints_file_name):
+ """Internal function to create Location objects of a breakpoints data file
+ """
+ # Read in the points to break at
+ logging.info("Reading breakpoints file '%s'", breakpoints_file_name)
+ breakpoints_file = open(breakpoints_file_name)
+ breakpoints = json.load(breakpoints_file)
+ logging.debug("breakpoints: %s", str(breakpoints))
+
+ #TODO: go over the breakpoints and try to set them as a sanity check
+
+ locations = {}
+
+ for loc_name in breakpoints['breakpoints']:
+ bkpt = breakpoints['breakpoints'][loc_name]
+ offset = 0
+
+ if 'file' in bkpt:
+ filename = bkpt['file']
+ else:
+ filename = None
+
+ if 'symbol' in bkpt:
+ symbol = bkpt['symbol']
+ if 'offset' in bkpt:
+ offset = bkpt['offset']
+ else:
+ offset = 0
+ else:
+ if 'offset' in bkpt:
+ logging.error("In location %s offset is included without a"
+ " symbol")
+ exit(2)
+ symbol = None
+
+ if 'line' in bkpt:
+ line = bkpt['line']
+ try:
+ int(line)
+ except ValueError:
+ logging.error("In location %s line is not a valid int",
+ loc_name)
+ exit(2)
+ else:
+ line = None
+
+ if symbol:
+ if line or filename:
+ logging.error("In location %s nor filename nor line should"
+ "be present when symbol is present", loc_name)
+ exit(2)
+
+ if (not line and filename) or (line and not filename):
+ logging.error("In location %s line and filename have to be "
+ "present the same time", loc_name)
+ exit(2)
+
+ if (not symbol) and (not filename):
+ logging.error("In location %s no symbol nor code location is "
+ "specified at all", loc_name)
+ exit(2)
+
+ loc = Location(symbol=symbol, offset=offset, filename=filename, line=line)
+
+ locations[loc_name] = loc
+
+ return locations
+
+class TestExecutor(object):
+ """ This class implements the test logic.
+
+ It reads the input files, and executes the steps of the testcase. It receives an
+ AbstractDebugger instance on creation. The test execution is implemented in the
+ execute function.
+ """
+
+ def __init__(self, debugger):
+ self.debugger = debugger
+
+ def execute(self, irqs_filename, breakpoints_filename, testcase_filename):
+ """ Execute a testcase
+
+ Execute the testcase defined in 'testcase_filename', using the IRQs and
+ breakpoints defined in irqs_filename and breakpoints_filename.
+ """
+ # Read in the list of IRQs
+ logging.info("Reading irqs file '%s'", irqs_filename)
+ irqs_file = open(irqs_filename)
+ irqs = json.load(irqs_file)
+ logging.debug("irqs: %s", str(irqs))
+
+ # read in the test sequence
+ logging.info("Reading test sequence file '%s'", testcase_filename)
+ test_file = open(testcase_filename)
+ test = json.load(test_file)
+ logging.debug("testcase: %s", str(test))
+
+ # TODO: crosscheck the tests file against the breakpoints and the irq's
+ # available
+
+ locations = create_locations_from_file(breakpoints_filename)
+
+ self.debugger.clear_breakpoints()
+
+ # execute the test
+ steps = test['steps']
+ for i, step in enumerate(steps):
+
+ logging.info("---- Step %d ----", i)
+
+ continue_execution = False
+
+ if 'wait_for' in step:
+ bp_name = step['wait_for']
+ self.debugger.set_breakpoint(bp_name, locations[bp_name])
+ next_to_break_at = bp_name
+ continue_execution = True
+ elif 'expect' in step:
+ bp_name = step['expect']
+ self.debugger.set_breakpoint(bp_name, locations[bp_name])
+ next_to_break_at = bp_name
+
+ # Find the next wait_for in the test sequence, and set a
+ # breakpoint for that as well. So that it can be detected if an
+ # expected breakpoint is missed.
+
+ wait_for_found = False
+ ii = i+1
+
+ while ii < len(steps) and not wait_for_found:
+ next_step = steps[ii]
+ if 'wait_for' in next_step:
+ next_bp_name = next_step['wait_for']
+ self.debugger.set_breakpoint(next_bp_name,
+ locations[next_bp_name])
+ wait_for_found = True
+ ii += 1
+
+ continue_execution = True
+
+
+ if 'trigger' in step:
+ irqs_dict = irqs['irqs']
+ irq = irqs_dict[step['trigger']]
+ line_nu = irq['line_num']
+ self.debugger.trigger_interrupt(line_nu)
+
+
+ if continue_execution:
+ self.debugger.continue_execution()
+
+ triggered_breakpoint = self.debugger.get_triggered_breakpoint()
+
+ if triggered_breakpoint is None:
+ logging.error("No breakpoint was hit?????")
+ exit(0)
+
+ if triggered_breakpoint != next_to_break_at:
+ logging.error("execution stopped at '%s' instead of '%s'",
+ triggered_breakpoint, next_to_break_at)
+ exit(0)
+ else:
+ logging.error("execution stopped as no breakpoint is set")
+ exit(1)
+
+ self.debugger.clear_breakpoints()
+
+ logging.info("All the steps in the test file are executed successfully"
+ " with the expected result.")
diff --git a/irq_test_tool/irq_test_gdb_debugger.py b/irq_test_tool/irq_test_gdb_debugger.py
new file mode 100644
index 0000000..930783d
--- /dev/null
+++ b/irq_test_tool/irq_test_gdb_debugger.py
@@ -0,0 +1,238 @@
+#-------------------------------------------------------------------------------
+# Copyright (c) 2020, Arm Limited. All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+#-------------------------------------------------------------------------------
+
+""" This module implements the debugger control interface for the GDB debugger
+"""
+
+import traceback
+import logging
+import re
+# pylint: disable=import-error
+import gdb
+# pylint: enable=import-error
+
+from irq_test_abstract_debugger import AbstractDebugger
+from irq_test_executor import TestExecutor
+
+class Breakpoint:
+ def __init__(self, number, hit_count):
+ self.number = number
+ self.hit_count = hit_count
+
+ def __str__(self):
+ return "(#" + str(self.number) + ": hit_count=" + str(self.hit_count) + ")"
+
+class GDBDebugger(AbstractDebugger):
+ """This class is the implementation for the debugger control interface for GDB
+ """
+ def __init__(self, use_sw_breakpoints):
+ super(GDBDebugger, self).__init__()
+ self.last_breakpoint_list = []
+ self.breakpoint_names = {}
+ self.use_sw_breakpoints = use_sw_breakpoints
+ self.last_breakpoint_num = 0
+ self.execute_output = None
+
+ def parse_breakpoint_line(self, lines):
+
+ number = 0
+ hit_count = 0
+
+ # look for a first line: starts with a number
+ m = re.match(r'^\s*(\d+)\s+', lines[0])
+ if m:
+ number = int(m.group(1))
+ lines.pop(0)
+
+ else:
+ logging.error('unexpected line in "info breakpoints": %s', lines[0])
+ exit (0)
+
+ # look for additional lines
+ if lines:
+ m = re.match(r'^\s*stop only', lines[0])
+ if m:
+ lines.pop(0)
+
+ if lines:
+ m = re.match(r'^\s*breakpoint already hit (\d+) time', lines[0])
+ if m:
+ hit_count = int(m.group(1))
+ lines.pop(0)
+
+ return Breakpoint(number, hit_count)
+
+ #TODO: remove this as it is unnecessary
+ def get_list_of_breakpoints(self):
+ breakpoints_output = gdb.execute('info breakpoints', to_string=True)
+
+ if breakpoints_output.find("No breakpoints or watchpoints.") == 0:
+ return []
+
+ breakpoints = []
+
+ m = re.match(r'^Num\s+Type\s+Disp\s+Enb\s+Address\s+What', breakpoints_output)
+ if m:
+ lines = breakpoints_output.splitlines()
+ lines.pop(0) # skip the header
+
+ while lines:
+ breakpoints.append(self.parse_breakpoint_line(lines))
+
+ return breakpoints
+
+ logging.error('unexpected output from "info breakpoints"')
+ exit(0)
+
+ def set_breakpoint(self, name, location):
+ # Using the gdb.Breakpoint class available in the gdb scripting
+ # environment is not used here, as it looks like the python API has no
+ # knowledge of the Hardware breakpoints.
+ # This means that the Breakpoint.__init__() be called with
+ # gdb.BP_HARDWARE_BREAKPOINT as nothing similar exists. Also the
+ # gdb.breakpoints() function seems to be returning the list of software
+ # breakpoints only (i.e. only breakpoints created with the 'break'
+ # command are returned, while breakpoints created with 'hbreak' command
+ # are not)
+ # So instead of using the python API for manipulating breakpoints,
+ # 'gdb.execute' is used to issue 'break', 'hbreak' and
+ # 'info breakpoints' commands, and the output is parsed.
+ logging.info("Add breakpoint for location '%s'", str(location))
+
+ if logging.getLogger().isEnabledFor(logging.DEBUG):
+ logging.debug("List the breakpoints BEFORE adding")
+ #gdb.execute("info breakpoints")
+ for b in self.get_list_of_breakpoints():
+ logging.debug(" " + str(b))
+ logging.debug("End of breakpoint list")
+
+ if self.use_sw_breakpoints:
+ keyword = "break"
+ else:
+ keyword = "hbreak"
+
+ if location.symbol:
+ if (location.offset != 0):
+ argument = '*(((unsigned char*)' + location.symbol + ') + ' + location.offset + ')'
+ else:
+ argument = location.symbol
+ else:
+ argument = location.filename + ":" + str(location.line)
+
+ command = keyword + " " + argument
+ logging.debug("Setting breakpoint with command '" + command + "'")
+
+ add_breakpoint_output = gdb.execute(command, to_string=True)
+
+ print add_breakpoint_output
+ m = re.search(r'reakpoint\s+(\d+)\s+at', str(add_breakpoint_output))
+ if (m):
+ self.last_breakpoint_num = int(m.group(1))
+ else:
+ logging.error("matching breakpoint command's output failed")
+ exit(1)
+
+ if logging.getLogger().isEnabledFor(logging.DEBUG):
+ logging.debug("List the breakpoints AFTER adding")
+ #gdb.execute("info breakpoints")
+ for b in self.get_list_of_breakpoints():
+ logging.debug(" " + str(b))
+ logging.debug("End of breakpoint list")
+
+ self.breakpoint_names[self.last_breakpoint_num] = name
+
+ def __triger_interrupt_using_STIR_address(self, line):
+ logging.debug("writing to STIR address %s", hex(line))
+
+ command = "set *((unsigned int *) " + hex(0xE000EF00) + ") = " + str(line)
+ logging.debug("calling '%s'", command)
+ gdb.execute(command)
+
+ def __triger_interrupt_using_NVIC_ISPR_address(self, line):
+ # write ISPR register directly
+ register_id = line//32
+
+ # straight
+ #register_offset = line%32
+ # reversed endianness
+ register_offset = ((3-(line%32)/8)*8)+(line%8)
+
+ # TODO: remove magic numbers
+ NVIC_ISPR_address = 0xE000E200
+ NVIC_ISPR_n_address = NVIC_ISPR_address + register_id * 4
+
+ value = 1 << register_offset # 0 bits are ignored on write
+
+ logging.debug("Writing to address 0x{:08x} register 0x{:08x}".
+ format(NVIC_ISPR_n_address, value))
+
+ command = "set *((unsigned int *) " + hex(NVIC_ISPR_n_address) + ") = " + hex(value)
+ logging.debug("calling '%s'", command)
+ gdb.execute(command)
+
+ def trigger_interrupt(self, interrupt_line):
+ logging.info("triggering interrupt for line %s", str(interrupt_line))
+
+ line = int(interrupt_line)
+
+ # self.__triger_interrupt_using_NVIC_ISPR_address(line)
+ self.__triger_interrupt_using_STIR_address(line)
+
+ def continue_execution(self):
+ logging.info("Continuing execution ")
+ # save the list of breakpoints before continuing
+ # self.last_breakpoint_list = gdb.breakpoints()
+ self.execute_output = gdb.execute("continue", to_string=True)
+
+ def clear_breakpoints(self):
+ logging.info("Remove all breakpoints")
+ gdb.execute("delete breakpoint")
+
+ def get_triggered_breakpoint(self):
+ logging.info("getting the triggered breakpoints")
+
+ if not self.execute_output:
+ logging.error("Execute was not called yet.")
+ exit(1)
+
+ m = re.search(r'Breakpoint\s+(\d+)\s*,', self.execute_output)
+ if m:
+ print "self.breakpoint_names: " + str(self.breakpoint_names)
+ return self.breakpoint_names[int(m.group(1))]
+ else:
+ logging.error("Unexpected output from execution.")
+ exit(1)
+
+class TestIRQsCommand(gdb.Command):
+ """This class represents the new command to be registered in GDB
+ """
+ def __init__(self, arg_parser):
+ # This registers our class as "test_irqs"
+ super(TestIRQsCommand, self).__init__("test_irqs", gdb.COMMAND_DATA)
+ self.arg_parser = arg_parser
+
+ def print_usage(self):
+ """Print usage of the custom command
+ """
+ print self.arg_parser.print_help()
+
+ def invoke(self, arg, from_tty):
+ """This is the entry point of the command
+
+ This function is called by GDB when the command is called from the debugger
+ """
+ args = self.arg_parser.parse_args(arg.split())
+
+ debugger = GDBDebugger(args.sw_break)
+ test_executor = TestExecutor(debugger)
+
+ try:
+ test_executor.execute(args.irqs, args.breakpoints, args.testcase)
+ except:
+ pass
+
+ traceback.print_exc()