With the upgrade of the RTCA DO178B to the RTCA DO178C, four so-called supplements have also been created. The RTCA DO330 is one of these four supplements and it describes the requirements for tool qualification in aerospace. This standard takes a certain special role compared to the RTCA DO331, DO332 and DO333, which were created at the same time. The topic of tool qualification has already been dealt with in RTCA DO178B, in contrast to model-based, object-oriented development or formal methods. The following article gives answers to the following questions:
– What is the idea behind tool qualification?
– How is the RTCA DO330 structured?
– What are the 3 most important aspects of the standard?
What is the idea behind tool qualification?
The 4-eyes principle is an essential aspect of the safety-relevant development according to RTCA DO178C in order to achieve a minimization of systematic errors. If software tools are used and they perform verification activities, they are an integral part of the dual control principle. An example is the automated verification of coding guidelines. If the tool does not detect a violation of the rules here, then this error will not be detected. So tools cannot detect errors in a system.
Even more serious are errors that can be introduced into the system by using tools. A source code generator can be such an example. If the code generator is faulty, then it generates errors in the executable code.
The basic idea of tool qualification is that one cannot „simply“ assume that a software tool will function correctly without further testing. The RTCA DO178C requires that a tool qualification is always necessary when a process defined in the standard is replaced by a software tool.
RTCA DO330 contains specifications for software tools in aerospace. The standard also claims to be applicable in other industrial sectors.
How is the RTCA DO330 structured?
The RTCA DO330 classifies the topic tool qualification in general form as follows. Software tools are different from the embedded software for which these software tools are used and form an independent domain; therefore, software tool-specific specifications are required for both tool developers and tool users. It is further explained that software tools are often developed by teams other than those using the software tools. These software tool development teams often have no background in the development of safety-related software according to RTCA DO178C or comparable standards.
The RTCA DO330 defines the following lifecycle for a tool:
The structure of DO330 is based on this life cycle.
Important aspect 1: Classification of the tool to be qualified
At the beginning of each DO330 tool qualification, there is the classification of the tool to be qualified. This classification is made according to the following matrix:
In order to determine the Tool Qualification Level, the Design Assurance Level (DAL) of the software for whose development the software tool to be qualified is to be used is required. In the matrix, this is Software Level A-E.
Furthermore, criteria 1-3 must be determined using the following definition:
Criteria 1: A tool whose output is part of the airborne software and thus could insert an error..
Criteria 2: A tool that automates verification processes and thus could fail to detect an error and whose output is used to justify the elimination or reduction of
- verification processes other than that automated by the tool.
- development processes that could have an impact on the airborne software.
Criteria 3: A tool that, within the scope of its intended use, could fail to detect an error.
The result is the Tool Qualification Level (1-5). The TQL is used to determine all the measures that need to be carried out.
Important aspect 2: Importance of Tool Qualification Level 5 (TQL-5)
If we look at the total set of all software tools for which the application of RTCA DO330 is considered, we find that the overwhelming majority of these tools meet Criteria 3. These are mainly verification tools for dynamic and static testing. According to the above matrix, for all these tools the methods and measures according to TQL-5 must be implemented in the tool qualification. These are essentially the following:
Required planning documents:
- Add to the PSAC (Plan for Software Aspects of Certification) the software tools to be qualified
- Define the tool qualification strategy in the Tool Qualification Plan (TQP)
- Tool-specific information in the Software Life Cycle Environment Configuration Index (SECI)
Required specification documents:
- Tool Operational requirements document / data
Required test data:
- Test spezification including traceability to Tool Operational Requirements
- Test procedures
- Test results for each software tool versions to be verified
- Tool Operational Requirements review with respect to completeness and correctness
Since no architecture and design documents need to be created for TQL-5 and only a few planning aspects need to be documented, commercial tool manufacturers can meet these requirements quite well. In practice, commercial tool vendors usually provide so-called tool qualification packages, which contain the above documents including automatically executable test cases. These test cases must be executed in exactly the same environment in which the software tool to be qualified is used. However, since the tests are automated, this can usually be done with a minimum of effort. If all tests can be successfully executed, the tool qualification is passed. If errors occur, they must be evaluated and suitable measures for further action must be derived.
Important aspect 2: Importance of Tool Qualification Levels 1-4 (TQL-1, TQL-2, TQL-3, TQL-4)
If software tools meet criteria 1, i.e. an error can be introduced into the flight software, then the TQL according to the decision matrix depends on the DAL level of the flight software. A software tool used for the development of DAL A flight software must then be qualified according to the methods and measures of TQL-1. A DAL B flight software means TQL-2 for the software tool, DAL-C corresponds to TQL-3 and DAL-D corresponds to TQL-4.
The number of software tools that meet these requirements in practice is small. So far only a few commercial tools have been qualified according to these requirements. One example is the Scade tool suite. One reason for this is that the effort required for such a tool qualification is certainly a factor of 3-10 greater than a qualification according to TQL-5. An analysis of the methods and measures for TQL-1 shows that they are almost identical to the measures that must be taken for a DAL A flight software development. The measures for TQL-2 to TQL-4 correspond accordingly to DAL B to DAL D.
Criteria 2 is to be applied to software tools which „only“ cannot find an error, but whose results are used in the further stages of development to omit further verification steps. The tool qualification for such software tools corresponds approximately to the effort of a DAL D flight software if these software tools are used for a DAL A or DAL B flight software development.
The RTCA DO 330 describes in great detail the measures that must be taken for a software tool qualification. Many misunderstandings, which were caused by the rather short description in the DO 178B, could be eliminated.
In comparison to ISO26262 the Functional Safety Standard of the automotive industry shows that in aerospace (DO330) only two methods are accepted. These are the validation of the software tool (corresponds to DO330 TQL-5) and the development of a software tool according to a Functional Safety Standard (corresponds to DO330 TQL-1 to TQL-4). The ISO26262 tool qualification measures: Increased confidence in use and the evaluation of the software tool development process are not accepted in aerospace.
The functional safety standards for the railway industry EN50128 and EN50657 define the tool classes T1, T2 and T3 for the software tool qualification. Tool class T2 is comparable to DO330 TQL-5. Tool class T3 is used for tools that can introduce errors into a system, but the measures are in no way comparable to those of DO330 (TQL-1 to TQL-4). Rather, the measures to be applied for T3 are also very similar to those of DO330 TQL-5. In practice, EN50128 and EN50657 thus leave a lot of freedom for interpretation of the measures to be applied, especially for software tools that can introduce errors into a system.
Related HEICON Blog posts
- Importance of Tool Qualification in the FuSa (part 1)!
- Tool qualification – The pain of functional safety (part 2)!
- EN 50128 and EN 50657 support tools
- ISO 26262 Confidence in the use of software tools – A feasible strategy!
- IEC 61508 – Tool qualification – When? Why? How?
I’ll be glad to help you also with any specific questions about your project. Send an email to: info [at] heicon-ulm.de. An overview of the HEICON services can also be found on the HEICON Homepage.