The compiler is the central „tool“, which is required for every software development. It forms the link between the human-readable high-level source code (e.g., C and C ++) and the machine code, interpretable for the hardware processor. For the development of safety critical software according to relevant functional safety standards such as ISO26262 (car), EN50128 (rail), IEC61508 (automation, general) or DO178C (aviation), special requirements apply for the tools used during the development (see also tool qualification blog 1 and blog 2). The compiler plays a special role here. On the one hand, it is the central tool for any development. On the other hand, the measures proposed in the standards can not be fully applied in practice. The blog shows a process from the aerospace industry for compiler verification / validation, which is also highly recommended for other industries.
Typical tool qualification measure, used in almost all functional safety standards:
- Data from the historical use of the compiler
- Assessment of the development process of the compiler
- Test the functionality of the compiler
- Complete development of the compiler according to a standard of functional safety
The most powerful method No. 4 for safety reasons is practically not useable for compilers as they are built by commercial manufacturers. The cost for a compiler that is developed according to safety standards is such high that selling it with a profit is hardly possible in the relative small safety market.
Method No. 3 can not be successfully applied, as a complete test of the functionality of a compiler is not possible for reasons of complexity. Either too many aspects remain untested, or the testing is just uneconomical.
Method No. 2 has not yet been really used in the safety market. The reason might be, that this method can not deliver sufficiently good results for compilers in particular.
This leaves only method 1. This measure is finally applied for compiler verification in all safety critical markets. I am not aware of any safety critical project that would use a compiler that is completely new on the market. In all functional safety projects, compilers are used which are on the market for many years and have already been used in similar projects.
However, since historical data does not provide a sufficient waranty for an individual projects, the aerospace industry has developed a process that significantly reduces the risk of remaining undetected compiler errors.
Minimising the risk of undetected compiler errors in the aerospace industry:
The process in the aerospace industry consists of two important additional elements:
- For each test (Unit- Integration- or Systemtest), exactly the same compiler settings are used, which are also used for the final operative source code.
- For particularly safety critcal projects (DAL A and B), a traceability from the object code to the high-level language source code (C and C ++) is required.
The experience of the last 20 – 30 years in aerospace clearly shows that the application of these two methods is very effective. In combination with the use of known compilers, the risk of undetected compiler errors is reduced to an absolute minimum. In all aerospace projects, which are known to me from the past 15 years, no errors have been found, even after years of field usage, which could be traced back to the compiler.
Therefore it is recommended for other industries to apply this process as well. All tests should be executed with the operating compiler and the corresponding settings. With a good test strategy and a thought through test environment at each level (unittest, integration test and system level), this is associated with hardly any extra effort.
On the other hand, as experience in aerospace shows, the risk of undetected compiler errors decreases dramatically when tests that show both a structural coverage and a requirement coverage of 95% and more, no compiler errors were found.
The complete functionality of a compiler can not be proved (see the discussion of method 3 above), but for a single project, the compiler functionality used is limited and therefore completely testable. This is exactly what is achieved with applying the recommended measures.
Die Erfahrung der letzten 20 – 30 Jahren in der Luftfahrt zeigen deutlich, dass die Anwendung dieser beiden Methoden sehr effektiv sind. In Kombination mit dem Einsatz bekannter Compiler wird das Risiko unentedeckter Compilerfehler auf ein absolutes Minimum gedrückt. In den Luftfahrtprojekten die mir aus den letzten 15 Jahren bekannt sind, wurden sind keine Fehler, auch nach langjährigem Feldeinsatz, festgestellt, die auf den Compiler zurückzuführen sind.
I’ll be glad to help you also with any specific questions about your project. The HEICON Starter as well as the HEICON Consulting products are designed to solve such open points. Send an email to: info[at] heicon-ulm.de!