Fast functional safety certification for your project

While there are many tweaks you can do to your development processes to speed your certification, everything starts with code quality. But how can you ensure code quality? Fortunately, there are some easy ways to almost instantly upgrade your code quality game with as little pain as possible.

Get help from standards

Did you know that in C99 there are about 190 different ambiguities in the code specification? What we mean is that in C99, there are 190 different syntactically legal C constructs that aren’t spelled out in the C language specification. It actually gets a bit worse when you go to C18 and even more so in C++ where you start introducing concepts of multiple and virtual inheritance. Of course, a compiler has to turn your source into concrete code, so it has to pick an interpretation of what your code means and run with it.

What this means in practical terms is that you can get different compilers that interpret your source code differently. In a high-reliability system, this is a nightmare scenario, especially since many companies pursuing functional safety-certification are cross-compiling their code on multiple platforms for ease of testing. As you can imagine, this can have a really bad impact on the time it takes to get your certification because you have to test around all of these situations to prove the repeatability and reliability of your code. How do we get through this? The short answer is that you avoid these ambiguities in your code. But how do we do that? Using coding standards like MISRA are a quick way to solve the problem because they are designed to keep you from those types of common pitfalls in code. They also promote safe and reliable coding practices to reduce the number of defects in your code. But how can we make sure we follow those standards? Fortunately, functional safety standards give us a way to do this.

Standards require code analysis

Virtually every functional safety standard requires that you do static analysis of your code, and they highly recommend that you do runtime (or dynamic) analysis of your code. The most broad-reaching of these standards is IEC 61508 which covers safety-related systems in general. In section C.4.2 of that standard, using C without coding standards that remove ambiguous and dangerous behavior is not recommended for anything above Safety Integrity Level (SIL) 1. In other words, you must use static analysis to make your code more robust if you are looking to get SIL 2-4 certification for your product. Why is this? These static analysis tools can force a developer to implement coding standards like MISRA. Moreover, static and runtime analysis helps you to up your code quality game by quickly pointing out when you are getting into risky coding behavior, especially with the aforementioned coding standard ambiguities.

However, when you use these types of automated tools can also have a huge impact on your certification timeline. Many organizations use hard-to-configure, hard-to-use code analysis tools that are relegated to running on a build server as part of a nightly build. That doesn’t really help you that much because the individual developer is not getting instantaneous feedback on what is wrong with the code they just wrote. Moreover, sometimes the warning messages that come from these tools are inscrutable and waste the developer’s time trying to figure out what they mean and how the code can be corrected to make the warning go away. If you can run code analysis during development – and before you check it into a formal build – then it’s like the defect never even happened. You get a lower defect injection rate for your project that certification entities love to see because it means you have a very mature development organization.

Making code analysis part of the daily workflow

My colleagues and I have met many different companies in different industries and what we have noted is that the easier you make it to configure and use code analysis tools, the more likely developers are to use them and therefore get the benefit sooner. Having these automated tools as part of the developer’s toolbox means that you can check and improve the code quality as you write your application, while you are “in the zone” of understanding what that section of code is meant to do and how it interacts with other modules in the system. To do this effectively, the tools have to be integrated into the daily workflow.

When looking through what others are saying in regards to integrated code analysis, I found that Google published an article in an ACM publication looking at the merits of code analysis. While the article takes a holistic view of their entire codebase including C, C++, and Java, their results are very clear: 

“Compiler errors are displayed early in the development process and integrated into the developer workflow. We have found expanding the set of compiler checks to be effective for improving code quality at Google.” 

The authors stated that moving static analysis checks into the compiler workflow and making them appear as errors drastically improved the attention paid to the tool’s findings and that it ultimately meant that their code was of a much higher quality. Further on, they talk about a survey sent to developers who recently encountered a compiler error and developers who had received a patch with a fix for the same problem:

"Google developers perceive that issues flagged at compile time (as opposed to patches for checked-in code) catch more important bugs; for example, survey participants deemed 74% of the issues flagged at compile time as "real problems," compared to 21% of those found in checked-in code."

The article also talks about the importance of having code analysis part of the workflow by stating that when they automatically ran commits through a static analysis tool and invited engineers to look at the analysis dashboard, very few engineers followed through. Having instant feedback in the compilation process made static analysis easier to use and harder to ignore. Therefore, they chose to integrate static analysis by default in everyone’s workflow. They believe that for code analysis tools to succeed, developers must feel they benefit from their use and enjoy using the tools.

But what kinds of results can you expect to see from adding code analysis to your workflow? One thing you can expect to get is an improvement in the overall security of your application as having high code quality can eliminate exploits like buffer overflows, illegal pointers and the like, as described in this article. While this in itself is a great reason to use code analysis, it is sometimes hard to convince people of the maxim that “an ounce of prevention is worth a pound of cure” and you need more significant results to convince developers and management alike of the merits of code analysis. 

A paper by Stefan Wagner and others used empirical data to calculate the benefit of code analysis tools vs. traditional testing on different code bases. Their results are very telling: out of the 769 identified defects, 76% were caught by code analysis tools and only 4% by traditional testing (the remaining 20% found by code review). How fast could you achieve your mean time to failure (MTTF) goal in software if you could eliminate 75% of your bugs before you even begin testing? The answer is “awfully quick”. The time and money savings on testing alone make the investment in code analysis tools worthwhile, not to mention the savings in time to market. These are the types of processes that functional safety certification entities love to see because it drastically reduces the risk of defects making it into the field in your final product.

High-quality code speeds your path to functional safety

The key to speeding your path to functional safety certification is to improve your code quality. By improving code quality, you lower your defect injection rate which means you more quickly reach your software release criteria and that makes your development organization look very mature to a functional safety-certification entity. While you can never know exactly how many defects are left in an application, you can minimize the number by using code analysis tools early and often.

Article written by Shawn Prestridge, Industry profile and Senior Field Application Engineer/Team Leader for US FAEs at IAR Systems