RSS Feed

Static Code Analysis

A static code analysis is commonly understood to be an automatic check by a tool. This is only a recent development. Only 15 years ago these tools were not as good as they are today and they were not available in such a variety as today. Static code analysis is simply part of what so far was called a static white-box test. This is a test which is performed on the source code without executing it. Thus it is called "static". The expression "white-box" stands for the kind of test which is concerned with the implementation of the code i.e. its structure and details, calculations and data operations. The overall functionality of the software is not in focus, but the details, quality and safety of its programming. A part of the test can be done automatically using analysis tools as e.g. PC-Lint or similar tools. But this should not be all it has to be followed by a manual code inspection, which is an inspection or review of the source code by specialists, according to defined rules. The output of the test is a proper test report and respective inputs for a defect data collection for statistical and process improvement purposes.

There are approximately 700 possible static problems or faults in C programming. In total there are approximately 350 to 400 fault conditions which can be detected by current state of the art static code analysis tools.

Types of Code Checkers

Not all the code checkers are doing the same kind of analysis. Again here I will focus on the code analysis tools which are suitable for C and the micro controller world. There are various tools available. They can be grouped into 3 main categories.

1. Simple Static Checkers

There is a commonly agreed area of C programming which is considered as dangerous programming. You will find valuable information in Andrew Koenig's book "C Traps and Pitfalls" and a number of other publications. Last but not least there is a so called MISRA standard which addresses the problem. There are a number of checkers which try to find as much as possible of these language transgressions. One of the most widely used is PC-Lint. However there are other similar ones, e.g. a tool called QA/C.

Some of these tools have a basic version for the more commonly agreed fault modes in programming and an extended version which offers additional MISRA checks. The MISRA standard is very strict and not all of their recommendations are commonly accepted by all industries. Therefore it makes sense to separate the basic and the MISRA version of the tools. However, you have to be aware that not all of the MISRA rules can be found by the checkers. There is a good amount of rules of which the transgressions can be hardly detected in an automatic way and they have to be enforced by reviews or inspections. Finally it has to be said that not all of the tools have the same performance.

2. Potential Runtime Error Checkers

During the recent years some promising attempts have been made to dig with static checkers into the domain of so far dynamic checks. One of the tools in this field is Polyspace. These checkers attempt to find runtime problems before the SW is executed. Imagine a code line like "unsigned short = unsigned short * unsigned short". For runtime and other resource reasons the programmer choose to use an unsigned short as variable for the result of the multiplication. Now this is potentially very dangerous. Depending on the values which have to be multiplied the calculation will overflow or not. Sophisticated checkers will attempt to find out and calculate the worst case inputs into such an operation. Depending on these worst case inputs they will issue a warning or not. The sophisticated checkers are usually very expensive. However not the price is the problem. These tools will usually come with a threefold result for critical code lines. They will give you a "green" in case everything is fine. They will give you a "red" in case they detected a real problem, and they will give you a "yellow" which basically means "I don't know you have to check by hand". Personally I would be grateful to use such a tool if they leave me with 10% or 20% of yellows. Then I have a big benefit. But if I get 80% to 90% yellow, I can not see the benefit. However, there is a future for these tools and as time goes on I am sure the benefit will rise.

3. Supporting Tools

There are also some supporting tools like SNIFF+ or CodeSurfer. They were made for re-engineering of software and better project overview. If you are confronted with an unknown source code for review or maintenance these tools make it much easier to dig into it and understand foreign code. Instead of only focusing on individual C expressions they visualize the logic and structure of a program. You will get call graphs and data flow charts out of these little helpers. They can not be considered as true automatic checkers but since automatic checks are only part of the bigger subject of static white-box tests, the supporting tools would be the bridge between automatic checks and manual checks, supplementing them with tool support.

Things to observe when you do automatic code checks

  1. Automatic code checks are part of proper software testing. Basically the same rules as for the other test activities also apply for automatic checks. E.g. they have to be specified and documented.
  2. You have to define a standard configuration for PC-Lint (or your other checker). Lint offers a wide range to customize the checks, but it makes no sense to switch off too many checks. The best solution is to define a standard set for your company. This set has then to be applied for every project. The only legitimate deviations may be due to compiler or controller related items as e.g. data size of "int" or the necessity to define own data types as e.g. "bit". However this has to be considered very carefully.
  3. The automatic checks have to be executed, and in case of detected errors they have to be reported. This has to be followed by a bug fix by the developer. This loop has to be repeated until no further errors are found. The End of Test Criteria is to be 100% error free on the defined Lint checks or MISRA related checks. In case this can not be reached and errors are remaining, an explanation has to be documented why the errors can not be fixed and that they do not constitute a risk.
  4. After identifying all errors they need to be eliminated by changing the code to comply with the Lint rules.
  5. Since automatic code checks can be done easily it is good working style if the developer checks his code by himself and immediately fixes his bugs. Some companies even have to rule that only Lint error free code can be checked in into the version management archives. If you adhere to this hint the testing activity using automatic code checks is only a quick and more formal activity. Verifying that the good working style was adhered to. The automatic check should be the first testing activity. It is the entry criteria for further testing. A code has to be rejected until it is "clean" and further testing as e.g. code inspections, module tests, etc. have to be done only on Lint error free code.
  6. You should know what your automatic code checker really checks and what is left to be checked by other tests. This is most important! Otherwise you may have gaps or you waste time in inspecting code for things which were already checked by the tool.