Code Quality and Risk Assessment for the Dutch ATC

The Issue at hand

The Dutch Air Traffic Control organization was considering outsourcing the further development and maintenance of one of its applications to an external party. As this is a public organization, they need to adhere to the European tender laws. Therefore, they wished to create a technical due diligence document to create a level playing field of the parties submitting for this contract.

The Service Provided

We carried out a Software Quality and Risk Assessment using CAST Imaging technology, measuring against all ISO standards (e.g., ISO 25010 and ISO 5055) and industry best practices. The application source code was shared via a secure connection, and we ran the analysis offsite.

The research questions included:

Assess and benchmark in detail and in an objective way:

• The maintainability of the software, especially regarding:

  • solving software bugs and

  • implementing new functionality

  • The reliability and Robustness of the software.

  • The efficiency of the software.

  • The security of the software.

  • The total quality of the software.

  • The number of Critical violations in the software.

  • The technical debt in the software.

  • The transferability of the software:

  • How hard is it to transition the maintenance activities to the customer?

  • How hard is it to transition the development activities to customer?

The outcomes achieved

We presented the results in a management session, where all the answers to the research questions were answered. We showed the results in the Management dashboard, explaining all high-level KPI’s of the software. Although the measurement results showed that several actions need to be carried out to improve the quality of the software to an acceptable level, the number of actions necessary and the associated effort are relatively low.

We delivered a practical Action Plan where the actions are listed regarding the critical violations that need to be fixed and where these violations reside in the code. This is part of the Engineering Dashboard, which was shown to the technical experts of Customer and Supplier in a separate technical meeting. The results of the Software Quality and Risk Assessment are input for the Customer management decision whether to acquire the software or not.

Example Health and Risk dashboard (score 1 to 4)

Example Engineering dashboard (critical issues)