Squish Coco

Code Coverage Measurement for Tcl, QML, C# and C/C++

Part I

Chapter 1  Squish Coco – A code coverage tool for Tcl, QML, C# and C/C++

Squish Coco is a complete code coverage tool chain for Tcl, QML, C# and C/C++ programs (including SystemC programs). It runs on macOS, Linux  and Microsoft® Windows. For a full list of supported compilers, see the company website.

Squish Coco analyzes the way an application runs, e.g. as part of a test suite. The results can then be used to make the tests more efficient and complete. In particular, Squish Coco helps to

Squish Coco can be used at every stage of testing and for all testing methodologies (unit tests, automatic tests, manual white box tests, black box tests, etc.). A test suite can be divided into independent parts: Squish Coco makes it possible to merge multiple execution reports into a single unified report.

The main components of the Squish Coco package are two separate tools and an add-in:

  1. CoverageScanner – a program to analyze and instrument a Tcl, QML, C# and C/C++ application.
  2. CoverageBrowser – a program to display and manage the results of the coverage analysis.
  3. The Visual Studio® Coco Wizard – an optional add-in to configure code coverage for C# or C++ projects that are developed under Microsoft® Visual Studio® and similar IDEs.

There are also several helper programs for report generation, coverage data manipulation and license management.

Figure 1.1: Overview of the code coverage toolchain for Tcl, QML, C# and C/C++

1.1  CoverageScanner – Instrumentation as part of the build process

CoverageScanner instruments the source code that is seen by the compiler without touching the original code. It generates an instrumented executable, shared library or plugin. The instrumentation keeps track of the code that is executed when the executable is run or when the library or plugin is used. This allows measuring e.g. how many times each line is executed, or whether a line was executed at all. When the instrumented program terminates, it produces an execution report.

In order to get high-quality coverage measurements, CoverageScanner not only instruments at the function and statement level but also records statement coverage and decision coverage information. Another kind of instrumentation supports measurements of code execution times.

Optionally it is also possible to insert the test name and execution status (i.e. “passed” or “failed”) into the report. This can be done by a script or directly from the application. CoverageScanner can then be integrated into a test framework (like CppUnit or CxxTest), where it generates separate coverage information for each test.

CoverageScanner is a command line tool that replaces the compiler normally used to build the executable, library, or plugin. It inserts special instrumentation instructions into the preprocessed source code and then uses the original compiler to compile the modified code. At the same time, a database (the .csmes file) is generated for later analysis. It contains the list of instrumentations and a copy of the instrumented source code.

After compilation, the instrumented executable can be run as usual (and an instrumented library or plugin can be used via the program that accesses them), but now, when the program terminates, an execution report is produced (the .csexe file).

CoverageScanner works with almost any standard compiler, including GNU gcc, GNU g++, Microsoft® Visual Studio® 6.0, .NET, Express, and Embedded C++, Intel® C++, etc. For most other compilers, CoverageScanner can be easily configured.

CoverageScanner has built-in support for the Qt Framework which makes it possible to disable the instrumentation of the moc-generated code.

1.2  CoverageBrowser – Viewing, analyzing and managing code coverage results

The execution report in the .csexe file can be analyzed and managed with the CoverageBrowser tool. This program has a GUI (graphical user interface) by which the user can browse and manage their execution reports. This makes it straightforward to locate untested or unreachable code and inefficient tests.

A summary of CoverageBrowser’s abilities:

Figure 1.2: CoverageBrowser: Code coverage screen shot