Squish Coco

Code Coverage Measurement for Tcl, QML, C# and C/C++

Part I
Overview

Chapter 1  Squish Coco – A code coverage tool for Tcl, QML, C# and C/C++

Squish Coco is a complete code coverage tool chain for Tcl, QML, C# and C/C++ programs that runs on macOS, Linux  and Microsoft® Windows. It analyzes the performance of an application being exercised (e.g. by a test suite), to reveal important information that can improve the testing of Tcl, QML, C# and C/C++ applications.

In particular, Squish Coco helps to

Squish Coco can be used at every stage of testing and for all testing methodologies (unit tests, automatic tests, manual white box tests, black box tests, etc.). A test suite can be divided into independent parts: Squish Coco makes it possible to merge multiple execution reports into a single unified report.

The main components of the Squish Coco package are two separate tools and an add-in:

  1. CoverageScanner – a program to analyze and instrument a Tcl, QML, C# and C/C++ application.
  2. CoverageBrowser – a program to display and manage the results of the coverage analysis.
  3. The Microsoft® Visual Studio® Add-In – an optional add-in to configure code coverage for C# or C++ projects that are developed under Microsoft® Visual Studio®.

There are also several helper programs for report generation, coverage data manipulation and license management.

pictures/manual.tmp001.png
Figure 1.1: Overview of the code coverage toolchain for Tcl, QML, C# and C/C++

1.1  CoverageScanner – Instrumentation as part of the build process

CoverageScanner instruments the source code that is seen by the compiler without touching the original code. It generates an instrumented executable, shared library or plugin. The instrumentation keeps track of the code that is executed when the executable is run or when the library or plugin is used. This allows to measure e.g. how many times each line is executed, or whether a line was executed at all. When the instrumented program terminates, it produces an execution report.

In order to get high-quality coverage measurements, CoverageScanner not only instruments at the function and statement level but also records statement coverage and decision coverage information. Optionally it is also possible to insert the test name and execution status (i.e. “passed” or “failed”) into the report. This can be done by a script or directly from the application. CoverageScanner can then be integrated into a test framework (like CppUnit or CxxTest), where it generates separate coverage information for each test.

CoverageScanner is a command line tool that replaces the compiler normally used to build the executable, library, or plugin. It inserts special instrumentation instructions into the preprocessed source code and then uses the original compiler to compile the modified code. At the same time, a database (the .csmes file) is generated for later analysis. It contains the list of instrumentations and a copy of the instrumented source code.

After compilation, the instrumented executable can be run as usual (and an instrumented library or plugin can be used via the program that accesses them), but now, when the program terminates, an execution report is produced (the .csexe file).

CoverageScanner works with almost any standard compiler, including GNU gcc, GNU g++, Microsoft® Visual Studio® 6.0, .NET, Express, and Embedded C++, Intel® C++, etc. For most other compilers, CoverageScanner can be easily configured.

CoverageScanner has built-in support for the Qt Framework which makes it possible to disable the instrumentation of the moc-generated code.

1.2  CoverageBrowser – Viewing, analyzing and managing code coverage results

The execution report in the .csexe file can be analyzed and managed with the CoverageBrowser tool. This program has a GUI (graphical user interface) by which the user can browse and manage their execution reports. This makes it straightforward to locate untested or unreachable code and inefficient tests.

A summary of CoverageBrowser’s abilities:

pictures/manual.tmp002.png
Figure 1.2: CoverageBrowser: Code coverage screen shot
email squish@froglogic.com