Menu

Squish Coco

Code Coverage Measurement for Tcl, QML, C# and C/C++

Part XI
Coco Internals

Chapter 41  File system and registry

This chapter is about the usage Squish Coco makes of the file system to store permantent settings.

41.1  Location of the installed files

The location of the installed files varies according to the platform.

41.1.1  Microsoft® Windows

The location of the programs can be chosen at installation time. The installer creates and environment variable SQUISHCOCO that contains the path to the installation directory. By default it is either C:\Program Files\squishcoco or C:\Program Files (x86)\squishcoco, depending on the platform. In this documentation we use the expression Windows Coco to refer to it.

This directory contains the following files and directories:

41.1.2  Linux

The location can be chosen at installation time; its default value is /opt/SquishCoco/. The directory has the following subdirectories:

bin/, lib/
The Squish Coco binaries, including the prefix versions of the compiler wrappers and their profile files.
wrapper/bin/
The compiler wrappers as files without the cs prefix.
doc/
Documentation files.
samples/
Example programs and files for the tutorials.

41.1.3  macOS

The programs are installed at /Applications/SquishCoco.

This directory contains the Squish Coco binaries, including the prefixed versions of the compiler wrappers and their profile files. It also contains the documentation. It has the following subdirectories:

wrapper/
The compiler wrappers as files without the cs prefix.
samples/
Example programs and files for the tutorials.

41.2  Location of the license

The professional and the non-commercial edition of Squish Coco use different naming schemes to locate the files and registry keys that contain information about the license. In the following description, the professional edition is primarily described, with the values that change in the noncommercial edition following in parentheses.

41.2.1  Node-locked licenses

When searching for a license, Squish Coco first tries to find a node-locked license.

  1. If the environment variable SQUISHCOCO_LICENSEKEY_DIR is set, Squish Coco treats the content as a directory name and tries to find there the license file. The license file is named .squishcoco-3-license.
  2. If Squish Coco runs under UNIX® or macOS, it then searches for a license file in the directories $HOMEPATH, $HOME and in the root directory, ‘/’.
  3. If Squish Coco runs under Microsoft® Windows, it searches instead in the registry under HKEY_CURRENT_USER\Software\squishcoco\LicenseKey.

It is possible that a license is present but that it has been deactivated. If Squish Coco finds such a license, it stops searching for another node-locked license and tries to find a license server instead.

Setting SQUISHCOCO_LICENSEKEY_DIR globally for all users is not recommended. It would force all users to use the same license, but each license key is specific to user and host.

41.2.2  Address of the license server

If no node-locked license is found, Squish Coco will try to find the location of a license server.

  1. First it tries to read the specification of a license server from the environment variable SQUISHCOCO_LICENSE_SERVER, if such a variable is present and its value is not the empty string.
  2. If Squish Coco runs under UNIX® or macOS, it then tries to read the specification from a file .squishcoco-3-licserver.

    It searches for this file in the directories $HOMEPATH, $HOME and the root directory, ‘/’.

  3. If Squish Coco runs under Microsoft® Windows, then it searches in the registry under HKEY_CURRENT_USER\Software\squishcoco\LicenseServer for a specification.

The specification of the license server consists of a host name or an IP address, possibly followed by a colon and a port number, like ‘myserver.com:49344’.

41.3  Location of the temporary files

During compilation, the CoverageScanner creates some temporary files, which it usually deletes automatically after use.

By default, the temporary files are created in the system temporary files directory. This is /tmp on UNIX® systems, and the directory given by the environment variable %TEMP% on Microsoft® Windows. It is possible to change this loaction by setting the environment variable SQUISHCOCO_TEMP_DIR to the path of another directory. This directory must already exist when the files are written, it is not created automatically.

The automatic deletion of temporary files can be switched off with the command line option --cs-keep-instrumentation-files (see Chapter 9.2.5).

41.4  Location of the program settings

If a program has other permanent settings that must be stored in a file, that file is located in the following directory:

On Windows
%APPDATA%\squishcoco
On Linux
$HOME/.config/squishcoco

41.5  Location of the installation log file

The Windows installer produces a log file at %TEMP%\SquishCoco_logfile\vsaddin.log. It can be consulted after installation problems.

Chapter 42  Supported compilers

The following compilers are currently supported by CoverageScanner:

42.1  C# compilers

The command line C# compiler of Microsoft® Visual Studio® .NET and Mono C# compiler are supported.

Native CommandCoverageScanner Command
mcscsmcs
gmcscsgmcs
dmcscsdmcs
csccscsc

42.2  Microsoft® Visual C++

The command line compiler and linker of Microsoft® Visual C++ and Microsoft® Visual C++ Toolkit 2003 are supported.

Native CommandCoverageScanner Command
clcscl
libcslib
linkcslink

42.3  Intel® C++ Compiler

The C and C++ command line compiler from Intel® is supported.

Native CommandCoverageScanner Command
iclcsicl
icccsicc
icpccsicpc

42.4  GNU gcc

Only the g++ and gcc command line compilers are directly supported.

Native CommandCoverageScanner Command
gcccsgcc
g++csg++
arcsar

42.4.1  Special compiler versions for cross-compiling

If a GNU compiler is used for cross-compiling, it often has a special name, like ‘arm-linux-gcc’ instead of ‘gcc’.

The installation script of Squish Coco tries to create automatically the compiler wrappers for every GNU compiler installed on the machine. In general, all necessary compiler wrappers are therefore present in the system. A cross-compiler may however also reside in a non-standard directory in the system, where it will not be found by the Squish Coco installer.

If a compiler configuration is missing, the following can be done:

  1. Copy ’coveragescanner’ to the file ’cscompiler⟩’.

    Example: On Microsoft® Windows, this would be something like:

    C:\Program Files\squishcoco>copy coveragescanner.exe csarm-linux-gcc.exe
  2. Copy the profile ’gcc.cspro’ or g++.cspro’ to ’⟨compiler.cspro’.

    Example

    C:\Program Files\squishcoco>copy gcc.cspro arm-linux-gcc.cspro

The new GNU cross-compiler ’cscompiler⟩’ can now be used. It instruments the code and then calls ’⟨compiler⟩’ for compilation.

Chapter 43  Code insertion

(This chapter is a continuation of the example in Chapter 5.)

CoverageScanner inserts the instrumentation code as illustrated in the following examples. (The inserted code is displayed in bold blue.)

Sequential Statement
The statements are instrumented before their execution. The instrumentation consists in allocating a Boolean variable which detects if the code was executed or not.

Example

a=foo();
a++;
break;

will be changed into:

a=foo();
a++;
{ inst[0]=1; break}

inst[0] is set to 1 if the ‘break’ statement is executed.

Conditional Statements and Boolean Expressions (full instrumentation)
For Boolean expressions the same principle applies except that in addition to recording the execution itself, the state (true or false) is also recorded.

Example

if ( a<b )
  return 1;

will be changed into:

if ( (a<b) ? inst[0]=1 : inst[1]=1,0 )
  return 1;

 inst[0]  is set to 1 if the Boolean expression a<b was true.

inst[1] is set to 1 if the Boolean expression a<b was false.

Conditional Statements and Boolean Expressions (partial instrumentation)
In some cases, recording the value of a Boolean expression is unnecessary for sequential statement instrumentation. For example, the statement if (b) return 0; else return 1; is completely covered by a statement coverage—after all, recording whether b becomes true or false does not provide any extra information. Similarly, for the statement if (b) return 0; it is only necessary to check if b was false. By default, Squish Coco suppresses the generation of redundant instrumentation in order to minimize the instrumented code’s size and to maximize execution speed.

Example

if ( a<b )
  return 1;

will be changed into:

if ( a<b )
  return 1;
else inst[0]=1 ;

inst[0] is set to 1 if the Boolean expression a<b was false. Since the statement coverage records the instruction return 1;, it is not necessary to record if a<b was true;

The generated instrumentation code that needs to be inserted to provide statement coverage for the earlier foo() function example means that the function’s code is transformed to the code shown here:

   1 char inst[5];
   2 void foo()
   3 {
   4   bool found=false;
   5   for (int i=0; (i<100) && (!found); ++i)
   6   {
   7     if (i==50 ) { inst[0]=1;break;}
   8     if (i==20 ) { inst[1]=1;found=true;}
   9     if (i==30 ) { inst[2]=1;found=true;}
  10   inst[3]=1; }
  11   printf("foo\n");
  12 inst[4]=1; }
Figure 43.1: Code coverage instrumentation at statement block level

If we insert the instrumentation code necessary to support decision coverage into this example, the resulting code will look like this:

   1 char inst[13];
   2 void foo()
   3 {
   4   bool found=false;
   5   for (int i=0; ((i<100 && !found)?inst[0]=1:inst[1]=1,0); ++i)
   6   {
   7     if ((i==50?inst[2]=1:inst[3]=1,0){ inst[4]=1; break;}
   8     if ((i==20?inst[5]=1:inst[6]=1,0){inst[7]=1; found=true;}
   9     if ((i==30?inst[8]=1:inst[9]=1,0){inst[10]=1; found=true;}
  10   inst[11]=1; }
  11   printf("foo\n");
  12 inst[12]=1; }
Figure 43.2: Code coverage instrumentation at decision level

If we insert the instrumentation code necessary to support condition coverage into this example, the resulting code will look like this (except that we have wrapped one line to fit better on the page):

   1 char inst[15];
   2 void foo()
   3 {
   4   bool found=false;
   5   for (int i=0;((i<100)?inst[0]=1:inst[1]=1,0) &&
   6        ((!found)?inst[2]=1:inst[3]=1,0); ++i) {
   7     if ((i==50?inst[4]=1:inst[5]=1,0){inst[6]=1; break;}
   8     if ((i==20?inst[7]=1:inst[8]=1,0){inst[9]=1; found=true;}
   9     if ((i==30?inst[10]=1:inst[11]=1,0){inst[12]=1; found=true;}
  10   inst[13]=1; }
  11   printf("foo\n");
  12 inst[14]=1; }
Figure 43.3: Full code coverage instrumentation at condition level

Here is what the code would look like if we inserted the partial instrumentation code for condition coverage (again, with one line wrapped):

   1 char inst[12];
   2 void foo()
   3 {
   4   bool found=false;
   5   for (int i=0; ((i<100)?inst[0]=1:inst[1]=1,0) &&
   6        ((!found)?inst[2]=1:inst[3]=1,0); ++i) {
   7     if (i==50 ) {inst[4]=1; break;} else inst[5]=1;
   8     if (i==20 ) {inst[6]=1; found=true;} else inst[7]=1;
   9     if (i==30 ) {inst[8]=1; found=true;} else inst[9]=1;
  10   inst[10]=1; }
  11   printf("foo\n");
  12 inst[11]=1; }
Figure 43.4: Partial code coverage instrumentation at condition level

Chapter 44  Code Coverage Benchmarks

44.1  Test Algorithm

The sorting algorithm used for the tests is quicksort.
Source code:

To download: sort.c pictures/zoom.png

44.2  Benchmarks

CompilerNormal Execution
(time)
Execution
Branch Coverage
or Function Coverage

(time)
Execution
Decision Coverage
or
Line Coverage

(time)
Execution
Condition Coverage

(time)
GCC without optimization 6840ms8030ms (+17.3%) 9400ms (+37.4%) 9830ms (+43.7%)
GCC with optimization -Os 4160ms4430ms (+6.4%) 6180ms (+48.5%) 6360ms (+52.8%)
GCC with optimization -O1 3530ms4250ms (+20.3%) 5420ms (+53.5%) 5970ms (+69.1%)
GCC with optimization -O2 3950ms4040ms (+2.2%) 5080ms (+28.6%) 5320ms (+34.6%)
GCC with optimization -O3 3860ms4030ms (+4.4%) 5060ms (+31%) 5230ms (+35.4%)
Table 44.1: Benchmark (sorting algorithm)
CompilerNative Compilation
(bytes)
Statement Block Coverage
or
Function Coverage

(bytes)
Decision Coverage
or
Line Coverage

(bytes)
Condition Coverage
(bytes)
GCC without optimization 2128 3452 (+1324) 3996 (+1868) 4052 (+1924)
GCC with optimization -Os 1940 3196 (+1256) 3716 (+1776) 3760 (+1820)
GCC with optimization -O1 1968 3172 (+1204) 3736 (+1768) 3792 (+1824)
GCC with optimization -O2 2100 3452 (+1352) 4096 (+1996) 4160 (+2060)
GCC with optimization -O3 3544 3548 (+4) 4364 (+820) 4428 (+884)
Table 44.2: Sorting algorithm code size of the object
email squish@froglogic.com