Menu

Squish Coco

Code Coverage Measurement for Tcl, QML, C# and C/C++

Part XII
Frequently Asked Questions

Installation

Is there a way to deploy Squish Coco on Windows without any user interactions?

The Squish Coco installer for Windows supports the following command line options:

/S
runs the installer in silent mode
/SD
runs the installer in silent mode and suppress all popup dialog
/D
specification of the installation directory

For example, to install Squish Coco in background, do:

SquishCocoSetup.exe /SD

For example, to install Squish Coco in background on c:\Program Files\squishcoco, do:

SquishCocoSetup.exe /SD "/D=c:\Program Files\squishcoco"

The Microsoft® Visual Studio® Add-In is not installed

To verify that Microsoft® Visual Studio® Add-In is loaded by Microsoft® Visual Studio®, do the following:

  1. Open "My Document" folder.
  2. Open the folder "Visual Studio 2005" (or "Visual Studio 2008")
  3. Open the folder "Addins"
  4. You should find the files Visual Studio 2005\SquishCocoVs2005AddIn.dll and SquishCocoVs2005AddIn.addin. If the files are missing, please reinstall Squish Coco.
  5. Start Microsoft® Visual Studio®.
  6. Click on "Tools->Options…". Select the entry Environment - Add-in/Macro Security. Verify that %VSMYDOCUMENTS%\addins is present in the path, and that the macros are enabled.
    pictures/manual.tmp056.png
    Figure @.1: Microsoft® Visual Studio® Option Dialog
  7. Click on "Tools->Add-In Manager…". The list of loaded addins is displayed. Verify that Squish Coco is checked and also that the startup flag is set.
    pictures/manual.tmp057.png
    Figure @.2: Add-In Manager
  8. Open a C or C++ project.
  9. Click on "Tools->Code Coverage Build Mode…". The Microsoft® Visual Studio® Add-In window should appear.

Microsoft® Visual Studio® Add-In is crashing when "My Documents" is on a network drive

Due to some security policy, loading a DLL in Microsoft® Visual Studio® from a network share is not allowed. The solution is to install the Microsoft® Visual Studio® Add-In on the local drive.

Proceed as follows:

  1. Create a folder called "C:\VisualStudioPlugins"
  2. Start Visual Studio. Open the option dialog and add "C:\VisualStudioPlugins":
    1. Click on Tools Menu->Options.
    2. Select Environment->Add-In/Macro Security.
    3. Add the path "C:\VisualStudioPlugins".
  3. Move the contents of "My Documents\Visual Studio 2010\Addins\*SquishCoco*" to "C:\VisualStudioPlugins".
  4. Restart Microsoft® Visual Studio®.

Installation problems on macOS 10.12

Squish Coco is not yet considered a trusted source by macOS, therefore it is not yet possible to install it directly on newer versions of macOSby clicking on the .pkg file.

However, there is a simple method to circumvent this problem:

  1. Locate the downloaded .pgk file in the Finder.
  2. Click on it with the Ctrl key pressed.
  3. A menu appears. In it, select the “Open” entry.

Then a dialog box appears which allows you to override the trust settings for this file. Afterwards, the installation begins as usual.

Code coverage for C# is no longer working

If the C# compiler is updated, the Squish Coco setup for code coverage is overwritten and code coverage no longer works. The Squish Coco installer must then be run again (see Chapter 2.3).

Compiling

Using CoverageScanner with ccache (Unix only)

ccache is a compiler cache which permits to speedup the recompilation of objects. To use CoverageScanner with ccache it is necessary to set the PATH variable as followings:

export PATH=/opt/SquishCoco/wrapper/bin:/opt/ccache/bin:$PATH

Create then a ccache wrapper got g++ and gcc as follows (as root):

mkdir -p /opt/ccache/bin
ln -s /usr/bin/ccache /opt/ccache/bin/g++
ln -s /usr/bin/ccache /opt/ccache/bin/gcc

Which this configuration, calling gcc will:

  1. call CoverageScanner’s gcc wrapper
  2. which call ccache’s gcc wrapper
  3. which finally call the real gcc binary.

Compiling issue when using the Boost library

When compiling a project using the boost library, the following errors occurs during the compilation:

5>c:\boost\boost_1_47_0\boost\type_traits\type_with_alignment.hpp(294):
   error : col 2:syntax error
5>Error:Could not insert insert instrumentation in file test.cpp

The project is instrumenting the boost library, it is necessary to disable the instrumentation of it. Simply add --cs-exclude-file-abs-wildcard="c:\boost\boost_1_47_0\*" to the list of the compiler command line arguments.

Microsoft® MSBuild Does Not call CoverageScanner on x86 Platforms

Proceed as follows:

Microsoft® MSBuild projects are not instrumented under Jenkins CI

The problem occurs when a project has build without problems under a user account, but the compilation fails when it is run under Jenkins CI.

The problem manifests in compiler errors like this one:

cl : Command line warning D9002: ignoring unknown option '--cs-on' [C:\Program Files\Jenkins\workspace\SquishCoco\example.vcxproj]

Here Microsoft® MSBuild has called the compiler directly instead of the wrapper, and the compiler did not understand the CoverageScanner options.

To correct this, it is necessary to set the environment variable LOCALAPPDATA at the beginning of the Jenkins build script. Its value must be the path to the AppData directory of another user, i.e. something like C:\Users\SomeUser\AppData\Local. The build script may then look like this:

set LOCALAPPDATA=C:\Users\SomeUser\AppData\Local
"C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe" "C:\Program Files\Jenkins\workspace\SquishCoco\example.sln" /t:ReBuild  /p:Configuration=DebugCoverage

Instrumentation

Compiling Issues of Lambda Expressions with Microsoft® Visual Studio® 2010?

Microsoft® Visual Studio® 2010 des not provide a complete support of C++11 and so expressions in the form “[](int x) { return x; }” may not compile once instrumented. To support such expressions, Squish Coco provides an experiemental instrumentation than can be activated through the command line option --cs-vs2010-lambda.

How to exclude a source file from the code coverage analysis?

Excluding some source files from the code coverage analysis needs to be performed during the compilation. Two methods are possible:

  1. Compile using CoverageScanner with the command line option --cs-exclude-file-regex, --cs-exclude-file-wildcard and --cs-exclude-path.
  2. Use CoverageScanner pragmas to deactivate the instrumentation, by adding the following lines after all #include commands.
    #ifdef __COVERAGESCANNER__
    #pragma CoverageScanner (cov-off)
    #endif
    Adding this portion of code at the top of the source file (and so before all #include commands), will also deactivate the coverage analysis of the headers.

My source code contains inline functions, are they instrumented?

Inline functions are instrumented like other functions. To avoid instrumenting templates and inline functions of 3rd party library or the standard C++ library, CoverageScanner instrument all functions and procedures of the source file and the headers which are not present in the system directories. On UNIX®, this correspond to /opt, /usr and $QTDIR. On Microsoft® Windows, this correspond to the standard application installation directory and the QTDIR path. If its header file is in these directories, it is necessary to tell CoverageScanner to instrument it too. The command line options --cs-include-file-regex, --cs-include-file-wildcard and --cs-include-path permits to force the instrumentation of additional files.

Is it normal that the .csmes file contains a copy of the source code?

Yes. The .csmes file contains all information necessary for CoverageBrowser. That’s why the source code and the preprocessed source file is also included.

CoverageBrowser

Is there a way to minimize the CPU usage?

CoverageBrowser needs a lot of CPU resources and disk accesses to calculate the statistics for each execution. This calculation is performed in background, but can also be deactivated. Just proceed as follows: hide the column Coverage of the docking window "Executions" using the context menu "Show/Hide Columns->Coverage".

When I’m exporting the statistics per methods I can only see the 65536 first entries

This is a limitation of Microsoft Excel or OpenOffice. Excel 97 can only display 16384 rows. Excel 2000 and Excel 2003 can display up to 65536 rows and newer version of Excel can display up to 1048576 rows. OpenOffice Calc is limited to using 32000 rows.

Unit Tests

How to get the code coverage of a library from my unit tests?

We suppose that the project contains N libraries in the directories libprojN. Each library generated are static libraries, and they are generated in libprojN. We have N unit tests, all of them are located in testprojM directories.

So the directory tree looks as followings:

libproj1
  |   libproj1.a
  +-- libproj1.a.csmes
libproj2
  |   libproj2.a
  +-- libproj2.a.csmes
......
libprojN
  |   libprojN.a
  +-- libprojN.a.csmes

testproj1
  |   testproj1.exe
  +-- testproj1.exe.csmes
testproj2
  |   testproj2.exe
  +-- testproj2.exe.csmes
.....
testprojM
  |   testprojM.exe
  +-- testprojM.exe.csmes

First we need to execute the unit tests:

for TESTDIR in testproj*
do
   cd $TESTDIR
   ./$TESTDIR.exe
   cd ..
done

After execting this bash script, a .csexe file is created for each unit test:

testproj1
  |   testproj1.exe
  |   testproj1.exe.csexe
  +-- testproj1.exe.csmes
testproj2
  |   testproj2.exe
  |   testproj2.exe.csexe
  +-- testproj2.exe.csmes
.....
testprojM
  |   testprojM.exe
  |   testprojM.exe.csexe
  +-- testprojM.exe.csmes

We import then the testprojM.exe.csexe file into its testprojM.exe.csmes:

for TESTDIR in testproj*
do
   cmcsexeimport -m $TESTDIR/$TESTDIR.exe.csmes -e $TESTDIR/$TESTDIR.exe.csexe \
                 --delete --title="$TESTDIR"
done

Now, the files testprojM.exe.csmes are containing the coverage information of the unit tests and the libraries tested. The file testprojM.exe.csexe is deleted since that it is no more used.

It is now possible to import the coverage information of the unit tests in the .csmes file of each library tested:

TESTPROJs=$(ls testproj*/testproj*.exe.csmes)
for LIBRARY_DIR in libproj*
do
   LIBPROJ=$LIBRARY_DIR/$LIBRARY_DIR.a.csmes
   cmmerge -a -o $LIBPROJ -i $LIBPROJ $TESTPROJs
done

This command imports all unit test results into each libprojN.a.csmes. The -i command line option permits to ignore the source files which are not present into libprojN.a.csmes. This permits to ignore the test code during the merge process.

Reports

Is is possible to compute a code coverage report of selection of files?

cmreport has the possibility to exclude files from a code coverage report. If we want for example to compute a report of all files present only in mysrc directory, then proceed as followings:

  1. First, exclude all sources files: the command --exclude-file-abs-wildcard="*" exclude all source files.
  2. Finally, include all sources files from the directory mysrc: the command --include-file-abs-wildcard="*/mysrc/*" include all source files whose absolute path constains /mysrc/.

A complete command would be:

cmreport  --csmes=project.csmes \
          --html=project.html \
          --exclude-file-abs-wildcard="*" \
          --include-file-abs-wildcard="*/mysrc/*"

Miscellaneous

Is it possible to debug the CoverageScanner API?

To debug the functions provided by the CoverageScanner API it is necessary to:

  1. force CoverageScanner to keep the generated source file after compiling and linking. Therefore it is necessary to add --cs-keep-instrumentation-files to the compiler and linker command line arguments.
  2. compile the CoverageScanner library with debug information. Therefore it is necessary to add --cs-libgen=-g for GCC (resp. --cs-libgen=/Zi for Microsoft® Visual Studio®) to the linker command line option.

Once the application is recompiled, it is possible to set a breakpoint in __coveragescanner_save() and step through the source code.

How to report an issue concerning CoverageScanner?

It is possible to generate a log file which allows froglogic to investigate issues concerning CoverageScanner.

Proceed as follows on Microsoft® Windows:

  1. Click on Start->Programs->Squish Coco->Support->CoverageScanner Log Recording This will install a debug version of CoverageScanner which will produce a set of log files.
  2. Reproduce your issue.
  3. In the console window, press enter to install the normal version of CoverageScanner.
  4. The log files are located in %SQUISHCOCO%\logfiles. Zip it together and send it to Squish Coco support (pictures/email.gif squish@froglogic.com).

Proceed as follows on Linux:

  1. In a console window:
    cd /opt/SquishCoco/{version of Squish Coco}
    mv coveragescanner coveragescanner.bak
    cp coveragescannerdbg coveragescanner

    This will install a debug version of CoverageScanner which will produce a set of log files.

  2. Reproduce your issue.
  3. Restore the release version of CoverageScanner:
    cd /opt/SquishCoco/bin
    mv coveragescanner.bak coveragescanner
  4. The log files are located in /tmp/squishcoco or %TEMP%\SquishCoco_logfiles. Zip it together and send it to Squish Coco support (pictures/email.gif squish@froglogic.com).
The log file may contain parts of your source code. If this is an issue, you may remove the concerned lines with your favorite editor.
email squish@froglogic.com