Code Coverage is an interesting topic – Wikipedia defines it as “the degree to which the source code of a program has been tested.” The idea is that you want to measure what percentage of the lines of code in your application are executed when you run your test suite. The “perfect” test suite would execute every single line of code in your codebase. Being able to definitively measure that goes a long way towards validating your testing suite and giving you confidence that your tests are thorough and testing everything that needs to be tested.
So I was really happy to run across the Delphi Code Coverage tool. I was intrigued and interested, so I downloaded it and gave it a look. And I really liked it. I decided to run it through its paces by checking the code coverage of the unit tests for my THTMLWriter project. It didn’t take me long to figure out how to use it. It’s written by Christer Fahlgren. He blogged about it back in 2010. It has a couple of other committers, and it’s been an active project throughout out 2012.
The project includes this note: “The 1.0 release was made possible through the generous support of DevFactory.” I thought that was cool, and so I encourage you to go to their website and check out their services.
Some things to note off the bat:
So what did I do? Well, first I pulled the source to make sure I had it. Then, I went ahead and downloaded the RC7 build, which has one file in it: codecoverage.exe. I took that file and put it in a folder along my path so I can call it wherever I need it.
Then, I read the nice simple documentation. I was just testing it out, so I ran everything manually via the IDE just to see how it all worked. Eventually, you’d want to integrate this into your continuous integration. So here is a quick rundown of what I did and what I found.
First, I opened up the project options for my THTMLWriter unit test application and told the Linker to create a MAP file for the project when I build it. The tools uses the MAP file to trace all the code paths that the application under test uses.
Next, I looked for the appropriate command line switches.
- I used it –e switch to name the executable that would run.
- I used the –m switch to name the *.map file.
- I used the –u switch to tell the tool what code unit I wanted analyzed. Note that the switch wants the unit name, not the file name. You can list as many units as you want after this switch. Alternatively, you can use the –uf switch to point to a text file that has a list of unit names (one per line) to be examined. In my case, I’m interested in the code coverage for the uHTMLWriter.pas unit.
- I used the –html switch to indicate that I wanted the output to be an HTML file. This would make it easier for me to look at the results in this “by hand” method of running things.
The resulting command line was as follows:
codecoverage -e HTMLWriterTestAppTests.exe -m HTMLWriterTestAppTests.map -u uHTMLWriter -html
Note that the above command line assumes that the current directory of your command prompt is the directory where the files in question are located. I actually put the command line in a small batch file in the same directory.
So then I executed the command. The tool popped up the GUI Runner for DUnit, I pressed the Run button, the tests ran, and when they were done, the tool finished up and produced some output in the form of an HTML file. (It also provided a FastMM dialog at the end of its run, which took a few seconds to appear).
As noted, the result was two HTML files -- one summary file:
and a file that shows me the coverage for the uHTMLWriter unit. That has a header in it that tells me that I have 98% code coverage. Not bad!
But the real fun part is that the file contains a complete listing of the unit, color-coded to indicate which lines were executed (in green) and which weren’t (in purple). So 2% of my code isn’t covered? So I scanned the file and came across this:
Looks like I never wrote a unit test for the OpenLabel method that takes a string parameter. (I have one for the parameter-less OpenLabel). Okay, so I went back, wrote a unit test for that one, and re-ran the code coverage tool, and now I have code coverage for that method:
Nice! The tool also pointed out that I don’t test that a number of exceptions get properly raised, so I’ll get working on those, too. I’d like to be able to run the report and have it return 100% for my coverage.
Overall, this is a nice and powerful tool to help you make sure that your unit tests really are running every line of code in whatever you are unit testing. It’s another great contribution to the Delphi community. Thanks, Christer.