.Net code coverage is most meaningful when it gives my team a full view of which portions of our code have been tested. While test runners and ad hoc coverage tools certainly help our developers refine testing and target methods within their .net apps, it’s this full coverage picture that is most instructive at the team level and most critically in quality assurance. It’s not possible or practical to run all testing for a given software build within a single process and code coverage session. Any code coverage tool for .net that doesn’t allow me to merge results from multiple samplings is limited in it’s ability to give me the big picture of how our code has been tested. Merging coverage results from multiple testing cycles becomes essential to strategic quality improvement.
Merging code coverage results from separate executions has been available in NCover 4 through the GUI, but today it makes it’s premier at the command line. This feature was first released with NCover v4.1.2056. Previous blog posts have described build integration, but a feature absent prior to now was the ability to in an unattended manner merge coverage results based on a build id or on an execution time frame. This dovetails nicely with the recently released .Net Code Coverage Summary Report as well.
Syntax of the merge command
> ncover merge Options --project=VALUE NCover project name. *REQUIRED* --buildId=VALUE Specify build-id for merging executions --start-time=VALUE Specify min start time of executions to merge --end-time=VALUE Specify max end time of executions to merge --timeout=VALUE Timeout request after number of seconds
It’s worth noting that the buildId option can be used in conjunction with the start and end time options, but it is not required. Project plus either a buildId or a start-time are the minimum requirements to initiate a merge.
Example command line merge by build Id
> set NCOVER_BUILD_ID=%date%_%time% // any build id will work
<<< Run tests and collect coverage >>>
> ncover merge --project="my project" --buildId=%NCOVER_BUILD_ID%
This strategy shows how I can set the reserved environment variable for NCover that will assign a build Id to executions as they are captured. Upon completion of that testing cycle the merge command gathers all the completed executions with a matching build id and creates a single merge. This command is safe to run repetitively so that executions for a single regression cycle, which may take days, can be continually merged.
Example command line merge all executions since a starting time
> set my_start_time_var=%date% %time%
<<< Run build and test scripts here >>>
> ncover merge --project="some ncover project" --start-time=%my_start_time_var%
This form of the command is particularly useful for build integration. This allows me to capture the starting date or time in an environment variable and then pass it to NCover once all other steps of the process are complete and code coverage executions have finalized. It’s not important how the date and time are captured only that the format of that parameter matches the localized date and time conventions of my windows setup. If I manually execute a merge by date and time on a machine using typical US English localization, that command looks much like the following:
> ncover merge --project="my project name" --start-time="10/24/2012 1:00pm" --end-time="10/24/2012 23:59"
This is not an exhaustive discussion of executions and merging for NCover, but there is more to read in the online documents.
Merging from NCover GUI
Merge command