Schema Location
Schema Location
Would it be possible to add an option to five the location of the XSL file used in the XML output? I would like to have the output file to point to a XSL file in a different directory.
Re: Weird 'Line' and 'End' values
This is a placeholder value that the compiler uses for code that it generates. I plan to exclude these sequence points by default before shipping. For now you can safely ignore them.
Re: Weird 'Line' and 'End' values
Hello,
i am not sure that i understand these weird lines correctly. You say i
can safely ignore them, but they influence the total coverage
percentage for the method they are in.
Do these lines actually exist? Are the line numbers just incorrect? If so, how can i find the correct line numbers?
Or are these lines "fake lines" that actually don't exist? If so, one
can not safely ignore them, because they influence the coverage
percentages.
Regards,
Daniel
Re: Weird 'Line' and 'End' values
They do influence the coverage percentage, yes. They are not really fake, but they do not have actual lines of code associated with them. This occurs when the compiler generates code for you that is "in between the lines" of your code. The using, foreach and lock statements are examples.
Re: NCover through CCNET pauses at the end
It does aggregate results at the end and write out the XML file, but it does the same thing whether you are running under CruiseControl or not. Do you get proper coverage reports? Are there any strange entries near the end of the log?
Re: NCover through CCNET pauses at the end
Hi Peter,
Yes, the coverage reports look fine. I actually haven't thought to check the log files (d'oh!), but I'll take a look on Monday when I'm back at work, and report back to you.
Thanks,
- Kim
Re: NCover through CCNET pauses at the end
Hi Peter,
Sorry about the delay, busy week...
There are no timestamps in the log, so it's hard to tell when the respective things happen, but there are no error messages of any kind at the end of the log.
All it says is:
EVENT: Profiler Shutdown MESSAGE: Signalling Shutdown. EVENT: Data link established. MESSAGE: Received driver ready event.
I don't have much time at the moment to troubleshoot this further (and honestly, it's not that big a problem for us), but let me know if there's something specific I can look out for.
Cheers,
- Kim
Re: NCover through CCNET pauses at the end
Hi again,
FWIW, I just tested this on my home machine, and here's what I observed:
1) When I run the test as non-admin, the profiler never manages to connect to the profiled nunit process. I can imagine this is reasonable behavior in a way, maybe admin privs are necessary for debugging/profiling
2) When I run it as admin, it works fine, regardless of whether I run it through CC.NET or not, so there's something else afoot.
Maybe something was changed in the later NCover betas that changes this behavior... I'll try and test the latest stuff at work moving forward.
Thanks,
- Kim
Re: Coverage "categories", exceptional code blocks
Hi mate,
Yeah it does sound a really nice feature - heck of a lot of work though! I was asked about something similar by someone many months ago and sadly no revelations to make it easy have come since then.
At the moment, NCoverExplorer has no knowledge whatsoever of the code being profiled - it just knows about start and end lines and columns, number of visits etc as provided by coverage.xml from NCover.
Perhaps Peter may have some thoughts on what if anything he can catch
at the profiler level. I do remember seeing a post by him previously
where he indicated he would like to automatically exclude private
constructors, and I thought he did mention exception catch blocks as well. Not sure
how much he reads the forums nowadays but I can try to ping him and see if it is indeed practical.
If not, then what you are requesting dramatically increases the complexity of NCoverExplorer. I guess the obvious way that springs to mind would require source code file parsing. Apart from the practicalities of that (keeping in mind that NCoverExplorer currently not only works with C# but also VB.Net, J# and Managed C++) it also means an extra "step" in the loading of coverage results for display to get the meaningful numbers you are after. That step would be pretty slow I imagine for large projects in particular - you would have to open every source code file that does not have 100% coverage, build a parse tree, traverse and record the results. I'm thinking ideally you would want it as another (optional) step in the coverage pipeline, so you could repeatedly open the results without having to repeat the parsing until you do another coverage run.
Are there other ways of doing this? Obviously Lutz Roeder does some funky stuff in Reflector from the IL - but even if such an approach were tried trying to reconcile that decompiled result against the original source code file would be problematic at best if not impractical/impossible.
The only other thing that comes to mind is a metadata style approach, which is something I will eventually be heading for once I add project support and method exclusions to NCoverExplorer. So you can setup a much more granular level of coverage exclusions and retain those each time you load the coverage file. This won't go to within a method level though which I would be really reluctant to do. Embedding line numbers of critical sections would be incredibly fragile and painful to maintain.
The "Attribute" approach such as NCover already has for coverage exclusions would be another option but subject to the same problems of fragility, and requires NCover or NCoverExplorer to reflect and work with those attributes. Plus lots of people dont like muddying their code with metadata for external tools like code coverage.
Certainly interested to hear of people's suggested approaches if they have any. At the moment this honestly falls into the "too hard" basket for the amount of time I have to work on this. That said, if Peter was able to "spread the load" and enhance the coverage.xml supplied then it would obviously be much simpler...
Re: Coverage "categories", exceptional code blocks
Hi,
Have anyone heard anything about if there's any progress in this subject?
I'm also interested in having another thresold level for exception blocks..
/Roy
Re: Coverage "categories", exceptional code blocks
Hi Roy,
As you can see by the deafening silence it looks like the answer is not yet... ;-)
I'm resolute in my determination not to do any source code parsing in NCoverExplorer, so that puts the onus on NCover. Knowing a little more about NCover's internal architecture now I think it would be possible in theory to treat catch blocks as a special case when the code is executed, but Peter knows this area far far better than myself. You could try posting it as a feature request in the relevant NCover forum referring to this thread and see what he says.
I would think you would want it as an optional switch - personally I like forcing the developers to try to hit the exception catch blocks because "most of the time" a catching of an exception is intentional to execute some other code in response, which would never ever get proven to be tested if you always exclude them from coverage. I do appreciate though there are certain exceptions and code which are very hard to simulate the exception from a unit test.
Regards,
Grant.
Re: Ncover Task for NAnt
It looks like in the fix I did recently someone had for multiple asemblies I broke something else - sorry! Let me fix it and will push a new build up shortly...
Re: Ncover Task for NAnt
Please try the new release you can find in this thread:
http://ncover.org/SITE/forums/thread/494.aspx
In addition, please try the new feature of generating the NAnt script automatically for you and see what difference that makes. Let me know how you get on.
Re: Ncover Task for NAnt
I'm now using all the v1.3.4.1754 assemblies (for console + NAnt tasks), and the problem still exists.
This is the generated block by NCoverExplorer:
<target name="coverage">
<exec program="regsvr32">
<arg value="/s" />
<arg value="C:\Program Files\NCover\coverlib.dll" />
</exec>
<ncover program="C:\Program Files\NCover\NCover.Console.exe"
version="1.5.4"
commandLineExe="\wo-rnd-build5\c$\views\mh_refactor_PCAS3_2\common\build\TestTmp\nunit-console.exe"
commandLineArgs="TestPlcmUtil.dll"
workingDirectory="\wo-rnd-build5\c$\views\mh_refactor_PCAS3_2\common\build\TestTmp"
logLevel="Normal"
/>
</target>
which I changed only slightly:
<exec program="regsvr32">
<arg value="/s" />
<arg value="${path::get-directory-name(path.ncover.console)}\coverlib.dll" />
</exec>
<ncover program="${path.ncover.console}"
version="1.5.4"
commandLineExe="${path.nunit.console}"
commandLineArgs="${path::get-file-name(filename)}"
coverageFile="${ncover.output.dir}\${basename}-ncover.result.xml"
workingDirectory="${test.temp.dir}"
logLevel="Normal"
/>
Thanks,
MH.
Re: Ncover Task for NAnt
hmmm... intriguing you still have problems. A few things:
1. What version of NAnt and NCover are you using?
2. Can you try adding verbose="true" to the NAnt task and post the start of the task output (up to before it runs the tests)?
3. Does it work when you run it directly inside NCoverExplorer using that new screen?
4. If you generate the "command line" version of the output, does that work if you run that from a command line?
Re: Ncover Task for NAnt
O.K. this will be a long one:
1. NAnt 0.85 RC4 (0.85.2344.0) and NCover 1.5.4.
2.
[ncover] Creating settings file: C:\Documents and Settings\build\Local Settings\Temp\1\tmp34B.tmp.ncoversettings
[ncover] Working directory: C:\build
[ncover] Arguments: //r "C:\Documents and Settings\build\Local Settings\Temp\1\tmp34B.tmp.ncoversettings"
[ncover] <?xml version="1.0" encoding="utf-8"?>
[ncover] <ProfilerSettings>
[ncover] <CommandLineExe>C:\Program Files\NUnit 2.2.6\bin\nunit-console.exe</CommandLineExe>
[ncover] <CommandLineArgs>TestTimerService.dll /xml=C:\build\test\TestTimerService-nunit.result.xml</CommandLi
neArgs>
[ncover] <WorkingDirectory>C:\build\bin</WorkingDirectory>
[ncover] <Assemblies />
[ncover] <CoverageXml>C:\build\coverage\coverage.xml</CoverageXml>
[ncover] <LogFile>coverage.log</LogFile>
[ncover] <VerboseLog>false</VerboseLog>
[ncover] <NoLog>false</NoLog>
[ncover] <DumpOnErrorNormal>false</DumpOnErrorNormal>
[ncover] <DumpOnErrorFull>false</DumpOnErrorFull>
[ncover] </ProfilerSettings>
[ncover] Starting 'C:\Program Files\NCover\NCover.Console.exe (//r "C:\Documents and Settings\build\Local Settin
gs\Temp\1\tmp34B.tmp.ncoversettings" )' in 'C:\build'
[ncover] NCover.Console v1.5.4 - Code Coverage Analysis for .NET - http://ncover.org
[ncover] Copyright (c) 2004-2005 Peter Waldschmidt
[ncover] Command: C:\Program Files\NUnit 2.2.6\bin\nunit-console.exe
[ncover] Command Args: TestTimerService.dll /xml=C:\build\test\TestTimerService-nunit.result.xml
[ncover] Working Directory: C:\build\bin
[ncover] Assemblies:
[ncover] Coverage Xml: C:\build\coverage\coverage.xml
[ncover] Coverage Log: coverage.log
3. No. but from another reason; It fails with "Could not find file 'C:\Documents'." which I think refers to the '//r' flag for config file path. This error does NOT occur when I run it within NAnt, it just runs and gives "empty" coverage.xml file.
4. Yes, but only when I get rid of the config file argument (//r ...)
Should you need any more info let me know.
Thanks,
MH
Re: Ncover Task for NAnt
Hi mate,
Thanks for all the info. You have clearly found a bug in that NCoverExplorer beta with the //r argument - I have just pushed a new build up now so if you wouldn't mind giving that a try from here:
http://www.kiwidude.com/dotnet/NCoverExplorer-1.3.5b.zip
[rant] Hell and damnation on Microsoft for their spaces in paths and appallingly idiotic "Documents and Settings" and "Program Files" folders... [/rant]
That should at least hopefully get it working from NCoverExplorer... but not solve your original problem.
When run using the <ncover> task, does the Nunit test file created have the same results in it as if it was run from the <exec> task?
Next question - how does the output you got above compare to the output of when you call the exec task (if you turn verbose=true on for that too). I am looking for where the command arguments are going wrong - you can see in the above what NCover was passed. You should be able to see the same thing for <exec>
Can you also try adding the "//s test.settings" argument to your exec task - and compare the contents of that file with the <ProfilerSettings> xml shown above?
Sorry for all the hassle - but I know a bunch of people who have the <ncover> task working without any problems so it's got me really intrigued. I appreciate you may have little time to "play" with this but it is much appreciated as you will hopefully save many others from having the same problems. Catching that NCoverExplorer bug was much appreciated...
Re: Ncover Task for NAnt
Hi kiwidude,
Thank you for the quick replies obviously it's our all concern getting these tools (NCover, NCoverExplorer, ...) better :)
Here are the tests I run:
- executing NCover from within the new wizard, F5 on the generated NAnt script - FAIL. maybe because the <ncover> task is still broken.
- executing NCover from within the new wizard, F5 on the generated command line - SUCCESS. all is fine the -nunit.result.xml & the coverage.xml are generated with accurate data
- switching the faulty <ncover> task with <exec> - the -nunit.result.xml are identical.
- The output is a bit different, mainly by dumping the <profilerSetting> to the console. You can see it below (produced by beyondcompare):
Left file: C:\build\exec-output.txt Right file: C:\build\task-output.txt
- I add "//s" to my NAnt script. The generated file is different from the <ProfilerSetting> xml dump. You can judge yourself (left side is the <ProfilerSetting> copy-pasted to text file):
Left file: C:\build\ncoversettings Right file: C:\build\test.settings
All in all I don't see a clue to what went wrrong with <ncover> task, I hope you do ;).
MH
Re: Ncover Task for NAnt
I sent you a message with the original (4) files I used for comparison. hope it helps.
Re: Ncover Task for NAnt
Great stuff - at least NUnit is getting those arguments correctly.
The <ncover> task simple prepares a .ncoversettings file and then executes ncover using the //r parameter. So the problem would appear to be caused by something not being written to that file that differs from how ncover.console.exe takes in it's command line parameters.
Diff #1. I just noticed your <exec> task is not specifying a working directory for NCover (using the //w parameter) - I had missed that from your first post. This should be okay as you are setting your working directory for the exec task itself to that directory - but it's a difference nonetheless.
Diff #2. I'm also writing an empty <Assemblies /> node into the xml from the <ncover> task - I know NCover 1.3.3 can cope with that but perhaps not 1.5.4?
Diff #3. The <ProfileIIS> being in there with a value of false. That should be the default and no concern but sometimes straws need to be grasped ;)
Alright - next plan on this. From a command line in your "test.temp.dir" (C:\build\bin right?) verify that the "temp.settings" works and gives you coverage output before we start fiddling:
"C:\Program Files\NCover\NCover.Console.exe" //r temp.settings
Now edit the temp.settings file, and add an empty <Assemblies/> node in there, try again.
Repeat for adding a <WorkingDirectory>C:\build\bin</WorkingDirectory>, try again.
Repeat for removing the <ProfileIIS> node.
Did any of those that fix it?
Incidentally interested in your comments on running NCover within NCoverExplorer and feedback from how you find it. When you hit F5 it always uses the same "ncover.console //r" trick to run NCover, so curious how you say it failed after clicking "NAnt" but passed after clicking "CommandLine". Did you change any options in between? All those buttons should be doing is producing some output on the RHS - it shouldnt effect the actual way NCoverExplorer itself executes (unless there's a bug of course!).
Re: Ncover Task for NAnt
So, here we go...
I started by adding the <WorkingDirectory> to temp.setting, a vous la coverage.xml now has content! Next, adding an empty <Assemblies/> node, and the file is empty again. adding or removing <ProfileIIS> node did nothing either way. Also switching exec "workingdir" property with //w does not have any effect. So maybe it is this <Assemblies/> node after all!?
Now back to new wizard. The error was of not able to parse "c:\Documents and Settings" path, now from some reason it does, but the same problem (empty coverage.xml) exists. BUT, I must say that I copied the command line (with rclick) to the console. you are right both F5 fails to create coverage.xml either from the wizard's NAnt script or the command-line window. The only difference that copying the command line works, but saving the NAnt script fails (because of the <ncover> task issue).
regards,
MH
Re: Ncover Task for NAnt
Ok, that all makes more sense now. The same code is used to run from F5 for what the NAnt task is doing and within NCoverExplorer - so they were both broken for that particular combination of no assemblies selected and NCover 1.5.4.
New build 1.3.5.1761 up on my site which should (fingers crossed) get you up and running at last:
http://www.kiwidude.com/dotnet/NCoverExplorer-1.3.5b.zip
http://www.kiwidude.com/dotnet/NCoverExplorer.Extras-1.3.5b.zip
I'm sure you will let me know if I've still got it wrong ;-)
Thanks again for all your patience and for helping to resolve this. There were too many combinations and permutations of NCover versions and parameters for me to test them all myself . When I get more time I might put together some unit and regression tests - I'm a bit stretched at the moment between a 60+ hr/wk day job, NCoverExplorer development and another little developer tool I'm looking to announce to the community in a few weeks time...
Re: Ncover Task for NAnt
Hooray,
Now the (new) wizard works great.
Do you plan to fix the <ncover> task as well, I noticed that NAnt.NCoverExplorer.Tasks.dll was missing from the zip file you gave, is it fixed in build 1761?
Thank you again Grant for great tool :-)
MH
Re: Ncover Task for NAnt
Sorry about that - I "moved" rather than "copied" the dll out of the build folder location for testing purposes - new zip file uploaded now. Using VNC to the home PC while stuffing down a sandwich at work let down the hand/eye coordination... ;-)
Re: Ncover Task for NAnt
Looks good! I haven't tested on the whole project yet, only on a tiny test build script, but for now it seems like <ncover> tasks is doing its job...
Thanks, Grant. I enjoyed help debugging these stuff.
MH
Re: Detailed Html Output
Hi Dmitry,
No it isn't... yet! I've had a couple of people request this. It is on the list for a future version... it's been a case of juggling priorities as always...
In the meantime - we have a similar situation with our CC.Net automated builds. However what we do is copy the coverage.xml artifacts to a shared drive location that the developers can open them up from within NCoverExplorer - this does the job ok and obviously gives you all the NCoverExplorer benefits of navigation, filtering, exclusions etc.
Re: Detailed Html Output
Grant,
Thanks for a quick reply. I'll stop searching for a black cat in a dark room then and I'll wait for the next release on NCover and NCover Explorer :)
We used to use Clover.NET, but it's a bit expensive and it does not use code profiling API, which means it has to inject your source code with all sorts of stuff to make a coverage run...
-- Dmitry