Integrating NDepend with Jenkins

Apparently it's National Jenkins Month here at Cyotek as we seem to be writing about it quite a lot recently. Previously I explained how I got fed up of manually building and publishing NuGet package projects, and got our Jenkins CI server to build and publish them for me.

This got me thinking - some time ago I received a license for NDepend and even wrote a post briefly covering some of its features.

Unfortunately while NDepend is a powerful tool, I have serious issues with it's UI, both in terms of accessibility (it's very keyboard unfriendly) and the way the UI operates (such as huge floating tool"tips"). Add to that having to manually run the tool meant a simple outcome - the tool was never used.

Note: The version I have is 6.3 which is currently 9 months out of date - while I was writing this post I discovered a new 2017 version is now available which I hope may have addressed some of the issues I previously raised

Despite the fact I wasn't hugely enamoured with NDepend, a static analysis tool of some sort is a good thing to have in your tool-belt for detecting issues you might miss or not be aware of. And as I've been spending so much time with Jenkins automation recently, I wondered how much of NDepend I could automate away.

Pipeline vs Freestyle

I'm going to be adding the NDepend integration to the Jenkins pipeline script that I covered in two articles available here and here, but if you're not using pipelines you can still do this with Freestyle jobs.

Tinkering the script

Once again I'm going to declare some variables at the top of my script so I can easily adjust them if need be. To avoid adding any more parameters, I'm going to infer the existence of a NDepend project *.ndproj by assuming it is named after the project being compiled, and located in the same directory as the solution.

groovy

def nDependProjectName  = "${libName}.ndproj"
def nDependProject      = slnPath + nDependProjectName
def nDependRunner       = "\"${WORKSPACE}\\tools\\ndepend\\NDepend.Console.exe\""

I have NDepend checked into version control in a tools directory so it is available on build agents without needing a dedicated installation. You'll need to adjust the path above to where the executable is located (or define a Jenkins tool reference to use)

Calling NDepend

As with test execution, I'm going to have a separate stage for code analysis that will only appear and execute if a NDepend project is detected. To perform the auto-detection I can make use of the built-in fileExists command

groovy

if(fileExists(slnPathRel + nDependProjectName))
{
  stage('Analyse')
  {
    bat("${nDependRunner} \"${nDependProject}\"")
  }
}

The path specified in fileExists must be relative to the current directory. Conversely, NDepend.Console.exe requires the project filename to be fully qualified.

I decided to place this new stage between the Build and Tests stages in my pipeline script, as there isn't much point running tests if an analysis finds critical errors.

Using absolute or relative paths in a NDepend project

By default, all paths and filenames inside the NDepend project are absolute. As Jenkins builds in temporary workspaces that could be different for each build agent it's usually preferable to use relative paths.

There are two ways we can work around this - the first is to use command line switches to override the paths in the project, and the second is to make them relative.

Overriding the absolute paths

The InDirs and OutDir arguments can be used to specify override paths - you'll need to specify both of these, as InDirs controls where all the source files to analyse are located, and OutDir specifies where the report will be written. Note that InDirs allows you to specify multiple paths if required.

groovy

bat("${nDependRunner} \"${nDependProject}\" /InDirs ${WORKSPACE}\\${binPath} /OutDir \"${slnPath}NDependOut\"")

Normally I always quote paths so that file names with spaces don't cause parsing errors. In this case the InDirs parameter is not quoted due to the path ending in a \ character. If I leave it quoted, NDepend seems to be treating as an escape for the quote, thus causing a different set of parsing errors

Configuring NDepend to use relative paths

These instructions apply to the stand alone tool, but should also work from the Visual Studio extension.

Open the Project Properties editor
Select the Paths Referenced tab
In the path list, select each path you want to make relative
Right click and select Set as Path Relative (to the NDepend Project File Location)
Save your changes

As I don't really want absolute paths in these files, I'm going to go with this option, although it would be better if I could configure the default behaviour of NDepend in regards to paths. As I already have some NDepend projects, I'm going to leave InDirs and OutDir arguments in the script until I have time to correct these existing projects with absolute paths.

To fail or not to fail, that is the question

Jenkins normally fails the build when a bat statement returns a non-zero exit code, which is usually the expected behaviour. If NDepend runs successfully and doesn't find any critical violations then it will return the expected zero. However, even if it has otherwise ran successfully, it will return non-zero in the event of critical violations.

It's possibly a good idea to leave this behaviour alone, but for the time being I don't want NDepend to be capable of failing my builds. Firstly because I'm attaching these projects to code that often has been in use for years and I need time to go through any violations, and secondly because I know from previous experience that NDepend reports false positives.

The bat command has an optional returnStatus argument. Set this to true and Jenkins will return the exit code for your script to check, but won't fail the build if it's non-zero.

groovy

bat(returnStatus: true, script: "${nDependRunner} \"${nDependProject}\" /InDirs ${WORKSPACE}\\${binPath} /OutDir \"${slnPath}NDependOut\"")

Publishing the HTML

Once NDepend has created the report, we need to get this into Jenkins. Unsurprisingly, Jenkins has a HTML Publisher plugin for just this purpose - we only have to specify the location of the report files, the default filename and the report name.

The location is whatever we set the OutDir argument to when we executed NDepend. The default filename will always be NDependReport.html, and we can call it whatever we want!

Adding the following publishHTML command to the anaylse stage will do the job nicely

groovy

publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: slnPathRel + 'NDependOut', reportFiles: 'NDependReport.html', reportName: 'NDepend'])

Security Breach!

Once the HTML has been published, it will appear in the sidebar menu for the job. On trying to view the report you might be in for a surprise though.

If you're using Blue Ocean, then the first part of the statement above is incorrect - the Blue Ocean UI doesn't show the HTML reports at all, to view the reports you need to use the Classic interface

Jenkins wraps the report in a frame so that you can get back to the original job page. The request that loads the document into the frame has the Content-Security-Policy:, X-Content-Security-Policy and X-WebKit-CSP headers set, which effectively lock the page down to external resources and script execution.

The NDepend report makes use of script and in-line CSS and so the policy headers completely break it, unless you're using an older version of Internet Explorer that doesn't process those headers.

As I'm much happier pretending that IE doesn't exist clearly that's not a solution for me. I did test it just to check though, and setting IE to an emulated mode worked after a fashion - the page was very unresponsive and several times stopped painting. Go IE!

Reconfiguring the Jenkins Content Security Policy

Update 03Feb2017. The instructions below only temporarily change the CSP and will be reverted when Jenkins is restarted. This follow-up post describes how to permanently change the CSP.

I don't want to be disabling security features without good cause and so although the Jenkins documentation does state how to disable the CSP (along with a warning of why you shouldn't!), I'm going to try adjusting it instead.

After some testing, the following policy would allow the report to work correctly

bat

sandbox allow-scripts; default-src 'self'; style-src 'self' 'unsafe-inline';

I'm not a security expert. I tinkered the CSP policy enough to allow it to work without turning it off fully, but that doesn't mean the settings I have chosen are either optimal or safe (for example, I didn't try using file hashes).

To change the CSP, open the Script Console in Jenkins administration section, and run the following command

groovy

System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "sandbox allow-scripts; default-src 'self'; style-src 'self' 'unsafe-inline';")

With this policy in place, refreshing the report (after clearing the browser cache) would display a fully functional report. I still have some errors regarding fonts the CSS is referencing, but as they don't even exist it seemed a little pointless adding a rule for them.

Another alternative to changing the CSP

One other possible alternative to avoid changing the CSP would be to replace the NDepend report - it's actually a feature of NDepend that you can specify a custom XSLT file used to generate the report. Assuming this is straight forward enough to do, that would actually be a pretty cool feature of NDepend and would mean a static report could be generated that would comply with a default CSP, not to mention trimming the report down a bit to just essentials.

Creating a rules file

Another NDepend default is to save all the rules in the project file. However, just like this Jenkins pipeline script I keep adapting, I don't want to keep dozens of copies of stock rules.

And NDepend delivers here too - it allows rules to be stored in external files, and so I used the NDepend GUI to create a rules file before deleting all the rules embedded in the project.

As none of my previous NDepend projects use rule files, I didn't add any overrides in the NDepend.Console.exe call above, but you can use the /RuleFiles and /KeepProjectRuleFiles parameters for overriding them if required.

Comparing previous results, a work in progress

One interesting feature of NDepend is that can automatically compare the current analysis with previous ones, allowing to you judge if code quality is improving (or not).

Of course, that will only work if the previous report data exist - which it won't if it's only stored in a temporary workspace. I also don't want that data in version control. I tried adding a public share on our server, but when ran via Jenkins, both NDepend and the HTML Publish claimed the directory didn't exist. I tried pasting the command line from the Jenkins log into a new console window which executed perfectly, so it's more than likely a permissions issue for the service the Jenkins agent runs under.

As the HTML Publisher plugin doesn't support exclusions, and as we probably don't want all that historical data being uploaded into Jenkins either, that would also mean copying the bits of the report we wanted to publish to another folder for the plugin to process.

All in all, for the time being I'll just stick with the current analysis report - at least it is a starting point for investigating my code.

Done, for now

And with this new addition my little script has become that much more powerful. While I still have to do a little more tinkering to the script by removing some of the parameters I've added and making more use of auto detection, I think the script is finished for the time being (at least until I revisit historical NDepend analyses, or find something else to plug into it!)