Introduction

A few days ago I blogged about a PowerShell script that imports a bulk of NUnit test result XML files into a TeamCity Continuous Integration Server. This blog post takes this concept a bit further by using TeamCity to start a “fake” build synchronously with a NUnit test run that was started on an external test system. The “fake” TeamCity build will then wait for a file system event and trigger the XML file import after the tests are finished. This way it seems like the TeamCity test build is running in real-time and executing the tests. The TeamCity build task that is waiting for a file system event is using a NAnt script that calls a PowerShell script. The PowerShell script takes care of the file system event handling.

Figure 1: The early stages of Windows 8

High level overview

Calling StartTeamCityBuildWithCurl.ps1 will kick off the TeamCity build. In my case I am using a batch file that you can find in the download zip package. The batch file passes build parameters to the TeamCity build. The build then uses the ImportTestResults.build NAnt script to call the BuildTestResultsWatcher.ps1 PowerShell script. This script sets up a file system event handler and calls ProcessXML.ps1 to import a the fresh test result XML file.

Script file listings

The main script files that are involved in this workflow chain are listed here:

StartTeamCityBuildWithCurl.ps1

Param([string] $BuildNumber="50")

Set-StrictMode -Version "Latest"
$DebugPreference = "Continue"


$URL = "`"http://localhost:8887/httpAuth/action.html?add2Queue=bt3&env.name=ExternalBuildID&env.value=$BuildNumber`""
Write-Debug $URL
Write-Debug "Starting teamcity via cURL and teamcity REST API"
$result = start-process -filePath "C:\TestResultsImporter\Tools\curl\curl" `
                        -ArgumentList "--user", "autoadmin:LuckyYou", $URL `
                        -NoNewWindow `
                        -RedirectStandardError "error.txt" `
                        -RedirectStandardOutput "output.txt" `
                        -wait `
                        -passthru
$result.HasExited
$result.ExitCode

ImportTestResults.build

<?xml version="1.0" ?>
<project name="TestResultsImporter" default="build" xmlns="http://nant.sf.net/release/0.90/nant.xsd">
  <target name="build" depends="send.nunit.report.import.message.to.teamcity" />
  <target name="wait.for.testresults.xml.file" description="Starts blocking PowerShell script that waits for FileSystemEvent and processes XML file">
    <echo message="Starting PowerShell Script to wait for testresult.xml file and then convert the format"></echo>
    <exec program="PowerShell" commandline="-file &quot;C:\TestResultsImporter\Scripts\LatestResultsImporter\TestResultsFileWatcher.ps1&quot;" />
  </target>
  <target name="send.nunit.report.import.message.to.teamcity" depends="wait.for.testresults.xml.file" description="Sends message via standard output to teamcity to trigger the import of a NUnit test result report">
    <echo message="##teamcity[importData type='nunit' path='C:\TestResultsImporter\Pickup\TestResults.xml' parseOutOfDate='true']"></echo>  
  </target>
</project>

BuildTestResultsWatcher.ps1

$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "C:\TestResultsImporter\PublishedTestResults\v0.1"
$watcher.IncludeSubdirectories = $true
$watcher.EnableRaisingEvents = $true
$watcher.Filter = "TestResults.xml"


function Process-TestResultsFile([string] $BuildFolder)
{
    Write-Host $BuildFolder
    $TestResultsFilePath = Join-Path -Path $watcher.Path -ChildPath $BuildFolder
    Write-Host $TestResultsFilePath
    Write-Host "Found TestResults.xml"
    try
    {
    . "C:\TestResultsImporter\Scripts\Common\ProcessXML.ps1" `
    -BeforeXMLFilePath $TestResultsFilePath `
    -AfterXMLFilePath "C:\TestResultsImporter\Pickup\TestResults.xml" `
    }
    catch
    {
      Write-Host "Error converting XML"
    }
}

$ProcessTestResultFileJob = Register-ObjectEvent -InputObject $watcher -EventName "Created" -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
$CreatedEvent = Wait-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
$BuildFolder = $CreatedEvent.SourceEventArgs.Name
Remove-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"
[System.Threading.Thread]::Sleep(1000)
Process-TestResultsFile -BuildFolder $BuildFolder

Unregister-Event -SourceIdentifier "TestResultsFileCreatedEventIdentifier"

ProcessXML.ps1

param([string] $BeforeXMLFilePath, [string] $AfterXMLFilePath, [string] $XMLTemplatePath)

function Load-XML([string] $Path)
{
    $xml = [xml](get-content -Path $Path -Force) 
    return $xml
}


$FUTestResultXML = Load-XML -Path $BeforeXMLFilePath

#Do some processing, if required.

$FUTestResultXML.Save($AfterXMLFilePath)

Step By Step setup guide

Folder structure

The first important step is to extract the zip file that is available as download and copy its content into the C:\ root folder of your PC.

The ToImport folder contains the NUnit test results XML files for each particular build.

Figure 2: Test results file to import

The PublishedTestResults folder is being watched by the FileSystemWatcher script, which is in this case triggered by the TeamCity NAnt build task. The associated TeamCity build will not complete until you copy manually one of the test result folders into the PublishedTestResults directory.

Figure 3: Folder being watched by the FileSystemWatcher script

TeamCity Configuration

You need to create a very simple build configuration. Add a NAnt build runner, select the NAnt executable in the tools folder and hook it up to the ImportTestResults.build file.

Figure 4: Create a new TeamCity project and a new build configuration

Figure 5: ImportTestResults.build is now hooked up

Running the import

Pictures

First start the ReRun.bat file. The file can be found in the folder C:\TestResultsImporter\Scripts\LatestResultsImporterMake. Make sure that the build number you pass in is valid. This batch file will trigger the TeamCity build via REST API and the cURL tool.

Figure 6: The rerun.bat file kicked off a new TeamCity build

The TeamCity NAnt build task is calling a PowerShell script that then waits for the test result XML file to be copied into the PublishedTestResults directory.

Figure 7: Waiting for NUnit tests to be complete

To simulate the completion of a test run we can just manually copy folder 77 for example into the PublishedTestResults directory. The PowerShell script that is waiting for a new XML file is going to pick it up and then import the XML file via TeamCity service messages into TeamCity.

Figure 8: New XML file is picked up and imported

Video

Watch the following YouTube video to see the import magic in action.

Download

The scripts and sample data can be downloaded here: C_TestResultsImporter.zip

Ausblick

The downside of this approach is that it consumes a TeamCity agent for the duration of the NUnit test run. During the wait, the Agent is actually idle. This approach is not scalable in the sense that results of tests that are running concurrently in different environments cannot be imported without the need of several agents.