MSBuild Transforms, Batching, Well-Known Metadata and MSTest

Thanks to a comment from Daniel Richardson on my previous MSTest post (and a lot more research, testing, & debugging), I’ve found a more flexible way of calling MSTest from MSBuild.  The main drawback of the solution I blogged about earlier was that new test assemblies added to the solution would not be run in MSBuild unless the Exec call to MSTest.exe was updated to include them.  But thanks to a combination of MSBuild transforms and batching, this is no longer necessary.

First, I needed to create a list of test assemblies.   The solution is structured in a way that makes this relatively simple.  All of our test assemblies live in a “Tests” folder, so there’s a root to start from.  The assemblies all have the suffix “.Test.dll” too.  The following CreateItem task does the rest:

<CreateItem Include=”$(TestDir)**bin$(Configuration)*.Test.dll” AdditionalMetadata=”TestContainerPrefix=/testcontainer:”>
<Output TaskParameter=”Include” ItemName=”TestAssemblies” />
</CreateItem>

The task above creates a TestAssemblies element, which contains a semicolon-delimited list of paths to every test assembly for the application.  Since the MSTest command line needs a space between each test assembly passed to it, the TestAssemblies element can’t be used as-is.  Each assembly also requires a “/testcontainer:” prefix.  Both of these issues are addressed by the combined use of transforms, batching, and well-known metadata as shown below:

<Exec Command=””$(VS90COMNTOOLS)..IDEmstest.exe” @(TestAssemblies->’%(TestContainerPrefix)%(FullPath)’,’ ‘) /runconfig:localtestrun.testrunconfig” />

Note the use of %(TestContainerPrefix) above.  I defined that metadata element in the CreateItem task.  Because it’s part of each item in TestAssemblies, I can refer to it in the transform.  The %(FullPath) is well-known item metadata.  For each assembly in TestAssemblies, it returns the full path to the file.  As for the semi-colon delimiter that appears by default, the last parameter of the transform (the single-quoted space) replaces it.

The end result is a MSTest call that works no matter how many test assemblies are added, with no further editing of the build script.

Here’s a list of the links that I looked at that helped me find this solution:

Comments

  1. Rob says:

    Scott

    A great article, I don’t have much experience with mstest or msbuild and have read with interest. I’am currently trying to solve a similar issue.

    I’m currently getting a error with:
    ’%(TestContainerPrefix)%(FullPath)’,’ ‘) /runconfig:localtestrun.testrunconfig” />

    Teamcity reports:
    The item metadata %(TestContainerPrefix) is being referenced without an item name. Specify the item name by using %(itemname.TestContainerPrefix).

    Any ideas would be appreciated

    Thanks

    Rob

  2. Scott says:

    As long as you’ve defined the transform in your exec task this way:

    @(TestAssemblies->’%(TestContainerPrefix)%(FullPath)’,’ ‘) /runconfig:localtestrun.testrunconfig” />

    it seems like what you have should work. TeamCity isn’t the build server we use, but I assume it’s running your MSBuild file, so hopefully it isn’t too different.

  3. Keith Woods says:

    Thanks Scott, nice one…

    Rob, try this:
    ‘%(TestContainerPrefix)%(FullPath)’,’ ‘) /runconfig:localtestrun.testrunconfig” />

    I think its basically an escaping issue, the text in the post doesn’t exactly copy and paste the best.

  4. Keith Woods says:

    ok, that didn’t post to well either lol…

    maybe this will, basically just try escaping the command correctly:

    <Exec Command=""$(VS90COMNTOOLS)..IDEmstest.exe"
    @(TestAssemblies->'%(TestContainerPrefix)%(FullPath)',' ')
    /runconfig:localtestrun.testrunconfig" />

  5. Keith Woods says:

    and again, whats missing from above is the quotes around $(VS90COMNTOOLS)..IDEmstest.exe are actually done using html markup… i.e. & lt; …. & gt;

  6. Richard says:

    Could you give an example of how this is used in the CC.NET configuration script? I’d like to see how this is implemented before pulling my hair out?

Leave a Reply to Keith Woods Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.