This post is about running MSTest for applications that target mixed platforms.
If you are lucky enough to be able to write your applications in pure .NET, then you may never encounter 32bit/64bit platform issues. However, if any dependent library or plug-in is compiled for a specific architecture, then your whole application must be run in that mode. This is why the default Window's Internet Explorer is still 32bit despite the 64bit version shipping since Vista: it has to be the same architecture as any legacy plug-ins. By contrast, Notepad doesn't have any plug-ins, so it can get away with being 64bit only.
My companies applications rely on many native libraries, which are obviously compiled for specific architectures (x86 and x64). Deploying an application for multiple target processors is a complex subject in itself that can be solved with a range of strategies from dynamic library linking to processor-specific installers, but however you deploy, your application will behave differently in these two different modes so they must both be tested.
For better or for worse, we use MSTest to control application quality. Since the release of Visual Studio 2010 this has been able to
run in 64-bit mode as well as 32-bit mode, but there are certain subtleties that complicate the practical aspects of administering your tests.
To understand the problem, consider the way MSTest works: Testing is done using two programs
MSTest.exe and
QTAgent32.exe.
MSTest is told what assemblies to load and it scans those assemblies (using reflection) to find any classes and methods annotated as tests using the
various test attributes. To do this it must be able to load the assembly and all its dependent assemblies and because
MSTest is a 32bit process, none of these assemblies can be exclusively 64bit. Once loaded,
MSTest instructs
QTAgent32 to run these tests, which means
QTAgent32 must load the assemblies itself and execute the test methods, but because it is also a 32bit process it cannot load 64bit assemblies either.
In Visual Studio 2010 a new version of QTAgent32 was added called QTAgent.exe, which can run 64bit assemblies. This means that even though MSTest is still 32bit, QTAgent can execute in full 64bit mode so that pure .NET assemblies can now be tested in 32bit and 64bit mode. However, it still doesn't easily allow applications with mixed-mode assemblies to be tested in 64bit mode because they cannot be loaded by MSTest in the first place.
One interesting solution to this is to
force MSTest.exe to be a 64bit application. This implies that
MSTest is actually pure .NET code anyway, but has been forced to run in x86 mode. If you are going down this road, note that
MSTest relies on various registry entries (
HKLM\SOFTWARE\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\TestType and
HKLM\SOFTWARE\Microsoft\VisualStudio\10.0\Licenses) to decide which extensions it can handle and what features are licensed for use, and that these are installed by default to the
WOW6432Node registry "shadow" branch. To run in 64bit mode you must
copy some of the registry entries over as well as editing the binaries themselves.
There is an alternative approach that doesn't involve editing executable files and local machine registry settings (which can be a pain across a large development team). Our application builds for two distinct platform targets x86 and x64 (note however that most assemblies are compiled as "Any CPU" except the ones that contain native code) and test projects are only built in the x86 solution configuration. This ensures that the code in their bin folder is 32bit compatible, and they can therefore be loaded into MSTest. Also, the tests are configured to run against the binaries in the actual application deployment folder rather than running in their binary folder using the new root folder feature of Visual Studio 2010:
In this case we have used an environment variable that signifies where to find the deployed binaries. Whether you do this or not, it seems to always want a full path for one reason or another, which can make supporting multiple development environments a challenge.
We make one of these test configurations for each target platform, remembering to change the Hosts section that controls 32bit and 64bit execution. Then we can run both configurations from the command line like this:
mstest /testsettings:WorkStation32.testrunconfig /testmetadata:SOLUTION.vsmdi
mstest /testsettings:WorkStation64.testrunconfig /testmetadata:SOLUTION.vsmdi
The trick here is that in both cases MSTest will load the 32bit binaries to decide what tests to run, but the different configuration files will control if QTAgent32 or QTAgent is used. Note that this cannot work with the /noisolation switch, because MSTest cannot host the 64bit binaries.
The disadvantage of running your unit tests against the deployment folder is that your tests are less "clean" and test failures could take longer to diagnose. The advantage is that the tests are being run on code as it will appear in the wild, which can include complex deployment features such as assembly obfuscation.
This system will work on development desktops and build servers. It may give the Visual Studio IDE pause for thought occasionally, but it is fundamentally compatible, which is one of the only real advantages of MSTest in the first place.
