Comparing Test Spec Frameworks

Up until now, I've been using a custom test spec framework geared towards helping write unit tests with a BDD style description: Given; When; Then, described in a previous post. The framework was built on top of NUnit, Moq, and Structuremap.AutoMocking nuget packages to make assertions, create mock objects and resolve the test target dependencies with mock objects respectively. There is no problem with the framework as it is, but I was seeking a way to reduce the language 'noise' that was incurred - a new method for each Given, When and Then step is cumbersome and gets tedious after a while.

This lead me to discover the Machine.Specifications (MSpec) framework. It allows you to write tests in the same BDD style, but does not require a full method for each step. Instead, you can just write a lambda function. This can reduce any one test down to a single line, by making sure that you stick to one assertion per test. It introduces some new types which enter the terminology: Establish,  Because and It. These are direct replacements for Given, When and Then, as the semantics are identical.

But what about the automocking feature of the old framework? This can be fulfilled by another package designed to work with MSpec: Machine.Fakes. This, too, automatically resolves any dependencies of the target with mocked objects, provided by your mocking framework of choice. I used the Machine.Fakes.Moq nuget package to do this, but since you also use the Fakes framework to get access to the mocked dependencies it has created for you, and also generate additional mocks you may need the fact that I'm using Moq becomes irrelevant - it is completely abstracted away. To get access to an mocked dependency you use The<T>, and to create a mock you use An<T>. I also like the nice feature that gives you a list of mocked objects: Some<T>.

That's the language hooks and mocks taken care of, but in the effort to reduce the syntax bloat as much as possible I brought in the Fluent.Assertions nuget package. This library is leaner than NUnit because it lets me make assertions using a set of extension methods, e.g. .ShouldEqual() or .ShouldBeTrue(), so I don't have to begin an assertion statement with NUnit, I can start with the object I want to make the assertion on. It also puts the final nail in the coffin for NUnit, as I have already done away with the [SetUp], [TearDown] or [Test] attributes.

Let's compare two test classes written for the same target class. The Target is shown below, followed by the before and after test classes.

Target
Before
After
As you will notice, the 'After' test class contains over 50% fewer lines. The tests are now more readable because there is less language noise to cut through when reviewing the code. The important statements jump right out, and they also take less time to write. The method bodies and calls to base methods in the Given and When steps have gone, replaced with single line statements. The MFakes framework has helped make each line that refers to a mock more concise be removing the need to call .Object. The Fluent.Assertions library has made the assertions simpler too.

There are a couple of downsides that I have discovered not shown in the example that I think are worth mentioning. Firstly, Tests (It field variables) cannot be run from a base class. They can be declared, but go undetected by the build runner. This means that you will have to duplicate any tests that are based on statements that come before a branch at an if statement, assuming you want to test both branches of the method. Whereas previously you could declare a test in an abstract class and have it executed in each text fixture class that inherits from it.

The other negative is that you cannot instantiate a mock of your own using the An<T> from MFakes at field level. You have to be inside the body of the Establish lambda/anonymous delegate before you can assign a value to the variable defined at field level. You aren't told about this error until run time which is annoying to begin with, but being forced to declare your mock variables on one line and assigning them on another doubles the amount of code needed to achieve what that the previous framework could do. So it goes against the purpose of this exercise somewhat.

All in all, I think the positives outweigh the negatives. The biggest factor being that there is less manual syntax to be typed to generate a test. Which is one less reason not to do TDD, which can only be a good thing!

Automock with StructureMap and Moq

This is a guide of how to use AutoMocking with StructureMap and Moq in conjunction with cucumber BDD style unit testing.

Taking the BDD GIVEN, WHEN & THEN Scenario approach to unit testing has a number of benefits. I find it makes tests easier to maintain as you stick to one assertion per test. The test scenarios are well partitioned making them cleaner than straight forward Arrange Act Assert style. It also produces more readable output of test results.

Automocking is a time-saving feature of StructureMap that allows you to easily instantiate the Class Under Test (the class that will be used in the GIVEN statement of the Scenario) by automatically resolving the constructor dependencies of the class with Mocked out instances. Test classes therefore become decoupled from the constructor(s) of the Class Under Test. Meaning that you'll get a test that fails if you change a constructor instead of a test that won't build. This is a debatable point but can be considered an advantage in TDD - you get feedback from your tests as opposed to the compiler, as it should be.

By installing the StructureMap.AutoMocking and Moq packages you get out-of-the-box support for AutoMocking. Given that I regularly use StructureMap and Moq as my IOC and mocking framework of choice respectively, it makes sense to take advantage.

An Example

In the example class to test there are two dependencies that will need to be automatically mocked, and a method that should be tested.


The base class provides us with the GIVEN & WHEN syntax. It uses the MoqAutoMocker provided by the StrutureMap.Automocking library to mock any dependencies on our class's constructor. It provides a handle to access any Mocked dependency that was injected when the class was instantiated. Also provided is a shortcut to Moq's Verify method as well as some syntactic sugar that lets you use [Then] instead of [Test] to round off the BDD style.


The example unit test class implements the abstract class and is named according to the method being tested. There should be one class per method being tested - i.e. one WHEN statement per scenario. If there is another public method it should be tested in a new class that also implements the base class. This practice is what makes the test output easy to digest.

References

• StructureMap Automocker Documentation
• StackOverflow question with handy Moq tip to access underlying Mock
• Scott Millet's post on BDD Unit Testing