Comparing Test Spec Frameworks

Up until now, I've been using a custom test spec framework geared towards helping write unit tests with a BDD style description: Given; When; Then, described in a previous post. The framework was built on top of NUnit, Moq, and Structuremap.AutoMocking nuget packages to make assertions, create mock objects and resolve the test target dependencies with mock objects respectively. There is no problem with the framework as it is, but I was seeking a way to reduce the language 'noise' that was incurred - a new method for each Given, When and Then step is cumbersome and gets tedious after a while.

This lead me to discover the Machine.Specifications (MSpec) framework. It allows you to write tests in the same BDD style, but does not require a full method for each step. Instead, you can just write a lambda function. This can reduce any one test down to a single line, by making sure that you stick to one assertion per test. It introduces some new types which enter the terminology: Establish,  Because and It. These are direct replacements for Given, When and Then, as the semantics are identical.

But what about the automocking feature of the old framework? This can be fulfilled by another package designed to work with MSpec: Machine.Fakes. This, too, automatically resolves any dependencies of the target with mocked objects, provided by your mocking framework of choice. I used the Machine.Fakes.Moq nuget package to do this, but since you also use the Fakes framework to get access to the mocked dependencies it has created for you, and also generate additional mocks you may need the fact that I'm using Moq becomes irrelevant - it is completely abstracted away. To get access to an mocked dependency you use The<T>, and to create a mock you use An<T>. I also like the nice feature that gives you a list of mocked objects: Some<T>.

That's the language hooks and mocks taken care of, but in the effort to reduce the syntax bloat as much as possible I brought in the Fluent.Assertions nuget package. This library is leaner than NUnit because it lets me make assertions using a set of extension methods, e.g. .ShouldEqual() or .ShouldBeTrue(), so I don't have to begin an assertion statement with NUnit, I can start with the object I want to make the assertion on. It also puts the final nail in the coffin for NUnit, as I have already done away with the [SetUp], [TearDown] or [Test] attributes.

Let's compare two test classes written for the same target class. The Target is shown below, followed by the before and after test classes.

Target
Before
After
As you will notice, the 'After' test class contains over 50% fewer lines. The tests are now more readable because there is less language noise to cut through when reviewing the code. The important statements jump right out, and they also take less time to write. The method bodies and calls to base methods in the Given and When steps have gone, replaced with single line statements. The MFakes framework has helped make each line that refers to a mock more concise be removing the need to call .Object. The Fluent.Assertions library has made the assertions simpler too.

There are a couple of downsides that I have discovered not shown in the example that I think are worth mentioning. Firstly, Tests (It field variables) cannot be run from a base class. They can be declared, but go undetected by the build runner. This means that you will have to duplicate any tests that are based on statements that come before a branch at an if statement, assuming you want to test both branches of the method. Whereas previously you could declare a test in an abstract class and have it executed in each text fixture class that inherits from it.

The other negative is that you cannot instantiate a mock of your own using the An<T> from MFakes at field level. You have to be inside the body of the Establish lambda/anonymous delegate before you can assign a value to the variable defined at field level. You aren't told about this error until run time which is annoying to begin with, but being forced to declare your mock variables on one line and assigning them on another doubles the amount of code needed to achieve what that the previous framework could do. So it goes against the purpose of this exercise somewhat.

All in all, I think the positives outweigh the negatives. The biggest factor being that there is less manual syntax to be typed to generate a test. Which is one less reason not to do TDD, which can only be a good thing!

New Tricks from an Old Pattern

Its great when you can still be surprised by a pattern you think you already know inside-out. I was pleased today to think about the BDD Unit Testing Style 'pattern' in a new way. Here's my previous post on the subject that will give you the necessary background for the rest of this post.

The pattern utilises object inheritance to make it powerful and descriptive. To illustrate, we must start with an example. The code below contains a method that tests two conditions and tries to return an object. If both conditions fail then the method throws an exception. It is worth noting that the sample class is not a good example of OO design, but rather a simplified case. The important point is that there is a path through the method that will cause an exception to be thrown.

Using the BDD Style Unit Testing pattern makes it pretty easy to test the first two conditions and the object that is returned in each case. Note the use of an abstract base class which provides us with the Result variable to hold our return object and importantly encapsulates away the call to the When() method. The Class is called WhenIGetScoredPlayer, meaning we should expect to find the call to GetScoredPlayer() on the Target inside of the overriden When() method (see the preceding article if you aren't following).

This means we don't have to repeat this call to the same target method in any of the classes that inherit from this base class. It makes the classes easy to navigate and the test output becomes straight forward as coherent sentences are produced: "When I Get Scored Player, Given The Player Has a Score Greater Than 8, Then the Result is a High Score Player". This is useful information when reviewing tests results.

However, you will now notice that its not so simple in the third path of this method. That's because in the case when a player scores below 2 an exception is thrown. If I inherit from the abstract WhenIGetScoredPlayer class and setup the player's score accordingly, then the exception will be thrown too early. That's because in the base GivenA<T> class a call to the virtual When() follows a call to the virtual Given() in a method tagged with [SetUp], to make sure they happen in sequence as part of the framework - its useful for us to have things happen in this order.

I could start from scratch and perform all the setup in a new class. Making the call to GetScoredPlayer() inside of a [Test] method instead of as part of [SetUp]. But I don't want to duplicate unnecessarily. This is when I thought about the old pattern in a new way - using an override to postpone the call instead of starting all over again.

What I did was to still implement the base class with the appropriately named GivenThePlayerHasAScoreLowerThan2. The trick is to simply override the base When() and leave it empty. This prevents it from calling the GetScoredPlayer() during the [SetUp] phase of the test. It can now occur in the [Test] method, wrapped in an NUnit assertion that the correct type of Exception is thrown. Producing the desired output without unnecessary duplication: "When I Get Scored Player, Given The Player Has a Score Lower Than 2, A Score Too Low Exception Is Thrown".

Automock with StructureMap and Moq

This is a guide of how to use AutoMocking with StructureMap and Moq in conjunction with cucumber BDD style unit testing.

Taking the BDD GIVEN, WHEN & THEN Scenario approach to unit testing has a number of benefits. I find it makes tests easier to maintain as you stick to one assertion per test. The test scenarios are well partitioned making them cleaner than straight forward Arrange Act Assert style. It also produces more readable output of test results.

Automocking is a time-saving feature of StructureMap that allows you to easily instantiate the Class Under Test (the class that will be used in the GIVEN statement of the Scenario) by automatically resolving the constructor dependencies of the class with Mocked out instances. Test classes therefore become decoupled from the constructor(s) of the Class Under Test. Meaning that you'll get a test that fails if you change a constructor instead of a test that won't build. This is a debatable point but can be considered an advantage in TDD - you get feedback from your tests as opposed to the compiler, as it should be.

By installing the StructureMap.AutoMocking and Moq packages you get out-of-the-box support for AutoMocking. Given that I regularly use StructureMap and Moq as my IOC and mocking framework of choice respectively, it makes sense to take advantage.

An Example

In the example class to test there are two dependencies that will need to be automatically mocked, and a method that should be tested.


The base class provides us with the GIVEN & WHEN syntax. It uses the MoqAutoMocker provided by the StrutureMap.Automocking library to mock any dependencies on our class's constructor. It provides a handle to access any Mocked dependency that was injected when the class was instantiated. Also provided is a shortcut to Moq's Verify method as well as some syntactic sugar that lets you use [Then] instead of [Test] to round off the BDD style.


The example unit test class implements the abstract class and is named according to the method being tested. There should be one class per method being tested - i.e. one WHEN statement per scenario. If there is another public method it should be tested in a new class that also implements the base class. This practice is what makes the test output easy to digest.

References

• StructureMap Automocker Documentation
• StackOverflow question with handy Moq tip to access underlying Mock
• Scott Millet's post on BDD Unit Testing

Specflow BDD test with HttpClient

I've been using Specflow for a while now for various BDD testing scenarios. However until now I'd not come across an example that I'd like to blog about.

Enter: HttpClient and HttpRequestMessage included in .net 4.5 under the System.Net.Http namespace.

Scenario

The story starts with the requirement to drop a client cookie so we know which users have visited the site before. And perform some basic logging about the visit details, query string information, referrer etc.

Straight forward enough, but the complexity arises because we want a single Visit Tracking Utility that spans multiple sites, each on a different platform and written in different languages. Some are even 3rd party sites, so we have minimal control over the html displayed - and zero control over whats going on at the server. In order to achieve this platform-agnostic solution we need to solve the problem with javascript as we accept will have at least this much control over each target site. We also have some specification about the name and value of the cookie as well as the expiry date date.

The technical requirement therefore is a url available to make requests to that will drop a cookie on the clients machine called VisitId, with a Guid value that expires in 7 days time.

The BDD Test

If you are not already, please familiarise with specflow. The plan here is to capture the Behaviour in a Feature file that we can make assertions from based on our code. We begin by citing the scenario we want to test using the Gherkin Given; When; Then format. Specflow then allows us to attach code to the steps we dictate, then we insert the required code and necessary NUnit assertions to validate our scenario.

Starting with a new IntegrationTest project in the solution we add a new VisitTracking feature file and enter the following scernario.

Feature: Visit Tracking
    In order to know which users have previously visited
    I want to drop a client cookie on each visit

Scenario: Setting client cookie for a new visit
    Given the api uri is local.trackmyvisit.com/api/trackvisit
    And the expected cookie name is VisitId
    When I hit the visit tracking uri
    Then the response HttpCode is OK
    And the response sets a cookie
    And the cookie name is correct
    And the cookie value is a valid Guid
    And the cookie expiry is 7 days from now

Each 'Then' and following 'And' gives a chance to make an assertion about the request we have made. We are testing exactly the things described in our specification.

If we build and try to run this now, then specflow will tell us that we are missing the steps specified in the scenario:

-> No matching step definition found for the step. Use the following code to create one:
[When(@"I hit the visit tracking uri")]
public void WhenIHitTheVisitTrackingUri()
{
    ScenarioContext.Current.Pending();
}

Copy from the test output all the necessary steps into a VisitTrackingSteps class. With a couple of extra helper methods and paramter arguments on your class should look like the following. Don't forget to use the [Binding] attribute before the class declaration or specflow won't know where to look for the steps. Also you will need to install the System.Net.Http package to get reference to the HttpClient object et al.

[Binding]
public class VisitTrackingSteps
{
    private string _uri;
    private string _cookieName;
    private HttpResponseMessage _result;

    [Given(@"the api uri is (.*)")]
    public void GivenTheApiUriIs(string uri)
    {
        _uri = "http://" + uri;
    }

    [Given(@"the expected cookie name is (.*)")]
    public void GivenTheExpectedCookieNameIs(string cookieName)
    {
        _cookieName = cookieName;
    }

    [When(@"I hit the visit tracking uri")]
    public void WhenIHitTheVisitTrackingUri()
    {
        var client = new HttpClient();
        var msg = new HttpRequestMessage(HttpMethod.Get, _uri);

        _result = client.SendAsync(msg).Result;
    }

    [Then(@"the response HttpCode is (.*)")]
    public void ThenTheResponseHttpCodeIs(HttpStatusCode statusCode)
    {
        Assert.That(_result.StatusCode, Is.EqualTo(statusCode));
    }

    [Then(@"the response sets a cookie")]
    public void ThenTheResponseSetsACookie()
    {
        var isSetCookieHeaderPresent =
            !String.IsNullOrEmpty(GetValueOfSetCookieHeader());
        Assert.IsTrue(isSetCookieHeaderPresent);
    }

    [Then(@"the cookie name is correct")]
    public void ThenTheCookieNameIsCorrect()
    {
        var firstKey = ParseSetCookieValue().AllKeys.FirstOrDefault();
        Assert.That(firstKey, Is.EqualTo(_cookieName));
    }

    [Then(@"the cookie value is a valid Guid")]
    public void ThenTheCookieValueIsAValidGuid()
    {
        Guid guid;
        var guidValue = ParseSetCookieValue()[_cookieName];
        var isValidGuid = Guid.TryParse(guidValue, out guid);

        Assert.IsTrue(isValidGuid);
    }

    [Then(@"the cookie expiry is (.*) days from now")]
    public void ThenTheCookieExpiryIsDaysFromNow(int days)
    {
        var expiresValue = ParseSetCookieValue()["expires"];
        var expires = Convert.ToDateTime(expiresValue);
        var rangeStart = DateTime.Now.AddDays(7).AddMinutes(-1);
        var rangeEnd = DateTime.Now.AddDays(7).AddMinutes(1);

        Assert.That(expires, Is.InRange(rangeStart, rangeEnd));
    }

    private string GetValueOfSetCookieHeader()
    {
        IEnumerable<string> values;
        _result.Headers.TryGetValues("Set-Cookie", out values);

        return values.FirstOrDefault();
    }

    private NameValueCollection ParseSetCookieValue()
    {
        var collection = new NameValueCollection();
        var cookieValArray = GetValueOfSetCookieHeader().Split(';');

        foreach (var arr in cookieValArray.Select(s => s.Split('=')))
        {
            collection.Add(arr[0].Trim(), arr[1].Trim());
        }

        return collection;
    }
}

You can see that in the 'When' method the HttpClient and HttpRequestMessage are used to initiate the call to the visit tracking site - at present not yet created. The SendAsync method of the HttpClient is called using a GET request to the path of the api controller method. This is replicating exactly what the script tag will do from each web page it appears on. So we have been able to quickly conjure up a http request without the use of selenium or other web driver. The other nice part is that we can interrogate the response from the request by accessing the Result property of the generic Task we get from calling SendASync.

The tests will build and run now but there is currently nothing to hit so the tests will fail. We can now implement the logic that will fulfil the spec.

Tracking Logic using an Api Controller

Our spec tells us we need a uri we can hit that will perform the visit tracking logic and write back a cookie to the client through the http response. Furthermore, as there will be now javascript or html returned in the response (i.e. nothing for the client to parse) this sounds like a candidate for using an MVC4 WebApi Controller. The controller implements the exact requirements of the spec.

public class TrackVisitController : ApiController
{
    // call using GET method to /api/trackvisit
        
    public HttpResponseMessage Get()
    {
        var cookie = new HttpCookie("VisitId")
        {
            Value = Guid.NewGuid().ToString(),
            Expires = DateTime.Now.AddDays(7)
        };

        HttpContext.Current.Response.Cookies.Add(cookie);
            
        return new HttpResponseMessage(HttpStatusCode.OK);
    }
}

The unit tests are already fathomable for this method, but this post is about integration testing, so thats what we'll focus on,

Script Tag

We then only need to tag each page we wish to track. The uri is used for the source of a script tag on each page we want our tracking system to work on.

<script src="http://test.trackmyvisit.com/api/trackvisit" />

All the source is available in my GitHub Repository. And you will have to setup the site locally and adjust the host binding yourself if you want this to run out-of-the-box :-).