Logging with AppHarbor Web & Background Workers

Last year AppHarbor introduced Background Workers into their repertoire. You can push a .net solution with a console application project that will be compiled and executed just as you would expect if executing the task locally. You can have a Background Worker and a Web Worker in the same application. The Background Worker will have the same access and to the adds-on used by the Web Worker so you can delegate any 'offline' processes to a Background Worker to allow your web app to concentrate on just serving responses. This provides a good platform for horizontal scaling.

A source of frustration with this approach however is logging. The standard AH add-on for logging is LogEntries. LogEntries is a generic SAAS consumed by many integrated hosting providers. It provides token-based TCP logging when used as another target or appender with NLog or Log4Net respectively. Logs are collected with almost zero latency and are visible in the snappy LogEntries interface.

The frustration arises when you are using LogEntries with the Web Worker + Background Worker architecture I described. As I mentioned, LogEntries by default is token-based, meaning your application gets one token to use to identify itself. This token is shared by the Web Worker and the Background Worker.

If your Background Worker (running your Console Application) and your Web Worker (running your Web Application) are dependent on shared domain projects - meaning that they both make consume the same classes - then your logs can be ambiguous as to the source of the calling code. Has your 'FATAL' entry been encountered in the Web Worker or the Background Worker? You cannot tell by the type alone!

Thankfully, LogEntries provides a neat way of solving this issue, though it is not obvious if you are starting with the AppHarbor add-on, as the configuration is performed behind the scenes - you're fast-forwarded through the basic introduction. LogEntries' Logs live under a default Host, the AppHarbor Host, which is added upon Initialisation. A Default Log is added to the AppHarbor Host, and this is where your logs end up when using the provided token. The token is automatically injected into your web/app.config settings when the Web Worker and Background Workers are deployed.

To make our Web Worker and Background Worker write to separate Logs, we simply have to add a new one in our AppHarbor Host. Make sure that you select Token TCP transport.

In doing so we get a new token. We can then use that token in the Console Application app.config, so that all logging is directed to the Background Worker log. The configuration of the app.config le_appender in Log4Net looks like the following:

  <log4net>
    <appender name="LeAppender" type="log4net.Appender.LogentriesAppender, LeLog4net">
      <Token value="some-thing-secret" />
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%level% %logger %m" />
      </layout>
    </appender>
    <root>
      <appender-ref ref="LeAppender" />
    </root>
  </log4net>

This forces the LeAppender to use the token you have provided - and not the default LOGENTRIES_TOKEN injected by AppHarbor into your app.config. Separate logs are yours!

Implement a C# Interface in F#

Whilst seeking ways to learn F# and weave it into predominantly C# projects I have started to implement interfaces written in C# with an F# version. That sometimes means re-writing a C# implementation with an F# implementation. It works particularly well for a number of reasons:

Motivation - Provides direction rather than just 'hacking about' with the language.

Confidence - If you have test coverage, you can use the same tests to be sure your code is working.

Syntax - Provides a sound introduction to the indentation and parenthesis use in F# by forcing you to translate your own C# code.

Insight - Helps to reinforce that F# still compiles to IL underneath and is executed on the CLR - its all .NET after all!

Communication - Helps others to learn by examining the 'before' (C#) and 'after' (F#) implementations.

Seamless - Integrates with your existing C# solution when using dependency injection and IoC maintaining your decoupled architecture.

In F#, interfaces are implemented Explicitly as opposed to Implicitly in C#. This means that we must provide pointers to the functions we wish to fulfil the interface per method or property. The Gist below provides examples of the various signatures we would expect to implement.

Specifying Environment Variables at Deploy Time not at Build Time

Over a month ago I posted an article in which I revelled in the ability to create a publish package in one step then deploy it in another. Last week I discovered a limitation to that process, as it was, and had to find a way to reconfigure it. This post is about that challenge and the conclusions I've drawn from it.

The difficulty with the process that I described in that article was that it is largely dependent on Configuration Transformations. These have been around for a while now and do a great job of specifying the environment variables with respect to the build configuration, normally Debug or Release. But that is exactly the problem (and my realisation):  Config Transforms blur the boundary between Building and Deploying.

When I'm compiling my application I want to be thinking about the build, about optimisation, about references and dependencies. So unless I'm developing locally and debugging I usually want to build in Release mode, as its probably going to get deployed for testing or production. I might be compiling following performance enhancements - I don't want to have to concern myself with the target environment's connection strings when I'm trying to make my app go faster. I always want to build in Release mode. But with config transformations that are tightly coupled to my build configuration I'm stuck with a built package with production connections strings - I can't performance test against that!

Build configurations are necessary, but I don't want to have to create a new one just so that I can apply a different Config Transformation at Build Time. Nor do I want to specify the Publish Profile at build time either. I want to separate Build configuration and Environment configuration concerns. What I want to do is specify the Environment settings at Deploy Time, after my deploy package has been built.

Thankfully, there is a way to do this. But its much more convoluted then adding a new config transformation file to your project. It seems Microsoft have unfortunately made it easier to do things the wrong way. MS have made it easier to deploy from the desktop, but in an enterprise this is usually not the case. Perhaps the new Publish Profile is more about making it easier to deploy to Azure?

The 'Package' MSBuild target that is only available for Web projects generates a directory containing a zipped up deployable package containing all the assets and dlls required, a MyProject.deploy.cmd file that is a friendly wrapper around the msdeploy.exe and, crucially for us, a MyProject.SetParamters.xml. The xml file gives us the ability to modify the contents of the package at Deploy Time (when we call the .cmd). This is what we will be editing:


The xml file can be manually edited before deploying, but this is rarely practical. The IIS App Name and any connection strings are pulled into this file by default. The connection strings are replaced at deploy time with the actual values found in whatever the result of your configuration transformation was. It is also possible to configure the IIS App Name in your .csproj file, but then we are again dependent on the build configuration. The way to override these settings when we deploy is to call msdeploy.exe via the MyProject.deploy.cmd and make use of the -setParam flag. The syntax of which is rather cumbersome:


The second problem is this: what if you want to specify Application Settings (appSettings) in your web.config? As stated, IIS App Name and Connections strings are all that you get to override out-of-the-box. To be able to configure an App Setting, you must add a new file to the root of your web project: parameters.xml. This file needs to contain any App Settings you want to specify when you deploy. Follow this article for a more detailed explanation of the syntax of a parameter.xml. Essentially, we must specify the XPath to the setting and provide an alias to it so we can refer to it in the generated MyProject.SetParameters.xml that we will go on to edit at Deploy Time:


Now, when we create the package at Build Time there is another parameter specified in our MyProject.SetParameters.xml generated along with the package. Like so:


We then apply the same overriding "-setParam" flag when calling the MyProject.deploy.cmd at Deploy Time to configure the "My Service Url" however we want for out target environment. The final call that overrides the parameters is shown below. I've parameterised the call so that I can use the Team City UI to run a custom build and override the parameters without the chance of missing a closing quote etc.

The added bonus of this configuration is that if you are manually deploying a built package by importing it into an IIS website and using the GUI, you are then prompted to provide the settings. This is useful when only certain members of IT are permitted to know the production connection strings or other sensitive information.

However, should you not be in an enterprise with such tight restrictions, then it might be preferable to store the various environment settings in source control with the project. The benefit of this is that there is less chance of misconfiguring a deployment with a typo. This could be achieved by using the "-setParamFile" flag when calling the cmd. However, the difficulty is that if your custom SetParameters.xml is in source control then you have to go fetch it out of the deployed package, which is already zipped up by the build step! Perhaps TC's artifacts can assist here...

This feels like a much more appropriate way to configure the deployment workflow: Always compiling the code in the most suitable configuration (Release), then dealing with the deployment configuration when the time comes to Deploy (or re-deploy). The trouble is that the steps shown above are a lot more tedious and error prone in comparison to the Configuration Transforms we are already used to. Even so, I think this is the right solution for the job.

Developing Shopify Dashing on Windows with Vagrant

We've started using Shopify's Dashing gem internally so we can publicise any moving figures people are interested in. You can go from zero to great-looking-dashboard in next to no time once you understand the API. The Dashing Wiki has enough documentation and the example dashboards that ship with it are enough to communicate the usage pattern. 

This post is about the biggest stumbling block we had when starting out. We develop on Windows machines as most of what we do is .net. Dashing is a solution written in Ruby so we wanted to develop on a Linux OS as we would be deploying to Heroku to host our production dashboards.

Enter Vagrant. This a really neat tool that lets you script the creation of a virtual machine. The Vagrant script sits in the root of your project so that it is under source control and can be shared across the team. Simply open the command line at your project directory and enter vagrant up. Within minutes a virtual machine running Ubuntu in text only mode is started.

Vagrant takes care of automatically sharing the project folder on your host machine with the guest virtual machine, so that you can continue development as normal and just execute the code inside the virtual machine. The port 3030 is forwarded so that we can view the Dashboard on our local browser. This is the content of my Vagrantfile:


Vagrant.configure("2") do |config|
  config.vm.box = "precise32"
  config.vm.network :forwarded_port, guest: 3030, host: 3030
end

So now we have in waiting a virtual machine running our target operating system, with a synced folder and the ability to hit any website running inside of it. However, that's not quite enough for Dashing. We need to update the version of Ruby to 1.9.1 and add some other essential apps. Here are the necessary steps that might hopefully save someone a lot of time:


From the Windows command line:

c:\MyDashing> vagrant up
c:\MyDashing> putty


Configure putty correctly and ssh into the VM.
Log in with username: vagrant
Enter commands:

$ cd ../../
$ sudo apt-get update
$ sudo apt-get install ruby1.9.1-full -y
$ exit

Back in the Windows command line:

c:\MyDashing> vagrant reload
c:\MyDashing> putty

Again, ssh into the VM:

$ cd ../../

$ sudo apt-get install build-essential -y
$ sudo apt-get install nodejs -y
$ sudo gem install bundler --no-rdoc --no-ri
$ sudo gem install dashing --no-rdoc --no-ri

$ dashing new MyDashing
$ cd MyDashing
$ bundle install
$ dashing start


Now you have initialized a dashing project. You should be able to enter dashing start at the command prompt and not be met with any error messages. Navigate to http://localhost:3030/sample on your local machine and hey presto! Development can now continue on your local machine, while serving the dashboards via your virtual box.

Gotchas:
- Make sure you update your Gemfile with any gems used in any new job you create. Then don't forget to run bundle install at the prompt!
- Any private key files or other assets required by code should reside in the root of the project.
- Keep an eye on the output when running vagrant up as Vagrant will cleverly accomodate any conflicting forwarded ports. This can affect the ports you use to ssh into the VM and view the dashboard.

P.s. I'm aware of the provisioning facilities of Vagrant, including the capability of running all of the above through a shell script during construction of the VM. However, I couldn't get it to work without including the restart so I have just left the raw commands as a the guide for now. If/when I figure it out I will update the post. However, these steps only need to be run once per machine (not every time you run vagrant up) so you might find its just about tolerable as it is.

AngularJS Directives and Isolate Scope

I've been learning the AngularJs framework recently and have come across a case study that highlights how Isolate Scope works.

The app we're developing needs two individual sliders. The values represented by each of the sliders are related as they form two variables in a simple equation: the Loan-to-value ratio of a mortgage (LTV). The LTV is a percentage of the Loan Required figure over the Property's Value figure. There is a slider for both the Loan Required and Property Value.

Currently, mortgage lenders are reluctant to offer mortgages with an LTV over 95%. We've therefore chosen to introduce this feature into our sliders, so that they may not be dragged to a position in which the LTV ration is greater than 100%. If a slider is dragged beyond the 100% mark, the other slider must be moved accordingly so as to maintain the ratio at 100%. Challenge accepted.

This seemed like a good fit for a Directive. A Directive in AngularJs is a way of describing a portion of the page. Using a directive to construct your page is what's known as being declarative. Your intentions for the purpose of that piece of mark up is clear. A Directive would be something that you might use often, so as to provide yourself with a short hand way of getting the element on the page.

We intend to create a 'Linked Slider' Directive, that is capable of communicating with other slider directives, so that their values may stay in-sync. To do this we must make use of Isolate Scope. Scope is very important in AngularJs. It is the 'glue' that allows controllers and views to share information. The Angular Magic permits us to manipulate the same values in both the controller and the view - its the Scope that keeps the values consistent across both, thus delivering a huge amount of value for developers.

Scope is normally associated with a Controller and a View. Directives are declared on the View and share the same Scope as the Controller and View. Isolate Scope enables the Directive to maintain its own Scope. This is useful for us for two reasons. Firstly, it gives us the ability to 'wire up' our directive to our outer Scope without tightly coupling it - making it reusable elsewhere. Secondly, it allows us to have multiple instances of the same slider Directive.

You might enjoy the working example to get a feel for what I'm talking about. The code for the directive is shown below, followed with discussion..


Starting from line 1, the directive is declared on the app global variable, and named as 'linkedSlider'. It is restricted for us as an attribute (A), and then has its Scope defined.

The Scope property is the Isolate Scope I was banging on about. There is a convention at play here. The keys of the scope property become attributes that can be used to configure the Directive when it is declared in the mark up. The '=', '@' and '&' values of these keys have special meaning.

'=' means the value is a two-way binding property. This means that its value can be changed by the Directive and by the Controller. This is why we use the '=' symbol to describe both the Value and Link properties. The Value property is the value that the slider is going to maintain. The Link property is the value that the slider is going to have to update in light of changes to its own Value when the LTV is pushed over 100%. It is this arrangement that lets us declare the property once and but be able to use it as many sliders as we like.

'@' is a straight forward attribute. It does not stay up to date in the Directive if it is changed in the Controller's scope. For this reason attributes are normally populated with string literals or numeric values. In our case, we specify all the necessary properties of the slider (max, min & step).

'&' represents an expression to be evaluated. It could be a pointer to a function on the Scope, or anything that can be executed. I have used this type of scope property to contain the test to be checked each time the slider is moved, to determine whether or not the other slider needs to be adjusted.

The Link function that follows is what binds the directive to the controller and kicks everything off. You will notice still the reliance on jQuery to create an instance of the jQuery UI slider. Nothing unusual there. As you can see I pass in the necessary configuration from the Isolate Scope of the instance of the Directive. When the slide event occurs, I have to call the special function scope.$apply() for the value change to be recognised in the Controller Scope. I then evaluate the Scope's condition property (remember that we defined it as an expression with '&') to find out if the new value would cause the LTV to exceed 100%. If so, I use the scope.$apply function again to update the Link property value in the Controller Scope.

As we have just seen, we have given our Directive the ability to change the value of it's Link property. This Link property will be a Value property in another instance of the same Directive, so we need to be able to listen for changes. scope.$watch is just the thing we need to be able to react to changes to our Value in the Controller Scope. When a change occurs, we use the jQuery UI slider API to update our slider's value, this also takes care of dragging the slider to the correct position.

There still remains two unexplained features for the moment. I do not understand why a setTimeout is required. Without it, the attribute ('@') values are not yet populated at the time of initialising the JQUI Slider. Also, I am also required to apply a scope.$watch to the Link value. Again, without this the Link property value will not stay up to date, even though there is no actual function content. A small price to pay for what is otherwise an elegant (enough) solution.

Accelerating Agile

I was at QCON earlier this month. I attended plenty of excellent talks. The talk I found myself nodding in agreement most often was Dan North's Accelerating Agile. In it, he describes his experience with a firm who were capable of producing valuable software in a much shorter time frame than he had previously though possible. The overall message was that the strict Agile Methodologies he was used to practising had been replaced with bare bones Agile Manifesto reasoning and very low ceremony processes. He distilled the processes the firm exhibited down to 12 steps. I pretty much agree with each one, below I have added why I do so based on experience. Presentation slides here.

Step 1: Learn the domain
Ironically, it is the first point that I have the biggest problem with! But first, I must agree it is hugely valuable to have developers become experts in the domain that they working in. However, I think the point is largely lifted from Eric Evan's Domain Driven Design Book; specifically the Crunching Knowledge chapter. Evans, too, speaks of immersion in this sense. It is the basis for building an accurate domain model and enables the tech team to speak the same ubiquitous language as the business experts. My issue with this point is that developer time spent learning the domain should be considered 'on-the-clock'. A developer who is sat at the desk of a business expert, engaged in learning the trade is obviously not designing or coding at this moment - but they have begun the project. The clock has started. And if the point of this talk is that these steps help you produce faster, then this time spent learning (1 or 2 weeks) must be factored into that.


My second issue is that this step is not always possible on every project. Learning the trade of your domain is not also as simple as taking a short course or shadowing a colleague. I have worked on a lot of e-commerce projects in which the end user is the customer - who could be anywhere in the world. This is obviously a hurdle to getting inside the mind of those using your software! Any project in which the end user is not on in the same building will have trouble with this first step and I don't think Dan did enough to address this point.

Step 2: Prioritise risky over valuable
Dan's memorable phrasing on this step was to 'Slay the Dragon's First'. And it makes absolute sense. The traditional Agile (note the capitalisation) approach is to prioritise business value and try to deliver the simplest, and most desired functionality first. However logical this sounds, its possible to run into trouble further down the line if you do not deal with the danger zones. Making sure that you have cracked the parts of the domain which you know could jeopardise the project later is a very wise decision. This point is mirrored in George Fairbanks' book Just Enough Software Architecture; A Risk-Driven Approach. In it, he stresses the importance of evaluating Risk to inform the complexity of individual components. He argues that this should prevent an over-engineered solution, whilst Dan's point is that trying to eliminate Risk enables you to fail faster (which is good!).

Step 3: Plan as far as you need
I really like this step. As much as it appeals to my lazy side, it also supports the way we have been organising ourselves at work recently. Dan backs the idea of having only a small Kanban board and planning no more that a week ahead. This immediately gives you back all the hours spent sprint planning and storyboarding. The prescription is to plan no more than a week in advance, but crucially to review the board often. 'Often' in this context is supposed to be imprecise. Just keep the board up to date and build tacit knowledge to be able to plan smarter. From experience I absolutely agree that a small board works well because it physically won't let you plan too far ahead, nor let you get distracted. It is most effective when used as a communication tool, to serve as a snap shot of what is going on at this moment. A key piece of practical advice was that if a story/piece of work has been on the board for too long is to analyse it a break it down to smaller stories, possibly to be worked on in parallel.

Step 4: Try something different
I cannot claim to be a polyglot programmer, but I agree with the sentiment here. Having more tools at your disposal enables you to solve more problems. Dan spoke explicitly about the languages and styles that you could try out for the sake of expanding your ability to solve problems. Also at QCON, Damian Conway gave an excellent keynote: Fun with Dead Languages. The moral of the story is that learning alternative languages helps us to become better problem solvers. He makes the point that a programming language is not just a tool for repeating operations, or even just communicating ideas between developers. Conway believes that a programming language's primary purpose is to provide a way to think about a problem. If we have more ways to think we have a better chance of solving a particular problem in the best way possible. I realise that not every problem is best solved with C#. But as the .net developer I am I have difficulty slicing the problem any other way. And as such I am currently learning F#!

Step 5: Fire, Aim, Ready
This isn't especially ground breaking, its an opinion that's been around for a while in fact. But nevertheless a brilliant title that communicates the message well. Don't wait to prettify, don't hesitate to show off, don't let uncertainty stop you from demonstrating. Negative possibilities are exactly what you should seek to confirm or disprove as early as you can by getting the product out there. Dan makes the case that this all about feedback from the right people. I believe it also helps maintain the right level of communication with your stakeholders and helps bake in a success story to your project. Sharing the experience of having ideas shot down or realisations occur in cross department meetings reinforces team mentality and keeps everyone's expectations aligned.

Step 6: Build small, separate pieces
This is good practical advice mostly in the name of maintainability. The benefit of having small separate pieces is that they can be replaced easily. Mapping out the geography of your project allows you to also think more clearly about the architecture. I think we generally have to think harder about how not to build monolithic objects/solutions. Its the old argument of easy versus simple. Dan points out that the DRY principle is the enemy of Decoupled. I was uncomfortable with this idea at first but Dan went on to give clear guidelines about when to think this way: by referencing Evans' DDD he used the concept of a Bounded Context. "Let this be the limit to which you should attempt to be DRY".

Step 7: Deploy small, separate pieces
This step is largely dependent on the previous step. In my experience smaller components permit smaller, lower risk, more frequent, quicker deploys. Hopefully resulting in less down time. Dan provides lots of practical DevOps advice here. He suggests never rolling backward... If you are deploying frequently then you are only ever deploying small changes. If you are only every incrementing in small steps then you can never break something so severely that you have to roll back, just fix the code and deploy again! This requires discipline that the whole team must achieve. Dan argues the case for consistent environments and consistent deployment plans that make it easy for new starters to get to grips with and less to remember when problems in deployment arise. To help with this I have recently looked at introducing Vagrant into our development work-flow.

Step 8: Prefer simple over easy
Dan offers some specific examples in this point but the main message is clear: Simple requires thought, but in the end the result is easy to communicate. A simple idea is one that you can hold in your head. 

Step 9: Make the trade-offs
Starting off with a framework might help you get some quick wins but it might be a burden in the long run. My feeling is to always rely on third party libraries to help you get started, but to revise the position on each component where necessary.

Step 10: Share the love
This is all about communication between team members. Make it common practice to pair program and to exploit those 'Aha!' moments for the team's benefit. Dan ventures that code reviews are valuable tools. My opinion is that that they can be replaced by continuous pair programming and pull-requests, but each team is different. 'On-boarding' was a process he described to initiate new starters into the process (if any!) and technologies in play. All the steps described so far I think contribute to making this journey for a beginner as easy as possible.

Step 11: Be ok with “failure”
As I mentioned earlier Failing Fast is a Good Thing! In my experience knowing what won't work can be just as valuable as knowing what will. Dan believes we should be more concerned with Product Delivery not Project Delivery. Progress, he believes, can also be defined as a series of experiments.

Step 12: There are always 12 steps
"Delivering this fast can be addictive." My final thought is that even though it is easy to jump on the SCRUM hate wagon right now, that this is could be the opposite end of the spectrum. Let's just remember that each team and each organisation is different. And as Ward Cunningham mentioned to me after the talk: "You're an engineer aren't you? Engineer your own process!" Wise words.

Separate Build and Deploy Configs with Team City and MS Web Deploy

Finally, after a nearly two years of using the technique laid out in Troy Hunt's Blog Post, I've finally learned how to split out Building and Deployment into separate build configurations.

As in the extremely detailed post, up until now I've only been able to build and deploy in a single, immutable step. And to be fair, it's worked pretty well. Deployments have been relatively fast, predictable and repeatable. It never really bugged me until we had to create a new deployment internally that couldn't live in AppHarbor. This time, this configuration, it had to be right.

The reason having separate, distinct build and deployment steps in your deployment pipeline is a good idea is because it makes the your application build output a single atomic unit. The build can then have its second-phase acceptance tests run in parallel against the same build, so that you get feedback sooner and fail faster. The same applies to subsequent phases. More information on can be found in Jez Humble's free chapter of Continuous Delivery.

The other benefit is that it makes deploying the build very quick. The deployment step itself, at the lowest level, is about transferring files. If you can reduce the deployment step to this then deploying should only take moments. Web Deploy is smart enough to only deploy the files that have changed. So only the minimum amount of network traffic and IO  has to be tolerated. This comes in especially handy if, a bug makes into production that cannot be fixed by rolling forward with a fix. When you need to roll back, the ability to quickly deploy a previously built package without having to compile again is valuable.

The Build Configuration

The Build Config should have a number of steps: build the application, run any tests, potentially generate meta data statistics, and, most importantly, generate deployable output - known as Artifacts. Team City affords a way to access Build Artifacts by letting you address them in the general settings of the build.

In your Build Configuration, find the Artifacts Paths of your General Settings and add the following line:

MyWebApplication\obj\%system.Configuration%\Package => DeployPackage

This places the deployable package in your collection of artifacts in a folder called DeployPackage. Now we have to tell MSBuild to create a package for us when we build the solution. Add a Visual Studio build step, and make sure the solution file path is correct.  Enter the Configuration as and then enter the magic Command Line parameters:

/p:DeployOnBuild=true;DeployTarget=Package

You just need to add the the value of the system.Configuration to the Build Parameters collection of your configuration and this Build Config is complete.

The Deploy Configuration

The deploy configuration has as lot less to do. It only needs to deploy the package that was created in the first build configuration. In Team City, to enable the Deploy Configuration to see the artifacts of another configuration, it needs an Artifact Dependency. Add a new one in the new Deploy configuration by selecting the first Build config in the 'Depend On' drop down. Choose the Last Successful build, and lastly, specify the Artificats Rules. If you provide the line below, you will copy the Artifacts from the DeployPackage folder into a local folder that the Deploy Config can use.

DeployPackage => DeployPackage

If you peek inside the contents of the Deploy Package Contents the Build Config generates, you'll see an executable inside the root folder, with the same name of the web project you want to deploy. This is the exe that will do the heavy lifting of file copying and web config transforming (if you have environment specific configs).

You need to add a single Command Line Build Step to your Deploy Config to run this executable. Importantly, specify the Working Directory to be the path of the Artifacts we copied over from the Dependency we added: DeployPackage. Provide the Command Executable name and lastly, the necessary parameters:

/y /M:https://stage.environment.com:8172/MsDeploy.axd /u:username /p:password /A:Basic -allowUntrusted

I won't go into the specifics of what the M parameter is, but you should have installed Web Deploy on your hosting server already, this is the Web Deploy listener address of that host. The remaining parameters are pretty straight forward.

The Deploy Config will now take whatever was generated from the Build Config and deploy it to the host web server. Two. Separate. Steps. For an extra touch of Auto-Magic you can add a Trigger to your Deploy Config so that it will run whenever the 'dependee', Build Config succeeds. If your Build config is triggered by a change in source control, this results in a deployment after each git push or svn commit. This can be handy, but is not always desirable in production or if you have testers working on specific builds.

Bonus: Custom Builds

If you need to roll back to a previous version, or you need to deploy to a new host server there is a way to make good use of your Deploy Config: the Custom Build. By clicking the '...' next to the Run button you can execute the build with specific, one-off, configuration. You can override the build parameters, for example, to point to a different server (you will need to update your command line properties to read from a build parameter). In the Dependencies tab you will find the option to deploy builds from any of the previous versions of the Build Configuration output, pictured below. This can save the day in an emergency, arms-flailing-hair-on-fire-roll-back scenario. Enjoy.