Decoupling your Web Deploy Build Script from Team City with Powershell

Continuing my (inadvertent) series on stream-lining your deployment process this next instalment is about bundling your deployment script in with your deployment package and breaking dependencies between Build and Deployment Configurations.

Previous Topics:
1 - App Settings are for Variables that Change with the Environment
2 - Separate Build and Deploy Configs with Team City and MS Web Deploy
3 - Specifying Environment Variables at Deploy Time not at Build Time

When I last left this topic, environment settings were captured in Team City's Build Parameters. They were applied at deploy time using a deployment script that called the web deploy command line executable. This helps keep sensitive settings out of source control. The bad news is that this deploy script was also maintained in Team City.

The reason this is a problem becomes clear when we need to add or remove a parameter from the configuration. A new parameter name and value must obviously be added to the configuration before it can be used in the deployment. However, we are also required to update the deploy script to apply this parameter during the deployment. What we have is a dependency on the build package on the deploy script. The deploy script cannot be updated long in advance of a new parameter being added, as a deployment in the meantime that uses the new parameter where it is not yet needed will cause the build to fail. We need to move away from a shared deploy script and towards one that is specific to the package it is going to deploy.

The answer here is to bring the deploy script into source control. That way, we can manage the script like any other code artifact with version history and also, crucially, configure the script such that it will apply all the build parameters to the deployment package at the time it should be deployed. We can then add the actual parameter name and value to Team City far in advance of the actual deployment time, as we know our build script will only try to make use of the new parameter when it is executed, whilst parallel deployments can continue unaffected.

A deployment script might look something like this:

MyWebApplication.deploy.cmd /y /M:%TargetDeployUrl% /u:%Username% /p:%Password% /A:Basic -allowUntrusted "-setParam:name='LoggingEmailAddress',value='%LoggingEmailAddress%'" "-setParam:name='ServiceEndPoint',value='%ServiceEndPoint%'"

This contains only two build parameters (environment variables), but you can see that its fiddly enough to warrant version history. All we are doing is calling a command line program, so the easiest thing to do would be to create another .cmd in the same directory. I'm going to use a Powershell script to do the same because it will make it easier to work with a collection of Target Deploy Urls that we shall see later on.

The plan is to capture this call in a powershell script which is then called by our deployment build configuration in Team City. Team City will need to pass the build parameters to the powershell deploy script so that the actual values can be substituted upon execution. The simplest way to do this is with Environment Variables. Enviroment Variables are read from powershell using the $env: prefix. You'll see these in action in the script below. Warning: Do not use periods or dashes in your environment variable names as powershell has trouble escaping these characters - use underscores if necessary. Don't worry about the 'env.' that is prepended to your variable. This does not affect your deploy script.

The powershell script should be added to the root of the solution and committed to source. When the Build Configuration is run (that's build, not deploy see my previous article on separating build and deploy steps) we will need to make sure that this script is 'bundled' in with the deployment package. The deployment package contains the web deploy .cmd program that we need to call, so we should put our script in the same directory as this. To do this, configure an additional Artifact Path in the General Settings of your build configuration in Team City. We'll call our deploy script Deployment.ps1.

The deploy build configuration now has access to this script. The single build step of our deploy configuration just has to invoke Deployment.ps1. The settings shown in the image below were all I needed to enable this. The environment variables do not have to be referenced directly - this is a good thing. They are all accessible by the deploy script throughout the deployment, permitting the script to make use of whatever environment variables it needs.

Lastly, the contents of Deployment.ps1 are shown below. The main reason for using a powershell script is now clear: the ability to deploy to any number of targets. Running through the script, we see that it first defines a function to get the working directory. This is needed in order to reference the web deploy .cmd by its absolute file path. Then, we read in the TargetDeployUrls environment variable - as defined in our deploy configuration in Team City - and split it into an array on the comma character. We then loop over the array, calling the web deploy .cmd once per target. You can see that the long command is split into several lines for readability using a here-string. The line breaks are then removed so a single line operation can be performed. Note where the $env:LoggingEmailAddress and $env:ServiceEndPoint parameters are referenced. The command is executed using the Invoke-Expression alias iex.


The array loop has the advantage of allowing us to deploy to environments with any number of host machines. For example, in our staging environment we have only one server that needs to be deployed to, whilst our production environment has two. This deployment script caters for both these scenarios - all you need to when deploying to production is provide a comma separated list of deploy target urls. However, in its current configuration the script applies the same username and password to all servers. This might not be the case in every environment, but that improvement will have to wait for another day!

Specifying Environment Variables at Deploy Time not at Build Time

Over a month ago I posted an article in which I revelled in the ability to create a publish package in one step then deploy it in another. Last week I discovered a limitation to that process, as it was, and had to find a way to reconfigure it. This post is about that challenge and the conclusions I've drawn from it.

The difficulty with the process that I described in that article was that it is largely dependent on Configuration Transformations. These have been around for a while now and do a great job of specifying the environment variables with respect to the build configuration, normally Debug or Release. But that is exactly the problem (and my realisation):  Config Transforms blur the boundary between Building and Deploying.

When I'm compiling my application I want to be thinking about the build, about optimisation, about references and dependencies. So unless I'm developing locally and debugging I usually want to build in Release mode, as its probably going to get deployed for testing or production. I might be compiling following performance enhancements - I don't want to have to concern myself with the target environment's connection strings when I'm trying to make my app go faster. I always want to build in Release mode. But with config transformations that are tightly coupled to my build configuration I'm stuck with a built package with production connections strings - I can't performance test against that!

Build configurations are necessary, but I don't want to have to create a new one just so that I can apply a different Config Transformation at Build Time. Nor do I want to specify the Publish Profile at build time either. I want to separate Build configuration and Environment configuration concerns. What I want to do is specify the Environment settings at Deploy Time, after my deploy package has been built.

Thankfully, there is a way to do this. But its much more convoluted then adding a new config transformation file to your project. It seems Microsoft have unfortunately made it easier to do things the wrong way. MS have made it easier to deploy from the desktop, but in an enterprise this is usually not the case. Perhaps the new Publish Profile is more about making it easier to deploy to Azure?

The 'Package' MSBuild target that is only available for Web projects generates a directory containing a zipped up deployable package containing all the assets and dlls required, a MyProject.deploy.cmd file that is a friendly wrapper around the msdeploy.exe and, crucially for us, a MyProject.SetParamters.xml. The xml file gives us the ability to modify the contents of the package at Deploy Time (when we call the .cmd). This is what we will be editing:


The xml file can be manually edited before deploying, but this is rarely practical. The IIS App Name and any connection strings are pulled into this file by default. The connection strings are replaced at deploy time with the actual values found in whatever the result of your configuration transformation was. It is also possible to configure the IIS App Name in your .csproj file, but then we are again dependent on the build configuration. The way to override these settings when we deploy is to call msdeploy.exe via the MyProject.deploy.cmd and make use of the -setParam flag. The syntax of which is rather cumbersome:


The second problem is this: what if you want to specify Application Settings (appSettings) in your web.config? As stated, IIS App Name and Connections strings are all that you get to override out-of-the-box. To be able to configure an App Setting, you must add a new file to the root of your web project: parameters.xml. This file needs to contain any App Settings you want to specify when you deploy. Follow this article for a more detailed explanation of the syntax of a parameter.xml. Essentially, we must specify the XPath to the setting and provide an alias to it so we can refer to it in the generated MyProject.SetParameters.xml that we will go on to edit at Deploy Time:


Now, when we create the package at Build Time there is another parameter specified in our MyProject.SetParameters.xml generated along with the package. Like so:


We then apply the same overriding "-setParam" flag when calling the MyProject.deploy.cmd at Deploy Time to configure the "My Service Url" however we want for out target environment. The final call that overrides the parameters is shown below. I've parameterised the call so that I can use the Team City UI to run a custom build and override the parameters without the chance of missing a closing quote etc.

The added bonus of this configuration is that if you are manually deploying a built package by importing it into an IIS website and using the GUI, you are then prompted to provide the settings. This is useful when only certain members of IT are permitted to know the production connection strings or other sensitive information.

However, should you not be in an enterprise with such tight restrictions, then it might be preferable to store the various environment settings in source control with the project. The benefit of this is that there is less chance of misconfiguring a deployment with a typo. This could be achieved by using the "-setParamFile" flag when calling the cmd. However, the difficulty is that if your custom SetParameters.xml is in source control then you have to go fetch it out of the deployed package, which is already zipped up by the build step! Perhaps TC's artifacts can assist here...

This feels like a much more appropriate way to configure the deployment workflow: Always compiling the code in the most suitable configuration (Release), then dealing with the deployment configuration when the time comes to Deploy (or re-deploy). The trouble is that the steps shown above are a lot more tedious and error prone in comparison to the Configuration Transforms we are already used to. Even so, I think this is the right solution for the job.

ClickOnce Automatic Update with Continuous Deployment

Problem

We need a task-scheduled desktop application to force update itself on start-up every time it runs, so that it always executes the latest version. There may or may not be updates available for the application. This should be part of an automated deployment process, such that the developer does not need to remember to 'bump' the version number on every release.

Solution

A console or windows forms application published as a ClickOnce application has the ability to solve this. We just need to configure the application accordingly. The other element of the solution is Continuous Deployment using Team City. This will provide the automatic versioning.

Click Once

Add the following to the .csproj file to be deployed in the first property group. The following settings worth noting:






<IsWebBootstrapper> & <BootstrapperEnabled>
Gives the application the ability to download other requirements of your application.


<Install>
States that the application will run from the installed files (as opposed to downloading a new executable each time - this is not possible in this scenario as we need the task to be automated).


<InstallFrom>
Tells the application where to look for installation source files. Could alternatively be CD-Rom based.


<UpdateEnabled>, <UpdateMode> & <UpdateRequired>
Needed to make sure the application will check for updates and to do so prominently on screen when updates are required.


<ApplicationVersion>
Tells the application which version it is. This setting will be overridden in when configuring continuous deployment. 


We could run the publishing wizard at this point and install the application locally. However we need this to run on a client/target machine. This deployment aspect needs to be handled by Team City.

Team City

We will assume a Build Configuration is already in place and that the files are made available from your source control. We can then start by adding a build step to 'Publish' the project in the ClickOnce format.


Build Number configuration under General Settings
Here we are setting up the build number to comply with [major].[minor].[build].[revision] format required to apply to .net assemblies. This build number will also be used to tell the published ClickOnce application which version it is currently running.

Assembly Info Patcher Build Feature
Under Build Steps you will need to enable the Assembly Info Patcher as an Additional Build Feature. This will set the assembly version of our project (and all projects in our solution) to be the Build Number set by Team City.

Build Step 1 (Publish)
Here, the only important setting is to specify the target of the MSBuild task is to 'Publish' the project we want to deploy.

System Properties under Build Parameters
These properties tell the build to set the Application Version and the Minimum Required Version to be the same property. This is what keeps the application up-to-date. By specifying that the minimum required version is the same as the latest version, the application always stays up to date.

Also use: system.Configuration = Release

Publish Url
The remaining element of this approach is to publish the necessary installation files. In the example I have specified http://mypublishurl.com/ as an example. In reality, this url could simple be the address of a virtual directory, pointing to a folder where the all the output of the build is copied to.


Build Step 2 (Copy files to Publish Path)
Therefore, the last step of the Team City build process is to copy the output to the directory our Publish Url is serving. The output required is found in the app.publish folder of our bin directory. 

Installing

You should now be able install your application. This is done by accessing the setup.exe generated by the Publish build step. Hit http://mypublishurl.com/setup.exe and you should be greeted with a download dialog. After continuing with the install the application will run immediately and you should be left with a start menu entry under the publisher name you specified.

Updating

Trigger a new build of the build configuration in Team City. This will increment your build number and re-deploy the application to the publish path. Wait for the build to complete and open your application from the start menu again. It should now attempt to update itself.

Scheduling

This is easily achived with windows task scheduler. You will just need to point to the 
.application file in your start menu.