Continuing my (inadvertent) series on stream-lining your deployment process this next instalment is about bundling your deployment script in with your deployment package and breaking dependencies between Build and Deployment Configurations.
Previous Topics:
1 - App Settings are for Variables that Change with the Environment
2 - Separate Build and Deploy Configs with Team City and MS Web Deploy
3 - Specifying Environment Variables at Deploy Time not at Build Time
When I last left this topic, environment settings were captured in Team City's Build Parameters. They were applied at deploy time using a deployment script that called the web deploy command line executable. This helps keep sensitive settings out of source control. The bad news is that this deploy script was also maintained in Team City.
The reason this is a problem becomes clear when we need to add or remove a parameter from the configuration. A new parameter name and value must obviously be added to the configuration before it can be used in the deployment. However, we are also required to update the deploy script to apply this parameter during the deployment. What we have is a dependency on the build package on the deploy script. The deploy script cannot be updated long in advance of a new parameter being added, as a deployment in the meantime that uses the new parameter where it is not yet needed will cause the build to fail. We need to move away from a shared deploy script and towards one that is specific to the package it is going to deploy.
The answer here is to bring the deploy script into source control. That way, we can manage the script like any other code artifact with version history and also, crucially, configure the script such that it will apply all the build parameters to the deployment package at the time it should be deployed. We can then add the actual parameter name and value to Team City far in advance of the actual deployment time, as we know our build script will only try to make use of the new parameter when it is executed, whilst parallel deployments can continue unaffected.
A deployment script might look something like this:
MyWebApplication.deploy.cmd /y /M:%TargetDeployUrl% /u:%Username% /p:%Password% /A:Basic -allowUntrusted "-setParam:name='LoggingEmailAddress',value='%LoggingEmailAddress%'" "-setParam:name='ServiceEndPoint',value='%ServiceEndPoint%'"
This contains only two build parameters (environment variables), but you can see that its fiddly enough to warrant version history. All we are doing is calling a command line program, so the easiest thing to do would be to create another .cmd in the same directory. I'm going to use a Powershell script to do the same because it will make it easier to work with a collection of Target Deploy Urls that we shall see later on.
The plan is to capture this call in a powershell script which is then called by our deployment build configuration in Team City. Team City will need to pass the build parameters to the powershell deploy script so that the actual values can be substituted upon execution. The simplest way to do this is with Environment Variables. Enviroment Variables are read from powershell using the $env: prefix. You'll see these in action in the script below. Warning: Do not use periods or dashes in your environment variable names as powershell has trouble escaping these characters - use underscores if necessary. Don't worry about the 'env.' that is prepended to your variable. This does not affect your deploy script.
The powershell script should be added to the root of the solution and committed to source. When the Build Configuration is run (that's build, not deploy see my previous article on separating build and deploy steps) we will need to make sure that this script is 'bundled' in with the deployment package. The deployment package contains the web deploy .cmd program that we need to call, so we should put our script in the same directory as this. To do this, configure an additional Artifact Path in the General Settings of your build configuration in Team City. We'll call our deploy script Deployment.ps1.
The deploy build configuration now has access to this script. The single build step of our deploy configuration just has to invoke Deployment.ps1. The settings shown in the image below were all I needed to enable this. The environment variables do not have to be referenced directly - this is a good thing. They are all accessible by the deploy script throughout the deployment, permitting the script to make use of whatever environment variables it needs.
Lastly, the contents of Deployment.ps1 are shown below. The main reason for using a powershell script is now clear: the ability to deploy to any number of targets. Running through the script, we see that it first defines a function to get the working directory. This is needed in order to reference the web deploy .cmd by its absolute file path. Then, we read in the TargetDeployUrls environment variable - as defined in our deploy configuration in Team City - and split it into an array on the comma character. We then loop over the array, calling the web deploy .cmd once per target. You can see that the long command is split into several lines for readability using a here-string. The line breaks are then removed so a single line operation can be performed. Note where the $env:LoggingEmailAddress and $env:ServiceEndPoint parameters are referenced. The command is executed using the Invoke-Expression alias iex.
The array loop has the advantage of allowing us to deploy to environments with any number of host machines. For example, in our staging environment we have only one server that needs to be deployed to, whilst our production environment has two. This deployment script caters for both these scenarios - all you need to when deploying to production is provide a comma separated list of deploy target urls. However, in its current configuration the script applies the same username and password to all servers. This might not be the case in every environment, but that improvement will have to wait for another day!
Showing posts with label webdeploy. Show all posts
Showing posts with label webdeploy. Show all posts
Monday, 27 January 2014
Monday, 11 March 2013
Separate Build and Deploy Configs with Team City and MS Web Deploy
Finally, after a nearly two years of using the technique laid out in Troy Hunt's Blog Post, I've finally learned how to split out Building and Deployment into separate build configurations.
As in the extremely detailed post, up until now I've only been able to build and deploy in a single, immutable step. And to be fair, it's worked pretty well. Deployments have been relatively fast, predictable and repeatable. It never really bugged me until we had to create a new deployment internally that couldn't live in AppHarbor. This time, this configuration, it had to be right.
The reason having separate, distinct build and deployment steps in your deployment pipeline is a good idea is because it makes the your application build output a single atomic unit. The build can then have its second-phase acceptance tests run in parallel against the same build, so that you get feedback sooner and fail faster. The same applies to subsequent phases. More information on can be found in Jez Humble's free chapter of Continuous Delivery.
The other benefit is that it makes deploying the build very quick. The deployment step itself, at the lowest level, is about transferring files. If you can reduce the deployment step to this then deploying should only take moments. Web Deploy is smart enough to only deploy the files that have changed. So only the minimum amount of network traffic and IO has to be tolerated. This comes in especially handy if, a bug makes into production that cannot be fixed by rolling forward with a fix. When you need to roll back, the ability to quickly deploy a previously built package without having to compile again is valuable.
The Build Configuration
The Build Config should have a number of steps: build the application, run any tests, potentially generate meta data statistics, and, most importantly, generate deployable output - known as Artifacts. Team City affords a way to access Build Artifacts by letting you address them in the general settings of the build.
In your Build Configuration, find the Artifacts Paths of your General Settings and add the following line:
MyWebApplication\obj\%system.Configuration%\Package => DeployPackage
This places the deployable package in your collection of artifacts in a folder called DeployPackage. Now we have to tell MSBuild to create a package for us when we build the solution. Add a Visual Studio build step, and make sure the solution file path is correct. Enter the Configuration as and then enter the magic Command Line parameters:
/p:DeployOnBuild=true;DeployTarget=Package
You just need to add the the value of the system.Configuration to the Build Parameters collection of your configuration and this Build Config is complete.
The Deploy Configuration
The deploy configuration has as lot less to do. It only needs to deploy the package that was created in the first build configuration. In Team City, to enable the Deploy Configuration to see the artifacts of another configuration, it needs an Artifact Dependency. Add a new one in the new Deploy configuration by selecting the first Build config in the 'Depend On' drop down. Choose the Last Successful build, and lastly, specify the Artificats Rules. If you provide the line below, you will copy the Artifacts from the DeployPackage folder into a local folder that the Deploy Config can use.
DeployPackage => DeployPackage
If you peek inside the contents of the Deploy Package Contents the Build Config generates, you'll see an executable inside the root folder, with the same name of the web project you want to deploy. This is the exe that will do the heavy lifting of file copying and web config transforming (if you have environment specific configs).
You need to add a single Command Line Build Step to your Deploy Config to run this executable. Importantly, specify the Working Directory to be the path of the Artifacts we copied over from the Dependency we added: DeployPackage. Provide the Command Executable name and lastly, the necessary parameters:
/y /M:https://stage.environment.com:8172/MsDeploy.axd /u:username /p:password /A:Basic -allowUntrusted
I won't go into the specifics of what the M parameter is, but you should have installed Web Deploy on your hosting server already, this is the Web Deploy listener address of that host. The remaining parameters are pretty straight forward.
The Deploy Config will now take whatever was generated from the Build Config and deploy it to the host web server. Two. Separate. Steps. For an extra touch of Auto-Magic you can add a Trigger to your Deploy Config so that it will run whenever the 'dependee', Build Config succeeds. If your Build config is triggered by a change in source control, this results in a deployment after each git push or svn commit. This can be handy, but is not always desirable in production or if you have testers working on specific builds.
Bonus: Custom Builds
If you need to roll back to a previous version, or you need to deploy to a new host server there is a way to make good use of your Deploy Config: the Custom Build. By clicking the '...' next to the Run button you can execute the build with specific, one-off, configuration. You can override the build parameters, for example, to point to a different server (you will need to update your command line properties to read from a build parameter). In the Dependencies tab you will find the option to deploy builds from any of the previous versions of the Build Configuration output, pictured below. This can save the day in an emergency, arms-flailing-hair-on-fire-roll-back scenario. Enjoy.
As in the extremely detailed post, up until now I've only been able to build and deploy in a single, immutable step. And to be fair, it's worked pretty well. Deployments have been relatively fast, predictable and repeatable. It never really bugged me until we had to create a new deployment internally that couldn't live in AppHarbor. This time, this configuration, it had to be right.
The reason having separate, distinct build and deployment steps in your deployment pipeline is a good idea is because it makes the your application build output a single atomic unit. The build can then have its second-phase acceptance tests run in parallel against the same build, so that you get feedback sooner and fail faster. The same applies to subsequent phases. More information on can be found in Jez Humble's free chapter of Continuous Delivery.
The other benefit is that it makes deploying the build very quick. The deployment step itself, at the lowest level, is about transferring files. If you can reduce the deployment step to this then deploying should only take moments. Web Deploy is smart enough to only deploy the files that have changed. So only the minimum amount of network traffic and IO has to be tolerated. This comes in especially handy if, a bug makes into production that cannot be fixed by rolling forward with a fix. When you need to roll back, the ability to quickly deploy a previously built package without having to compile again is valuable.
The Build Configuration
The Build Config should have a number of steps: build the application, run any tests, potentially generate meta data statistics, and, most importantly, generate deployable output - known as Artifacts. Team City affords a way to access Build Artifacts by letting you address them in the general settings of the build.
In your Build Configuration, find the Artifacts Paths of your General Settings and add the following line:
MyWebApplication\obj\%system.Configuration%\Package => DeployPackage
This places the deployable package in your collection of artifacts in a folder called DeployPackage. Now we have to tell MSBuild to create a package for us when we build the solution. Add a Visual Studio build step, and make sure the solution file path is correct. Enter the Configuration as and then enter the magic Command Line parameters:
/p:DeployOnBuild=true;DeployTarget=Package
You just need to add the the value of the system.Configuration to the Build Parameters collection of your configuration and this Build Config is complete.
The Deploy Configuration
The deploy configuration has as lot less to do. It only needs to deploy the package that was created in the first build configuration. In Team City, to enable the Deploy Configuration to see the artifacts of another configuration, it needs an Artifact Dependency. Add a new one in the new Deploy configuration by selecting the first Build config in the 'Depend On' drop down. Choose the Last Successful build, and lastly, specify the Artificats Rules. If you provide the line below, you will copy the Artifacts from the DeployPackage folder into a local folder that the Deploy Config can use.
DeployPackage => DeployPackage
If you peek inside the contents of the Deploy Package Contents the Build Config generates, you'll see an executable inside the root folder, with the same name of the web project you want to deploy. This is the exe that will do the heavy lifting of file copying and web config transforming (if you have environment specific configs).
You need to add a single Command Line Build Step to your Deploy Config to run this executable. Importantly, specify the Working Directory to be the path of the Artifacts we copied over from the Dependency we added: DeployPackage. Provide the Command Executable name and lastly, the necessary parameters:
/y /M:https://stage.environment.com:8172/MsDeploy.axd /u:username /p:password /A:Basic -allowUntrusted
I won't go into the specifics of what the M parameter is, but you should have installed Web Deploy on your hosting server already, this is the Web Deploy listener address of that host. The remaining parameters are pretty straight forward.
The Deploy Config will now take whatever was generated from the Build Config and deploy it to the host web server. Two. Separate. Steps. For an extra touch of Auto-Magic you can add a Trigger to your Deploy Config so that it will run whenever the 'dependee', Build Config succeeds. If your Build config is triggered by a change in source control, this results in a deployment after each git push or svn commit. This can be handy, but is not always desirable in production or if you have testers working on specific builds.
Bonus: Custom Builds
If you need to roll back to a previous version, or you need to deploy to a new host server there is a way to make good use of your Deploy Config: the Custom Build. By clicking the '...' next to the Run button you can execute the build with specific, one-off, configuration. You can override the build parameters, for example, to point to a different server (you will need to update your command line properties to read from a build parameter). In the Dependencies tab you will find the option to deploy builds from any of the previous versions of the Build Configuration output, pictured below. This can save the day in an emergency, arms-flailing-hair-on-fire-roll-back scenario. Enjoy.
Subscribe to:
Posts (Atom)




