Deploying Azure web applications with FAKE


The Azure App Service is a great service that makes hosting web-facing applications extremely easy, with support for many value adds out of the box e.g. scale out, A/B testing and authentication are all included. I’ve recently been looking at how you can use this service within the context of some F# frameworks and libraries e.g. Suave. I’ll blog about the Suave side of things in another post – there’s a lot to it – but one of the other parts I wanted to mention was that FAKE now has support for Kudu, the Azure App Service SCM deployment engine.

What is Kudu?

One of the features that App Service offers is a multitude of deployment options, including FTP, HTTP and also source control web hooks. The latter supports a number of providers, including GitHub, BitBucket, VSTS and even a locally hosted git repository. The App Service listens to push events on a specific branch, downloads the source code onto the web server (into a sandboxed location) and then copies it into the website proper. The latter stage – the copy – is the one of most interest. Essentially, the app service runs a batch file which can do whatever is needed to do a build and deploy. For a .NET application, this typically includes: –

  1. Perform an MSBuild of the application.
  2. Copy the outputs to the “staging” directory on the web site.
  3. Run KuduSync to deploy to the actually web application folder.

KuduSync itself essentially does a few things: –

  • Does a diff of the current files to deploy from the previous deployment.
  • Removes any obsolete files from the app.
  • Copies over any new / updated files from the staging directory to the app.
  • Makes a list of the deployed files for comparison the next time it runs.

The Azure CLI extensions come with some commands to “pre-generate” a batch file for specific, common use cases e.g. ASP .NET application etc.., but you’ll often need to do something more than just that – and this means getting your hands dirty with a kudu script.

A Sample Kudu script

So here’s a standard Kudu build script (which I’ve actually minimised as much as possible) which deploys some raw web assets (HTML, JS etc.), builds a .NET application and deploys a web job : –

:: Restore NuGet packages
.paket\paket.bootstrapper.exe
.paket\paket.exe restore

:: Copy static site content over - note the "excludes.txt" which contains file types to ignore....
xcopy src\webhost "%DEPLOYMENT_TEMP%\" /Y /E /Q /EXCLUDE:excludes.txt
IF !ERRORLEVEL! NEQ 0 goto error

:: Deploy an F# script as a continuously running Web Job
xcopy src\Sample.fsx "%DEPLOYMENT_TEMP%\app_data\jobs\continuous\Sample\" /Y
IF !ERRORLEVEL! NEQ 0 goto error

:: Build to the temporary path
cd "%DEPLOYMENT_SOURCE%"
call :ExecuteCmd "%MSBUILD_PATH%" /m /t:Build /p:Configuration=Release;OutputPath="%DEPLOYMENT_TEMP%";UseSharedCompilation=false %SCM_BUILD_ARGS% /v:m
IF !ERRORLEVEL! NEQ 0 goto error
cd ..

:: KuduSync
call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_TEMP%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
IF !ERRORLEVEL! NEQ 0 goto error

There’s actually a whole host of things here: –

  1. First I’m pulling down my Nuget dependencies with Paket (this could just as easily be Nuget.exe) before moving onto our main build.
  2. xcopy across the website assets. There are some files I don’t want, so I also have to add an “excludes.txt” file which contains file types I don’t want as an argument to xcopy. This was a real pain to figure out and to use the correct arguments to xcopy.
  3. Copy across an .fsx file as a web job. I needed to figure out how web jobs are stored on the app service in order to know the path to build up, of course.
  4. Do some jumping around of folders before doing an MSBuild of my application.
  5. Call Kudu Sync to do the final deploy, passing in the set of folder locations needed for the tool.

Using batch files for a built pipeline probably isn’t the best way to go. The problem with this is that managing a set of build sets quickly becomes a pain with a batch file – you have GOTOs everywhere, labels etc., you can’t do complex control flow etc. etc. Imagine you wanted to now run unit tests, and if that failed, do some set of tasks, but if it passed, do something else etc. etc. – it quickly becomes a nightmare.

Enter FAKE

On the other hand, FAKE is an excellent library and DSL designed to act manage a build pipeline. Not only does it have loads of helpers for e.g. file system access, config file rewriting, environment variables and MSBuild etc. but it allows us to define build pipelines with dependencies – even conditional stages. Finally, because FAKE is just F# and runs on the full .NET framework, you can always break out and just run any .NET code you want directly from within a FAKE script. With FAKE, you can have a single build script for e.g. local builds, CI builds and also now supports Kudu deployment builds through the newly-added Kudu module in FAKE. Let’s see what the above build script looks like in FAKE: –

You can see here that there are several distinct build steps, which are composed together as dependencies on one another at the very end using the ==> operator. Note that the code above is actually just F# although we’re using a specific DSL with custom operators to set up a “build chain”. So if any of the stages fail, the whole build will fail and we’ll be presented with a summary log (which we can see directly in the Azure portal) of the results of the build. Notice also the lack of environment variables etc. – the Kudu helper module takes care of all of that for us – whilst we don’t need gotos anymore because FAKE handles the build pipeline.

Now our Kudu script is much simpler, because we’re delegating control of the main build orchestration to a language better able to reason about and define program flow: –

:: Restore NuGet packages
.paket\paket.bootstrapper.exe
.paket\paket.exe restore

:: Start main build script
packages\FAKE\tools\FAKE.exe build.fsx

Conclusion

Kudu and Azure App Service are great tools. By plugging FAKE into the mix, we get both a succinct and easy to use scripting experience with the power of the .NET framework and a fantastic language like F# as well.

Visual Studio Team Services and FAKE


What is VSTS?

Visual Studio Team Services (VSTS) is Microsoft’s cloud-based source control / CI build / work item tracking system (with a nice visual task board). It’s a platform that is evolving relatively quickly, with lots of new features being added all the time. It also comes with a number of plans including a free plan which entitles you to an unlimited number of private repositories with up to 5 users (and MSDN users do not count), plus a fixed number of hours for centralized builds.

The catch is that there is (at least, I couldn’t find!) any way to host a completely public repository with unlimited users – obviously a big problem if you want to have an open source project with lots of contributors. But for a private team, it might be a good model for you. You can also use the CI build facilities of VSTS with GitHub repositories – so in this sense you can treat it as a competitor to something like AppVeyor perhaps.

Contrary to common opinion, VSTS is completely compatible with Git as a source control repository. Yes, you can opt to use TFS as a source control model, but (in my opinion) you’d have to be crazy to do this or have a team that are really used to the TFS way of working – I find Git to be a much, much more effective source control system.

Why FAKE?

I wanted to try and see whether it was possible to get VSTS working with FAKE.

One of the best things about FAKE is, in addition to the ease of use, flexibility and power that you get by creating build tasks directly within F# (and therefore with the full .NET behind it), is that because you are not dependent on hosting a bespoke build server with custom tasks etc., such as Team City, it’s extremely rare (hopefully never) that you have a situation where a build runs locally but fails to run on the build server.

Unlike relying on e.g. Team City to orchestrate your build, you delegate the entire CI build steps to FAKE – MSBuild, Unit Tests, configuration file rewriting etc. etc.. So if the build fails, you don’t have to e.g. log into the Team City box, check some log files etc. – your FAKE script does all the heavy lifting so you can run the exact same steps locally.

Putting it all together

So my goal was to write a simple FAKE build script which pulled down any dependencies, performed a build and ran unit tests – all integrated within VSTS. As it turns out, it wasn’t very difficult at all.

Firstly, we hook up the build to source control. In our case, it’s the Git repository of the Team Project, so works straight out of the box, but you can point to another Git repository e.g. GitHub as well. You can also select multiple branches. We then set a trigger to occur on each commit.

1.pngSecondly, we have to set up the actual build steps. As we’re delegating to FAKE to perform the whole build + tests, we want to use as few “custom” VSTS tasks as possible. In fact, we actually only need two steps.

  1. Some way to download Paket or Nuget, and then initiate the FAKE build.
  2. Some way of tieing in the results of XUnit that we’re going to run in FAKE into the VSTS test reports.

Unlike old-school TFS etc., VSTS now has an extensible and rich set of build tasks that you can chain together – no need for Workflow Foundation etc. at all here: –

3.png

Notice the “Batch Script” task above – perfect for our needs, as we can use it to perform our first build task to download Paket and then start FAKE.

We can now see what the FAKE script does – this is probably nothing more than what you would do normally with FAKE anyway to clean the file system, perform a build and then run unit tests: –

Notice that when we run unit tests, we also emit the results as an XML file. This is where the second build task comes into VSTS (Publish Test Results), which is used to parse the XML results and tie into VSTS’ build report.

2.png

So when we next perform a commit, we’ll see a build report that looks something like this: –

4.png

Notice that the chart on the right shows that I’ve run 2 unit tests that were successful – this is the second build task parsing the XUnit output. Of course we can also drill into the different stages if needed to see the output: –

5.png

Conclusion

This post isn’t as much about either VSTS or FAKE features per se, as it is about illustrating how both VSTS and FAKE are flexible enough that we can plug the two together. What’s great about this approach is that we’re not locked in to VSTS as a build system – we’re just using FAKE and running it centrally – but if we’re using VSTS we also can benefit from the integration that VSTS offers with e.g. Visual Studio and the build system e.g. creating work items, associating commits to work items and viewing from VS etc. etc. – whilst still using FAKE for our build.