Last word in .NET build systems

Has not been said yet, I do not think.

In the F# world you may be looking at:

• MSBuild
• FAKE
• IntelliFactory.Build

I am not going to do a detailed pro/con analysis of these just yet, but note that every one of them is currently missing abstractions relevant for this problem domain - building. A build system should allow you to do at least what veritable Makefile does - optimal rebuilds, but at an abstract level, as a library.

The best system I have seen so far that gives you these abstractions is Shake by Neil Mitchell (coded in Haskell). It goes beyond Makefiles by allowing dynamic dependencies. I did not study the specifics very closely, but the overall design is vastly useful, brilliant.

Do not have time at this moment to get it into the shape it deserves, but here is some work I have been drafting now and then to build a similar library in F#: fshake. If this scratches an itch, let me know, I would be interested in contributors - I have not put the license in yet, but this project will be under Apache license.

Introducing FAKE boot workflow

FAKE is a tool to automate your builds with F# scripts. Use the newly available FAKE boot workflow to get started quickly, automatically reference packages, and plug in your own build logic from NuGet.

FAKE with NuGet support

UPDATE: the proposal made it into pre-release FAKE, see the more recent article: http://t0yv0.blogspot.com/2013/04/introducing-fake-boot-workflow.html

Intended F# build workflow: start from an empty folder, write build.fsx, and run fake - and get anything building with software from NuGet.

Draft Implementation  Discussion

Automate, automate, automate..

Latest snapshot of the WebSharper repositories (Bitbucket and at GitHub) showcases some build automation I finally managed to get working. It starts from MSBuild that uses NuGet to pull dependencies, including build dependencies, and then jumps to FAKE - using the alpha pre-release FAKE version has solved some F# version problems I faced earlier. It builds a lot of code, creates an output NuGet package, and also creates several Visual Studio templates that it packages up into a VSIX extensibility package - using the free to use API we are releasing under IntelliFactory.Build.

Things are looking a bit better since the build succeeds from scratch inside the AppHarbor environment that is currently hooked up - as a simple build server.

Despite the modest progress, build automation is still a nightmare. I think we need a much simpler story, one that would definitely involve the NuGet repository as a global binaries library and F# as the language for all build logic, with MSBuild in support role (generating MSBuild scripts for Visual Studio users).

See also the discussion on ScriptCS - this project attempts to do something very similar based on C#. I like their authors'  emphasis on Node Package Manager as the model.

A thought that occurred to me frequently when debugging the builds was the simple question - why do all the parts have to work with process isolation barriers? It seems like we are trying to play the UNIX game on Windows, where the filesystem simply does not keep up, and processes have an unbearably slow cold start. For example, in principle, why invoke NuGet or MSBuild for that matter as a process through the command line, when there is .NET API for both? It seems that PowerShell at least allows Cmdlet's to keep some state in-memory, so reusing the same Cmdlet is vastly faster than invoking the same functionality in a separate process. I might be working on the tool along these lines as time permits.

A big win would be to have sane F# compiler (without the famous toxic nuclear waste that Haskell avoids by controlling side effects in the type system).. Then we would not have to invoke it in a separate process for each project in the solution. This could drastically reduce build times.

Multi-targeting .NET projects with F# and FAKE

I am currently working on simplifying build configurations for a bunch of projects including WebSharper, IntelliFactory.Build, IntelliFactory.FastInvoke and eventually multiple WebSharper extensions. I am now using F# and FAKE when possible instead of MSBuild, and relying more heavily on the public NuGet repository.

The good news is that abstracting things in F# and sharing common build logic in a library via NuGet really works well, and feels a lot more natural than MSBuild. Consider this Build.fsx file from FastInvoke:



The FAKE-based build (well, together with some MSBuild boilerplate that I generate) accomplishes quite a few chores:

• Bootstraps NuGet.exe without any binaries in the source repo
• Resolves packages specified in solution packages.config, including pulling in FAKE and build logic such as IntelliFactory.Build
• Determines current Mercurial hash or tag
• Constructs AutoAssemblyInfo.fs with company metadata and mercurial tag
• Constructs MSBuild boilerplate to help projects find NuGet-installed dependencies without specifying the version, for easy dependency version updates
• Builds specified projects in multiple framework configurations

And you can, of course, do more inside the FAKE file.

The bad news is that I expected quite a bit more from FAKE, and I end up fighting it more than using it - note that these can be either legit problems with FAKE or else my limited understanding of it.

• Running FAKE.exe drops your code into the 2.0 runtime, even if the host process was in 4.0 - not acceptable for me, had to replace FAKE.exe invocation with FSI.exe invocation - UPDATE: when using pre-release alpha version of FAKE, the scripts default to the 4.0 runtime and use FSharp.Core 4.3.0.0 - problem solved
• FAKE had no easy support for dependency tracking, such as not overwriting a file unless necessary (useful to prevent say the MSBuild it invokes from doing work twice) - had to roll a few or my own helpers
• Surprisingly FAKE MSBuild helpers are calling MSBuild process instead of using the in-process MSBuild API. By using MSBuild API myself, I am able to speed things up a bit.
• On a similar note, I toyed with invoking either FAKE or Fsi.exe in a slave AppDomain (should be faster than a separate process, right?) from the host MSBuild process that my build starts with. The approach failed miserably. Fsi.exe is reading System.Environment.GetCommandLineArgs() instead of reading the EntryPoint args, so that it does not see the args I pass to the slave AppDomain, but instead sees the args that MSBuild receives. And FAKE, again, drops me into the 2.0 runtime, probably starting another system process too.

In the end my impression is that there is tremendous value in automating build logic in F# instead of MSBuild. As to FAKE, it seems most of its value comes from the helper library - some shared direct-style imperative recipes. I would like to see that released separately, say as FAKE.Lib NuGet package, to make it easier to use standalone. Also, it seems that FAKE could really benefit from some extra standard features for dependency management such as comparing target input and output files by checksum or date, to be on par with rake.