Friday, March 14, 2014

Concurrent ML and HOPAC

One of these days I got a hold of a copy of Concurrent Programming in ML by John Reppy, an excellent book. One of the many virtues: it is the missing documentation piece for Concurrent ML (CML) system, explaining design motivations and giving helpful examples.

I wish it was in open domain. Why? Simply to push CML design further, bring more attention to it. It is so much better than what is being commonly used. This is not to say that there is no space for other concurrency abstractions. But the CML has the advantage of offering a small set of features from which all other useful abstractions are easily built. I cannot do a better job than the book for advocating CML, but I particularly like:

  • Simple to understand semantics - programs in PI or other process calculi map very closely to CML code
  • Makes it easy to build other abstractions - asynchronous communications, locks, buffers, reactive systems, you name it
  • Selective communication seems VERY important
  • Plays well with ML-style languages (my favorite)
  • Admits moderately efficient implementation - should frequently be good-enough for production, and definitely excellent for prototyping

In a tweet I mentioned that having the book in open domain would "help fight actor/reactive nonesense" and was asked to elaborate. So here are some statements I came to disagree with:

  • Concurrent software should be written exclusively with Erlang-style asynchronous message-passing
  • It should be written with reactive systems of RX/IObservable flavor
  • It should be written in the FRP paradigm
  • F# has excellent support for concurrency, it provides `async` and "actors"..

Obviously, Erlang-style message passing, reactivity, and carefully designed FRP systems (such as Elm language, AFRP) are all good for certain problems. RX/IObservable systems, IMHO, are a net liability. In general, take any of these paradigms as the default, and you will quickly find programs that you want to write fit the paradigm no better than a saddle would fit a cow. I think CML stands out as a much better foundational choice, subsuming the others.

Note to F# users - thanks to valiant effort by Vesa Karvonen, you can now use CML-style primitives in F#. See the Hopac project - it needs a bit more love!

Note to lovers of F# async. Async is a hack, but a good one. It is what you do when you want cheap by-the-million threads but do not have the time to rewrite the runtime (CLR). However, with a proper runtime, there is no need. See Racket or CML, where blocking syntax is used to orchestrate millions of lightweight threads.

Note on deadlock: CML-style channels use sync write, but async write is easy to write too. The book argues why sync writes are a better default. Yes you can program deadlock, but you can do the same within any sufficiently expressive concurrency paradigm. You definitely can in Erlang, try it as homework.

Tuesday, October 15, 2013

Last word in .NET build systems

Has not been said yet, I do not think.

In the F# world you may be looking at:

I am not going to do a detailed pro/con analysis of these just yet, but note that every one of them is currently missing abstractions relevant for this problem domain - building. A build system should allow you to do at least what veritable Makefile does - optimal rebuilds, but at an abstract level, as a library.

The best system I have seen so far that gives you these abstractions is Shake by Neil Mitchell (coded in Haskell). It goes beyond Makefiles by allowing dynamic dependencies. I did not study the specifics very closely, but the overall design is vastly useful, brilliant.

Do not have time at this moment to get it into the shape it deserves, but here is some work I have been drafting now and then to build a similar library in F#: fshake. If this scratches an itch, let me know, I would be interested in contributors - I have not put the license in yet, but this project will be under Apache license.

Friday, October 4, 2013

WebSharper vs FunScript

Got asked to compare WebSharper to TypeScript on StackOverflow, doing it here instead. Disclaimer: I work for IntelliFactory and in fact develop WebSharper, so I am obviously biased.

Good things to say about FunScript:

  • seems to be entirely unrestricted (I could not even locate a license file)
  • has some very talented people hacking on it (but not as the day job)
  • has this wonderful idea of using TypeScript definition files to provide typed API for JavaScript libraries (however, at this moment it does not work with latest TypeScript version so no luck there)

Why would you be interested in WebSharper instead? The quick answer is that it actually gets used a lot more, and is known to work quite well on large projects (I will cite FPish and CloudSharper as the ones we have been using it on at IntelliFactory), there is a team working on it day-to-day, and you can get actual support. It has been in use longer and it might have more issues ironed out, and it definitely supports a larger portion of F# standard library out-of-the-box.

NOTE: Tomas pointed me to a FunScript way of doing the following in the comments. Jon Harrop sites problems with using sqrt in FunScript. In WebSharper, it works. Suppose it did not, and you know how the function looks in JavaScript, you do:

[<Inline "Math.sqrt($x)">]
let sqrt (x: double) = sqrt x

And there you have it.. Also, rest assured that there is no string splicing going on - the inline is parsed as JavaScript, then a large subset of JS like the above is lifted to our Core form, which assures things like evaluation order are preserved, and lets optimizations work smoothly, including ones we have not written yet :)

That being said, I believe we have made quite a few mistakes in the past, including:

  • hiding sources (it is open-source now)
  • mismanaging the community - actually I am reading books now on how to facilitate an open-source project community better
  • making it too hard to get started and not giving enough documentation
    (presently we are writing manual chapters at our github repo) and preparing a website update
  • trying to provide all library bindings ourselves - TypeScript is definitely a great idea and we will add support for it (have a working prototype for 0.8 but need to upgrade to latest TS - same as FunScript
  • attempting to solve too many problems at once, and not focusing on important problems first. This leaving us with a large and somewhat difficult to change codebase
  • some engineering mistakes in organizing code or designing APIs

If you feel like making more suggestions, please do..

People also typically bring up licensing and how the output code looks like as problems. I do not think those are show-stopper issues:

  • License (AGPL) - it is actually free for open-source use. If you need to close your app, but cannot afford the listed license fees, just talk to us and we can work out a deal. Note that having this license might be annoying but it is essential for securing funding and commercial support for the project - for it to have a future.
  • Code output - we are working out a better optimizer with Andras Janko that will help a lot, in both shrinking output and improving performance, but really, how the code looks - this is not an issue. In months of programming with WebSharper I never remember looking at the JavaScript output. Look at Emscripten, can you read its output? Yet Emscripten is extremely useful. Once you get past a certain stability level, this does not matter anymore.

I also privately think both WebSharper and FunScript make this fundamental mistake - using F# quotations. I think the future is simply not there. There is exciting stuff going on with projects such as ASM.js and Emscripten, and I strongly suspect the future successful project in this area will provide a much more compatible CLR implementation on top of JS, plus perhaps some specialized optimizations for functional code as F# produces. I have seen a few CLR-to-JS compilers, but am not sure which is the best today. I tried writing one myself and hope to come back to it, it certainly was challenging and interesting. The key is to stop thinking of such projects as source-to-source compilers and start treating JS for what it is, portable assembly.

Thursday, June 6, 2013

Generic programming in F# - another take

Playing a bit more with generics, I stumbled upon some fairly compact combinators that can derive n-ary conjunctions from a binary conjunction. Here is a self-explanatory F# example (disregard the ugliness involved in F#'s lack of higher kinds, Haskell or OCaml code would not need this):

In case you are wondering about the Coq file in the gist: a good objection to this approach is efficiency. Instances computed in this way are not efficient at all. Just think of all the allocated tuples! It turns out, we can remedy the situation with automated program transformation. In F# it would involve computing over quotations or similar quoted forms. It is hard to get started since little support for optimizing those is available out of the box. In Coq there is more batteries. As you can see in the gist, Coq seems to tackle the above example trivially with the "compute" tactic, and derive an instance equivalent to a hand-written one.

Wednesday, April 3, 2013

Introducing FAKE boot workflow

FAKE is a tool to automate your builds with F# scripts. Use the newly available FAKE boot workflow to get started quickly, automatically reference packages, and plug in your own build logic from NuGet.

Friday, March 29, 2013

FAKE with NuGet support

UPDATE: the proposal made it into pre-release FAKE, see the more recent article:

Intended F# build workflow: start from an empty folder, write build.fsx, and run fake - and get anything building with software from NuGet.

Draft Implementation  Discussion

Wednesday, March 27, 2013

Generalizing records combinators a bit

Towards generic programming in F#: thoughts on generalizing the earlier combinators over records and unions...