People 101

After a long stretch of working from a cave, I am discovering the joys and horrors of working with a team of multiple people and one robot. How does one function? Where is the manual to people?

I am still looking for a good one. In the meanwhile, some field notes:

1. People reach different conclusions from the same data due to differences in judgement - disagreement does not always result from lack of understanding or poor communication.
2. People argue, and mostly about unimportant things, because due to (1) what they judge important you consider trivial.
3. Being an expert in a field does not entitle you to expert opinion on everything, something I have to remind myself often.
4. People make mistakes, both of professional and interpersonal nature.
5. People do a lot more of all the above when lacking proper sleep and rest.
6. I am no exception.

I am making some ghastly mistakes, requiring a sincere apology. I wish there was an easy way to cover these with regression tests.

Tachyus

I am joining Tachyus F# team starting next week, and moving to San Mateo, CA. I am both excited and a little sad. During my time at IntelliFactory I learned pretty much all the skills I now have as a programmer. I met, worked with and learned from amazing hackers. It is time for me to move on, but IntelliFactory will always be a very special memory for me, and Budapest a special place. Thanks Adam Granicz, Diego Echeverri, Joel Bjornson, Loic Denuziere, Andras Janko, and many others for an amazing time.

UI.Next available

UI.Next is available for experimentation as a public WebSharper.UI.Next NuGet package. You can build examples from source - just get WebSharper first.

UI.Next addresses most shortcomings we felt WebSharper had for single-page JavaScript applications. The most interesting part is a dataflow model integrated with DOM for defining reactive UI, but we also provide support for client-side routing and animation.

If you play with it, Simon and I will be very interested in your feedback.  Next week we plan to do a few more samples, to cover more animation, interaction with mouse and keyboard.  We find doing these examples is a very helpful way of coming up with better API.

The fatal attraction of FRP

For about a year or so, I have made Functional Reactive Programming (FRP) taboo. I cringed at every mention of it. I think this was a mental self-defense reaction, my extended immune system was sending me a signal that I spent too much time thinking about this subject without any tangible results.

The fatal attraction of FRP is its simplicity. Semantics are beautiful and clear. There are Behaviors, functions of time, Events, timed occurrences, and they all dance together. You write causal transformations (future values depend on past values, not vise versa), and it works. You use equational reasoning to transform your program. You get inspired by Conal Elliott's papers.

However, how do we implement it? Many a ship has foundered in these waters.

Perhaps you do not even have to. People do different things to really "get," obtain an operational understanding of something. Mathematicians like denotational semantics. They understand object A by explaining it in terms of object B we already know. Only programmers like to implement things, but then they can use an existing implementation to play with. Once I play with Impl(A), I "understand" A.

At the Budapest functional programming beer night I used to go to, we had the fortune of having Gergely Patai and Csaba Hurska. These guys have built simply amazing Haskell projects, the Elerea FRP library, and the LambdaCube purely functional 3d engine. Gergely is a mathematician. He explained FRP to me by reducing things to Cartesian closed categories.  As you guess, it did not help me - to this day my understanding of those things is shaky. Csaba is a little more of a programmer, he told be Haskell is a hacker's language. Both, now that I think of it, made fun of me for talking about dependent types being the future of programming, with good reason, as I now know. But I digress.

At a very high-level, problems with implementing FRP in a language like Haskell or F# are these:

1. It is quite hard to enforce causality of transformations with a vanilla type system. AFAIK most practical systems either design a new language and type system, or just leave this invariant unchecked. 
2. If event streams are first class values, this can easily create memory leaks in a higher-order program, if the entire unfolding history is retained. Solutions here are: do not give the user first-class event streams (but only their transformers, or something like that), create a new language and type system that rules out those nasty higher-order programs (Elm.js), or use some kind of combination of convention, types, and clever implementation (weak pointers etc), to make it work.

Much as I despair getting FRP right, I am coming back to the topic as we are working on WebSharper UI.Next with Simon Fowler.  While trying to implement a sub-FRP system, something that is not quite as general but hopefully much simpler to implement, I also realize I start slowly thinking about general FRP again.

Briefly, the compromises we are making in UI.Next are:

1. No first-class event streams. You want to transform event occurrences? Do it with imperative state and callbacks. That stops working? Do it with agents and communicating micro-processes. That stops working too? Use Concurrent ML (or Hopac). We are not quite there yet with providing CML primitives in WebSharper/JS but this is planned, probably as a proxy for Hopac.
2. We do, however, provide a dataflow abstraction that is almost, but not quite, approaching FRP behaviors. A View<'T> is something that varies over time, in discrete steps. It is now computed from zero or more Var<'T>, which are a reactive form of ref cells.
3. Using the mental model of communicating processes, we designed a protocol for View<'T> processes to synchronize in a way that is friendly to GC for most programs, even without relying on weak pointers (not at this point, at least). The protocol also does not preserve occurrences, it synchronizes to the "latest" value, which is nice.
4. Views can be observed imperatively or by constructing reactive DOM documents.

It is tempting to add Behaviors too, perhaps as View'T>.

As I am playing with this, I have one rebellious thought.  Are FRP event streams or their transformers even worth the trouble? Behaviors are clearly wonderful. But do you really want first-class "mouseClicks"? Or a fold on those? In the context of an ML type system such as F#. 

It remains to be seen. So far I like that our system is working well with embedding typical higher-order, stateful widget code, as found left and right in the JavaScript ecosystem. It is also simple enough so we could implement it.. Reasoning laws are NOT the ones for pure FP, but things are mostly tractable if you stick to the paradigm of communicating processes.  CML might just fill the spot, but some cross-pollination might also happen. That is, we might end up adding process combinators that look a lot like FRP event and event/behavior combinators.

Dangers of react and VirtualDom

This is a brain dump from a Twitter/IRC discussion with @panesofglass and @bryanedds.

So there is this brand-new thing called Facebook React and hype wave about Virtual DOM. I only skimmed this library, but here are some reasons why I remain sceptical:

• Identity! Typical DOM trees are a lot more than the markup. They are full of pieces of state that are *opaque* - abstract types, and *mutable*. This is especially true if you use 3rd-party libraries, but even with vanilla DOM, input tag, once rendered, interacts with the user and updates its focus state, the position of the cursor, and so on. Can all of it be captured in a snapshot as a pure value? Does React do this? I will be interested to find out.
• If the answer is yes, I would be interested in playing more with VDom. Taking this as a foundation for design, means you throw out all other UI libraries (Ext JS, Kendo, etc). It is like a campaign to rewrite the Internet, without mutable state. But who cares - still might be interesting.
• WebSharper Formlets library we designed with Joel Bjornson a while back assumes the answer is NO, so it does work with third-party libraries. The cost is some care is taken to manage identity of things. As far as change propagation through a tree on *pure* values, hey, we have done that in Formlets too, there is nothing too magical there. There are some nasty things when you do higher-order combinators, such as "Formlet.Many", and we try to do a sensible thing but I am not entirely confident in the theory. Loic Denuziere can comment on Piglets better than I can, but I think is quite similar as far as VDom aspect is concerned.
• Formlets do not have or need a Diff function. You only need a Diff function if you describe transformations from a Model type to your UI type (VDom or what have you) as functions. If you have combinators with more structure, this can be done implicitly. Check the source.
• Before, managing identity made me feel really bad. Am I a functional programming or what? These days, I am slowly accepting the concurrent paradigm. This is a different worldview, where process calculi (Pi, CSP) play a foundation similar to lambda calculus in FP. The paradigm makes working with identity essential, natural, and effective. Excellent fit for the world of UI!
• There is also the Dataflow/FRP paradigm. I tried a few times, but my brain is still to small to digest it. Things get quite crazy when you allow dynamic switching combinators Signal (Signal t) -> Signal t. So, in this space I most sympathize with the Elm language which, from my limited understanding, seems to take a stand and prohibit such combinators, or in fact even possibility of constructing a type Signal (Signal t) in its type system. Looks like a great design, cleaner implementation and many useful things you can build. Some Haskellers though claim that the more general design is a "solved problem". Perhaps.
• Self-Adjusting Computation was recently pointed out to me. I am still working through the paper. Looks highly relevant.
• What would I like for Formlets/Piglets vNext, given time? Mostly, I would like a solid *model*, one that even I can understand and explain, that gives a meaning to what is going on, and is reasonably cleanly mapped to the implementation. For that, looks like I still need to catch up on some studying.

Chat logs:

#websharper on Freenode

Does syntax matter? Or how I started progrmamming.

How I started programming: well there was some Basic, Pascal and PHP, but I started getting serious only after encountering and trying to understand Haskell, while doing grad school for an unrelated subject. Haskell challenged me, somehow. It looked like nothing I have seen before. And playing with it had that feeling of figuring out a puzzle.

How I found Haskell: going through the list of languages that Kate or some such editor supported highlighting for. Yeah, I know.. In alphabetic order.

F# was not on the radar then, but OCaml was. I looked at it (among with Python, Ruby, you name it). I remember thinking a couple of things:

1. Syntax looks so painful
2. How do I use these command-line tools?? (it is better these days with ocalmbuild)
3. It has objects - must be another PHP-like disaster of a language
4. Given 1-3, no point in learning it

Huge mistake. OCaml is a gem. But you can see how a beginner might miss the point.

So for all good languages out there - I guess syntax does not matter that much, but great documentation does. If the syntax is strange, but the documentation explains why the language is worth learning nonetheless, a beginner might stick with it.

Concurrent ML and HOPAC

One of these days I got a hold of a copy of Concurrent Programming in ML by John Reppy, an excellent book. One of the many virtues: it is the missing documentation piece for Concurrent ML (CML) system, explaining design motivations and giving helpful examples.

I wish it was in open domain. Why? Simply to push CML design further, bring more attention to it. It is so much better than what is being commonly used. This is not to say that there is no space for other concurrency abstractions. But the CML has the advantage of offering a small set of features from which all other useful abstractions are easily built. I cannot do a better job than the book for advocating CML, but I particularly like:

• Simple to understand semantics - programs in PI or other process calculi map very closely to CML code
• Makes it easy to build other abstractions - asynchronous communications, locks, buffers, reactive systems, you name it
• Selective communication seems VERY important
• Plays well with ML-style languages (my favorite)
• Admits moderately efficient implementation - should frequently be good-enough for production, and definitely excellent for prototyping

In a tweet I mentioned that having the book in open domain would "help fight actor/reactive nonesense" and was asked to elaborate. So here are some statements I came to disagree with:

• Concurrent software should be written exclusively with Erlang-style asynchronous message-passing
• It should be written with reactive systems of RX/IObservable flavor
• It should be written in the FRP paradigm
• F# has excellent support for concurrency, it provides `async` and "actors"..

Obviously, Erlang-style message passing, reactivity, and carefully designed FRP systems (such as Elm language, AFRP) are all good for certain problems. RX/IObservable systems, IMHO, are a net liability. In general, take any of these paradigms as the default, and you will quickly find programs that you want to write fit the paradigm no better than a saddle would fit a cow. I think CML stands out as a much better foundational choice, subsuming the others.

Note to F# users - thanks to valiant effort by Vesa Karvonen, you can now use CML-style primitives in F#. See the Hopac project - it needs a bit more love!

Note to lovers of F# async. Async is a hack, but a good one. It is what you do when you want cheap by-the-million threads but do not have the time to rewrite the runtime (CLR). However, with a proper runtime, there is no need. See Racket or CML, where blocking syntax is used to orchestrate millions of lightweight threads.

Note on deadlock: CML-style channels use sync write, but async write is easy to write too. The book argues why sync writes are a better default. Yes you can program deadlock, but you can do the same within any sufficiently expressive concurrency paradigm. You definitely can in Erlang, try it as homework.