Dive into Erlang

Reia: Now creating modules on-the-fly

Wed, 06 Aug 2008 14:47:00 -0500

Note: This is cross-posted from my new blog. The original article is available at tonyarcieri.org.

One of the complaints I’ve heard about so many languages is an inability to do certain things from the interactive interpreter which you can do otherwise in content loaded from files. Erlang does not allow module declarations from the interactive interpreter. And while it’s not entirely impossible, it’s indicative of a wide gulf between the language as evaluated on the fly and a language which is fundamentally designed to be compiled.

I greatly enjoyed how Ruby allows you do to virtually anything from the interactive interpreter that you otherwise could in normal program code, including defining functions as well as declaring modules and classes. I sought to do the same in Reia, my dynamic scripting language for Erlang’s VM. Reia now allows on-the-fly module declarations from anywhere, including both scripts and the interactive interpreter. For example, the latest version on github allows you to do declare a module from the interactive interperer like this:

  >> module Foo
  ..   def bar
  ..     42
  .. 
  => ~ok
  >> Foo.bar()
  => 42

This declares a Reia module named ‘Foo’ and defines one function under it, which returns 42. After being declared, the ‘Foo’ module is compiled to BEAM bytecode and loaded directly into Erlang’s code server. The "~ok" you see returned (which is Reia syntax for an atom called ‘ok’) is actually coming from the Smerl library by Yariv Sadan, and indicates that the module was successfully compiled and loaded into the code server. Smerl allows for simple metaprogramming in Erlang by allowing you to build "meta modules" on-the-fly which then outputs all the Erlang forms needed to pass to compile:forms in order to buld a BEAM (or HiPE) module.

When Reia evaluates module declarations, it first compiles the Reia code to Erlang forms then passes them to Smerl to build modules, which invokes the Erlang compiler and loads the resulting bytecode into the Erlang code server. The end result: on-the-fly module declaration. You can call methods from compiled Reia modules just as you would any Erlang function, just keeping in mind that the module name is capitalized. From eshell, you could call ‘Foo’:bar() to invoke the same function.

This allows a number of things which Erlang presently does not, and that doesn’t just mean declaring modules from the interactive interpreter. Reia allows multiple module declarations from the same file, and furthermore provides a toplevel scope which exists outside of modules and is evaluated as the file is being loaded. This means you can declare modules in a file then invoke functions in them immediately afterward, just like the above example from the interactive interpreter.

There’s a number of limits at present. All module declarations need to be done from toplevel scope, i.e. you can’t conditionally declare a module. You also can’t place statements besides defs inside a module declaration, so you can’t conditionally declare functions/methods either. However, that said, module declaration is no longer limited to compiled code in Reia. You can now do it anywhere.

There’s several possible optimizations down the road. The compiler could be optimized to:

Automatically produce HiPE versions of the modules if HiPE is available
Keep a cache of generated BEAM/HiPE bytecode and only recompile if the original source file changes
Precompile multiple modules in the same source file, employing cross-module optimizations

Revactor: Erlang-style concurrency for Ruby 1.9

Mon, 11 Feb 2008 22:18:00 -0600

I've released Revactor, a Ruby implementation of some of the features of Erlang-style concurrency. You can read more about it on the web site here:

http://revactor.org/

Those of you who primarily program in Erlang might be curious why I've done this. If so, please read on...

Erlang has been the most amazing language I've ever learned. As I read over Erlang creator Joe Armstrong's reasoning behind every single semantic decision in the language, I couldn't help but be awed at how cohesive it was in regard to building highly concurrent systems which must by their very nature be highly fault tolerant. After all: both the program and the underlying hardware are prone to breakage at any time. Any large system built out of a collection of smaller components run across multiple computer systems has to deal with this reality, and Erlang, more than any other language, has provided a clean solution.

Unfortunately, while the semantic decisions were spot on for the task at hand, they introduced vast differences which set Erlang far apart from most other languages. This can pose problems when trying to introduce Erlang into a setting where no one has a background in functional programming or the Actor model. For the majority of programmers, who work in imperative languages and think about concurrency in von Neumann terms (i.e. threads), Erlang is just "too weird."

The best way to ween imperative, thread-thinking programmers off the von Neumann style and onto pervasive, asynchronous, shared-nothing parallelism is to present the concept gradually. Many simply can't make all the conceptual leaps Erlang requires at once (an ironic statement for a blog called "Dive into Erlang"). I've tried to resolve this by implementing "Erlang-style concurrency" in a language with the imperative, shared-state model that the majority of programmers are familiar with.

Conceptually, Erlang provides a sledgehammer for the problems of concurrent programming. It's wrapped up the problem of concurrent programming in such a way that you can build massive, multi-million line codebases with an incomprehensible amount of functionality in a highly fault-tolerant manner that would be impossible in other languages. But, sometimes you don't need a sledgehammer... just a flyswatter will do. And while a sledgehammer may be a damn good way of killing a fly, it's an awful lot easier for your average person to wield a flyswatter.

Revactor is a flyswatter to Erlang's sledgehammer.

What does Erlang-style concurrency mean? Well, here it is, straight from the horse's mouth:

Creating and destroying processes is very fast.
Sending messages between processes is very fast.
Processess behave the same way on all operating systems.
We can have very large numbers of processes.
Processes share no memory and are completely independent.
The only way for processes to interact is through message passing.

Clearly one of the key ingredients to Erlang's fault tolerance is the idea that: "Processes share no memory and are completely independent." This is not possible in languages with a shared heap without resorting to heavyweight OS processes, which would require foregoing the "Creating and destroying processes is very fast" requirement. A shared heap dramatically impacts fault tolerance, especially in languages with mutable state. But for small programs, with a certain degree of hair pulling bugs relating to shared state can be resolved. Problems surrounding scaling across multiple CPU cores can be partially resolved too: shared-state languages can run a shared-nothing virtual machine per CPU, and the VMs can communicate with asynchronous message passing. Think of this as sort of like an Erlang node per CPU. This isn't nearly as elegant a solution as the Erlang SMP scheduler, but it does eliminate any locks or contentions which would prevent scaling across N CPUs in a linear fashion. It affords a bit of shared-nothing parallelism in languages where it would otherwise be infeasible.

With that giant caveat, can the rest of the problems be addressed? The answer is clearly: not nearly as well as Erlang, but you don't always need a sledgehammer...

Revactor implements the Actor model on top of the Ruby 1.9 VM. This VM brought with it much needed speed improvements, native threading which allows for high-performance I/O bindings, and a new "semi-coroutine" concurrency primitive known as a Fiber. Fibers are not nearly as lightweight as Erlang processes: the baseline size of a Fiber is two orders of magnitude larger than an Erlang process, and the time to spawn one is two orders of magnitude slower than Erlang. But, like Erlang processes Fibers run in userspace so context switching is very fast. Furthermore, since they're coroutines, they eliminate many of the concurrency headaches of threads. Batch state mutations can be performed in an atomic manner, because only one coroutine is ever running at the same time, and it's easy to reason about when context switches occur, as opposed to threads where it's impossible outside of mutexes and conditions.

The "Sending messages between processes is very fast" requirement is probably where Revactor shines the best. The raw interprocess messaging speed between two Actors is a bit less than half that of Erlang. That may not seem like much, but Erlang is one of the most optimized languages in that regard, and furthermore messaging throughput is arguably the most important attribute of systems with Erlang-like concurrency. However, messaging throughput benchmarks are meaningless unless we take into account another important attribute of Erlang messaging: "It is possible to selectively receive messages." Revactor provides selective message reception complete with patterns, guards, timeouts, and an associated code block for each message-matching expression, just like Erlang. Even with these features, messaging is fast.

"We can have very large numbers of processes" is another area of weakness. For practical purposes Revactor processes will range in the tens of thousands, whereas Erlang ranges in the tens of millions. This is another case where you'll have to decide if a sledgehammer is needed.

Revactor models many of the design attributes of Erlang. Message passing is asynchronous, and "processes" can monitor each other. The linking model of Erlang is incorporated: Actors can link themselves in graphs. Exceptions are propagated throughout all Actors in the graph, and Actors are capable of trapping exits.

Revactor's only strength over Erlang is that it allows for Erlang's ideas to be used in a language which is more familiar to the large imperative programming community and also one with a rich assortment of software packages aimed at a variety of tasks, which, at present, puts CEAN to shame.

Also, an up and coming new Ruby VM, Rubinius, features Actors as one of its fundamental concurrency primitives, and can hopefully address many of the performance problems which make Revactor a flyswatter next to Erlang.

I see Revactor benefitting the Erlang community in two ways: first, it provides an easy introduction to the Actor model for Ruby programmers. Rather than having to shift from a mutable shared-state imperative language to an immutable shared-nothing functional language all at once while at the same time learning the Actor model, programmers can get their feet wet learning the Actor model in a familiar language.

Secondly, I think Revactor is a great prototyping tool for building Erlang applications. While Erlang has generated a great deal of buzz in the Ruby community, one of the persistent problems that seems to have affected everyone (including myself) is: I thought Erlang is great, but I've never built anything big in it. For any Rubyist wanting to build their first large scale Actor-based application, I'd highly suggest playing around with the general structure in Ruby first, then translating it into Erlang.

Third, I have immense hope for applications written using a combination of Ruby and Erlang. There have been many attempts to do this: language bridges like Erlectricity and RBridge, and Fuzed for using Erlang to do heavy lifting for Rails. However, all of these approaches bend Erlang to Ruby's terms. I think the best way to use Ruby and Erlang in combination is to make Ruby more Erlang-like. Scala, an Erlang-like concurrent language for the JVM, can talk the Erlang wire protocol using the JInterface library and I'd very much like for Revactor to as well.

With this approach, it'd be possible to use any combination of Ruby, Erlang, and JVM-based languages like Scala with JInterface support to build distributed, massively-concurrent systems with Actor-based concurrency. This gets rid of the boolean decision of "do I use Erlang for this?" Instead, you can use Erlang where it makes the most sense sense: backend components which require high concurrency and fault tolerance, while still being free to use other languages that might be better suited for frontend components.

The black art of Erlang's parameterized modules

Wed, 31 Oct 2007 19:43:00 -0500

Those coming from an object oriented background may long for something more lightweight than a process to hold state and provide functions which can act on it. Recent releases of Erlang provide an undocumented and unsupported feature which can provide just that: parameterized modules. They've seen use in the MochiWeb toolkit, but should be considered something of a hidden feature which may lose support in new releases. In other words: use at your own risk.

To declare a parameterized module, simply specify some variable names in the module declaration:

-module(foo,[Bar, Baz]).

That's it! You're done. You've created a parameterized module. You can now use Bar and Baz in the scope of any functions defined in your module.

When you compile your parameterized module, the function ModuleName:new will automatically be exported, with the arity of the number of variables specified in your module declaration. This will return a value (which is actually a tuple containing your module name and the parameters you passed to new) which you can use to call functions in your module, e.g.:

Module = foo:new("Bar", "Baz"),
Module:quux().

Parameterized modules give you what are effectively constants which can be declared per "instance" of a module. They seem ideal for storing a large amount of state which can be used by a set of functions which act upon it. For example: MochiWeb uses it to store request data, which gets automatically populated by mochiweb_http:start. You hand MochiWeb a loop fun, and it will call that fun with a new parameterized request module each time an HTTP request has been received.

Some Erlang purists will likely look at this and wonder "Why don't you just use a record?" and consider it a gimpy attempt to hack modules into pseudo-objects. I can't say I disagree, but given how unwieldy the record syntax is, parameterized modules seem like they might be syntactically cleaner for managing complex state that's acted upon solely by a given module.

If you'd like to learn more about parameterized modules, there's a paper from their creator (PDF) which describes them in depth.

One important bit of advice: you can't invoke functions in parameterized modules from Eshell. You must invoke them from another module. If you've hopped into Eshell, compiled your module, and wondered why you end up with badfun when you try to invoke it, you've discovered the pitfalls of dealing with a second class citizen in a language.

Erlang vs. Trolls

Wed, 29 Aug 2007 19:59:00 -0500

Those of you who subscribe to Erlang-related newsfeeds may have noticed a number of disparaging posts regarding Erlang from a feed called "Rebel Science News". I hate to draw any more attention to its author, Louis Savain, than it is already receiving, as it is quite evident that the author is little more than a troll and a crackpot. This is made abundantly clear if you search Usenet for his posts, where he goes by the nickname of Traveler.

Louis Savain has been leveling increasingly harsh language towards Erlang in an apparent attempt to provoke a response. Sadly, I'm biting. I prefer not to feed trolls, but what can I say, I'm doing this against my better judgement.

In his most recent post Louis Savain asserts: "You want to know what I really think of Erlang? Erlang is crap. That’s what I think." He then goes into proactive defense mode, saying "Now don’t feel bad if you are an Erlang fanatic. And please don’t get mad. I always tell it like I see it. That’s my style. Besides, I think all programming languages are crap. Right now, I just have a specific bone to pick with the functional programming model and Erlang in particular. It is not the panacea that its supporters make it out to be."

In lieu of Erlang, Louis Savain proposes a language he calls COSA. However, under any kind of scrutiny, it becomes painfully clear what COSA is: an elaborate troll.

There is no workable implementation. The only implementation in existence, built atop Java, is at version 0.0.1, was written in Java, created by a single programmer (unsurprisingly not Louis Savain himself), and has not been maintained since 2004.

The specifics of the language are not fleshed out whatsoever. The only whitepaper available on the language is little more than a poorly fleshed out conceptual paper which is sorely lacking in detail. It speaks more of a troll who is trying to draw attention to an elaborate farce than actually trying to develop a language, and most of the attention grabbers come in the form of hurling insults at Erlang than they do putting forth novel ideas.

The "Parallel QuickSort in COSA" article is a canonical example. Here we see the troll's typical M.O.: hurl insults at Erlang, suggest an alternative, but don't provide any specifics. A parallel quicksort implementation is certainly an interesting idea which I'd take a look at in any language. However, what Louis Savain puts forth is a simple block diagram lacking any detail. He provides a page which allegedly describes the details, but when I go to look: surprise, surprise, they're not there. He suggests that algorithms are not a workable programming model, the entire time claiming to have implemented the quicksort algorithm, while not providing any workable implementation. Does he have any clue how many times he contradicts himself?

His ultimate argument seems to be that Erlang is not granular enough, when Erlang provides an extremely lightweight concurrency model which works well on modern processors. One would assume that increasingly the granularity of concurrency beyond the object/Actor level would bring diminishing returns on top of traditional multicore processors, an idea Louis Savain does not challenge. Instead, he suggests everyone is wrong (except himself, of course) and that CPU design must radically change to support his model.

In the end, this man is a huckster and a charlatan with nothing to show for his arguments except a web site laced with insults backed by proposed solutions with no practical implementation or any kind of evidence that his solutions are workable. Sadly, he's been polluting Erlang-related RSS feeds with his nonsense, comparing an unworkable concept against a proven language with a decades long track record of successes. Erlang has the technology to leverage the power of existing multicore processors today, not empty promises that it could if only the architects of CPUs would change their entire model.

If you want a great language for concurrent programming on now and future multicore CPUs, or writing fault tolerant programs which work across distributed computing clusters with minimal work, stick to Erlang. If you enjoy being hit in the face by a shit tornado, then by all means, check out COSA.

Programming Erlang

Fri, 20 Jul 2007 01:32:00 -0500

I received my dead tree copy of Programming Erlang today.

Many thanks to Joe Armstrong, both for writing an excellent book and creating an excellent language. This is perhaps the best guide to a language I've ever read.

Erlang and EC2: A match made in heaven

Wed, 18 Jul 2007 12:01:00 -0500

Obie Fernandez, one of the more prominent voices in the Rails community, recently blogged about hosting a web application on the EC2 platform. For those of you who aren't familiar, EC2 is Amazon's grid computing solution and allows clients to commission server instances on the fly through a web service, charging you for the amount of time, bandwidth, and space utilized through its S3 storage service.

Oddly enough, Evan Weaver, another prominent voice in the Rails community, blogged more or less contemporaneously on the "Ultimate Architecture", which he depicts as a reverse proxy to an app server cluster which uses S3 as its backing store. Using EC2 for the app server is the next logical step given S3's pairing with the EC2 platform (Amazon allows free data transfer between EC2 and S3)

Obie's post focuses on the scalability of Facebook applications. This is somewhat coincidental in that I recently consulted with a local Facebook-based startup who created an application called StickyNotes, who after having difficulty scaling their existing Rails application to meet the Facebook load expressed interest in hosting on EC2.

I gave my usual list of caveats regarding hosting a web service on EC2:

EC2 does not provide static IP addresses - you will still need a server to distribute traffic among app servers in the EC2 cluster. This can either be done with a reverse proxy (which means you pay twice for bandwidth) or a server which issues HTTP redirects to idle nodes in the cluster (which means you'll most likely end up with ugly hostname prefixes)
EC2 is stateless - As soon as an instance is gone, it's gone. It may have local storage, but you lose everything on it as soon as the instance shuts down. This makes it less-than-ideal for hosting a database. A Rails-oriented EC2 image tries to solve this with a script that automatically dumps the database to S3 every 10 minutes, which is a less-than-elegant solution.
Individual EC2 instances are limited - EC2 images are actually Xen virtual servers. Amazon's claimed specs are a "1.7GHz Intel CPU" and 1.75GB of RAM. Given Rails hunger for your precious system RAM, this means that you'll likely be able to host a maximum of 8 Rails instances per EC2 image.
EC2 does not provide a database solution - You might consider MySQL's NDB cluster spread across multiple EC2 instances (with periodic S3 backups) or colocating your database server elsewhere, but either way EC2 does not provide a good solution for the database problem.
EC2 is expensive - at least compared to other virtual server providers. The base cost of keeping an EC2 instance up for a single month is approximately $75, excluding bandwidth and S3 storage costs. While these numbers may make more sense when you're dynamically creating instances on-the-fly to respond to load spikes and thus not paying for excess capacity during traffic lulls, the up front cost is somewhat prohibitive if you only need one or two instances, at least if you're a frugal shopper.

I've considered EC2 for several purposes in the past. It seems ideally suited to batch jobs, particularly if you already have servers processing your baseline load of CPU intensive jobs and need to be able to quickly respond to spikes in your load. I do not really consider it useful for the purposes of web services, for the above reasons.

While I can't say Erlang even addresses half of these problems, Erlang does have a number of properties which make it ideal for targeting EC2:

With Mnesia, the app server is the database server - Mnesia provides a database designed from the ground up with fault tolerance and distribution in mind. You have your choice of an in-memory (ram_copies) or disk-based (disc_copies) approach, and both can be periodically logged to S3 (using mnesia:dump_tables). This approach also means you have a single EC2 image to worry about, rather than separate images for app servers and database servers.
Commissioning / decomissioning instances is easy - The same could be argued with Rails when using a reverse proxy like Swiftiply which allows app servers to register themselves, but Erlang and OTP are specifically designed to allow for seamlessly growable, fault tolerant clusters. When you want to take a node down, just fail it over then shut down the instance.
Footprint is lower - This is perhaps the biggest problem with Rails on EC2: the memory footprint of a single Rails instance, combined with single-threaded request processing. You will simply need more EC2 images to process the same amount of traffic simply because of the overhead of a framework where concurrent request processing is not possible. With no plans to make the Rails framework thread safe, and with each version increasing the framework's memory requirements, Rails simply isn't suited for systems without a high RAM : CPU ratio. Erlang frameworks like Erlyweb allow concurrent request processing with a single application instance per node, and distributed Erlang and OTP allow seamless scaling of a distributed app cluster with relatively no work on the part of the programmer.

That's not to say that I think it's foolish to run a Rails application on EC2, I just see a number of difficult challenges which aren't present in an Erlang framework. OTP was designed with the sort of scalability EC2 provides in mind.

That said, I hope to target EC2 with an Erlang application soon.

"Why do you like Erlang?"

Tue, 17 Jul 2007 18:09:00 -0500

After having turned into something of an Erlang zealot about two months ago, I've been asked this question quite frequently. I think the best way to explain this is as a corollary to Greenspun's Tenth Rule:

Any sufficiently advanced concurrent C program contains an ad hoc, informally-specified, bug ridden, slow implementation of half of Erlang

I've always had a slight obsession with developing network servers which I intended to process high numbers of concurrent connections. For years and years I tried to do this in C, exploring many different methods of event monitoring, as well as exploring different ways of implementing network servers as described in such books as Unix Network Programming. Sadly I found many of these models and approaches out-of-date. When glibc came out for Linux, threading became the de-facto standard.

Years later, I would try to implement a massively concurrent network server which used a thread-per-request model. By then I had built up quite a home-brewed server framework, with abstract interfaces to many OS facilities, and had written a network server with over 20,000 lines of code.

This is where I began to hit the wall with threads. I had cross-platform threading abstractions between Win32 and pthreads, and had been testing on a number of platforms, and started noticing that on certain platforms I was running into deadlocks, but not others. As the data structures grew increasingly advanced I started to notice more deadlocks. The locking became increasingly more sophisticated, more difficult to debug, and the performance began to decrease as important shared structures needed to be synchronized across all threads.

I began reading more on the overhead of mutexes, which for a performance-obsessed C programmer was quite maddening. Locking a mutex on Win32, for example, required over 600 CPU instructions! I soon came to the realization that the best approach was really one or two threads per CPU core, and that some lighter weight concurrency mechanism which avoids the synchronization problems of threads was the better approach to addressing high numbers of concurrent connections.

I first discovered Knuth's Coroutines as described in The Art of Computer Programming. These provide for lightweight cooperative scheduling without the need of a system call to switch between them. The GNU Pth library implemented something quite similar. I found several similar approaches to userspace threading, including the idea of "microthreads" as implemented by Win32's Fibers and *IX's ucontexts.

The end goal of all of these approaches was separating behavior from function. Each native thread (provided you have more than one) has its behavior isolated into an inner event loop, and the function can be implemented as a microthread.

Provided you got this far, subsequent problems occur when you discover you need a way to send messages between microthreads in a manner which abstracts processing network I/O events, and if you're using threads also lets you send messages between them. I experimented for years with this problem, coming up with some pretty complex abstractions between a multitude of event sources and threading models.

In the end, after years of struggling to build a framework for concurrent network programming, I never reached a point where I was satisfied. I never felt I had truly abstracted away the problems I was trying to solve.

Eventually I realized shared-nothing was the way to go. I became quite enamored with Dan Bernstein's qmail application, which used a collection of shared-nothing OS processes which communicate over pipes in order to process network traffic. Sick of threads, sick of microthreads, sick of trying to synchronize shared information, this is where I ended up. The wisdom of the Unix philosophy: "Do one thing and do it well," finally manifested itself in my eyes.

Finally, I found Erlang. The more I read about it, the more I realized it was an absolutely brilliant implementation of the very model that I had, for years, tried to implement myself. Furthermore, it combined the models of multiple native threads per CPU core which in turn run multiple "microthreads" with a messaging system far more elegant than I ever mustered myself.

For those of you who have not read it, here is a description of Erlang message processing, excerpted from the recently released Programming Erlang:

When we enter a receive statement, we start a timer (but only if an after section is present in the expression).

Take the first message in the mailbox and try to match it against Pattern1, Pattern2, and so on. If the match succeeds, the message is removed from the mailbox, and the expressions following the pattern are evaluated.

If none of the patterns in the receive statement matches the first message in the mailbox, then the first message is removed from the mailbox and put into a “save queue.” The second message in the mailbox is then tried. This procedure is repeated until a matching message is found or until all the messages in the mailbox have been examined.

If none of the messages in the mailbox matches, then the process is suspended and will be rescheduled for execution the next time a new message is put in the mailbox. Note that when a new message arrives, the messages in the save queue are not rematched; only the new message is matched.

As soon as a message has been matched, then all messages that have been put into the save queue are reentered into the mailbox in the order in which they arrived at the process. If a timer was set, it is cleared.

If the timer elapses when we are waiting for a message, then evaluate the expressions ExpressionsTimeout and put any saved messages back into the mailbox in the order in which they arrived at the process.

After reading this I realized this was what I was trying to invent all along. I showed this approach to Zed Shaw (whom you might recognize as the creator of the Mongrel web server) and he expressed that it was quite similar to the way he implemented message handling in his Myriad framework. I'm not going to speak for Zed in terms of Myriad being an ad hoc reimplementation of Erlang, but just point it out to show that when dealing with these sorts of problems long enough people come to the same sort of conclusions about how to solve them.

All that said, I felt naturally comfortable with the Erlang way of doing things. This is how I've been writing programs all along. It's feels so nice to finally find a language that abstracts away all the complex behaviors of concurrent network processing. Furthermore, distributed Erlang and the OTP framework are beyond my wildest dreams in terms of what you can build once the underlying problems are all abstracted away.

Now I can finally stop trying to build an ad hoc, informally-specified, bug-ridden framework, and start focusing on what I've actually cared about all along: the function.

Unicode and Erlang: Handling UTF-8

Fri, 29 Jun 2007 15:11:00 -0500

Erlang's strings are just lists of integers. You'd think this would make it incredibly easy to support Unicode. And it can!

There's just one problem: UTF-8 has become the definitive Internet standard due to its backwards compatibility with ASCII and Latin 1 character encoding. In order to represent the entire Unicode character plane, it's variable with: certain characters can be up to 32-bits.

A much nicer encoding to work with in Erlang is UTF-32. Unlike UTF-8, UTF-32 is fixed with, guaranteeing a 1:1 mapping between values in a list and represented characters. This is exactly how we work with strings in Erlang.

While I've seen bindings for iconv, this seems like an unnecessary dependency, considering that UTF-8 is quickly becoming the definitive standard and a UTF-8 <-> UTF-32 codec should be relatively trivial.

A friend of mine put together such a codec in pure-Erlang. It takes UTF-8 encoded binaries and outputs UTF-32 lists, and vice versa. Check it out:

http://langweenie.blogspot.com/2007/06/utf8-for-erlang.html

UPDATE: (7/13/07) The code has gone through a few iterations since I originally posted it. Here's the latest, most concise version:

http://user.it.uu.se/~pergu/utf8.erl

module(utf8).
-export([from_binary/1, to_binary/1]).

%% Given a binary of UTF-8 encoded text, return a UTF-32 String
%% (i.e. each element is a unicode code point).
from_binary(Bin) ->
    decode_binary(Bin, []).

decode_binary(<<>>, Str) ->
    {ok, lists:reverse(Str)};
%% 0-7F  0zzzzzzz
decode_binary(<<2#0:1,CodePoint:7,Rest/binary>>, Str) ->
     decode_binary(Rest, [CodePoint|Str]);
%% 110yyyyy 10zzzzzz
decode_binary(<<2#110:3,Y:5,2#10:2,Z:6,Rest/binary>>, Str) ->
    << CodePoint:16 >> = << 0:5,Y:5,Z:6 >>,
    decode_binary(Rest, [CodePoint|Str]);
%% 1110xxxx 10yyyyyy 10zzzzzz
decode_binary(<<2#1110:4,X:4,2#10:2,Y:6,2#10:2,Z:6,Rest/binary>>, Str) ->
    << CodePoint:16 >> = << X:4,Y:6,Z:6 >>,
    decode_binary(Rest, [CodePoint|Str]);
%% 11110www 10xxxxxx 10yyyyyy 10zzzzzz
decode_binary(<<2#11110:5,W:3,2#10:2,X:6,
	       2#10:2,Y:6,2#10:2,Z:6,Rest/binary>>, Str) ->
    << CodePoint:24 >> = << 0:3,W:3,X:6,Y:6,Z:6 >>,
    decode_binary(Rest, [CodePoint|Str]).

%% Given a list of unicode code points, return a binary of UTF-8
%% encoded text.
to_binary(Str) ->
    encode_utf32(Str, []).

encode_utf32([], Utf8) ->
    {ok, list_to_binary(lists:reverse(Utf8))};
 %% 0-7F  0zzzzzzz -> 0zzzzzzz
encode_utf32([U32|Str], Utf8) when U32 < 16#80 ->
    encode_utf32(Str, [U32|Utf8]);
%% 80-7FF yyy yyzzzzzz -> 110yyyyy 10zzzzzz
encode_utf32([U32|Str], Utf8) when U32 < 16#800 ->
    << _:5,Y:5,Z:6 >> = << U32:16 >>,
    encode_utf32(Str, [<<2#110:3,Y:5,2#10:2,Z:6>>|Utf8]);
%% 800-FFFF xxxxyyyy yyzzzzzz -> 1110xxxx 10yyyyyy 10zzzzzz
encode_utf32([U32|Str], Utf8) when U32 < 16#10000 ->
    << X:4,Y:6,Z:6 >> = << U32:16 >>,
    encode_utf32(Str, [<<2#1110:4,X:4,2#10:2,Y:6,2#10:2,Z:6>>|Utf8]);
%% 10000-10FFFF wwwxx xxxxyyyy yyzzzzzz -> 11110www 10xxxxxx 10yyyyyy 10zzzzzz
encode_utf32([U32|Str], Utf8) when U32 < 16#110000 ->
    << 0:3,W:3,X:6,Y:6,Z:6 >> = << U32:24 >>,
    encode_utf32(Str, [<<2#11110:5,W:3,2#10:2,X:6,
			 2#10:2,Y:6,2#10:2,Z:6>>|Utf8]).

So how's about that metaprogramming?

Mon, 25 Jun 2007 01:01:00 -0500

Rubyists are hooked on metaprogramming like crack fiends. It's certainly a defensible position. "DSLs", a.k.a. Ruby syntax repurposed to produce or emulate some other type of language, are all the rage among the Ruby community. Now granted, that only begins to scratch the surface of what Ruby metaprogramming can do for you, but it's perhaps one of the foremost examples (at least where you're using Ruby to metaprogram something other than Ruby)

Being able to simply express that which on the backend represents some sort of nastiness is one of the greatest features Ruby can provide. ActiveRecord has been excellent in this respect, metaprogramming SQL in a manner which is largely abstracted against the backend database software.

For example, in Active Record, we could:

Foo.find(:all)

or

Foo.find(:all, :conditions => "bar = baz")

and get back a "list" of all records in the "Foo" table. These records act like objects, responding to methods and trying their best to behave as if they weren't just an object representation of the result set.

We can iterate the list with standard list traversal commands from Ruby, such as #each, #map, and #inject, applying transformations and reducing the results to a single value if we so desire.

However, under a little scrutiny this approach is suboptimal. For example, let's consider the example:

Foo.find(:all).map { |obj| ... }

Here we wish to apply a transformation to the results set for the entirety of Foos. What if there are tens of millions of Foos in our table? Each of these must get mapped to an object representing the record. Only after we've mapped them all to objects can we begin to apply our transformation, which iterates against the result set we just returned. What if we were just after a list of one particular field from each record? We could use:

Foo.find(:all, :select => 'bar').map(&:bar)

to obtain the result set, but clearly this is suboptimal in that it still builds objects to encapsulate each record, then has to iterate through the result set again pulling a particular instance variable from each object.

Wouldn't it be nice if we could use standard syntax to query the database, but not have it be monumentally inefficient? If that standard syntax could express exactly what a given operation is doing, but use some sneaky backend tricks to speed up the process?

For example, imagine ActiveRecord returned a "magic array" that didn't actually load the results until you ask for them. What if #find(...).map { |obj| ... } built the objects as they were getting read from the database?

Erlang does this with a feature of its Mnesia database called qlc, or Query List Comprehensions. For those of you unfamiliar with the concept (present in Python and many functional languages, but unfortunately not Ruby), #map (with no block) would look something like this:

[X || X <- List]

This iterates through the given list, applying no transformation to the output. We can also add conditions to the list comprehension to look for particular things:

[X || X <- List, X#rec.bar == baz]

Now let's compare an ActiveRecord example to a Mnesia example:

ActiveRecord: Foo.find(:all, :select => 'bar', :conditions => 'bar = baz').map(&:bar)

Mnesia: [X#rec.bar || X <- mnesia:table(foo), X#rec.bar == baz]

Here the two approaches look similar, except Erlang is using native syntax to perform the query, and ActiveRecord is accepting an SQL fragment as a string.

With ActiveRecord, we'd think of this approach as being more similar to:

Foo.find(:select => 'bar').select { |obj| obj.bar == baz }.map(&:bar)

which gets rid of the conditions string, would be monumentally inefficient as Ruby would be filtering the records as opposed to the database.

So is the Erlang approach inefficient? If we're working on a table with hundreds of millions of records, is mnesia:table going to return them all before we can even begin selecting for the ones we want?

The answer is no, because there's some craftiness going on in the background you're not seeing:

qlc:q([X#rec.bar || X <- mnesia:table(foo), X#rec.bar == baz])

So what's going on here? qlc:q is actually constructing an efficient Mnesia query which applies the given transform as the results of the list comprehension are read out of the database. The list comprehension as written is never executed, rather the instructions from the parse tree are used to perform a substantially more efficient operation.

In Erlang, this approach is called a parse transform, and is somewhat reminiscent of a Lisp macro. It's effectively mucking through the AST, looking at what you wrote and building something entirely different from that. With Mnesia, it means you can get optimized database queries which provide a transformation on the results set using only standard Erlang list comprehension syntax. Sweet!

Hi, I'm the Actor model. You may remember me from such models as OOP...

Fri, 08 Jun 2007 16:02:00 -0500

In my initial examination of Erlang some months ago, I discovered something horrible about it: It isn't object oriented! Looking at a basic module definition, it looked like any other functional language. While a combination of functional and OO paradigms is perhaps a match made in hell, a pure-OO language like Ruby had turned me into something of an OO junkie.

After reading a bit more about Erlang, I discovered it used something similar to but arguably much more powerful than OO: the Actor Model. Actors accomplish many of the same jobs as objects do in OO languages: they provide an elegant way to partition code, they process messages, and they can also encapsulate state.

So what makes an Actor different from an object? Every Actor runs simultaneously. One way to think of Actors is as an unholy fusion of objects and threads. However, in Erlang, all the traditional problems of multithreaded programming, particularly locking and serialization of shared resources are conceptually abstracted away. Erlang provides effectively error-free concurrent programming compared to the error-prone leaky abstractions provided in threaded programming, particularly in the realm on error-handling.

The Actor model was conceptually born out of languages like Smalltalk, which arguably provided the underlying paradigm for all modern OO languages. Smalltalk's most fundamental language construct is a message, with objects representing message senders and receivers.

The Actor model functions in a similar manner, with messages sent from and to Actors. However, unlike Smalltalk the message processing is not implicitly synchronous. Actors, unlike objects, are not required to respond to messages. This means Actors can fire off and forget about messages they wish for other Actors to process, whereas in an OOP language one object must wait for another to complete the current request before proceeding.

Another OOP deficiency you may have noticed are state-holder classes which lack invariants. These classes are more-or-less replacements for structs, loaded with getter and setter methods, and nothing else. Such classes are typically used as internal state-holders by other proxy classes, and in languages which support them can be implemented as inner classes. Some languages, such as Ruby, even provide ways to metaprogram getter and setter methods, so prevalent is the pattern.

Erlang replaces these simple getter/setter objects with what are effectively messages. Since these structures don't need to preserve an invariant, there's no need to use objects. Erlang abstracts them to "records", which are effectively a struct, and in Erlang are implemented as tuples with named members. Using records, and Erlang's database, Mnesia, no object relational mapper thunking layer is needed to translate database records into objects like is needed in an OO language. In this sense, I consider Actor a more elegant abstraction than the OO paradigm.

The Actor model certainly isn't unique to Erlang, and predates Erlang by a few decades. It's found its way into other languages, such as Scala. As the number of cores in modern processors continues to increase I expect we'll see a continued proliferation of the Actor model. However, given the concepts Erlang embodies, I still expect Erlang to parallelize better than other languages which implement Actor. As Joe Armstrong says:

Your Erlang program should just run N times faster on an N core processor

Apologies to Mark Pilgrim

Thu, 07 Jun 2007 18:43:00 -0500

I've been asked if "Dive into Erlang" is intentionally stolen from Mark Pilgrim's book "Dive into Python" and his associated blog "Dive into Mark". While I have read Mark's blog before (I particularly enjoyed his translation of DHH) I did not even think of this when naming Dive into Erlang.

So, let's just call it an unintentional rip-off...

And with that, I'll resume my regularly scheduled coverage of Erlang.

Erlang vs. Rails concurrency, part 2

Thu, 07 Jun 2007 12:19:00 -0500

I see my previous post regarding DHH's comments on Rails concurrency set off something of a flamewar on programming.reddit.com, and at least one angry rebuttal:

...the comparison is not naive It's Bullshit He's basically comparing the memory usage of a Rails process filled with data necessary for serving requests to the management overhead of an Erlang process.

Point taken, and conceded. However, I'll hope you'll at least concede my point that Rails is a memory hungry framework, and when operating system processes are your concurrency primitive, that bloat must be replicated at least once, if not twice, for each CPU core in your system.

With several load-balanced Rails processes serving similar requests, you stand an extremely good chance of duplicating a lot of information between them, particularly in regard to the framework AST. With Rails, you're paying a minimum penalty of 50MB worth of RAM for each concurrent connection you wish to handle in terms of framework overhead alone, and that's before ActiveSupport starts loading in additional dependencies. Whatever happened to DRY?

Both Ruby and Rails necessitate this approach. Rails isn't thread safe, and even if it were, Ruby cannot effectively leverage multiple CPUs or CPU cores using threads. Ruby 1.8 cannot because of its green threads implementation, and Ruby 1.9 will not due to a giant lock on the VM. An O'Reilly blog critical of DHH's recent remarks paints a less dire picture of the future of multithreaded Ruby, but as things stand a multi-VM approach seems like the most practical route for leveraging concurrency in Ruby 1.9.

While an Erlang web framework will certainly use hundreds if not thousands of Erlang processes to handle the equivalent workload of what Rails would handle using dozens of OS processes, there's no framework duplication necessary for concurrent request processing.

Is DHH right about concurrency?

Wed, 06 Jun 2007 20:56:00 -0500

Ruby on Rails creator David Heinemeier Hansson opined today on the subject of concurrency in Rails. As a Rails programmer dealing with the severe limitations of both Ruby and the lack of thread safety in the Rails framework, and having "seen the light" of Erlang, I felt the need to opine myself on DHH's take on the matter:

"the fear of that transition [to multicore CPUs] has bled into places where it's largely not relevant, like web-application development. [...] Multiple cores are laughably easy to utilize for web applications because our problems are rarely in the speed of serving 1 request."

DHH is advocating using the OS process model in lieu of better approaches. There are some clear flaws with this approach. Specifically, a Rails instance, due to both its unithreaded nature and limitations of the Ruby VM inherently limit each Rails instance to a single CPU. This means computationally expensive requests to the framework can never use more resources in a system than a single CPU core, unless they're leveraging some external service such as a database server. This leads to asymmetrical distribution of load in request processing: heavy requests are bound to a single CPU core bottleneck.

Perhaps this is what DHH is implying when he says "our problems". I guess 37signals isn't dealing with any CPU intensive requests. I'd say this is something to keep in mind when picking your web framework: If you expect to do anything CPU intensive behind a given request, Erlang is far and away the better choice.

He continues:

Threads are not the only way to do that. Processes do the job nearly as well with a drop of the complexity. And that's exactly how Rails is scaling to use all the cores you can throw at it. [...] The 37signals suite is currently using some ~25 cores for the application servers that all the applications have dips on. We'd welcome a 64-core chip any day.

I can't argue with a shared-nothing process model, which is precisely what Erlang advances in lieu of threads. However, in Rails, each process is an island. Facilities which would be "shared" in Erlang (as a single process which processes messages in its shared-nothing architecture) must be duplicated across each process in Rails.

This argument also completely omits the memory overhead of Rails. With a 50-70MB baseline and a real-world Rails process growing to nearly 200MB over time, a 64 core CPU will require at least 13GB of RAM to even run a single Rails instance per CPU core. Even with a less pessimistic estimate of 150MB per Rails instance, you must still take into account that running two Rails instances per core will likely lead to greater throughput, meaning the optimum configuration for a 64 core CPU is closer to 20GB of RAM.

Now, let's compare to Erlang processes, which have approximately 300 bytes of overhead... perhaps a naive comparison, but you get the idea. I also suspect that when an Erlang web framework (perhaps ErlyWeb) emerges into the spotlight that concurrency will be touted as a major advantage over Rails.

What makes Erlang so great?

Tue, 05 Jun 2007 01:43:00 -0500

I've wondered this since I first heard out about Erlang. Excitement about Erlang began as some stray posts on various Internet message boards and news sites, but soon thereafter came a genuine buzz of excitement in the Ruby community. But what was this excitement all about?

Well, turns out Erlang has solved a great number of problems that have been plaguing programmers since programming's inception. These problems result in complex software "stacks" where multiple components written in different languages are assembled to solve a given problem. Erlang eliminates the need for complex, multifaceted software stacks, providing more generalized solutions to the common problems facing both programmers and computing in general.

Erlang's claim to fame lies in concurrency. Designed from the ground up for massive parallelism, Erlang is finally starting to make the dream of simple scaling by throwing more hardware at the problem possible. Before, bottlenecks of all kinds plagued this approach. While it was possible to scale horizontally with most types of software, generally it was a complex process that requires complex changes to databases and to the way software is written.

The traditional LAMP stack, comprising four different software packages: The Linux operating system, Apache, MySQL, and PHP (and typically a fifth package, memcached) can all be replaced by only Linux and the Erlang environment. This is because Erlang can completely replace the "AMP" part of the stack, in a way that is substantially superior to the other components and provides less error prone and more effective scalability.

YAWS, the Erlang web server, replaces Apache. In the following benchmark, YAWS beats Apache hands down managing a throughput of 800kB/s even with 80 thousand simultaneous connections. Apache failed at only 4,000:

Mnesia, the Erlang database, replaces MySQL. One might question choosing anything other than a well-known, traditional RDBMS. Relational databases won the database war in the '80s, and thus far there's been no replacement worth noting.

However, anyone using a modern web framework in conjunction with an object oriented language knows the pitfalls of a popular database abstraction: the Object Relational Mapper, or ORM. Since WebObjects these have become an integral part of modern web frameworks, but alas, they are not an ideal solution. There are many places where the two approaches cannot be abstracted into a single, unified model, and the abstraction leaks.

Erlang's Mnesia solves this problem by working exclusively with Erlang's native data types. With a package called Mnemosyne, you can even query it using Erlang's native pattern matching, although there are other query interfaces available that are a little more practical. However, the thunking layer between native data types and the database is gone. Furthermore, like Erlang itself, Mnesia supports high degrees of concurrency and scales quite well. It's already being used for production telecom databases.

Finally, PHP is replaced with Erlang itself, and web frameworks like ErlyWeb. PHP is generally scorned by experienced programmers, with a number of foibles, everything from its weak, error-prone type system to its highly inconsistent, redundant, and poorly-designed API. On the other hand, Erlang provides a dynamically-typed, functional language, built around the Actor model, which together make Erlang both a concise and elegant language to work in, but also provide for concurrency.

Together these features make Erlang more attractive as a web development language than Ruby. While concurrency is Erlang's greatest strength, it is Ruby's greatest weakness. The Ruby VM uses green threads, which mean it is currently bound to a single CPU core. Ruby's VM is also slower than most, and a project to improve its performance isn't due out until the end of the year. Even after the VM is upgraded, it retains a giant lock which means the only sure path to high concurrency is to run multiple copies of the VM. This provides for few advantages over simply running multiple Ruby processes.

Erlang runs its processes inside of threads themselves. They are lightweight and scheduled entirely by the Erlang compiler, meaning they don't need to make any heavy system calls to be created, destroyed, or to communicate. Erlang uses what can be considered an M:N threading model, with native threads running on each CPU core which can run a shared pool of Erlang processes. Furthermore, Erlang features a high performance JIT compiler called HiPE, which compiles Erlang modules to native code on-the-fly, ensuring excellent performance.

Finally, there is a great book about Erlang available now, in beta form. From the Pragmatic Programmers, to whom the massive spike in Ruby's popularity is ultimately rooted (and to whom the popularity of Erlang among the Ruby community is owed) comes their newest book, Programming Erlang, which provides an excellent guide to learning the language. I'm presently working my way through this book, but everything I've read so far has been quite exciting.

I'm looking forward to the day that Erlang can provide a practical means to get the massive scalability needed to build increasingly complex modern web applications. I see this painfully lacking in Ruby and Rails, and a clear sign that the Rails approach abstracts many common pitfalls away but leaves the problems of concurrency, performance, and overall scalability unaddressed.