The Python Paradox is now the Scala Paradox
Published by Martin Kleppmann on 18 Sep 2009.
Paul Graham has written a bunch of good essays on entrepreneurship,
laying down much of the philosophy behind
Y Combinator. If you’ve not read any of them yet, you should go and read
them now – they contain lots of wisdom.
In his 2004 short essay
The Python Paradox, PG argues (perhaps controversially) that
a company can hire smarter programmers if it chooses to write its code in a “comparatively esoteric”
programming language. At the time, Python was probably considered by most people to be esoteric in
comparison to Java – in the sense that not many people would learn it at university or for career
purposes. Therefore, the programmers who knew Python were people who learnt it for fun; and learning
languages for fun is an activity which typically only the bright and motivated people engage in.
Which makes the language a good “quality filter” for people.
Of course times have moved on, and
Python (and Ruby, for that matter) are definitely entering the mainstream. They are still fine
languages, but they no longer carry as much of an early adopter aura about them. The culture of
testing and code beauty which is embraced by the Ruby and Python communities is still something
special, and that is now the primary reason why I would choose Ruby and Python over Java or C#.
PG’s observation still holds true. But what are the new technologies to look out for? What can
you find in 2009 which has the same role as Python did in 2004?
Choosing a programming language
A few months ago, when I was designing the system architecture for
Go Test It, our awesome cross-browser testing product, I had a lot of design
choices to make. For some parts of the system it was fairly clear what we were going to use: for
instance, the frontend web application was almost certainly going to be Rails, because I had some
existing tools like
the invoicing gem which I wanted to reuse, and because it was
simply a good fit for the job.
But what about the actual test management infrastructure?
I chose to write it in Scala. There were plenty of reasons why this
could have seemed a bad idea:
- I had hardly any experience in using Scala, so I would have to invest a lot of time
learning it as I went along;
- nobody else I knew had any experience in using Scala at all, so anyone I hired
would probably also have to learn it (at the cost of slower progress);
- some of the tools (particularly the Eclipse plugin) were still packed with bugs;
- although I had access to Java libraries, I knew that I would have to write library
wrappers of my own to make use of the Scala language features;
- setting up a proper build process was
Scala also had some technical merits; particularly the actor model for multithreaded programming was a
good fit for our problem. Still, in a purely technical consideration, it probably wouldn’t have made
much sense. But my heart still said yes. The prospect of working with a language which looked
pretty, had static type inference, nice functional programming features, and runs on the JVM
(battle-tested and optimised over many years)… my heart just said yes. It wasn’t a rational
choice, but an instinctive, emotional one.
Paul Graham was right
Shortly after working this
out and drawing my architecture diagram (a pretty insane-looking tangle of boxes and arrows on a
sheet of paper), I was talking to
Sam Stokes, a freelance developer sitting next to me in our shared
office. Sam is bright, motivated and interested in the things going on in the software world. We
were talking casually about Go Test It and I showed him my messy diagram.
And when I had talked him through the architecture (“I think I’ll write these three components in
Scala, and they will talk to each other via a
RabbitMQ message queue”), Sam said, in effect, “Hey, this is cool, I
want to work on this”.
And he did. Over the course of a few weekends, Sam spent several days
learning Scala and contributing to the
Go Test It codebase. When you use Go Test It today, you are using the Scala
code he wrote. He did this simply because he found it interesting and wanted to learn something new.
The code he wrote was good, production-quality stuff. And he didn’t want a penny for it.
It’s not often that you get high-quality contributions to a closed-source project from a developer
who is busy enough with several other projects already, for free.
Of course Sam cannot live on technical
stimulation and my gratitude alone, and since then, we have actually contracted him to do paid work
on Go Test It. But I found his initial reaction, and his approach to the project, a great example to
prove Paul Graham’s point.
I would say Scala in 2009 has the place which Python had in 2004.
There are a few other languages I would also consider for this title:
Clojure come to mind (but don’t take that as a recommendation – I’ve not yet
used any of them seriously myself). What these languages all have in common is that they’ve been
around for long enough to come out of the purely academic space, are stable enough to be
production-worthy, but are also new and exciting enough to attract good
In an article about non-relational databases
which Ryan Carson asked me to write a few months ago, I suggested that
fashion can and should play a role in choosing which technologies to use. I got some criticism for
this remark, but I still stand by my view. It is effectively a different way of looking at PG’s
statement, provided you look for the fashion in the right circles (i.e. amongst experienced
developers working at the cutting edge).
I wouldn’t go with fashion against all technical merit,
but provided the technology is suitable and won’t increase your costs disproportionately, why not do
something fashionable and adventurous? In an innovation-based technology business, the quality of
your developers is key. Investments into things which make your good developers happy will pay off