|  
  
May 2003  
  
 _(This essay is derived from a guest lecture at Harvard, which incorporated
an earlier talk at Northeastern.)_  
  
When I finished grad school in computer science I went to art school to study
painting. A lot of people seemed surprised that someone interested in
computers would also be interested in painting. They seemed to think that
hacking and painting were very different kinds of work-- that hacking was
cold, precise, and methodical, and that painting was the frenzied expression
of some primal urge.  
  
Both of these images are wrong. Hacking and painting have a lot in common. In
fact, of all the different types of people I've known, hackers and painters
are among the most alike.  
  
What hackers and painters have in common is that they're both makers. Along
with composers, architects, and writers, what hackers and painters are trying
to do is make good things. They're not doing research per se, though if in the
course of trying to make good things they discover some new technique, so much
the better.  
  
  
  
I've never liked the term "computer science." The main reason I don't like it
is that there's no such thing. Computer science is a grab bag of tenuously
related areas thrown together by an accident of history, like Yugoslavia. At
one end you have people who are really mathematicians, but call what they're
doing computer science so they can get DARPA grants. In the middle you have
people working on something like the natural history of computers-- studying
the behavior of algorithms for routing data through networks, for example. And
then at the other extreme you have the hackers, who are trying to write
interesting software, and for whom computers are just a medium of expression,
as concrete is for architects or paint for painters. It's as if
mathematicians, physicists, and architects all had to be in the same
department.  
  
Sometimes what the hackers do is called "software engineering," but this term
is just as misleading. Good software designers are no more engineers than
architects are. The border between architecture and engineering is not sharply
defined, but it's there. It falls between what and how: architects decide what
to do, and engineers figure out how to do it.  
  
What and how should not be kept too separate. You're asking for trouble if you
try to decide what to do without understanding how to do it. But hacking can
certainly be more than just deciding how to implement some spec. At its best,
it's creating the spec-- though it turns out the best way to do that is to
implement it.  
  
  
  
Perhaps one day "computer science" will, like Yugoslavia, get broken up into
its component parts. That might be a good thing. Especially if it meant
independence for my native land, hacking.  
  
Bundling all these different types of work together in one department may be
convenient administratively, but it's confusing intellectually. That's the
other reason I don't like the name "computer science." Arguably the people in
the middle are doing something like an experimental science. But the people at
either end, the hackers and the mathematicians, are not actually doing
science.  
  
The mathematicians don't seem bothered by this. They happily set to work
proving theorems like the other mathematicians over in the math department,
and probably soon stop noticing that the building they work in says ``computer
science'' on the outside. But for the hackers this label is a problem. If what
they're doing is called science, it makes them feel they ought to be acting
scientific. So instead of doing what they really want to do, which is to
design beautiful software, hackers in universities and research labs feel they
ought to be writing research papers.  
  
In the best case, the papers are just a formality. Hackers write cool
software, and then write a paper about it, and the paper becomes a proxy for
the achievement represented by the software. But often this mismatch causes
problems. It's easy to drift away from building beautiful things toward
building ugly things that make more suitable subjects for research papers.  
  
Unfortunately, beautiful things don't always make the best subjects for
papers. Number one, research must be original-- and as anyone who has written
a PhD dissertation knows, the way to be sure that you're exploring virgin
territory is to to stake out a piece of ground that no one wants. Number two,
research must be substantial-- and awkward systems yield meatier papers,
because you can write about the obstacles you have to overcome in order to get
things done. Nothing yields meaty problems like starting with the wrong
assumptions. Most of AI is an example of this rule; if you assume that
knowledge can be represented as a list of predicate logic expressions whose
arguments represent abstract concepts, you'll have a lot of papers to write
about how to make this work. As Ricky Ricardo used to say, "Lucy, you got a
lot of explaining to do."  
  
The way to create something beautiful is often to make subtle tweaks to
something that already exists, or to combine existing ideas in a slightly new
way. This kind of work is hard to convey in a research paper.  
  
  
  
So why do universities and research labs continue to judge hackers by
publications? For the same reason that "scholastic aptitude" gets measured by
simple-minded standardized tests, or the productivity of programmers gets
measured in lines of code. These tests are easy to apply, and there is nothing
so tempting as an easy test that kind of works.  
  
Measuring what hackers are actually trying to do, designing beautiful
software, would be much more difficult. You need a good sense of design to
judge good design. And there is no correlation, except possibly a negative
one, between people's ability to recognize good design and their confidence
that they can.  
  
The only external test is time. Over time, beautiful things tend to thrive,
and ugly things tend to get discarded. Unfortunately, the amounts of time
involved can be longer than human lifetimes. Samuel Johnson said it took a
hundred years for a writer's reputation to converge. You have to wait for the
writer's influential friends to die, and then for all their followers to die.  
  
I think hackers just have to resign themselves to having a large random
component in their reputations. In this they are no different from other
makers. In fact, they're lucky by comparison. The influence of fashion is not
nearly so great in hacking as it is in painting.  
  
  
  
There are worse things than having people misunderstand your work. A worse
danger is that you will yourself misunderstand your work. Related fields are
where you go looking for ideas. If you find yourself in the computer science
department, there is a natural temptation to believe, for example, that
hacking is the applied version of what theoretical computer science is the
theory of. All the time I was in graduate school I had an uncomfortable
feeling in the back of my mind that I ought to know more theory, and that it
was very remiss of me to have forgotten all that stuff within three weeks of
the final exam.  
  
Now I realize I was mistaken. Hackers need to understand the theory of
computation about as much as painters need to understand paint chemistry. You
need to know how to calculate time and space complexity and about Turing
completeness. You might also want to remember at least the concept of a state
machine, in case you have to write a parser or a regular expression library.
Painters in fact have to remember a good deal more about paint chemistry than
that.  
  
I've found that the best sources of ideas are not the other fields that have
the word "computer" in their names, but the other fields inhabited by makers.
Painting has been a much richer source of ideas than the theory of
computation.  
  
For example, I was taught in college that one ought to figure out a program
completely on paper before even going near a computer. I found that I did not
program this way. I found that I liked to program sitting in front of a
computer, not a piece of paper. Worse still, instead of patiently writing out
a complete program and assuring myself it was correct, I tended to just spew
out code that was hopelessly broken, and gradually beat it into shape.
Debugging, I was taught, was a kind of final pass where you caught typos and
oversights. The way I worked, it seemed like programming consisted of
debugging.  
  
For a long time I felt bad about this, just as I once felt bad that I didn't
hold my pencil the way they taught me to in elementary school. If I had only
looked over at the other makers, the painters or the architects, I would have
realized that there was a name for what I was doing: sketching. As far as I
can tell, the way they taught me to program in college was all wrong. You
should figure out programs as you're writing them, just as writers and
painters and architects do.  
  
Realizing this has real implications for software design. It means that a
programming language should, above all, be malleable. A programming language
is for thinking of programs, not for expressing programs you've already
thought of. It should be a pencil, not a pen. Static typing would be a fine
idea if people actually did write programs the way they taught me to in
college. But that's not how any of the hackers I know write programs. We need
a language that lets us scribble and smudge and smear, not a language where
you have to sit with a teacup of types balanced on your knee and make polite
conversation with a strict old aunt of a compiler.  
  
  
  
While we're on the subject of static typing, identifying with the makers will
save us from another problem that afflicts the sciences: math envy. Everyone
in the sciences secretly believes that mathematicians are smarter than they
are. I think mathematicians also believe this. At any rate, the result is that
scientists tend to make their work look as mathematical as possible. In a
field like physics this probably doesn't do much harm, but the further you get
from the natural sciences, the more of a problem it becomes.  
  
A page of formulas just looks so impressive. (Tip: for extra impressiveness,
use Greek variables.) And so there is a great temptation to work on problems
you can treat formally, rather than problems that are, say, important.  
  
If hackers identified with other makers, like writers and painters, they
wouldn't feel tempted to do this. Writers and painters don't suffer from math
envy. They feel as if they're doing something completely unrelated. So are
hackers, I think.  
  
  
  
If universities and research labs keep hackers from doing the kind of work
they want to do, perhaps the place for them is in companies. Unfortunately,
most companies won't let hackers do what they want either. Universities and
research labs force hackers to be scientists, and companies force them to be
engineers.  
  
I only discovered this myself quite recently. When Yahoo bought Viaweb, they
asked me what I wanted to do. I had never liked the business side very much,
and said that I just wanted to hack. When I got to Yahoo, I found that what
hacking meant to them was implementing software, not designing it. Programmers
were seen as technicians who translated the visions (if that is the word) of
product managers into code.  
  
This seems to be the default plan in big companies. They do it because it
decreases the standard deviation of the outcome. Only a small percentage of
hackers can actually design software, and it's hard for the people running a
company to pick these out. So instead of entrusting the future of the software
to one brilliant hacker, most companies set things up so that it is designed
by committee, and the hackers merely implement the design.  
  
If you want to make money at some point, remember this, because this is one of
the reasons startups win. Big companies want to decrease the standard
deviation of design outcomes because they want to avoid disasters. But when
you damp oscillations, you lose the high points as well as the low. This is
not a problem for big companies, because they don't win by making great
products. Big companies win by sucking less than other big companies.  
  
So if you can figure out a way to get in a design war with a company big
enough that its software is designed by product managers, they'll never be
able to keep up with you. These opportunities are not easy to find, though.
It's hard to engage a big company in a design war, just as it's hard to engage
an opponent inside a castle in hand to hand combat. It would be pretty easy to
write a better word processor than Microsoft Word, for example, but Microsoft,
within the castle of their operating system monopoly, probably wouldn't even
notice if you did.  
  
The place to fight design wars is in new markets, where no one has yet managed
to establish any fortifications. That's where you can win big by taking the
bold approach to design, and having the same people both design and implement
the product. Microsoft themselves did this at the start. So did Apple. And
Hewlett-Packard. I suspect almost every successful startup has.  
  
  
  
So one way to build great software is to start your own startup. There are two
problems with this, though. One is that in a startup you have to do so much
besides write software. At Viaweb I considered myself lucky if I got to hack a
quarter of the time. And the things I had to do the other three quarters of
the time ranged from tedious to terrifying. I have a benchmark for this,
because I once had to leave a board meeting to have some cavities filled. I
remember sitting back in the dentist's chair, waiting for the drill, and
feeling like I was on vacation.  
  
The other problem with startups is that there is not much overlap between the
kind of software that makes money and the kind that's interesting to write.
Programming languages are interesting to write, and Microsoft's first product
was one, in fact, but no one will pay for programming languages now. If you
want to make money, you tend to be forced to work on problems that are too
nasty for anyone to solve for free.  
  
All makers face this problem. Prices are determined by supply and demand, and
there is just not as much demand for things that are fun to work on as there
is for things that solve the mundane problems of individual customers. Acting
in off-Broadway plays just doesn't pay as well as wearing a gorilla suit in
someone's booth at a trade show. Writing novels doesn't pay as well as writing
ad copy for garbage disposals. And hacking programming languages doesn't pay
as well as figuring out how to connect some company's legacy database to their
Web server.  
  
  
  
I think the answer to this problem, in the case of software, is a concept
known to nearly all makers: the day job. This phrase began with musicians, who
perform at night. More generally, it means that you have one kind of work you
do for money, and another for love.  
  
Nearly all makers have day jobs early in their careers. Painters and writers
notoriously do. If you're lucky you can get a day job that's closely related
to your real work. Musicians often seem to work in record stores. A hacker
working on some programming language or operating system might likewise be
able to get a day job using it.   
  
When I say that the answer is for hackers to have day jobs, and work on
beautiful software on the side, I'm not proposing this as a new idea. This is
what open-source hacking is all about. What I'm saying is that open-source is
probably the right model, because it has been independently confirmed by all
the other makers.  
  
It seems surprising to me that any employer would be reluctant to let hackers
work on open-source projects. At Viaweb, we would have been reluctant to hire
anyone who didn't. When we interviewed programmers, the main thing we cared
about was what kind of software they wrote in their spare time. You can't do
anything really well unless you love it, and if you love to hack you'll
inevitably be working on projects of your own.   
  
  
  
Because hackers are makers rather than scientists, the right place to look for
metaphors is not in the sciences, but among other kinds of makers. What else
can painting teach us about hacking?  
  
One thing we can learn, or at least confirm, from the example of painting is
how to learn to hack. You learn to paint mostly by doing it. Ditto for
hacking. Most hackers don't learn to hack by taking college courses in
programming. They learn to hack by writing programs of their own at age
thirteen. Even in college classes, you learn to hack mostly by hacking.   
  
Because painters leave a trail of work behind them, you can watch them learn
by doing. If you look at the work of a painter in chronological order, you'll
find that each painting builds on things that have been learned in previous
ones. When there's something in a painting that works very well, you can
usually find version 1 of it in a smaller form in some earlier painting.  
  
I think most makers work this way. Writers and architects seem to as well.
Maybe it would be good for hackers to act more like painters, and regularly
start over from scratch, instead of continuing to work for years on one
project, and trying to incorporate all their later ideas as revisions.  
  
The fact that hackers learn to hack by doing it is another sign of how
different hacking is from the sciences. Scientists don't learn science by
doing it, but by doing labs and problem sets. Scientists start out doing work
that's perfect, in the sense that they're just trying to reproduce work
someone else has already done for them. Eventually, they get to the point
where they can do original work. Whereas hackers, from the start, are doing
original work; it's just very bad. So hackers start original, and get good,
and scientists start good, and get original.  
  
  
  
The other way makers learn is from examples. For a painter, a museum is a
reference library of techniques. For hundreds of years it has been part of the
traditional education of painters to copy the works of the great masters,
because copying forces you to look closely at the way a painting is made.  
  
Writers do this too. Benjamin Franklin learned to write by summarizing the
points in the essays of Addison and Steele and then trying to reproduce them.
Raymond Chandler did the same thing with detective stories.  
  
Hackers, likewise, can learn to program by looking at good programs-- not just
at what they do, but the source code too. One of the less publicized benefits
of the open-source movement is that it has made it easier to learn to program.
When I learned to program, we had to rely mostly on examples in books. The one
big chunk of code available then was Unix, but even this was not open source.
Most of the people who read the source read it in illicit photocopies of John
Lions' book, which though written in 1977 was not allowed to be published
until 1996.  
  
  
  
Another example we can take from painting is the way that paintings are
created by gradual refinement. Paintings usually begin with a sketch.
Gradually the details get filled in. But it is not merely a process of filling
in. Sometimes the original plans turn out to be mistaken. Countless paintings,
when you look at them in xrays, turn out to have limbs that have been moved or
facial features that have been readjusted.  
  
Here's a case where we can learn from painting. I think hacking should work
this way too. It's unrealistic to expect that the specifications for a program
will be perfect. You're better off if you admit this up front, and write
programs in a way that allows specifications to change on the fly.  
  
(The structure of large companies makes this hard for them to do, so here is
another place where startups have an advantage.)  
  
Everyone by now presumably knows about the danger of premature optimization. I
think we should be just as worried about premature design-- deciding too early
what a program should do.  
  
The right tools can help us avoid this danger. A good programming language
should, like oil paint, make it easy to change your mind. Dynamic typing is a
win here because you don't have to commit to specific data representations up
front. But the key to flexibility, I think, is to make the language very
abstract. The easiest program to change is one that's very short.  
  
  
  
This sounds like a paradox, but a great painting has to be better than it has
to be. For example, when Leonardo painted the portrait of Ginevra de Benci in
the National Gallery, he put a juniper bush behind her head. In it he
carefully painted each individual leaf. Many painters might have thought, this
is just something to put in the background to frame her head. No one will look
that closely at it.  
  
Not Leonardo. How hard he worked on part of a painting didn't depend at all on
how closely he expected anyone to look at it. He was like Michael Jordan.
Relentless.  
  
Relentlessness wins because, in the aggregate, unseen details become visible.
When people walk by the portrait of Ginevra de Benci, their attention is often
immediately arrested by it, even before they look at the label and notice that
it says Leonardo da Vinci. All those unseen details combine to produce
something that's just stunning, like a thousand barely audible voices all
singing in tune.  
  
Great software, likewise, requires a fanatical devotion to beauty. If you look
inside good software, you find that parts no one is ever supposed to see are
beautiful too. I'm not claiming I write great software, but I know that when
it comes to code I behave in a way that would make me eligible for
prescription drugs if I approached everyday life the same way. It drives me
crazy to see code that's badly indented, or that uses ugly variable names.  
  
  
  
If a hacker were a mere implementor, turning a spec into code, then he could
just work his way through it from one end to the other like someone digging a
ditch. But if the hacker is a creator, we have to take inspiration into
account.  
  
In hacking, like painting, work comes in cycles. Sometimes you get excited
about some new project and you want to work sixteen hours a day on it. Other
times nothing seems interesting.  
  
To do good work you have to take these cycles into account, because they're
affected by how you react to them. When you're driving a car with a manual
transmission on a hill, you have to back off the clutch sometimes to avoid
stalling. Backing off can likewise prevent ambition from stalling. In both
painting and hacking there are some tasks that are terrifyingly ambitious, and
others that are comfortingly routine. It's a good idea to save some easy tasks
for moments when you would otherwise stall.  
  
In hacking, this can literally mean saving up bugs. I like debugging: it's the
one time that hacking is as straightforward as people think it is. You have a
totally constrained problem, and all you have to do is solve it. Your program
is supposed to do x. Instead it does y. Where does it go wrong? You know
you're going to win in the end. It's as relaxing as painting a wall.  
  
  
  
The example of painting can teach us not only how to manage our own work, but
how to work together. A lot of the great art of the past is the work of
multiple hands, though there may only be one name on the wall next to it in
the museum. Leonardo was an apprentice in the workshop of Verrocchio and
painted one of the angels in his Baptism of Christ. This sort of thing was the
rule, not the exception. Michelangelo was considered especially dedicated for
insisting on painting all the figures on the ceiling of the Sistine Chapel
himself.  
  
As far as I know, when painters worked together on a painting, they never
worked on the same parts. It was common for the master to paint the principal
figures and for assistants to paint the others and the background. But you
never had one guy painting over the work of another.  
  
I think this is the right model for collaboration in software too. Don't push
it too far. When a piece of code is being hacked by three or four different
people, no one of whom really owns it, it will end up being like a common-
room. It will tend to feel bleak and abandoned, and accumulate cruft. The
right way to collaborate, I think, is to divide projects into sharply defined
modules, each with a definite owner, and with interfaces between them that are
as carefully designed and, if possible, as articulated as programming
languages.  
  
  
  
Like painting, most software is intended for a human audience. And so hackers,
like painters, must have empathy to do really great work. You have to be able
to see things from the user's point of view.  
  
When I was a kid I was always being told to look at things from someone else's
point of view. What this always meant in practice was to do what someone else
wanted, instead of what I wanted. This of course gave empathy a bad name, and
I made a point of not cultivating it.  
  
Boy, was I wrong. It turns out that looking at things from other people's
point of view is practically the secret of success. It doesn't necessarily
mean being self-sacrificing. Far from it. Understanding how someone else sees
things doesn't imply that you'll act in his interest; in some situations-- in
war, for example-- you want to do exactly the opposite.   
  
Most makers make things for a human audience. And to engage an audience you
have to understand what they need. Nearly all the greatest paintings are
paintings of people, for example, because people are what people are
interested in.  
  
Empathy is probably the single most important difference between a good hacker
and a great one. Some hackers are quite smart, but when it comes to empathy
are practically solipsists. It's hard for such people to design great software
, because they can't see things from the user's point of view.  
  
One way to tell how good people are at empathy is to watch them explain a
technical question to someone without a technical background. We probably all
know people who, though otherwise smart, are just comically bad at this. If
someone asks them at a dinner party what a programming language is, they'll
say something like ``Oh, a high-level language is what the compiler uses as
input to generate object code.'' High-level language? Compiler? Object code?
Someone who doesn't know what a programming language is obviously doesn't know
what these things are, either.  
  
Part of what software has to do is explain itself. So to write good software
you have to understand how little users understand. They're going to walk up
to the software with no preparation, and it had better do what they guess it
will, because they're not going to read the manual. The best system I've ever
seen in this respect was the original Macintosh, in 1985. It did what software
almost never does: it just worked.   
  
Source code, too, should explain itself. If I could get people to remember
just one quote about programming, it would be the one at the beginning of
_Structure and Interpretation of Computer Programs._

> Programs should be written for people to read, and only incidentally for
> machines to execute.

You need to have empathy not just for your users, but for your readers. It's
in your interest, because you'll be one of them. Many a hacker has written a
program only to find on returning to it six months later that he has no idea
how it works. I know several people who've sworn off Perl after such
experiences.   
  
Lack of empathy is associated with intelligence, to the point that there is
even something of a fashion for it in some places. But I don't think there's
any correlation. You can do well in math and the natural sciences without
having to learn empathy, and people in these fields tend to be smart, so the
two qualities have come to be associated. But there are plenty of dumb people
who are bad at empathy too. Just listen to the people who call in with
questions on talk shows. They ask whatever it is they're asking in such a
roundabout way that the hosts often have to rephrase the question for them.  
  
  
  
So, if hacking works like painting and writing, is it as cool? After all, you
only get one life. You might as well spend it working on something great.  
  
Unfortunately, the question is hard to answer. There is always a big time lag
in prestige. It's like light from a distant star. Painting has prestige now
because of great work people did five hundred years ago. At the time, no one
thought these paintings were as important as we do today. It would have seemed
very odd to people at the time that Federico da Montefeltro, the Duke of
Urbino, would one day be known mostly as the guy with the strange nose in a
painting by Piero della Francesca.  
  
So while I admit that hacking doesn't seem as cool as painting now, we should
remember that painting itself didn't seem as cool in its glory days as it does
now.  
  
What we can say with some confidence is that these are the glory days of
hacking. In most fields the great work is done early on. The paintings made
between 1430 and 1500 are still unsurpassed. Shakespeare appeared just as
professional theater was being born,  and pushed the medium so far that every
playwright since has had to live in his shadow. Albrecht Durer did the same
thing with engraving, and Jane Austen with the novel.  
  
Over and over we see the same pattern. A new medium appears, and people are so
excited about it that they explore most of its possibilities in the first
couple generations. Hacking seems to be in this phase now.  
  
Painting was not, in Leonardo's time, as cool as his work helped make it. How
cool hacking turns out to be will depend on what we can do with this new
medium.  
  
  
  
  
**