gh

Want to start a startup?  Get funded by
Y Combinator.

July 2004(This essay is derived from a talk at Oscon 2004.)
A few months ago I finished a new
book,
and in reviews I keep
noticing words like "provocative'' and "controversial.'' To say
nothing of "idiotic.''I didn't mean to make the book controversial.  I was trying to make
it efficient.  I didn't want to waste people's time telling them
things they already knew.  It's more efficient just to give them
the diffs.  But I suppose that's bound to yield an alarming book.EdisonsThere's no controversy about which idea is most controversial:
the suggestion that variation in wealth might not be as big a
problem as we think.I didn't say in the book that variation in wealth was in itself a
good thing.  I said in some situations it might be a sign of good
things.  A throbbing headache is not a good thing, but it can be
a sign of a good thing-- for example, that you're recovering
consciousness after being hit on the head.Variation in wealth can be a sign of variation in productivity.
(In a society of one, they're identical.) And that
is almost certainly a good thing: if your society has no variation
in productivity, it's probably not because everyone is Thomas
Edison.  It's probably because you have no Thomas Edisons.In a low-tech society you don't see much variation in productivity.
If you have a tribe of nomads collecting sticks for a fire, how
much more productive is the best stick gatherer going to be than
the worst?  A factor of two?  Whereas when you hand people a complex tool
like a computer, the variation in what they can do with
it is enormous.That's not a new idea.  Fred Brooks wrote about it in 1974, and
the study he quoted was published in 1968.  But I think he
underestimated the variation between programmers.  He wrote about productivity in lines
of code:  the best programmers can solve a given problem in a tenth
the time.  But what if the problem isn't given? In programming, as
in many fields, the hard part isn't solving problems, but deciding
what problems to solve.  Imagination is hard to measure, but
in practice it dominates the kind of productivity that's measured
in lines of code.Productivity varies in any field, but there are few in which it
varies so much.  The variation between programmers
is so great that it becomes a difference in kind.  I don't
think this is something intrinsic to programming, though.  In every field,
technology magnifies differences in productivity.  I think what's
happening in programming is just that we have a lot of technological
leverage.  But in every field the lever is getting longer, so the
variation we see is something that more and more fields will see
as time goes on.  And the success of companies, and countries, will
depend increasingly on how they deal with it.If variation in productivity increases with technology, then the
contribution of the most productive individuals will not only be
disproportionately large, but will actually grow with time.  When
you reach the point where 90% of a group's output is created by 1%
of its members, you lose big if something (whether Viking raids,
or central planning) drags their productivity down to the average.If we want to get the most out of them, we need to understand these
especially productive people.  What motivates them?  What do they
need to do their jobs?  How do you recognize them? How do you
get them to come and work for you?  And then of course there's the
question, how do you become one?More than MoneyI know a handful of super-hackers, so I sat down and thought about
what they have in common.  Their defining quality is probably that
they really love to program.  Ordinary programmers write code to pay
the bills.  Great hackers think of it as something they do for fun,
and which they're delighted to find people will pay them for.Great programmers are sometimes said to be indifferent to money.
This isn't quite true.  It is true that all they really care about
is doing interesting work.  But if you make enough money, you get
to work on whatever you want, and for that reason hackers are
attracted by the idea of making really large amounts of money.
But as long as they still have to show up for work every day, they
care more about what they do there than how much they get paid for
it.Economically, this is a fact of the greatest importance, because
it means you don't have to pay great hackers anything like what
they're worth.  A great programmer might be ten or a hundred times
as productive as an ordinary one, but he'll consider himself lucky
to get paid three times as much.  As I'll explain later, this is
partly because great hackers don't know how good they are.  But
it's also because money is not the main thing they want.What do hackers want?  Like all craftsmen, hackers like good tools.
In fact, that's an understatement.  Good hackers find it unbearable
to use bad tools.  They'll simply refuse to work on projects with
the wrong infrastructure.At a startup I once worked for, one of the things pinned up on our
bulletin board was an ad from IBM.  It was a picture of an AS400,
and the headline read, I think, "hackers despise
it.'' [1]When you decide what infrastructure to use for a project, you're
not just making a technical decision.  You're also making a social
decision, and this may be the more important of the two.  For
example, if your company wants to write some software, it might
seem a prudent choice to write it in Java.  But when you choose a
language, you're also choosing a community.  The programmers you'll
be able to hire to work on a Java project won't be as
smart as the
ones you could get to work on a project written in Python.
And the quality of your hackers probably matters more than the
language you choose.  Though, frankly, the fact that good hackers
prefer Python to Java should tell you something about the relative
merits of those languages.Business types prefer the most popular languages because they view
languages as standards. They don't want to bet the company on
Betamax.  The thing about languages, though, is that they're not
just standards.  If you have to move bits over a network, by all
means use TCP/IP.  But a programming language isn't just a format.
A programming language is a medium of expression.I've read that Java has just overtaken Cobol as the most popular
language.  As a standard, you couldn't wish for more.  But as a
medium of expression, you could do a lot better.  Of all the great
programmers I can think of, I know of only one who would voluntarily
program in Java.  And of all the great programmers I can think of
who don't work for Sun, on Java, I know of zero.Great hackers also generally insist on using open source software.
Not just because it's better, but because it gives them more control.
Good hackers insist on control.  This is part of what makes them
good hackers:  when something's broken, they need to fix it.  You
want them to feel this way about the software they're writing for
you.  You shouldn't be surprised when they feel the same way about
the operating system.A couple years ago a venture capitalist friend told me about a new
startup he was involved with.  It sounded promising.  But the next
time I talked to him, he said they'd decided to build their software
on Windows NT, and had just hired a very experienced NT developer
to be their chief technical officer.  When I heard this, I thought,
these guys are doomed.  One, the CTO couldn't be a first rate
hacker, because to become an eminent NT developer he would have
had to use NT voluntarily, multiple times, and I couldn't imagine
a great hacker doing that; and two, even if he was good, he'd have
a hard time hiring anyone good to work for him if the project had
to be built on NT. [2]The Final FrontierAfter software, the most important tool to a hacker is probably
his office.  Big companies think the function of office space is to express
rank.  But hackers use their offices for more than that: they
use their office as a place to think in.  And if you're a technology
company, their thoughts are your product.  So making hackers work
in a noisy, distracting environment is like having a paint factory
where the air is full of soot.The cartoon strip Dilbert has a lot to say about cubicles, and with
good reason.  All the hackers I know despise them.  The mere prospect
of being interrupted is enough to prevent hackers from working on
hard problems.  If you want to get real work done in an office with
cubicles, you have two options: work at home, or come in early or
late or on a weekend, when no one else is there.  Don't companies
realize this is a sign that something is broken?  An office
environment is supposed to be something that helps
you work, not something you work despite.Companies like Cisco are proud that everyone there has a cubicle,
even the CEO.  But they're not so advanced as they think; obviously
they still view office space as a badge of rank.  Note too that
Cisco is famous for doing very little product development in house.
They get new technology by buying the startups that created it-- where
presumably the hackers did have somewhere quiet to work.One big company that understands what hackers need is Microsoft.
I once saw a recruiting ad for Microsoft with a big picture of a
door.  Work for us, the premise was, and we'll give you a place to
work where you can actually get work done.   And you know, Microsoft
is remarkable among big companies in that they are able to develop
software in house.  Not well, perhaps, but well enough.If companies want hackers to be productive, they should look at
what they do at home.  At home, hackers can arrange things themselves
so they can get the most done.  And when they work at home, hackers
don't work in noisy, open spaces; they work in rooms with doors.  They
work in cosy, neighborhoody places with people around and somewhere
to walk when they need to mull something over, instead of in glass
boxes set in acres of parking lots.  They have a sofa they can take
a nap on when they feel tired, instead of sitting in a coma at
their desk, pretending to work.  There's no crew of people with
vacuum cleaners that roars through every evening during the prime
hacking hours.  There are no meetings or, God forbid, corporate
retreats or team-building exercises.  And when you look at what
they're doing on that computer, you'll find it reinforces what I
said earlier about tools.  They may have to use Java and Windows
at work, but at home, where they can choose for themselves, you're
more likely to find them using Perl and Linux.Indeed, these statistics about Cobol or Java being the most popular
language can be misleading.  What we ought to look at, if we want
to know what tools are best, is what hackers choose when they can
choose freely-- that is, in projects of their own.  When you ask
that question, you find that open source operating systems already
have a dominant market share, and the number one language is probably
Perl.InterestingAlong with good tools, hackers want interesting projects.  What
makes a project interesting?  Well, obviously overtly sexy
applications like stealth planes or special effects software would
be interesting to work on.  But any application can be interesting
if it poses novel technical challenges.  So it's hard to predict
which problems hackers will like, because some become
interesting only when the people working on them discover a new
kind of solution.  Before ITA
(who wrote the software inside Orbitz),
the people working on airline fare searches probably thought it
was one of the most boring applications imaginable.  But ITA made
it interesting by
redefining the problem in a more ambitious way.I think the same thing happened at Google.  When Google was founded,
the conventional wisdom among the so-called portals was that search
was boring and unimportant.  But the guys at Google didn't think
search was boring, and that's why they do it so well.This is an area where managers can make a difference.  Like a parent
saying to a child, I bet you can't clean up your whole room in
ten minutes, a good manager can sometimes redefine a problem as a
more interesting one.  Steve Jobs seems to be particularly good at
this, in part simply by having high standards.  There were a lot
of small, inexpensive computers before the Mac.  He redefined the
problem as: make one that's beautiful.  And that probably drove
the developers harder than any carrot or stick could.They certainly delivered.  When the Mac first appeared, you didn't
even have to turn it on to know it would be good; you could tell
from the case.  A few weeks ago I was walking along the street in
Cambridge, and in someone's trash I saw what appeared to be a Mac
carrying case.  I looked inside, and there was a Mac SE.  I carried
it home and plugged it in, and it booted.  The happy Macintosh
face, and then the finder.  My God, it was so simple.  It was just
like ... Google.Hackers like to work for people with high standards.  But it's not
enough just to be exacting.  You have to insist on the right things.
Which usually means that you have to be a hacker yourself.  I've
seen occasional articles about how to manage programmers.  Really
there should be two articles: one about what to do if
you are yourself a programmer, and one about what to do if you're not.  And the
second could probably be condensed into two words:  give up.The problem is not so much the day to day management.  Really good
hackers are practically self-managing.  The problem is, if you're
not a hacker, you can't tell who the good hackers are.  A similar
problem explains why American cars are so ugly.  I call it the
design paradox.  You might think that you could make your products
beautiful just by hiring a great designer to design them.  But if
you yourself don't have good taste,
how are you going to recognize
a good designer?  By definition you can't tell from his portfolio.
And you can't go by the awards he's won or the jobs he's had,
because in design, as in most fields, those tend to be driven by
fashion and schmoozing, with actual ability a distant third.
There's no way around it:  you can't manage a process intended to
produce beautiful things without knowing what beautiful is.  American
cars are ugly because American car companies are run by people with
bad taste.Many people in this country think of taste as something elusive,
or even frivolous.  It is neither.  To drive design, a manager must
be the most demanding user of a company's products.  And if you
have really good taste, you can, as Steve Jobs does, make satisfying
you the kind of problem that good people like to work on.Nasty Little ProblemsIt's pretty easy to say what kinds of problems are not interesting:
those where instead of solving a few big, clear, problems, you have
to solve a lot of nasty little ones.  One of the worst kinds of
projects is writing an interface to a piece of software that's
full of bugs.  Another is when you have to customize
something for an individual client's complex and ill-defined needs.
To hackers these kinds of projects are the death of a thousand
cuts.The distinguishing feature of nasty little problems is that you
don't learn anything from them.   Writing a compiler is interesting
because it teaches you what a compiler is.  But writing an interface
to a buggy piece of software doesn't teach you anything, because the
bugs are random.  [3] So it's not just fastidiousness that makes good
hackers avoid nasty little problems.  It's more a question of
self-preservation.  Working on nasty little problems makes you
stupid.  Good hackers avoid it for the same reason models avoid
cheeseburgers.Of course some problems inherently have this character.  And because
of supply and demand, they pay especially well.  So a company that
found a way to get great hackers to work on tedious problems would
be very successful.  How would you do it?One place this happens is in startups.  At our startup we had
Robert Morris working as a system administrator.  That's like having the
Rolling Stones play at a bar mitzvah.  You can't hire that kind of
talent.  But people will do any amount of drudgery for companies
of which they're the founders.  [4]Bigger companies solve the problem by partitioning the company.
They get smart people to work for them by establishing a separate
R&D department where employees don't have to work directly on
customers' nasty little problems. [5] In this model, the research
department functions like a mine. They produce new ideas; maybe
the rest of the company will be able to use them.You may not have to go to this extreme.
Bottom-up programming
suggests another way to partition the company: have the smart people
work as toolmakers.  If your company makes software to do x, have
one group that builds tools for writing software of that type, and
another that uses these tools to write the applications.  This way
you might be able to get smart people to write 99% of your code,
but still keep them almost as insulated from users as they would
be in a traditional research department.  The toolmakers would have
users, but they'd only be the company's own developers.  [6]If Microsoft used this approach, their software wouldn't be so full
of security holes, because the less smart people writing the actual
applications wouldn't be doing low-level stuff like allocating
memory.  Instead of writing Word directly in C, they'd be plugging
together big Lego blocks of Word-language.  (Duplo, I believe, is
the technical term.)ClumpingAlong with interesting problems, what good hackers like is other
good hackers.  Great hackers tend to clump together-- sometimes
spectacularly so, as at Xerox Parc.   So you won't attract good
hackers in linear proportion to how good an environment you create
for them.  The tendency to clump means it's more like the square
of the environment.  So it's winner take all.  At any given time,
there are only about ten or twenty places where hackers most want to
work, and if you aren't one of them, you won't just have fewer
great hackers, you'll have zero.Having great hackers is not, by itself, enough to make a company
successful.  It works well for Google and ITA, which are two of
the hot spots right now, but it didn't help Thinking Machines or
Xerox.  Sun had a good run for a while, but their business model
is a down elevator.  In that situation, even the best hackers can't
save you.I think, though, that all other things being equal, a company that
can attract great hackers will have a huge advantage.  There are
people who would disagree with this.  When we were making the rounds
of venture capital firms in the 1990s, several told us that software
companies didn't win by writing great software, but through brand,
and dominating channels, and doing the right deals.They really seemed to believe this, and I think I know why.  I
think what a lot of VCs are looking for, at least unconsciously,
is the next Microsoft.  And of course if Microsoft is your model,
you shouldn't be looking for companies that hope to win by writing
great software.  But VCs are mistaken to look for the next Microsoft,
because no startup can be the next Microsoft unless some other
company is prepared to bend over at just the right moment and be
the next IBM.It's a mistake to use Microsoft as a model, because their whole
culture derives from that one lucky break.  Microsoft is a bad data
point.  If you throw them out, you find that good products do tend
to win in the market.  What VCs should be looking for is the next
Apple, or the next Google.I think Bill Gates knows this.  What worries him about Google is
not the power of their brand, but the fact that they have
better hackers. [7]
RecognitionSo who are the great hackers?  How do you know when you meet one?
That turns out to be very hard.  Even hackers can't tell.  I'm
pretty sure now that my friend Trevor Blackwell is a great hacker.
You may have read on Slashdot how he made his
own Segway.  The
remarkable thing about this project was that he wrote all the
software in one day (in Python, incidentally).For Trevor, that's
par for the course.  But when I first met him, I thought he was a
complete idiot.  He was standing in Robert Morris's office babbling
at him about something or other, and I remember standing behind
him making frantic gestures at Robert to shoo this nut out of his
office so we could go to lunch.  Robert says he misjudged Trevor
at first too.  Apparently when Robert first met him, Trevor had
just begun a new scheme that involved writing down everything about
every aspect of his life on a stack of index cards, which he carried
with him everywhere.  He'd also just arrived from Canada, and had
a strong Canadian accent and a mullet.The problem is compounded by the fact that hackers, despite their
reputation for social obliviousness, sometimes put a good deal of
effort into seeming smart.  When I was in grad school I used to
hang around the MIT AI Lab occasionally. It was kind of intimidating
at first.  Everyone there spoke so fast.  But after a while I
learned the trick of speaking fast.  You don't have to think any
faster; just use twice as many words to say everything.  With this amount of noise in the signal, it's hard to tell good
hackers when you meet them.  I can't tell, even now.  You also
can't tell from their resumes.  It seems like the only way to judge
a hacker is to work with him on something.And this is the reason that high-tech areas
only happen around universities.  The active ingredient
here is not so much the professors as the students.  Startups grow up
around universities because universities bring together promising young
people and make them work on the same projects.  The
smart ones learn who the other smart ones are, and together
they cook up new projects of their own.Because you can't tell a great hacker except by working with him,
hackers themselves can't tell how good they are.  This is true to
a degree in most fields.  I've found that people who
are great at something are not so much convinced of their own
greatness as mystified at why everyone else seems so incompetent.
But it's particularly hard for hackers to know how good they are,
because it's hard to compare their work.  This is easier in most
other fields.  In the hundred meters, you know in 10 seconds who's
fastest.  Even in math there seems to be a general consensus about
which problems are hard to solve, and what constitutes a good
solution.  But hacking is like writing.  Who can say which of two
novels is better?  Certainly not the authors.With hackers, at least, other hackers can tell.  That's because,
unlike novelists, hackers collaborate on projects.  When you get
to hit a few difficult problems over the net at someone, you learn
pretty quickly how hard they hit them back.  But hackers can't
watch themselves at work.  So if you ask a great hacker how good
he is, he's almost certain to reply, I don't know.  He's not just
being modest.  He really doesn't know.And none of us know, except about people we've actually worked
with.  Which puts us in a weird situation: we don't know who our
heroes should be.  The hackers who become famous tend to become
famous by random accidents of PR.  Occasionally I need to give an
example of a great hacker, and I never know who to use.  The first
names that come to mind always tend to be people I know personally,
but it seems lame to use them.  So, I think, maybe I should say
Richard Stallman, or Linus Torvalds, or Alan Kay, or someone famous
like that.  But I have no idea if these guys are great hackers.
I've never worked with them on anything.If there is a Michael Jordan of hacking, no one knows, including
him.CultivationFinally, the question the hackers have all been wondering about:
how do you become a great hacker?  I don't know if it's possible
to make yourself into one.  But it's certainly possible to do things
that make you stupid, and if you can make yourself stupid, you
can probably make yourself smart too.The key to being a good hacker may be to work on what you like.
When I think about the great hackers I know, one thing they have
in common is the extreme
difficulty of making them work
on anything they
don't want to.  I don't know if this is cause or effect; it may be
both.To do something well you have to love it.
So to the extent you
can preserve hacking as something you love, you're likely to do it
well.  Try to keep the sense of wonder you had about programming at
age 14.  If you're worried that your current job is rotting your
brain, it probably is.The best hackers tend to be smart, of course, but that's true in
a lot of fields.  Is there some quality that's unique to hackers?
I asked some friends, and the number one thing they mentioned was
curiosity.
I'd always supposed that all smart people were curious--
that curiosity was simply the first derivative of knowledge.  But
apparently hackers are particularly curious, especially about how
things work.  That makes sense, because programs are in effect
giant descriptions of how things work.Several friends mentioned hackers' ability to concentrate-- their
ability, as one put it, to "tune out everything outside their own
heads.''  I've certainly noticed this.  And I've heard several
hackers say that after drinking even half a beer they can't program at
all.   So maybe hacking does require some special ability to focus.
Perhaps great hackers can load a large amount of context into their
head, so that when they look at a line of code, they see not just
that line but the whole program around it.  John McPhee
wrote that Bill Bradley's success as a basketball player was due
partly to his extraordinary peripheral vision.  "Perfect'' eyesight
means about 47 degrees of vertical peripheral vision.  Bill Bradley
had 70; he could see the basket when he was looking at the floor.
Maybe great hackers have some similar inborn ability.  (I cheat by
using a very dense language,
which shrinks the court.)This could explain the disconnect over cubicles.  Maybe the people
in charge of facilities, not having any concentration to shatter,
have no idea that working in a cubicle feels to a hacker like having
one's brain in a blender.  (Whereas Bill, if the rumors of autism
are true, knows all too well.)One difference I've noticed between great hackers and smart people
in general is that hackers are more
politically incorrect.  To the
extent there is a secret handshake among good hackers, it's when they
know one another well enough to express opinions that would get
them stoned to death by the general public.  And I can see why
political incorrectness would be a useful quality in programming.
Programs are very complex and, at least in the hands of good
programmers, very fluid.  In such situations it's helpful to have
a habit of questioning assumptions.Can you cultivate these qualities?  I don't know.  But you can at
least not repress them.  So here is my best shot at a recipe.  If
it is possible to make yourself into a great hacker, the way to do
it may be to make the following deal with yourself: you never have
to work on boring projects (unless your family will starve otherwise),
and in return, you'll never allow yourself to do a half-assed job.
All the great hackers I know seem to have made that deal, though
perhaps none of them had any choice in the matter.Notes
[1] In fairness, I have to say that IBM makes decent hardware.  I
wrote this on an IBM laptop.[2] They did turn out to be doomed.  They shut down a few months
later.[3] I think this is what people mean when they talk
about the "meaning of life."  On the face of it, this seems an
odd idea.  Life isn't an expression; how could it have meaning?
But it can have a quality that feels a lot like meaning.  In a project
like a compiler, you have to solve a lot of problems, but the problems
all fall into a pattern, as in a signal.  Whereas when the problems
you have to solve are random, they seem like noise.
[4] Einstein at one point worked designing refrigerators. (He had equity.)[5] It's hard to say exactly what constitutes research in the
computer world, but as a first approximation, it's software that
doesn't have users.I don't think it's publication that makes the best hackers want to work
in research departments.  I think it's mainly not having to have a
three hour meeting with a product manager about problems integrating
the Korean version of Word 13.27 with the talking paperclip.[6] Something similar has been happening for a long time in the
construction industry. When you had a house built a couple hundred
years ago, the local builders built everything in it.  But increasingly
what builders do is assemble components designed and manufactured
by someone else.  This has, like the arrival of desktop publishing,
given people the freedom to experiment in disastrous ways, but it
is certainly more efficient.[7] Google is much more dangerous to Microsoft than Netscape was.
Probably more dangerous than any other company has ever been.  Not
least because they're determined to fight.  On their job listing
page, they say that one of their "core values'' is "Don't be evil.''
From a company selling soybean oil or mining equipment, such a
statement would merely be eccentric.  But I think all of us in the
computer world recognize who that is a declaration of war on.Thanks to Jessica Livingston, Robert Morris, and Sarah Harlin
for reading earlier versions of this talk.


mod

December 2019There are two distinct ways to be politically moderate: on purpose
and by accident. Intentional moderates are trimmers, deliberately
choosing a position mid-way between the extremes of right and left.
Accidental moderates end up in the middle, on average, because they
make up their own minds about each question, and the far right and
far left are roughly equally wrong.You can distinguish intentional from accidental moderates by the
distribution of their opinions. If the far left opinion on some
matter is 0 and the far right opinion 100, an intentional moderate's
opinion on every question will be near 50. Whereas an accidental
moderate's opinions will be scattered over a broad range, but will,
like those of the intentional moderate, average to about 50.Intentional moderates are similar to those on the far left and the
far right in that their opinions are, in a sense, not their own.
The defining quality of an ideologue, whether on the left or the
right, is to acquire one's opinions in bulk. You don't get to pick
and choose. Your opinions about taxation can be predicted from your
opinions about sex. And although intentional moderates
might seem to be the opposite of ideologues, their beliefs (though
in their case the word "positions" might be more accurate) are also
acquired in bulk. If the median opinion shifts to the right or left,
the intentional moderate must shift with it. Otherwise they stop
being moderate.Accidental moderates, on the other hand, not only choose their own
answers, but choose their own questions. They may not care at all
about questions that the left and right both think are terribly
important. So you can only even measure the politics of an accidental
moderate from the intersection of the questions they care about and
those the left and right care about, and this can
sometimes be vanishingly small.It is not merely a manipulative rhetorical trick to say "if you're
not with us, you're against us," but often simply false.Moderates are sometimes derided as cowards, particularly by
the extreme left. But while it may be accurate to call intentional
moderates cowards, openly being an accidental moderate requires the
most courage of all, because you get attacked from both right and
left, and you don't have the comfort of being an orthodox member
of a large group to sustain you.Nearly all the most impressive people I know are accidental moderates.
If I knew a lot of professional athletes, or people in the entertainment
business, that might be different. Being on the far left or far
right doesn't affect how fast you run or how well you sing. But
someone who works with ideas has to be independent-minded to do it
well.Or more precisely, you have to be independent-minded about the ideas
you work with. You could be mindlessly doctrinaire in your politics
and still be a good mathematician. In the 20th century, a lot of
very smart people were Marxists  just no one who was smart about
the subjects Marxism involves. But if the ideas you use in your
work intersect with the politics of your time, you have two choices:
be an accidental moderate, or be mediocre.Notes[1] It's possible in theory for one side to be entirely right and
the other to be entirely wrong. Indeed, ideologues must always
believe this is the case. But historically it rarely has been.[2] For some reason the far right tend to ignore moderates rather
than despise them as backsliders. I'm not sure why. Perhaps it
means that the far right is less ideological than the far left. Or
perhaps that they are more confident, or more resigned, or simply
more disorganized. I just don't know.[3] Having heretical opinions doesn't mean you have to express
them openly. It may be
easier to have them if you don't.
Thanks to Austen Allred, Trevor Blackwell, Patrick Collison, Jessica Livingston,
Amjad Masad, Ryan Petersen, and Harj Taggar for reading drafts of this.


rootsoflisp

May 2001

(I wrote this article to help myself understand exactly
what McCarthy discovered.  You don't need to know this stuff
to program in Lisp, but it should be helpful to
anyone who wants to
understand the essence of Lisp  both in the sense of its
origins and its semantic core.  The fact that it has such a core
is one of Lisp's distinguishing features, and the reason why,
unlike other languages, Lisp has dialects.)In 1960, John
McCarthy published a remarkable paper in
which he did for programming something like what Euclid did for
geometry. He showed how, given a handful of simple
operators and a notation for functions, you can
build a whole programming language.
He called this language Lisp, for "List Processing,"
because one of his key ideas was to use a simple
data structure called a list for both
code and data.It's worth understanding what McCarthy discovered, not
just as a landmark in the history of computers, but as
a model for what programming is tending to become in
our own time.  It seems to me that there have been
two really clean, consistent models of programming so
far: the C model and the Lisp model.
These two seem points of high ground, with swampy lowlands
between them.  As computers have grown more powerful,
the new languages being developed have been moving
steadily toward the Lisp model.  A popular recipe
for new programming languages in the past 20 years
has been to take the C model of computing and add to
it, piecemeal, parts taken from the Lisp model,
like runtime typing and garbage collection.In this article I'm going to try to explain in the
simplest possible terms what McCarthy discovered.
The point is not just to learn about an interesting
theoretical result someone figured out forty years ago,
but to show where languages are heading.
The unusual thing about Lisp  in fact, the defining
quality of Lisp  is that it can be written in
itself.  To understand what McCarthy meant by this,
we're going to retrace his steps, with his mathematical
notation translated into running Common Lisp code.


siliconvalley

May 2006(This essay is derived from a keynote at Xtech.)Could you reproduce Silicon Valley elsewhere, or is there something
unique about it?It wouldn't be surprising if it were hard to reproduce in other
countries, because you couldn't reproduce it in most of the US
either.  What does it take to make a silicon valley even here?What it takes is the right people.  If you could get the right ten
thousand people to move from Silicon Valley to Buffalo, Buffalo
would become Silicon Valley.
[1]That's a striking departure from the past.  Up till a couple decades
ago, geography was destiny for cities.  All great cities were located
on waterways, because cities made money by trade, and water was the
only economical way to ship.Now you could make a great city anywhere, if you could get the right
people to move there.  So the question of how to make a silicon
valley becomes: who are the right people, and how do you get them
to move?Two TypesI think you only need two kinds of people to create a technology
hub: rich people and nerds.  They're the limiting reagents in the
reaction that produces startups, because they're the only ones
present when startups get started.  Everyone else will move.Observation bears this out: within the US, towns have become startup
hubs if and only if they have both rich people and nerds.  Few
startups happen in Miami, for example, because although it's full
of rich people, it has few nerds.  It's not the kind of place nerds
like.Whereas Pittsburgh has the opposite problem: plenty of nerds, but
no rich people.  The top US Computer Science departments are said
to be MIT, Stanford, Berkeley, and Carnegie-Mellon.  MIT yielded
Route 128.  Stanford and Berkeley yielded Silicon Valley.  But
Carnegie-Mellon?  The record skips at that point.  Lower down the
list, the University of Washington yielded a high-tech community
in Seattle, and the University of Texas at Austin yielded one in
Austin.  But what happened in Pittsburgh?  And in Ithaca, home of
Cornell, which is also high on the list?I grew up in Pittsburgh and went to college at Cornell, so I can
answer for both.  The weather is terrible,  particularly in winter,
and there's no interesting old city to make up for it, as there is
in Boston.  Rich people don't want to live in Pittsburgh or Ithaca.
So while there are plenty of hackers who could start startups,
there's no one to invest in them.Not BureaucratsDo you really need the rich people?  Wouldn't it work to have the
government invest in the nerds?  No, it would not.  Startup investors
are a distinct type of rich people.  They tend to have a lot of
experience themselves in the technology business.  This (a) helps
them pick the right startups, and (b) means they can supply advice
and connections as well as money.  And the fact that they have a
personal stake in the outcome makes them really pay attention.Bureaucrats by their nature are the exact opposite sort of people
from startup investors. The idea of them making startup investments
is comic.  It would be like mathematicians running Vogue-- or
perhaps more accurately, Vogue editors running a math journal.
[2]Though indeed, most things bureaucrats do, they do badly.   We just
don't notice usually, because they only have to compete against
other bureaucrats.  But as startup investors they'd have to compete
against pros with a great deal more experience and motivation.Even corporations that have in-house VC groups generally forbid
them to make their own investment decisions.  Most are only allowed
to invest in deals where some reputable private VC firm is willing
to act as lead investor.Not BuildingsIf you go to see Silicon Valley, what you'll see are buildings.
But it's the people that make it Silicon Valley, not the buildings.
I read occasionally about attempts to set up "technology
parks" in other places, as if the active ingredient of Silicon
Valley were the office space.  An article about Sophia Antipolis
bragged that companies there included Cisco, Compaq, IBM, NCR, and
Nortel.  Don't the French realize these aren't startups?Building office buildings for technology companies won't get you a
silicon valley, because the key stage in the life of a startup
happens before they want that kind of space.  The key stage is when
they're three guys operating out of an apartment.  Wherever the
startup is when it gets funded, it will stay.  The defining quality
of Silicon Valley is not that Intel or Apple or Google have offices
there, but that they were started there.So if you want to reproduce Silicon Valley, what you need to reproduce
is those two or three founders sitting around a kitchen table
deciding to start a company.  And to reproduce that you need those
people.UniversitiesThe exciting thing is, all you need are the people.  If you could
attract a critical mass of nerds and investors to live somewhere,
you could reproduce Silicon Valley.  And both groups are highly
mobile.  They'll go where life is good.  So what makes a place good
to them?What nerds like is other nerds.  Smart people will go wherever other
smart people are.  And in particular, to great universities.  In
theory there could be other ways to attract them, but so far
universities seem to be indispensable.  Within the US, there are
no technology hubs without first-rate universities-- or at least,
first-rate computer science departments.So if you want to make a silicon valley, you not only need a
university, but one of the top handful in the world.  It has to be
good enough to act as a magnet, drawing the best people from thousands
of miles away.  And that means it has to stand up to existing magnets
like MIT and Stanford.This sounds hard.  Actually it might be easy.  My professor friends,
when they're deciding where they'd like to work, consider one thing
above all: the quality of the other faculty.  What attracts professors
is good colleagues.  So if you managed to recruit, en masse, a
significant number of the best young researchers, you could create
a first-rate university from nothing overnight.  And you could do
that for surprisingly little.  If you paid 200 people hiring bonuses
of $3 million apiece, you could put together a faculty that would
bear comparison with any in the world.  And from that point the
chain reaction would be self-sustaining.  So whatever it costs to
establish a mediocre university, for an additional half billion or
so you could have a great one.
[3]PersonalityHowever, merely creating a new university would not be enough to
start a silicon valley. The university is just the seed.  It has
to be planted in the right soil, or it won't germinate.  Plant it
in the wrong place, and you just create Carnegie-Mellon.To spawn startups, your university has to be in a town that has
attractions other than the university.  It has to be a place where
investors want to live, and students want to stay after they graduate.The two like much the same things, because most startup investors
are nerds themselves.  So what do nerds look for in a town?  Their
tastes aren't completely different from other people's, because a
lot of the towns they like most in the US are also big tourist
destinations: San Francisco, Boston, Seattle.   But their tastes
can't be quite mainstream either, because they dislike other big
tourist destinations, like New York, Los Angeles, and Las Vegas.There has been a lot written lately about the "creative class." The
thesis seems to be that as wealth derives increasingly from ideas,
cities will prosper only if they attract those who have them.  That
is certainly true; in fact it was the basis of Amsterdam's prosperity
400 years ago.A lot of nerd tastes they share with the creative class in general.
For example, they like well-preserved old neighborhoods instead of
cookie-cutter suburbs, and locally-owned shops and restaurants
instead of national chains.  Like the rest of the creative class,
they want to live somewhere with personality.What exactly is personality?  I think it's the feeling that each
building is the work of a distinct group of people.  A town with
personality is one that doesn't feel mass-produced.  So if you want
to make a startup hub-- or any town to attract the "creative class"--
you probably have to ban large development projects.
When a large tract has been developed by a single organization, you
can always tell.
[4]Most towns with personality are old, but they don't have to be.
Old towns have two advantages: they're denser, because they were
laid out before cars, and they're more varied, because they were
built one building at a time.  You could have both now.  Just have
building codes that ensure density, and ban large scale developments.A corollary is that you have to keep out the biggest developer of
all: the government.  A government that asks "How can we build a
silicon valley?" has probably ensured failure by the way they framed
the question.  You don't build a silicon valley; you let one grow.NerdsIf you want to attract nerds, you need more than a town with
personality.  You need a town with the right personality.  Nerds
are a distinct subset of the creative class, with different tastes
from the rest.  You can see this most clearly in New York, which
attracts a lot of creative people, but few nerds.
[5]What nerds like is the kind of town where people walk around smiling.
This excludes LA, where no one walks at all, and also New York,
where people walk, but not smiling. When I was in grad school in
Boston, a friend came to visit from New York.  On the subway back
from the airport she asked "Why is everyone smiling?"  I looked and
they weren't smiling.  They just looked like they were compared to
the facial expressions she was used to.If you've lived in New York, you know where these facial expressions
come from.  It's the kind of place where your mind may be excited,
but your body knows it's having a bad time.  People don't so much
enjoy living there as endure it for the sake of the excitement.
And if you like certain kinds of excitement, New York is incomparable.
It's a hub of glamour, a magnet for all the shorter half-life
isotopes of style and fame.Nerds don't care about glamour, so to them the appeal of New York
is a mystery.  People who like New York will pay a fortune for a
small, dark, noisy apartment in order to live in a town where the
cool people are really cool.  A nerd looks at that deal and sees
only: pay a fortune for a small, dark, noisy apartment.Nerds will pay a premium to live in a town where the smart people
are really smart, but you don't have to pay as much for that.  It's
supply and demand: glamour is popular, so you have to pay a lot for
it.Most nerds like quieter pleasures.  They like cafes instead of
clubs; used bookshops instead of fashionable clothing shops; hiking
instead of dancing; sunlight instead of tall buildings.  A nerd's
idea of paradise is Berkeley or Boulder.YouthIt's the young nerds who start startups, so it's those specifically
the city has to appeal to.  The startup hubs in the US are all
young-feeling towns.  This doesn't mean they have to be new.
Cambridge has the oldest town plan in America, but it feels young
because it's full of students.What you can't have, if you want to create a silicon valley, is a
large, existing population of stodgy people.  It would be a waste
of time to try to reverse the fortunes of a declining industrial town
like Detroit or Philadelphia by trying to encourage startups.  Those
places have too much momentum in the wrong direction.  You're better
off starting with a blank slate in the form of a small town.  Or
better still, if there's a town young people already flock to, that
one.The Bay Area was a magnet for the young and optimistic for decades
before it was associated with technology.  It was a place people
went in search of something new.  And so it became synonymous with
California nuttiness.  There's still a lot of that there.  If you
wanted to start a new fad-- a new way to focus one's "energy," for
example, or a new category of things not to eat-- the Bay Area would
be the place to do it.  But a place that tolerates oddness in the
search for the new is exactly what you want in a startup hub, because
economically that's what startups are.  Most good startup ideas
seem a little crazy; if they were obviously good ideas, someone
would have done them already.(How many people are going to want computers in their houses?
What, another search engine?)That's the connection between technology and liberalism.  Without
exception the high-tech cities in the US are also the most liberal.
But it's not because liberals are smarter that this is so.  It's
because liberal cities tolerate odd ideas, and smart people by
definition have odd ideas.Conversely, a town that gets praised for being "solid" or representing
"traditional values" may be a fine place to live, but it's never
going to succeed as a startup hub.  The 2004 presidential election,
though a disaster in other respects, conveniently supplied us with
a county-by-county
map of such places.
[6]To attract the young, a town must have an intact center.  In most
American cities the center has been abandoned, and the growth, if
any, is in the suburbs.  Most American cities have been turned
inside out. But none of the startup hubs has: not San Francisco,
or Boston, or Seattle.  They all have intact centers.
[7]
My guess is that no city with a dead center could be turned into a
startup hub.  Young people don't want to live in the suburbs.Within the US, the two cities I think could most easily be turned
into new silicon valleys are Boulder and Portland.  Both have the
kind of effervescent feel that attracts the young.  They're each
only a great university short of becoming a silicon valley, if they
wanted to.TimeA great university near an attractive town.  Is that all it takes?
That was all it took to make the original Silicon Valley.  Silicon
Valley traces its origins to William Shockley, one of the inventors
of the transistor.  He did the research that won him the Nobel Prize
at Bell Labs, but when he started his own company in 1956 he moved
to Palo Alto to do it.   At the time that was an odd thing to do.
Why did he?  Because he had grown up there and remembered how nice
it was.  Now Palo Alto is suburbia, but then it was a charming
college town-- a charming college town with perfect weather and San
Francisco only an hour away.The companies that rule Silicon Valley now are all descended in
various ways from Shockley Semiconductor.  Shockley was a difficult
man, and in 1957 his top people-- "the traitorous eight"-- left to
start a new company, Fairchild Semiconductor.  Among them were
Gordon Moore and Robert Noyce, who went on to found Intel, and
Eugene Kleiner, who founded the VC firm Kleiner Perkins.  Forty-two
years later, Kleiner Perkins funded Google, and the partner responsible
for the deal was John Doerr, who came to Silicon Valley in 1974 to
work for Intel.So although a lot of the newest companies in Silicon Valley don't
make anything out of silicon, there always seem to be multiple links
back to Shockley.  There's a lesson here: startups beget startups.
People who work for startups start their own.  People who get rich
from startups fund new ones.  I suspect this kind of organic growth
is the only way to produce a startup hub, because it's the only way
to grow the expertise you need.That has two important implications.  The first is that you need
time to grow a silicon valley.  The university you could create in
a couple years, but the startup community around it has to grow
organically.   The cycle time is limited by the time it takes a
company to succeed, which probably averages about five years.The other implication of the organic growth hypothesis is that you
can't be somewhat of a startup hub.  You either have a self-sustaining
chain reaction, or not.  Observation confirms this too: cities
either have a startup scene, or they don't.  There is no middle
ground.  Chicago has the third largest metropolitan area in America.
As source of startups it's negligible compared to Seattle, number 15.The good news is that the initial seed can be quite small.  Shockley
Semiconductor, though itself not very successful, was big enough.
It brought a critical mass of experts in an important new technology
together in a place they liked enough to stay.CompetingOf course, a would-be silicon valley faces an obstacle the original
one didn't: it has to compete with Silicon Valley.  Can that be
done?  Probably.One of Silicon Valley's biggest advantages is its venture capital
firms.  This was not a factor in Shockley's day, because VC funds
didn't exist.  In fact, Shockley Semiconductor and Fairchild
Semiconductor were not startups at all in our sense.  They were
subsidiaries-- of Beckman Instruments and Fairchild Camera and
Instrument respectively.  Those companies were apparently willing
to establish subsidiaries wherever the experts wanted to live.Venture investors, however, prefer to fund startups within an hour's
drive.  For one, they're more likely to notice startups nearby.
But when they do notice startups in other towns they prefer them
to move.  They don't want to have to travel to attend board meetings,
and in any case the odds of succeeding are higher in a startup hub.The centralizing effect of venture firms is a double one: they cause
startups to form around them, and those draw in more startups through
acquisitions.  And although the first may be weakening because it's
now so cheap to start some startups, the second seems as strong as ever.
Three of the most admired
"Web 2.0" companies were started outside the usual startup hubs,
but two of them have already been reeled in through acquisitions.Such centralizing forces make it harder for new silicon valleys to
get started.  But by no means impossible.  Ultimately power rests
with the founders.  A startup with the best people will beat one
with funding from famous VCs, and a startup that was sufficiently
successful would never have to move.  So a town that
could exert enough pull over the right people could resist and
perhaps even surpass Silicon Valley.For all its power, Silicon Valley has a great weakness: the paradise
Shockley found in 1956 is now one giant parking lot.  San Francisco
and Berkeley are great, but they're forty miles away.  Silicon
Valley proper is soul-crushing suburban sprawl.  It
has fabulous weather, which makes it significantly better than the
soul-crushing sprawl of most other American cities.  But a competitor
that managed to avoid sprawl would have real leverage.  All a city
needs is to be the kind of place the next traitorous eight look at
and say "I want to stay here," and that would be enough to get the
chain reaction started.Notes[1]
It's interesting to consider how low this number could be
made.  I suspect five hundred would be enough, even if they could
bring no assets with them.  Probably just thirty, if I could pick them,
would be enough to turn Buffalo into a significant startup hub.[2]
Bureaucrats manage to allocate research funding moderately
well, but only because (like an in-house VC fund) they outsource
most of the work of selection.  A professor at a famous university
who is highly regarded by his peers will get funding, pretty much
regardless of the proposal.  That wouldn't work for startups, whose
founders aren't sponsored by organizations, and are often unknowns.[3]
You'd have to do it all at once, or at least a whole department
at a time, because people would be more likely to come if they
knew their friends were.  And you should probably start from scratch,
rather than trying to upgrade an existing university, or much energy
would be lost in friction.[4]
Hypothesis: Any plan in which multiple independent buildings
are gutted or demolished to be "redeveloped" as a single project
is a net loss of personality for the city, with the exception of
the conversion of buildings not previously public, like warehouses.[5]
A few startups get started in New York, but less
than a tenth as many per capita as in Boston, and mostly
in less nerdy fields like finance and media.[6]
Some blue counties are false positives (reflecting the
remaining power of Democractic party machines), but there are no
false negatives.  You can safely write off all the red counties.[7]
Some "urban renewal" experts took a shot at destroying Boston's
in the 1960s, leaving the area around city hall a bleak wasteland,
but most neighborhoods successfully resisted them.Thanks to Chris Anderson, Trevor Blackwell, Marc Hedlund,
Jessica Livingston, Robert Morris, Greg Mcadoo, Fred Wilson,
and Stephen Wolfram for
reading drafts of this, and to Ed Dumbill for inviting me to speak.(The second part of this talk became Why Startups
Condense in America.)


corpdev

January 2015Corporate Development, aka corp dev, is the group within companies
that buys other companies. If you're talking to someone from corp
dev, that's why, whether you realize it yet or not.It's usually a mistake to talk to corp dev unless (a) you want to
sell your company right now and (b) you're sufficiently likely to
get an offer at an acceptable price.  In practice that means startups
should only talk to corp dev when they're either doing really well
or really badly.  If you're doing really badly, meaning the company
is about to die, you may as well talk to them, because you have
nothing to lose. And if you're doing really well, you can safely
talk to them, because you both know the price will have to be high,
and if they show the slightest sign of wasting your time, you'll
be confident enough to tell them to get lost.The danger is to companies in the middle.  Particularly to young
companies that are growing fast, but haven't been doing it for long
enough to have grown big yet.  It's usually a mistake for a promising
company less than a year old even to talk to corp dev.But it's a mistake founders constantly make.  When someone from
corp dev wants to meet, the founders tell themselves they should
at least find out what they want.  Besides, they don't want to
offend Big Company by refusing to meet.Well, I'll tell you what they want.  They want to talk about buying
you.  That's what the title "corp dev" means.   So before agreeing
to meet with someone from corp dev, ask yourselves, "Do we want to
sell the company right now?"  And if the answer is no, tell them
"Sorry, but we're focusing on growing the company."  They won't be
offended.  And certainly the founders of Big Company won't be
offended. If anything they'll think more highly of you.  You'll
remind them of themselves.  They didn't sell either; that's why
they're in a position now to buy other companies.
[1]Most founders who get contacted by corp dev already know what it
means.  And yet even when they know what corp dev does and know
they don't want to sell, they take the meeting.  Why do they do it?
The same mix of denial and wishful thinking that underlies most
mistakes founders make. It's flattering to talk to someone who wants
to buy you.  And who knows, maybe their offer will be surprisingly
high.  You should at least see what it is, right?No.  If they were going to send you an offer immediately by email,
sure, you might as well open it.  But that is not how conversations
with corp dev work.  If you get an offer at all, it will be at the
end of a long and unbelievably distracting process.  And if the
offer is surprising, it will be surprisingly low.Distractions are the thing you can least afford in a startup.  And
conversations with corp dev are the worst sort of distraction,
because as well as consuming your attention they undermine your
morale.  One of the tricks to surviving a grueling process is not
to stop and think how tired you are.  Instead you get into a sort
of flow.
[2]
Imagine what it would do to you if at mile 20 of a
marathon, someone ran up beside you and said "You must feel really
tired.  Would you like to stop and take a rest?"  Conversations
with corp dev are like that but worse, because the suggestion of
stopping gets combined in your mind with the imaginary high price
you think they'll offer.And then you're really in trouble.  If they can, corp dev people
like to turn the tables on you. They like to get you to the point
where you're trying to convince them to buy instead of them trying
to convince you to sell.  And surprisingly often they succeed.This is a very slippery slope, greased with some of the most powerful
forces that can work on founders' minds, and attended by an experienced
professional whose full time job is to push you down it.Their tactics in pushing you down that slope are usually fairly
brutal. Corp dev people's whole job is to buy companies, and they
don't even get to choose which.  The only way their performance is
measured is by how cheaply they can buy you, and the more ambitious
ones will stop at nothing to achieve that. For example, they'll
almost always start with a lowball offer, just to see if you'll
take it. Even if you don't, a low initial offer will demoralize you
and make you easier to manipulate.And that is the most innocent of their tactics. Just wait till
you've agreed on a price and think you have a done deal, and then
they come back and say their boss has vetoed the deal and won't do
it for more than half the agreed upon price. Happens all the time.
If you think investors can behave badly, it's nothing compared to
what corp dev people can do.  Even corp dev people at companies
that are otherwise benevolent.I remember once complaining to a
friend at Google about some nasty trick their corp dev people had
pulled on a YC startup."What happened to Don't be Evil?" I asked."I don't think corp dev got the memo," he replied.The tactics you encounter in M&A conversations can be like nothing
you've experienced in the otherwise comparatively
upstanding world
of Silicon Valley.  It's as if a chunk of genetic material from the
old-fashioned robber baron business world got incorporated into the
startup world.
[3]The simplest way to protect yourself is to use the trick that John
D. Rockefeller, whose grandfather was an alcoholic, used to protect
himself from becoming one.  He once told a Sunday school class

  Boys, do you know why I never became a drunkard?  Because I never
  took the first drink.

Do you want to sell your company right now?  Not eventually, right
now.  If not, just don't take the first meeting.  They won't be
offended.  And you in turn will be guaranteed to be spared one of
the worst experiences that can happen to a startup.If you do want to sell, there's another set of
techniques
 for doing
that.  But the biggest mistake founders make in dealing with corp
dev is not doing a bad job of talking to them when they're ready
to, but talking to them before they are.  So if you remember only
the title of this essay, you already know most of what you need to
know about M&A in the first year.Notes[1]
I'm not saying you should never sell.  I'm saying you should
be clear in your own mind about whether you want to sell or not,
and not be led by manipulation or wishful thinking into trying to
sell earlier than you otherwise would have.[2]
In a startup, as in most competitive sports, the task at hand
almost does this for you; you're too busy to feel tired.  But when
you lose that protection, e.g. at the final whistle, the fatigue
hits you like a wave.  To talk to corp dev is to let yourself feel
it mid-game.[3]
To be fair, the apparent misdeeds of corp dev people are magnified
by the fact that they function as the face of a large organization
that often doesn't know its own mind.  Acquirers can be surprisingly
indecisive about acquisitions, and their flakiness is indistinguishable
from dishonesty by the time it filters down to you.Thanks to Marc Andreessen, Jessica Livingston, Geoff
Ralston, and Qasar Younis for reading drafts of this.


langdes

May 2001

(These are some notes I made
for a panel discussion on programming language design
at MIT on May 10, 2001.)1. Programming Languages Are for People.Programming languages
are how people talk to computers.  The computer would be just as
happy speaking any language that was unambiguous.  The reason we
have high level languages is because people can't deal with
machine language.  The point of programming
languages is to prevent our poor frail human brains from being
overwhelmed by a mass of detail.Architects know that some kinds of design problems are more personal
than others.  One of the cleanest, most abstract design problems
is designing bridges.  There your job is largely a matter of spanning
a given distance with the least material.  The other end of the
spectrum is designing chairs.  Chair designers have to spend their
time thinking about human butts.Software varies in the same way. Designing algorithms for routing
data through a network is a nice, abstract problem, like designing
bridges.  Whereas designing programming languages is like designing
chairs: it's all about dealing with human weaknesses.Most of us hate to acknowledge this.  Designing systems of great
mathematical elegance sounds a lot more appealing to most of us
than pandering to human weaknesses.  And there is a role for mathematical
elegance: some kinds of elegance make programs easier to understand.
But elegance is not an end in itself.And when I say languages have to be designed to suit human weaknesses,
I don't mean that languages have to be designed for bad programmers.
In fact I think you ought to design for the
best programmers, but
even the best programmers have limitations.  I don't think anyone
would like programming in a language where all the variables were
the letter x with integer subscripts.2. Design for Yourself and Your Friends.If you look at the history of programming languages, a lot of the best
ones were languages designed for their own authors to use, and a
lot of the worst ones were designed for other people to use.When languages are designed for other people, it's always a specific
group of other people: people not as smart as the language designer.
So you get a language that talks down to you.  Cobol is the most
extreme case, but a lot of languages are pervaded by this spirit.It has nothing to do with how abstract the language is.  C is pretty
low-level, but it was designed for its authors to use, and that's
why hackers like it.The argument for designing languages for bad programmers is that
there are more bad programmers than good programmers.  That may be
so.  But those few good programmers write a disproportionately
large percentage of the software.I'm interested in the question, how do you design a language that
the very best hackers will like?  I happen to think this is
identical to the question, how do you design a good programming
language?, but even if it isn't, it is at least an interesting
question.3. Give the Programmer as Much Control as Possible.Many languages
(especially the ones designed for other people) have the attitude
of a governess: they try to prevent you from
doing things that they think aren't good for you.  I like the
opposite approach: give the programmer as much
control as you can.When I first learned Lisp, what I liked most about it was
that it considered me an equal partner.  In the other languages
I had learned up till then, there was the language and there was my
program, written in the language, and the two were very separate.
But in Lisp the functions and macros I wrote were just like those
that made up the language itself.  I could rewrite the language
if I wanted.  It had the same appeal as open-source software.4. Aim for Brevity.Brevity is underestimated and even scorned.
But if you look into the hearts of hackers, you'll see that they
really love it.  How many times have you heard hackers speak fondly
of how in, say, APL, they could do amazing things with just a couple
lines of code?  I think anything that really smart people really
love is worth paying attention to.I think almost anything
you can do to make programs shorter is good.  There should be lots
of library functions; anything that can be implicit should be;
the syntax should be terse to a fault; even the names of things
should be short.And it's not only programs that should be short.  The manual should
be thin as well.  A good part of manuals is taken up with clarifications
and reservations and warnings and special cases.  If you force
yourself to shorten the manual, in the best case you do it by fixing
the things in the language that required so much explanation.5. Admit What Hacking Is.A lot of people wish that hacking was
mathematics, or at least something like a natural science.  I think
hacking is more like architecture.  Architecture is
related to physics, in the sense that architects have to design
buildings that don't fall down, but the actual goal of architects
is to make great buildings, not to make discoveries about statics.What hackers like to do is make great programs.
And I think, at least in our own minds, we have to remember that it's
an admirable thing to write great programs, even when this work
doesn't translate easily into the conventional intellectual
currency of research papers.  Intellectually, it is just as
worthwhile to design a language programmers will love as it is to design a
horrible one that embodies some idea you can publish a paper
about.1. How to Organize Big Libraries?Libraries are becoming an
increasingly important component of programming languages.  They're
also getting bigger, and this can be dangerous.  If it takes longer
to find the library function that will do what you want than it
would take to write it yourself, then all that code is doing nothing
but make your manual thick.  (The Symbolics manuals were a case in
point.)  So I think we will have to work on ways to organize
libraries.  The ideal would be to design them so that the programmer
could guess what library call would do the right thing.2. Are People Really Scared of Prefix Syntax?This is an open
problem in the sense that I have wondered about it for years and
still don't know the answer.  Prefix syntax seems perfectly natural
to me, except possibly for math.  But it could be that a lot of
Lisp's unpopularity is simply due to having an unfamiliar syntax.
Whether to do anything about it, if it is true, is another question.

3. What Do You Need for Server-Based Software?

I think a lot of the most exciting new applications that get written
in the next twenty years will be Web-based applications, meaning
programs that sit on the server and talk to you through a Web
browser.  And to write these kinds of programs we may need some
new things.One thing we'll need is support for the new way that server-based
apps get released.  Instead of having one or two big releases a
year, like desktop software, server-based apps get released as a
series of small changes.  You may have as many as five or ten
releases a day.  And as a rule everyone will always use the latest
version.You know how you can design programs to be debuggable?
Well, server-based software likewise has to be designed to be
changeable.  You have to be able to change it easily, or at least
to know what is a small change and what is a momentous one.Another thing that might turn out to be useful for server based
software, surprisingly, is continuations.  In Web-based software
you can use something like continuation-passing style to get the
effect of subroutines in the inherently
stateless world of a Web
session.  Maybe it would be worthwhile having actual continuations,
if it was not too expensive.4. What New Abstractions Are Left to Discover?I'm not sure how
reasonable a hope this is, but one thing I would really love to
do, personally, is discover a new abstraction-- something that would
make as much of a difference as having first class functions or
recursion or even keyword parameters.  This may be an impossible
dream.  These things don't get discovered that often.  But I am always
looking.1. You Can Use Whatever Language You Want.Writing application
programs used to mean writing desktop software.  And in desktop
software there is a big bias toward writing the application in the
same language as the operating system.  And so ten years ago,
writing software pretty much meant writing software in C.
Eventually a tradition evolved:
application programs must not be written in unusual languages.
And this tradition had so long to develop that nontechnical people
like managers and venture capitalists also learned it.Server-based software blows away this whole model.  With server-based
software you can use any language you want.  Almost nobody understands
this yet (especially not managers and venture capitalists).
A few hackers understand it, and that's why we even hear
about new, indy languages like Perl and Python.  We're not hearing
about Perl and Python because people are using them to write Windows
apps.What this means for us, as people interested in designing programming
languages, is that there is now potentially an actual audience for
our work.2. Speed Comes from Profilers.Language designers, or at least
language implementors, like to write compilers that generate fast
code.  But I don't think this is what makes languages fast for users.
Knuth pointed out long ago that speed only matters in a few critical
bottlenecks.  And anyone who's tried it knows that you can't guess
where these bottlenecks are.  Profilers are the answer.Language designers are solving the wrong problem.  Users don't need
benchmarks to run fast.  What they need is a language that can show
them what parts of their own programs need to be rewritten.  That's
where speed comes from in practice.  So maybe it would be a net
win if language implementors took half the time they would
have spent doing compiler optimizations and spent it writing a
good profiler instead.3. You Need an Application to Drive the Design of a Language.This may not be an absolute rule, but it seems like the best languages
all evolved together with some application they were being used to
write.  C was written by people who needed it for systems programming.
Lisp was developed partly to do symbolic differentiation, and
McCarthy was so eager to get started that he was writing differentiation
programs even in the first paper on Lisp, in 1960.It's especially good if your application solves some new problem.
That will tend to drive your language to have new features that
programmers need.  I personally am interested in writing
a language that will be good for writing server-based applications.[During the panel, Guy Steele also made this point, with the
additional suggestion that the application should not consist of
writing the compiler for your language, unless your language
happens to be intended for writing compilers.]4. A Language Has to Be Good for Writing Throwaway Programs.You know what a throwaway program is: something you write quickly for
some limited task.  I think if you looked around you'd find that
a lot of big, serious programs started as throwaway programs.  I
would not be surprised if most programs started as throwaway
programs.  And so if you want to make a language that's good for
writing software in general, it has to be good for writing throwaway
programs, because that is the larval stage of most software.5. Syntax Is Connected to Semantics.It's traditional to think of
syntax and semantics as being completely separate.  This will
sound shocking, but it may be that they aren't.
I think that what you want in your language may be related
to how you express it.I was talking recently to Robert Morris, and he pointed out that
operator overloading is a bigger win in languages with infix
syntax.  In a language with prefix syntax, any function you define
is effectively an operator.  If you want to define a plus for a
new type of number you've made up, you can just define a new function
to add them.  If you do that in a language with infix syntax,
there's a big difference in appearance between the use of an
overloaded operator and a function call.1. New Programming Languages.Back in the 1970s
it was fashionable to design new programming languages.  Recently
it hasn't been.  But I think server-based software will make new
languages fashionable again.  With server-based software, you can
use any language you want, so if someone does design a language that
actually seems better than others that are available, there will be
people who take a risk and use it.2. Time-Sharing.Richard Kelsey gave this as an idea whose time
has come again in the last panel, and I completely agree with him.
My guess (and Microsoft's guess, it seems) is that much computing
will move from the desktop onto remote servers.  In other words,
time-sharing is back.  And I think there will need to be support
for it at the language level.  For example, I know that Richard
and Jonathan Rees have done a lot of work implementing process
scheduling within Scheme 48.3. Efficiency.Recently it was starting to seem that computers
were finally fast enough.  More and more we were starting to hear
about byte code, which implies to me at least that we feel we have
cycles to spare.  But I don't think we will, with server-based
software.   Someone is going to have to pay for the servers that
the software runs on, and the number of users they can support per
machine will be the divisor of their capital cost.So I think efficiency will matter, at least in computational
bottlenecks.  It will be especially important to do i/o fast,
because server-based applications do a lot of i/o.It may turn out that byte code is not a win, in the end.  Sun and
Microsoft seem to be facing off in a kind of a battle of the byte
codes at the moment.  But they're doing it because byte code is a
convenient place to insert themselves into the process, not because
byte code is in itself a good idea.  It may turn out that this
whole battleground gets bypassed.  That would be kind of amusing.1. Clients.This is just a guess, but my guess is that
the winning model for most applications will be purely server-based.
Designing software that works on the assumption that everyone will
have your client is like designing a society on the assumption that
everyone will just be honest.  It would certainly be convenient, but
you have to assume it will never happen.I think there will be a proliferation of devices that have some
kind of Web access, and all you'll be able to assume about them is
that they can support simple html and forms.  Will you have a
browser on your cell phone?  Will there be a phone in your palm
pilot?  Will your blackberry get a bigger screen? Will you be able
to browse the Web on your gameboy?  Your watch?  I don't know.
And I don't have to know if I bet on
everything just being on the server.  It's
just so much more robust to have all the
brains on the server.2. Object-Oriented Programming.I realize this is a
controversial one, but I don't think object-oriented programming
is such a big deal.  I think it is a fine model for certain kinds
of applications that need that specific kind of data structure,
like window systems, simulations, and cad programs.  But I don't
see why it ought to be the model for all programming.I think part of the reason people in big companies like object-oriented
programming is because it yields a lot of what looks like work.
Something that might naturally be represented as, say, a list of
integers, can now be represented as a class with all kinds of
scaffolding and hustle and bustle.Another attraction of
object-oriented programming is that methods give you some of the
effect of first class functions.  But this is old news to Lisp
programmers.  When you have actual first class functions, you can
just use them in whatever way is appropriate to the task at hand,
instead of forcing everything into a mold of classes and methods.What this means for language design, I think, is that you shouldn't
build object-oriented programming in too deeply.  Maybe the
answer is to offer more general, underlying stuff, and let people design
whatever object systems they want as libraries.3. Design by Committee.Having your language designed by a committee is a big pitfall,
and not just for the reasons everyone knows about.  Everyone
knows that committees tend to yield lumpy, inconsistent designs.
But I think a greater danger is that they won't take risks.
When one person is in charge he can take risks
that a committee would never agree on.Is it necessary to take risks to design a good language though?
Many people might suspect
that language design is something where you should stick fairly
close to the conventional wisdom.  I bet this isn't true.
In everything else people do, reward is proportionate to risk.
Why should language design be any different?