WINNERINTEGRATED CIRCUITS

Sun’s
Big Splash
he Sunnyvale, Calif., campus of Sun
Microsystems Inc. is a quiet andpeaceful place with six low-rise buildings connected by tree-lined walkways.
But the tranquility masks a frightening reality—Sun is in serious economic trouble. The
company was badly splattered by the burst of
the dot-com bubble of 2000. Revenues forthis once towering colossus of the server
industry went south, and its stock plunged
from more than US $60 in 2000 to less than
$3 in 2002. Recently, the stock has been slowly
but steadily climbing, and at press time it was
selling at more than $5—a sign that the worst
may be over for the company.

T

56 IEEE Spectrum | January 2005 | NA

But Sun, headquartered in Santa Clara, Calif., is still
far from its glory days of the last decade. It could use asmall miracle to get back solidly on its feet, and at last
the company may have one: a new microprocessor chip
intended for the volume servers that are the heart ofdata centers running the information and Web processing for businesses, universities, hospitals, factories,
and the like. Sun’s engineers have had working chips
since last spring and are now heavily into testing anddebugging them and making design changes for the next
fabrication run in early 2005.
The server business generates $50 billion a year,
according to Jessica Yang, a research analyst at IDC,
Framingham, Mass., and Sun’s share recently is about
12 percent—down from 17 percent just four years ago.
Sun’s new chip, called Niagara for the torrent of dataand instructions that flow between the chip and its
memory, was designed from the ground up to do awaywith the impact of latency—the idle time a microprocessor spends waiting for data or instructions to arrive
from memory. This latency is one of the biggest impediments to the microprocessor’s ability to do real work.
Niagara was not conceived at Sun. It started life
in classic Silicon Valley fashion, as the brainchild ofa Sunnyvale start-up called Afara Websystems Inc.

ROBERT HOUSER

THE NIAGARA MICROPROCESSOR CHIP IS SUN’S BESTHOPE FOR A COMEBACK BY LINDA GEPPERTCHIP TO COME:
Few thought that
Kunle Olukotun’s
idea of putting
multiple threads intoa microprocessor
would lead to better
server performance.
Sun Microsystems
believed and is readying the Niagara chip[shown] for a newline of servers.

January 2005 | IEEE Spectrum | INT 57

“ ‘Afara’ means ‘bridge’ in the west African language Yoruba,” explains
Stanford University professor Kunle Olukotun, one of the company’s
founders [see photo, “Chip to Come”]. Completing the founding trio are
Les Kohn, a microprocessor guru who has designed microprocessors
for both Sun and Intel Corp., and industry insider Fermi Wang. The company did not plan to sell microprocessors but instead to sell complete
servers built around its approach to microprocessor design.
When funding all but vanished after the terrorist attacks of 9/11 andthe recession that began in 2001, Afara began negotiations with Sun, andin 2002, the server giant acquired the start-up.

NIAGARA DIVERGES FROM OTHER MICROPROCESSORS in the way it pro-data is. If it’s in the high-speed on-chip cache memory, the wait could be only a few clock cycles. But if
the data is not in the cache, its retrieval from off-chipmain memory may take hundreds or even thousandsof clock cycles. It’s no wonder that minimizing thiscrippling latency is perhaps the most important challenge facing microprocessor designers.
To improve throughput, designers have created evermore complex mechanisms, such as executing some
instructions in a different order from the way they occur
in the instruction stream (out-of-order execution) or
beginning the execution of instructions that may never
be needed (speculative execution). “But no matter what
you do,” says Fred DeSantis, vice president of engineering for Horizontal Systems at Sun, “eventually you
still have to go to memory.” And wait.
Niagara’s architects adopted a different approach—
one that had been kicking around in the corridors ofuniversity engineering departments for decades. The
idea, called multithreading, consists of dividing the
instruction stream into several smaller streams, known
as threads. The concept was first developed for Control
Data Corp.’s CDC 6600 supercomputer in the 1960s.
In the Niagara design, each pipeline can handle four
threads. In each cycle, the pipeline begins the executionof an instruction from a different thread. So, for example, when an instruction from thread one is at stage three
in the pipeline, an instruction from thread two will beat stage two, and one from yet a different thread will
be at stage one.
If the pipeline cannot continue to execute the threadone instruction because it needs data from memory, it

Clock cycle

cesses instructions—the individual commands that come from software
applications like databases and spreadsheets. These applications enter
the microprocessor as a stream of instructions, which the microprocessor executes. The instructions tell the microprocessor what data to operate on, what operation to perform on them, and what to do with theresult. The instructions travel through a series of circuits called a pipeline,
which resembles an automobile assembly line.
Each stage of the pipeline performs one step of the instruction execution every clock cycle. In the first stage of the Niagara design, the pipeline
gets an instruction from memory, an ADD instruction, for example. The second stage selects that instruction for execution. The third stage determines
what kind of instruction it is—in this case, an ADD instruction. The fourthstage executes the instruction. The fifth stage is used for getting data frommemory. ADD instructions, however, do not access memory, but get theirdata from registers. So the instruction passes through the memory stage
and on to the final stage, during which the pipeline writes the results of theoperation back into a register.
Every clock cycle, a new instruction enters the pipeline. If all goes well,
the instructions march through the pipeline in lock step, and one instruction is completed with each tick of the microprocessor’s clock. So a sixstage pipeline can have six instructions at different
stages of execution at one time.
The rate at which a microprocessor executes instrucThread 1
Thread 2
Thread 3
Thread 4
tions is the most important measure of its performance. It is simply the product of its clock frequency and
PIPELINE
PIPELINE
the number of instructions it executes in each clock
Stage 12
3
4
5
6 Stage 123456cycle. Ever since the first microprocessor was invented
22
1
1
1 Instruction 1
2
2
1
1
1
1
in the early 1970s, architects have been on a relentless
1
scramble to improve performance.
Typically, the designers have pursued their goal on
2
2
2
1
1
1
2
2
2
1
1
1
two fronts: increasing the clock frequency and in2
creasing the number of instructions the processor can
2
2
2
2
1
1
3
2
2
2
1
execute in one clock cycle. Thanks largely to the semi3
conductor industry’s ability to produce ever smallertransistors, clock frequencies have soared over the past
3
3
2
2
2
1
3
2
2
2
2
1
30-plus years by five orders of magnitude—from tens
4of kilohertz to more than 4 gigahertz.
To increase the number of instructions per clock
cycle, architects have added more pipelines. Some microprocessors today,
THREADS ON THE MARCH: The Niagara microprocessor’s
such as Intel’s Pentium 4, have eight pipelines, allowing these chips, in prinsix-stage pipeline operates on six instructions at a time from four
different instruction threads [color-coded]. At each clock cycle,
ciple, to complete eight instructions in parallel during a single clock cycle.
Niagara is working on at least one and sometimes two instructions
In the current generation of microprocessors, some designs have gonefrom each thread in different pipeline stages. For example, in clock
a step further and put the essential elements of two microprocessors on one
cycle 1 [left, top], instruction 1 of the yellow thread is in the final
piece of silicon. Such a dual-core microprocessor, with eight pipelines instage [No. 6] of the pipeline. Each clock cycle, the instructions
each core running at 4 GHz, could pump out an astounding 64 billion instrucmove one stage forward in the pipeline [to the right in the diagram].
All processing need not stop and wait when information musttions per second if it could complete one instruction per pipeline per cycle.
be fetched from main memory—as in a microprocessor with a sinBut unfortunately, such microprocessors are rarely so efficient. If an
gle thread. See, for example, instruction 1 in the blue thread [right].
instruction in a pipeline needs data from memory, it has to wait until theInstead, Niagara does not execute any more instructions from thedata arrives before it can actually execute the instruction. So pipelines spend
blue thread until the needed information has arrived from memory.
But, in the meantime, it continues to execute instructions from thea lot of time stuck in neutral—as much as 85 percent of the time, in fact.
three other threads.
The length of the individual delays depends on where the sought-after

58 IEEE Spectrum | January 2005 | NAstores the information about the stalled instruction in a special type of onchip memory called a register file. At the same time, it continues with theexecution of the thread-two instruction, rotating among the three threads
that are available [see diagram, “Threads on the March”]. Then, when theneeded data for the other thread becomes available, the pipeline jumps backto that thread, using the register file to pick up exactly where it left off.
In conventional microprocessors, architects obtain multigigahertz
speeds by increasing the number of stages in the pipeline. Basically, a processing step that could be completed in one clock cycle in a slower pipeline
now needs two or more clock cycles to do the same job in a faster chip.
But because of Niagara’s ability to keep its pipelines running almost all of
the time, the architects do not have to run the microprocessor at multigigahertz speeds in order to get good performance. And the slower speeds
translate to a simpler pipeline. The simpler pipeline and lower frequency
let Niagara run at much lower power than comparable microprocessors.
The set of instructions that a microprocessor executes defines its
architecture. Designs based on that architecture may differ in many ways,
such as the size of their memories or the number of pipelines, but if they
execute the same set of instructions, they are of the same architecture.
Afara chose to use the Sparc architecture, developed by Sun, as the starting point of its design because of the many applications that run on it,
explains Olukotun. Sparc is the basis for most of the microprocessors
in Sun products.
“Afara’s approach was to build a very simple, straightforward pipeline,
maximize the number of threads the pipeline can efficiently handle, put
it into a microprocessor core, and then maximize the number of cores you
can put on a chip,” says DeSantis. Each Niagara chip has eight cores, which
use the Sparc instruction set, and the pipeline in each core can switch amongfour threads. So a single Niagara chip can handle 32 threads and execute
eight instructions per clock cycle [see figure, “Built for Speed”]. The chip
rarely wastes time waiting for data to come back from memory.

SUN HAS LONG SEEN THE MERITS of multicore, multithreaded microprocessors. The company’s first such design was called Majc, developed
for network communications in the late 1990s [see “Microprocessors: The
Off-beat Generation,” IEEE Spectrum, July 2000]. At the time of the Afara
acquisition, Sun engineers were at work on another such design, codenamed Honeybee. Clearly it made little sense for Sun engineers to pursue two similar designs slated for the same applications.
Both the Afara design and Honeybee were the responsibility of DeSantis.
Within weeks of the acquisition, upper management and key technicaldevelopers decided to pursue the Afara design, now code-named Niagara.
But Sun is not the only company to pursue a multithreaded approach.
One example is a recent version of Pentium, from Intel Corp., in Santa
Clara, Calif., that has two-way multithreading, which Intel calls hyperthreading. “The difference,” says Poonacha Kongetira, a senior engineering
manager at Sun, “is that Intel’s chips are first designed for a single thread,because that is consistent with desktop microprocessors, their main business. Then they extend the design to add another thread.”
Niagara, on the other hand, was optimized for multithreading fromthe very beginning. “It’s not meant for desktop applications,” says
Kongetira, “but for commercial workloads like databases and Web
servers—applications with a lot of independent queries that can be easily broken up into threads.”
To further reduce latency, Niagara’s architects gave the chip a fat, fastinterface between the processor cores and its two different kinds of onchip memory. Each chip has two levels of memory. Every core has its own
level-one memory, and the whole group of cores share a larger, level-two
memory. The level-two memory can send data to the cores at an aggregate rate of 100 gigabytes per second.
The idea for what became Niagara came to Olukotun in 2000, when hebegan talking with people who were running data centers. Servers werebeing equipped with ever more power-hungry microprocessors, and data-

NIAGARAGOAL: Design and build a microprocessor that works 10 times as
efficiently as existing devices for processing applications and at
half the power consumption of comparable devices
WHY IT’S A WINNER: Data center operators, with their thousandsof power-hungry servers, should find Niagara’s power and processing efficiency to be just what the doctor ordered
ORGANIZATION: Sun Microsystems Inc.
CENTER OF ACTIVITY: Sunnyvale, Calif.
NUMBER OF PEOPLE ON THE PROJECT: 100
BUDGET: Confidential

center managers “were complaining that they were running out of power and space,” he says.
Olukotun realized that multiple processors on a chip,
multithreading, and data centers were made for eachother. A data center contains row upon row of racks,
each about the size of a home refrigerator. Each rackholds anywhere from one server to hundreds, depending on their type, size, and power consumption. Data
centers need to do many things efficiently at the sametime, which is exactly the point of multithreading.
In endless rounds of discussions with venture capitalists, Olukotun swore that his approach would
improve server performance by a factor of 10 over what
Intel could do at the time.
Not surprisingly, few believed him, Olukotun told
Spectrum. They argued that microprocessor design
is mature and that it would be difficult to competewith the incumbents. But he stubbornly stayed onmessage: “Here’s a market that needs a new approach, and it’s an approach that none of the incumbents is taking,” he remembers saying. And theapproach fitted nicely with the capabilities of a start-up,
because it could vastly improve performance whileat the same time making the design simpler thanthat of conventional microprocessors.
Early in the process of establishing the start-up, theventure capitalists introduced Olukotun to Les Kohn.
“That was critical,” Olukotun recalls, “because he wasa master microprocessor architect and we meshed reallywell.” Kohn told him that he was done designing microprocessors. “But I managed to convince him that hereally needed to do just one more.”

WITH NIAGARA, KONGETIRA CAME FULL CIRCLE. He had
left Sun in 2000 to go to Afara and came back to Sun
after the acquisition. He remembers how closely the
Afara team worked together: “If a problem came up,
it was like a red flag to a bull,” he told Spectrum. “Fiveguys would be sitting in their cubicles thinking about
the problem, and one of them would pop his head up
over his cubicle wall with a possible solution. Then
there would be a discussion, and if it didn’t work,
the heads would go back down again.”
No microprocessor had ever pushed multithreading as far as Niagara would, so no designers had everfaced some of the problems that Olukotun and his
team did. One of their biggest challenges was making sure there were no bottlenecks in the flow of data
January 2005 | IEEE Spectrum | NA 59

Memory
controllers
and I/O

Crossbar switch

Multithreaded cores

Cache memory

BUILT FOR SPEED: Niagara has eight
Sparc microprocessor cores, each witha single pipeline processing instructions fromfour threads and completing one instruction ineach cycle. All together, the chip handles 32 threads.
Each core also has its own level-one cachememory, and they are connected to a sharedsecond-level cache via a high-speed crossbar switch.
On-chip memory controllers direct the flow of data
between the chip and off-chip memory.

between memory and the pipelines. “It’s a bandwidth game,” Olukotun
explains. “It follows the dictum that you can always buy bandwidth,
but latency is fixed by the speed of light, and you can’t bribe God.”
Early on at Afara, progress on the microprocessor was swift. It took
less than three months from the time they started designing the chip until
they were able to execute instructions through a software model of thepipeline. But then in 2001, funding started to dry up. Olukotun remembers ruefully how he, Kohn, and Wang were supposed to meet withpotential backers in the World Trade Center in New York City in thelate summer of 2001. But on the day of the meeting—11 September—they
learned from a taxi driver that a plane had hit the World Trade Center.
“We could see the smoke coming out of the WTC,” recalls Olukotun. Afterbeing stuck in a motel for two days, they finally got to meet with theirpotential funders. “And in the end, I think those guys signed on.”Afara’s plan to sell servers based on the new chip meant that the startup needed a lot of money to get to market. But after 9/11, the funding climate turned foul. Venture capitalists were steering clear of huge projects.
That’s when Afara began negotiations with Sun.
They found an early champion in Rick Hetherington, currently Sun’schief architect for Horizontal Systems, which develops and sells servers fordata centers and Web systems. Like Sun’s engineering manager Kongetira,Hetherington had left Sun to join a start-up. His start-up was funded by
Sequoia Capital, the same venture capital firm that was funding Afara. So
when Sequoia invited Hetherington to meet with Olukotun, he did so andwas impressed with Afara’s technology.
After Hetherington’s start-up foundered, he returned to Sun, which wasthen considering acquiring Afara. He started writing memos and sending
e-mails to Mike Splain, chief technology officer for the processor group,and other senior managers, urging the acquisition. “I told them, ‘We’ve got
to get this technology in and build it,’ ” he recalls.
After the acquisition, Hetherington put together a road map for thedevelopment of subsequent microprocessors based on Niagara. He would
not elaborate on the details, but apparently the development of Niagara 2,
the second generation of the design, is already staffed and under way.
“Acquisitions are tough to do,” admits Hetherington. Although Afara
was the first Sun acquisition he was directly involved with, he has noticed
that relatively few such acquisitions succeed. “Only about one in four makeit from beginning to end and turn into successful products,” he notes.
Any number of things might have derailed the Afara acquisition. First,
Afara was planning to have its chips made at TSMC, the huge Tai-

60 IEEE Spectrum | January 2005 | NA

I/O andsharedfunctionswanese chip-making foundry. But Dallas-based Texas
Instruments builds Sun’s microprocessors. It wasn’t
just a problem of loyalties: every foundry has its own
circuit libraries and design rules, and a change infoundries would mean a change in the design.
Second, Afara was planning to use Linux as the operating system, while Sun’s microprocessors run Solaris,
the company’s proprietary operating system. In manycases, too, Afara was using computer-aided-design software products that were different from Sun’s, forcing
decisions over which software to use. And then there
was the merging of two different corporate cultures.
Sun got its way on the foundry and the operating
system issues. The two groups compromised on thedesign software, using some of Sun’s software and somethat the Afara designers had used.
“Once we got past those issues, things started toclick,” adds Allan Strong, senior engineering director
for Niagara, who came to Sun with the Afara acquisition. For Strong, the high point in getting the new chip
to market was when the first packaged chips came backfrom the foundry last spring. Engineers quickly fired up
the chips and were able to run the operating systemon them. “Until that happens, you never really know,”
Strong says. “You’ve done the simulations, but all ittakes is one missing rectangle” out of untold millions
on one of the masks to make the chip fail.
When the chip is rolled out, Niagara will go into a newline of Sun’s Web and application servers. The company
plans to sell them as single servers and as two servers
combined into a single chassis, says Hetherington. They
will run in large systems that are built up by connecting
many servers together. The difference between this typeof system, which Sun calls “horizontal,” and large multiprocessor systems, which the company calls “vertical,” isthat in a horizontal system, each server runs its own copyof the operating system and has its own memory. In a
vertical system, on the other hand, servers run a single
operating system and share memory.
The company expects Niagara systems to be available in 2006. And if the chips hold up to expectations,■
we could see Sun rising once again.