Intelligent RAM (IRAM): The Industrial Setting, Applications and Architectures

The goal of Intelligent RAM (IRAM) is to design a cost-effective computer by designing a processor in a memory fabrication process, instead of in a conventional logic fabrication process, and include memory on-chip. To design a processor in a DRAM process one must learn about the business and culture of the DRAMs, which is quite different from microprocessors. We describe some of those differences, and then our current vision of IRAM applications, architectures, and implementations. 1. Potential and Challenges of IRAM Intelligent RAM (IRAM) may lead to a different style of computer than those based on conventional microprocessors. IRAM technology offers the following potential: • Improve memory latency by factors of 5 to 10 and memory bandwidth by factors of 50 to 100, by redesigning the memory interface and exploiting the proximity of on-chip memory [1][2]; • Improve energy efficiency of memory by factors of 2 to 4, primarily by going off-chip less frequently [3][4]; • Reduce design effort tenfold by filling the die with replicated memory rather than with custom logic ; • Make the memory size and organization fit the intended workload; • Reduce board area by factors of 4 or much greater by integrating many components on a single chip; and • Improve I/O bandwidth by factors of 4 to 8 by replacing the conventional I/O bus with multiple high-speed, point-to-point, serial lines.  This list makes IRAM an exciting opportunity. One IRAM challenge is matching the performance of microprocessors. Performance obstacles include: • IRAM is fabricated in a process that has been oriented towards small memory size and low charge leakage rather than fast transistor speed; • This same DRAM fabrication process offers fewer metal layers than a logic process to lower costs since routing speed is less of an issue in a memory; • DRAMs are designed to work in plastic packages and dissipate less than 2 watts, while desktop microprocessors dissipate 20 to 50 watts using ceramic packages; • DRAM refresh rates go up with operating temperature, approximately doubling for every 10 degrees C raise; • Some applications may not fit within the on-chip memory of an IRAM, and hence IRAMs must access either conventional DRAMs or other IRAMs over a much slower path than on-chip accesses. Another major IRAM challenge is matching the cost of DRAM memory. Cost obstacles include: • DRAMs include redundant memory so that fabrication flaws can be circumvented to improve yield and therefore lower cost. Microprocessors traditionally have no redundant logic to improve yield. Hence the on-chip logic may effectively determine the yield of the IRAM. • Testing time affects chip costs. Given both logic and DRAM on a the same die, an IRAM die may need to be tested on both logic and memory testers. • To help close the performance gap for logic in a DRAM process, merged-logic DRAM processes are being created with faster transistors and more metal layers which increasing the cost per wafer by 10% to 30%. The business model for IRAM also has challenges. Although an IRAM may be classified as a single chip computer and sold like desktop or embedded microprocessors, the initial companies most interested in pursuing IRAMs are DRAM companies, and they generally have little experience in the microprocessor market. Some challenges are: • DRAMs are “generic” parts, used in many places without impacting the software. Putting a processor in the DRAM limits the software that can run on the IRAM. • The DRAM economic model depends on producing a very high volume of parts––billions of DRAMs are made each year––while some microprocessors sell less than a million per year. • DRAMs companies do not need to worry about a supply of support and application software for their chips. IRAM would change that requirement. This paper first goes into more depth on the DRAM industry to motivate initial solutions to the IRAM challenges, looks at potential IRAM applications and architectures, and then concludes with our target implementation alternatives that could be taped out in 1999. 2. DRAM and Microprocessor Industries Figure 1 highlights some of the differences between the DRAM industry and the desktop microprocessor industry. DRAM companies agree on new standard interfaces for new generations and configurations of DRAMs. These standards include almost everything: pinout, package, addressing, refresh rates, and so on. Each microprocessor manufacturer generally sets their own instruction set standards to ensure software compatibility with prior generations, but is free to invent new interfaces with different packages and pins, different memory interfaces, and so on. Whereas microprocessors follow their own architecture standards with varying implementations over time, DRAM manufacturers standardize at the package level and innovate in the size of the memory cell and in efficiency of manufacturing process. 2.1. Differing Design Targets Not only do the multiple source versus single source business model affect the design of the chips, the figures of merit vary between the two cultures. DRAM designers pride themselves on improving storage capacity per chip by fourfold every three years (60%/year) and by having the smallest memory cell so as to have the lowest cost per bit. The capacity increases are generally achieved by reducing cell size by about a factor of 2.5 and increasing the die size by a factor of 1.5. The increasing die size is a major reason that the cost per bit changes more slowly than the capacity per chip. A secondary design target is bandwidth in the fast access mode, and the trailing concern is latency to access a random bit in memory. A DRAM company’s business goal is typically to supply 10% of a single DRAM generation. As there were 6.25 billion DRAMs shipped in 1996, such an apparently modest target can lead to hundreds of millions of chips. Desktop microprocessor designers tend to have a design cost target, expressed as die size, and then build the fastest chip they can for that size. Microprocessor volumes have more to do with an instruction set target than with the actual final performance, but given that microprocessor designers generally do not get to pick the instruction set, they aim for the highest performance. Of secondary importance is the cost. Recently the power dissipation has become so high that it is now a concern. Embedded microprocessor designers have much lower cost targets and power budgets, and more likely to sacrifice performance to ensure meeting the cost/power budgets than designers of desktop microprocessors. The different figures of merit for memory designers and microprocessor designers have resulted in a performance gap between processor and memory in computer systems. The primary approach to bridging this gap has been increasing the amount of SRAM on a microprocessor to act as a cache. Today, many microprocessors dedicate between one-third and two-thirds of the area on chip to these caches.[2] Moreover, today there are often external SRAM chips to build secondary caches. Such chips add to cost and increase board area. 2.2. Differing Generation Strategies Traditionally, DRAM manufacturers would design a new memory cell and a new fabrication process simultaneously. The company then produces tens of thousands of “engineering samples” until both the fabrication process and memory cell design are fully “characterized.” Characterization means that the resulting dies will operate at minimum refresh rates over the full temperature range supplying data with acceptable bit error rates. Once characterized, the subsequent chips are at the “first customer ship” milestone. There may also be a separate milestone of “mass production” when the part achieves the high volume that DRAM manufacturers strive for. Given that all DRAM manufacturers use the same semiconductor fabrication equipment and same wafers, the time to these milestones can determine what share of the market a company will achieve. DRAM Microprocessor Standards pinout, package, refresh rate, addressing, capacity, width, fast transfer mode, failure rate binary compatibility,  Floating Point, I/O bus Sources multiple single Key figures of merit 1) capacity, cost/bit 2) bandwidth 3) latency 1) performance on standard benchmarks 2) cost Rate of improvement 1) 60%/year, 25%/year 2) 20%/year 3) 7%/year 1) 60%/year 2) little change Figure 1. Business models of DRAM and desktop microprocessor industries. The size of the die, testing time, and yield determine profit of a company that has a sizeable market share. As a result, DRAM manufacturers are much more secretive about Spice parameters and design rules than microprocessor companies. To lower costs they shrink the die to increase the number of chips per wafer and improve the fabrication process to improve yield. As they better understand a process, they will reduce the testing time and may even reduce the number of spare rows and columns to get slightly smaller dies. DRAMs typically go through 3 to 4 generations of die sizes over a 4 to 6 year lifetime. Recently, DRAM manufactures have separated the process and memory cell size from the capacity of the die. Hence the same line might make third generation 64 Mbit and first generation 256 Mbit parts depending on the demands of the market. Today, it makes more sense today to talk about the generations of memory cell size and process rather than just the generation of, say, a 64 Mbit part. Once in mass production, DRAM die yields below 60% are considered disastrous. Such high yields comes from small die, low defect density, and using redundant rows and columns to repair some flaws. Although real yields are closely guarded secrets, yields of 80% or 90% are apparently achieved by some efficient manufacturers. Microprocessor manufacturers generally are not as tightly tied to the fabrication process as are DRAM designers. In fact, there are several “fabless” microprocessor manufacturers, but no