RETROSPECTIVE: Interconnections in Multi-Core Architectures: Understanding Mechanisms, Overheads, and Scaling

By summer of 2004, commercial multi-core processors had just started appearing in the market. These were dual-core processors designed to improve the throughput of processors. There was considerable excitement as it was apparent to many that most future processors were going to be multi-core. It was also clear that the number of cores on a multi-core processor would increase some were predicting tens of processors on the same chip. Putting a large number of cores on the same chip presented a wide variety of problems and opportunities. What should each core look like? How should the memory hierarchy of a multi-core processor be organized? How should the cores be connected? How should the cores interact with each other to support coherence and consistency at low cost? How should one build software and hardware to ensure high utilization of the cores? A flurry of work had started both in industry and academia researching different aspects of multicore processor architecture, design, and programming. At that time, we had been looking at heterogeneous multicore architectures and conjoined-core architectures as ways to improve multi-core processor efficiency. The work on heterogeneous multi-cores was primarily asking the question: should the cores on a multi-core processor be homogeneous or heterogeneous given that there exists considerable diversity among workloads? Also, what should be the architecture of each core to maximize efficiency. The work on conjoinedcore architectures was asking the question: what is the right degree of sharing across the different cores of a multi-core processor. For these works, issues surrounding interconnections between cores were largely ignored. Victor Zyuban had been working on IBM’s next generation server processor POWER6 especially in the context of power modeling and optimization. He had previously done a considerable amount of work on microarchitectural and circuit techniques for power optimization of single-core processors. Rakesh was invited to IBM TJ Watson Research Center as a research intern for the summer of 2004. Victor was assigned as his mentor. While it was clear that Rakesh would work on some research project related to multi-core processors, it took a few days of discussion after Rakesh reached Yorktown Heights to decide that he would work on interconnections for multi-core architectures. This was a component of a multi-core processor that neither Rakesh, Victor, or Dean had looked at before. At that time, there was limited understanding, at least in the public domain, of the design space of the interconnection framework for multi-core architectures, especially how it interacts with the rest of the architecture. The cost of implementing the interconnect for multi-cores was also unclear, especially for processors with a large number of cores – commercial multicore processors then had only two cores while the research projects that studied larger number of cores (Stanford Hydra CMP studied a four core processor, DEC/WRL Piranha was an eight core processor) did not have silicon prototypes. The value of co-design of the interconnection network and the rest of the chip was not known. We decided to perform one of the earliest studies examining interconnect design issues and costs for multi-core processors.