Compiling for parallel multithreaded computation on symmetric multiprocessors

Shared-memory symmetric multiprocessors (SMP's) based on conventional microprocessors are by far the most common parallel architecture today, and will continue to be so for the forseeable future. This thesis describes techniques to compile and schedule Id-S, a dialect of the implicitly parallel language Id, for execution on SMP's. We show that previous implementations of Id for conventional microprocessors incurred an overhead of at least 40-300% over an efficient sequential implementation of Id-S. We break down this overhead into various presence-tag checking and scheduling overheads. Given this overhead, we conclude that a fine-grained, element-wise synchronizing implementation of Id is not suitable for use on small-scale SMP's. We then describe a parallelization technique for Id-S that discovers both DAG and loop parallelism. Our parallelization exploits Id-S's single-assignment semantics for data structures. We show that for many programs, our technique can discover ample parallelism, without need for Id's traditional nonstrict, fine-grained, producer-consumer semantics. Because our parallelization eliminates the need for presence-tag checking and creates coarser-grained units of work, the parallelized codes only incur a small overhead versus sequential execution. Finally, we describe code-generation and scheduling techniques which produce efficient parallel executables which we run on a Sun Ultra HPC 5000 SMP. We compare speedups of parallelized Id-S codes using two different schedulers: an SPMD scheduler, and a more general multithreaded scheduler. We describe the advantages and disadvantages of each scheduler, and quantify the limitations in speedups for each scheduler which are due to parallelization, code generation, and scheduling. Thesis Supervisor: Arvind Title: Professor of Computer Science and Engineering Acknowledgements It is an honor and a privilege to have been a member of the Computation Structures Group (CSG) at the MIT Laboratory for Computer Science. From the "old" generation, I'd like to thank Steve Heller, Ken Traub, and Greg Papadopoulos, who brought me into CSG. From the "next" generation, I'd like to thank Boon Ang, Derek Chiou, and Alex Caro for all of the battles we've fought together all these years. Thanks also to Xiaowei Shen for many late-night talks in the lab, and James Hoe and Mike Ehrlich for teaching me a little about hardware. Thanks to Shail Aditya for many long discussions about dataflow, functional languages, and advice about surviving grad school. R. Paul Johnson and Andy Boughton have taken good care of us, through thick and thin. I have relied on your help and friendship too many times to count. Thank you. I would like to thank my advisor, Arvind, who is often exasperatingly uncompromising, but who has taught me to think clearly, to focus on fundamentals and first principles, and to build a strong foundation for my arguments and ideas. I am proud to have been one of your students. I'm very grateful to my readers, Rishiyur Nikhil and Saman Amarasinghe, who both read my thesis carefully, asked penetrating questions, pointed out holes and ambiguities, and helped me wrestle it into shape. The work in this thesis was motivated by ideas from two research projects: TAM and Cilk. From the Berkeley TAM project, David Culler, Klaus Schauser, Seth Goldstein, and Thorsten von Eiken bridged the gap between dataflow and conventional architectures, and laid the groundwork for the partitioning and code generation approach used in this thesis. From Cilk and its predecessor, PCM, Yuli Zhou, Michael Halbherr and Chris Joerg showed me the importance of scheduling; my early code to implement Cilk on SMP's was the basis of the compiler used in this thesis. Special thanks to Chris Joerg, whose questions eventually led me to the work in this thesis, and who directly or indirectly helped me solve some of the hard problems, and who encouraged me in my work when I needed it. I'd like to thank my family my sister Nini, and my parents Julie Yueh-ho and Chong-Kuang Shaw who have supported me throughout my years at MIT. Everything I have, I owe to you. Finally, thanks to my wife Atsuko, who brought true happiness to my life your optimism and patience have pulled me through, and I am so lucky to have shared these years with you. I am also happy to have shared my final year at MIT with my daughter Marie, whose energy and thirst for knowledge inspire me anew. For my parents, Julie Yueh-ho and Chong-Kuang Shaw