Finally we end with a summary of the lessons learned from these applications and predictions for the future of industrial parallel computing. By supporting hundreds of concurrent threads of computation and switching every cycle to a ready thread, these architectures can hide long memory latencies and synchronization delays. While a thread waits for a load to complete or a synchronization event to occur, other threads execute. The location of data becomes irrelevant and very fine-grain algorithms execute efficiently. Parallel programs are much easier to write, simpler, and easier to maintain.
Performance is limited only by the availability of parallelism and system resources.
In this tutorial, we will review the history of this class of architectures, current hardware designs, and future trends. We will discuss how multithreading influences and shapes the software tools used to program, debug, and monitor performance. We will give examples of how programs are developed and optimized for this new type of architecture. Emphasis will be on practical information, with a series of example programs being used to guide newcomers through the important stages in writing and tuning message-passing codes. The tutorial will not address details of parallel architectures, algorithms, or theoretical models.