Scalable Automatic Parallelization of Irregular Reductions on Shared Memory Multiprocessors

ScalableAutomaticParallelizationofIrregularReductionsonSharedMemoryMultipro cessorsE.GutierrezO.PlataE. L.ZapataDepartmentofComputerArchitecture,UniversityMalagaP.O. Box4114,E-29080Malaga,Spainfeladio,oscar,ezapatag@ac.um a.e sAbstractThispaperpresentsanewaral lelizationmethodforreductionsofarrayswithsubscriptedsubscriptsonscal-ablesharedmemorymultiprocessors.Themappingofcomputationsisbasedongroupingreductionloopit-erationsintosetsthatarefurtherdistributedacrossprocessors.Iterationsbelongingtothesamesetarechoseninsuchawaythatupdatedi erententriesthereductionarray.Thatis,loopdistributionim-pliesaconict-frewritedistributionoftherductionarray.Theiterationsetsaresetupbybuildingaloop-indexprefetchingarraythatal lowstoreorderoperlytheloopiterations.Theprosedmethoisgeneral,scalable,andeasytoimplementonacompiler.Inad-ditionitdealsinauniformwaywithoneandmultiplesubscriptarrays.Incaseofmultipleindirectionar-rays,writesontheeductionvectora ectingdi erentsetsaresolvedbyde ningconict-frsupersets.Aperformanceevaluationandcomparisonwithotherex-istingtechniquesispresented.Fromtheexperimentalresultsandperformanceanalysis,theproposedmethoappearsasaclearalternativetothearrayexpansionandprivatizedbu ertechniques,usualonstate-of-the-artparal lelizingcompilers,likePolarisorSUIF.Thescalabilityproblemthatthosetechniquesexhibitismiss-inginourmethod,asthememoryoverheadpresenteddoesnotdependonthenumberofprocessors.1Intro ductionParallelcomputationisincreasinglyb eingusedasameantoobtainenoughcomputingp owersolvelargescienti candengineeringproblems.Thehighcom-plexityoftheseproblems,aswellthecurrenthigh-p erformancearchitectures,stronglysupp ortthenecessityofdevelopingcompilertechniquestoe-cientlymapsuchapplicationsontoaparallelarchitec-Thisworkassupp ortedbytheMinistryofEducationandScience(CICYT)ofSpain(TIC96-1125-C03)ture.However,manyoftheseco desexhibitirregularaccesspatternstothedata.Currentcommercialcom-pilers[17,18]areinsucientlydevelop edtodealwiththisdataaccesses,leadingtolowparalleleciencieswhentheyareusedonsuchprograms.Reductionop erationsarefrequentlyfoundinthecoreoftheseapplications,asinthenextsimplelo op,doi=1,NA(f(i))=oprexpenddoThesymb oloprrepresentsanasso ciativeop eration(likesum,pro duct,maximum,minim.),andexprisanexpressionthatshouldnotcontainyreferencetothereductionarrayA(). Thesubscriptf()de-p endsonthelo opindexi,andapp earsinrightandlefthandsideoftheassignmentsentence.Suchpatternistermedashistogramreduction[16,11]. Duetothelo op-variantnatureofsubscriptarrayf(),lo op-carrieddep endencesmayb epresentatrun-time(ifitisnotap ermutationarray). Ageneralapproachtoparallelizeirregularco des,in-cludingreductions,isbasedontheinspector-executormodel.CHAOS[15]isawellknownimplementationofthismo delfordistributedmemorymachines.Irregu-larreductionsareparallelizedbyusinganinsp ectortolo catenon-lo caldataforeachpro cessor.Afterwards,anexecutormustgathernon-lo caldatab eforethere-duction,andmustscattertheresultsafterit.Thisstrategyintro ducesasigni cantoverheadduemainlytoitsgenerality,causedbtheinsp ector(communica-tionschedule,global-to-lo caladdresstranslation)andtheexecutor(communicationsandlo cal-to-globalad-dresstranslation). Thisoverheadisprop ortionaltothenumb erofnon-lo caldataaccesses.Inasharedmemorycontext,academicparallelizerslikePolaris[2]andSUIF[8]recognizeirregularreduc-tionsandparallelizethemusingthereplicatedbu erorthearrayexpansiontechniques.The rstmetho drepli-catesthereductionarrayonallpro cessors.Eachpro cessorcomputesap ortionofthereductiononitsprivatebu er.Lateraglobalreductionisobtainedby