SnapShot: Visualization to Propel Ice Hockey Analytics Abstract-Sports analysts live in a world of dynamic games flattened into tables of numbers, divorced from the rinks, pitches, and courts where they were generated. Currently, these professional analysts use R, Stata, SAS, and other statistical software packages for uncovering insights from game data. Quantitative sports consultants seek a competitive advantage both for their clients and for themselves as analytics becomes increasingly valued by teams, clubs, and squads. In order for the information visualization community to support the members of this blossoming industry, it must recognize where and how visualization can enhance the existing analytical workflow. In this paper, we identify three primary stages of today's sports analyst's routine where visualization can be beneficially integrated: 1) exploring a dataspace; 2) sharing hypotheses with internal colleagues; and 3) communicating findings to stakeholders.Working closely with professional ice hockey analysts, we designed and built SnapShot, a system to integrate visualization into the hockey intelligence gathering process. SnapShot employs a variety of information visualization techniques to display shot data, yet given the importance of a specific hockey statistic, shot length, we introduce a technique, the radial heat map. Through a user study, we received encouraging feedback from several professional analysts, both independent consultants and professional team personnel.Index Terms-Visual knowledge discovery, visual knowledge representation, hypothesis testing, visual evidence, human computer interaction. Sports are inherently competitive. Teams seek championships in their leagues and players strive to be the highest ranking athlete at their positions. In professional sports, the money flows to those who win. Winning dictates player salaries, team salary caps, and prioritizes television airtime.. In sports such as baseball, game data has both been collected and studied for over a century. An increasing number of professional sports teams are incorporating statistical analysis of game data into their organizations' strategies. The use of analytics is cited as a major factor in the American League record 20-game win streak of the 2002 Oakland Athletics of Major League Baseball and the National Basketball Association Championship of the 2011 Dallas Mavericks [12, 9] .. Although data analytics are becoming much more prevalent throughout sports, data visualization is still relatively unexplored. This surprises us because sports data lends itself well to visualization. A timeless example is Baseball Hall of Famer Ted Williams's and John Underwood's 1969 strike-zone graphic accompanying the Sports Illustrated article "The Science of Batting" [28] . The iconic graphic places Williams himself next to a theorized map of pitch locations, colored according to his perceived success in hitting a pitch in that location. A few teams in the National Hockey League, the highest league of professional ice hockey in North America, have begun to utilize analytics, but not visualization. There is evidence that visualization could be embraced by these teams. As Dan McKinnon, Director of Personnel for the Pittsburgh Penguins, shared with us, "We are always seeking out advantages to improve our strategy and thus our performance.". In cooperation with a professional hockey analyst, we have began to explore whether visualization could be advantageous for competitive hockey strategy. We utilized a user-centered methodology for designing a system, with our consulting analyst representing the "user". As a group, we focused our efforts on three potential stages where visualization could integrate into the analyst's workflow:. 1. Exploration of a dataspace. 2. Discussion of hypotheses amongst analysts. In our introduction, we explained that in hockey, winning on the ice starts with strong strategies and that hockey analysts contribute to these strategies through statistical analysis. In our related work, we described the current state of sports visualization. Through interactions with a professional hockey analyst, we developed and modelled a typical sports analyst workflow and we identified those areas where visualization can add value. We presented a hockey visualization system, SnapShot, designed to augment this workflow. Finally, we outlined and discussed three case studies of domain experts using SnapShot and the effectiveness of visualization for enhancing analysts' exploration process and business intelligence features.. To the information visualization community, this paper contributes: a model workflow of a sports analyst, the SnapShot system for visualizing 2010-2011 National Hockey League shot data, and three case studies of hockey analysts exploring their data visually.. We propose several areas of future work for ourselves and the visualization community as a whole. We hope to expand the number of event types that SnapShot supports to include events such as hits, passes, and penalties. SnapShot only visualizes the data provided; we would like to add analytic capacity to the system in order to support visual analytics methods. Our second analyst expressed a desire to filter based on whether a shooter's team is winning or losing, so we plan to accommodate this and other such analytic functions in later versions of SnapShot. We hope our documentation of a sports analysts' workflow will inform future sports visualization system design. Our workflow model provides a foundation for designing systems to support sports analysts. There is opportunity to refine this model through investigation of the exploration stage collaboration amongst sports analysts. Finally, we intend for our case studies to highlight the potential for incorporating visualization into the growing ice hockey analytics community. In order to offer hockey analysts effective visualization capabilities, we designed the SnapShot system around nuances of their data and three stages of their existing workflow: exploration, collaboration, and presentation.. The case studies demonstrate SnapShot effectively allowing for visual exploration of data. As Analyst B explained, "I like that this allows me to go from abstract thoughts to specific images very quickly. Sometimes I don't know what it is telling me initially, but I like the possibilities.". During the evaluation of SnapShot, two participating analysts preferred radial heat maps because the view visualizes the variable they were most interested in, shot length. Analyst B told us in his closing comments,"I was suspicious at first, but I've fallen for it." The Analyst C preferred shot maps because that view visualizes the variable he was most interested in, shot locations.. The analysts utilized small multiples representations, despite initial reservations. Upon his first encounter of them, Analyst A remarked, "I have to tell you, I am overwhelmed and very impressed all at once." Yet because many of hockey's statistics are categorical, the view proved its utility over the course of his investigation. By the end of his session, Analyst A was constantly referring back to it.. Analyst C shared promising evidence that SnapShot may provide the presentation capabilities for which it was designed, "I know many traditionalists in our sport who resist analytics, but maybe wouldn't with more pictures involved. " We were surprised that none of the evaluation participants expressed interest in sharing their visual findings with their peers, despite collaboration being a main design goal of SnapShot. However, this does align with the initial exploration stage of the model workflow being an individual process, in contrast to the collaborative brainstorming exploration stage. The use of visualization to help fans, team officials, and analysts better understand sports data has grown in popularity recently. Many static infographics about sports data can be found on various websites and in sport publications. More comprehensive interactive visualization systems are less common, however. In the information visualization community, a few projects and systems have been developed including the use of treemaps to summarize tennis match results [11] , sparkline-style visualizations to show baseball [25] team performance with different pitchers [5] , and flexible representations of tournament brackets/ladders [22] . Major League Baseball (MLB) and the National Basketball Association (NBA) have pioneered the gathering, analysis, and visualization of game data acquired through computer vision technologies applied to video. Sportvision's PITCHf/x [19] is a system for automatic statistical collection in baseball parks using two cameras mounted in the stadium to track the speed and location of a pitched baseball. Using PITCHf/x, statistics such as the pitcher with the fastest fastball or the pitcher with the sharpest-breaking curve can be analyzed.. A well-known example of baseball data visualization utilizing the PITCHf/x data is a video from the New York Times New Media Desk highlighting the pitching tendencies of Mariano Rivera, one of the best closing pitchers due to his "signature cutter" [4] . The NY Times video uses heat maps of Rivera's pitch locations to explain why his pitches are so difficult to hit and why he is such a successful pitcher. The company Stats LLC is using computer vision technology called SportVU [20] to capture movement and game action data from the National Basketball Association (NBA). Using this data, Goldsberry [7] created an application for visualizing several seasons worth of NBA data as a complex heat map called CourtVision (Figure 3) . The heat map regions are sized by shot attempts and colored by points per attempt, resulting in a series of court heat maps that help compare different teams and explore conventional wisdom about the teams. Fig. 3 . Goldsberry's heat map of NBA field goal attempts over a five year span [7] . Glyph size represents the number of shot attempts and color represents points per shot attempt. Image courtesy of Kirk Goldsberry.. Another basketball project [13] , built with Pat Summitt's mantra that "Rebounding wins championships" in mind, utilizes the SportVU optical tracking data in a multi-view system. Figure 4 shows two of the views, pairing scatterplot and heat map visualizations of the spatial distribution data for on-court rebounding. Soccer (called Football by most countries other than the United States) is similar to ice hockey in terms of game structure and style. Games in both sports have a continuous flow with passing and player position structured by physical aspects of the pitch (or rink). Because soccer is so popular in Europe, it has garnered significant analytical exploration and a growing body of work has been published regarding performance improvement systems, many including visualization. One such example is a system called Soccer Scoop [15] [16] (Figure 5 ), which pairs a statistical tool with the ability to visualize goalkeepers through glyphs. Team managers can use the tool to visualize one goalie between games, or compare two separate goalies to find opportunities for improvement.. The Attribute Explorer [18] , a well-known general purpose information visualization system, was adapted in 2008 to display soccer game events [1] (Figure 6 ). The tool was used for observing single matches and for collecting game events cumulatively for later analysis through coordinated views.. Most of the analytics work being done for ice hockey does not include visualization capabilities. A few hockey analysts have used systems such as Tableau [21] and Spotfire [23] for selected presentations of analysis data. Such visualizations obviously are general purpose, however, and do not show the data in the context of a rink.. Beyond in-house team analytics and hired consultants, news outlets and their web affiliates provide visual game data for consumption The National Hockey League created the Ice Tracker visualization system [14] for displaying events as they occur in real-time throughout a hockey game, displaying each event as a small glyph on a rink as well as on a timeline. If the event has a film clip associated with it, the glyph reflects this and allows the user to view the clip. The system provides filtering to display shots, goals, hits, penalties, and fights on the ice. ESPN hosts a similar visualization system called GameCast [6] . GameCast differs from Ice Tracker in that it normalizes events on the ice so that the away team's events are on the left end and the home team's events are on the right end and the glyphs displayed are in the relevant team's color scheme. Like Ice Tracker, GameCast can run in real-time during games or display a recap of a game. The audience for both Ice Tracker and GameCast is not a team or organization, but rather the public at large. While the full visualization of a single game is available after it has concluded, neither system allows games to be combined for longer-term analysis, and thus neither are utilized by professional analysts in the course of their work. SnapShot is written entirely in Javascript, making use of the jQuery toolkit 1 . The interactive visualizations are made up of SVG elements generated using the D3 2 , which also handles the data importing. The toggle buttons, calendar widget, and slider widget are all from the jQueryUI widget library 3 and the remaining widgets are native html. The exporting feature makes use of the canvg package 4 , which allows for converting SVG elements to PNG images. The GET-based bookmarking code is greatly simplified by Stephen Morley's QueryData.js scripts 5 . The radial heat map colorscale and a selection of the categorical color schemes were chosen using ColorBrewer2 [3] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I B I B I B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Accelerating generators of iterative methods for finding multiple roots of nonlinear equations Two accelerating generators that produce iterative root-finding methods of arbitrary order of convergence are presented. Primary attention is paid to algorithms for finding multiple roots of nonlinear functions and, in particular, of algebraic polynomials. First, two classes of algorithms for solving nonlinear equations are studied: those with a known order of multiplicity and others with no information on multiplicity. We also demonstrate the acceleration of iterative methods for the simultaneous approximations of multiple roots of algebraic polynomials. A discussion about the computational efficiency of the root-solvers considered and three numerical examples are given. In this paper we consider various classes of iterative methods for finding multiple roots of nonlinear equations. We derive several new root-finding methods of a higher order for multiple roots of nonlinear functions and all multiple roots of algebraic polynomials in parallel mode. All of these methods are produced by suitable accelerating generators of iterative functions. An iterative method ϕ r+1 of order r + 1 is generated from the previous method ϕ r of order r using a special transformation. The sequence of methods ϕ 2 , ϕ 3 , . . . , ϕ n , . . . can be derived automatically using symbolic computation (in Mathematica, MATLAB or Maple).. In Section 2 we present an accelerating formula for generating a sequence of iterative methods for determining multiple roots of equations of the form f (x) = 0. Using this generating formula, in Section 3 we derive several higher-order iterative methods, some of which are new ones. We distinguish two kinds of methods; those which deal with a known order of multiplicity and others with no information on multiplicity. In Section 4 we describe a new technique for generating some classes of iterative methods, based on the Weierstrass function, for the simultaneous determination of multiple roots of algebraic polynomials. Numerical examples are given in Section 5 to demonstrate the convergence behavior of the methods considered. This section also contains an analysis of the computational efficiency of the methods presented.

O O O B I O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Conditional hardness of precedence constrained scheduling on identical machines Already in 1966, Graham showed that a simple procedure called list scheduling yields a 2-approximation algorithm for the central problem of scheduling precedence constrained jobs on identical machines to minimize makespan. Till this date it has remained the best algorithm and whether it can or cannot be improved has become a major open problem in scheduling theory.We address this problem by establishing a quite surprising relation between the approximability of the considered problem and that of scheduling precedence constrained jobs on a single machine to minimize weighted completion time. More specifically, we prove that if the single machine problem is hard to approximate within a factor of 2 − then the considered parallel machine problem, even in the case of unit processing times, is hard to approximate within a factor of 2 − ζ, where ζ tends to 0 as tends to 0. Combining this with Bansal & Khot's recent hardness result for the single machine problem gives that it is NP-hard to improve upon the approximation ratio obtained by Graham, assuming a new variant of the unique games conjecture. One of the first approximation algorithms with a worstcase analysis can be traced back to 1966, when Graham [20] studied the following central scheduling problem (known as P |prec|Cmax in standard scheduling notation [19] ): There is a set N of n jobs to be scheduled on m identical parallel machines. Each machine can process at most one job at a time Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. and each job j ∈ N requires pj uninterrupted units of processing on one of the machines. Jobs also have precedence constraints between them that are specified by a partial order P on N , where (i, j) ∈ P implies that job i must be completed before job j can be started. The goal is to find a schedule that minimizes the makespan Cmax = maxj Cj , where Cj is the time at which job j completes in the given schedule.. Arguably the simplest algorithm for many scheduling problems is the so called list scheduling procedure, where we order the jobs in a preference list and schedule the first available job(s) from the list whenever a machine falls idle. In his seminal paper [20] , Graham showed that for P |prec|Cmax, the list scheduling procedure has a worst-case performance guarantee of 2 − 1/m.. Considering the special case with no precedence constraints, denoted by P ||Cmax, Graham [21] later refined his analysis and showed that the list scheduling procedure has a worst-case performance guarantee of 4/3−1/(3m), assuming the priority list is obtained by arranging jobs according to non-increasing processing times. Using more complex techniques, Hochbaum & Shmoys [22] improved upon the 4/3 − 1/(3m) approximation guarantee by giving a polynomial time approximation scheme for P ||Cmax. This gives a tight result since P ||Cmax is known to be strongly NP-hard and hence does not admit a fully polynomial time approximation scheme [15] . We remark that the problem where the number of machines is fixed, i.e., not part of the input, is also resolved in terms of approximability: Sahni [33] gave a fully polynomial approximation scheme and it is easily seen to be weakly NP-hard by a reduction from partition. In summary, the considered problem is well understood in the absence of precedence constraints.. Unfortunately, understanding the complexity in the presence of precedence constraints has turned out to be much more challenging and remains limited. The mentioned (2 − 1/m)-approximation algorithm obtained by Graham in 1966 , remains the algorithm of choice for this problem. The last progress in understanding whether it can or cannot be improved, was made over 30 years ago by Lenstra & Rinnooy Kan [29] who showed that it is NP-hard to approximate P |prec|Cmax within a factor less than 4/3. Closing this gap is mentioned as "Open Problem 1" in Woeginger and Schuurman's list of ten open problems in scheduling theory [35] . Furthermore, the computational complexity of finding an exact solution to P m|prec, pj = 1|Cmax (the problem with a fixed number m of machines and unit processing times), open problem "OPEN8" from the original list of Garey and Johnson [16] , is still open.. We remark that Du, Leung & Young [11] showed that P 2|prec|Cmax is strongly NP-hard. Whereas, P 2|prec, pj = 1|Cmax is known to be polynomially solvable by techniques from matching theory (Fujii, Kasami & Ninomiya [13] ). The best algorithm for problem P 3|prec, pj = 1|Cmax is a 4/3-approximation algorithm by Lam & Sethi [27] 1 who analyzed an algorithm by Coffman & Graham [9] . In a recent paper [14] , Gangal & Ranade showed that Graham's (2 − 1/m)-approximation algorithm can be improved to a (2 − 7 3m+1 )-approximation algorithm in case of unit processing times and m > 3. Older results by Hu [23] and Garey, Johnson, Tarjan, & Yannakakis [17] give polynomial time algorithms in the case of unit processing times and special cases of precedence constraints. We refer the interested reader to the surveys [7] and [19] for further information on these and other algorithms for special cases of P |prec|Cmax.

O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Occlusion

in

Collaborative

Augmented

Environments

Augmented

environments

superimpose

computer

enhancements

on

the

real

world.

Such

augmented

environments

are

well

suited

for

collaboration

of

multiple

users.

To

improve

the

quality

and

consistency

of

the

augmentation

the

occlusion

of

real

objects

by

computergenerated

objects

and

vice

versa

has

to

be

implemented.

We

present

methods

how

this

can

be

done

for

a

tracked

user's

body

and

other

real

objects

and

how

irritating

artifacts

due

to

misalignments

can

be

reduced.

Our

method

is

based

on

simulating

the

occlusion

of

virtual

objects

by

a

representation

of

the

user

modeled

as

kinematic

chains

of

articulated

solids.

Registration

and

modeling

errors

of

this

model

are

being

reduced

by

smoothing

the

border

between

virtual

world

and

occluding

real

object.

An

implementation

in

our

augmented

environment

and

the

resulting

improvements

are

presented.

We

are

planning

to

introduce

a

calibration

scheme

for

rescaling

the

phantom

avatar

to

fit

different

users.

The

basic

idea

for

this

is

to

let

the

user

strike

a

small

number

of

predefined

poses,

which

should

enable

the

system

to

infer

body

measurements..

We

plan

eventually

to

get

rid

of

the

sensor

on

the

users

back

by

additional

constraints

in

the

head/back

joint

and

to

introduce

certain

assumption

of

the

body

posture

(vertical)

for

our

preferred

work

situations,

standing

and

seated..

A

minimum

sensor

situation,

using

only

PIP

and

pen

and

the

information

on

the

users

handedness

(right-or

left-handed)

to

infer

avatar

posture

is

being

looked

into..

For

further

information

contact

via

email

fuhrmann|hesina|faure|gervautz@cg.tuwien.ac.at

or

access

the

latest

results

on

The

Studierstube-setup

As

mentioned

before,

each

participant

in

our

collaborative

augmented

environment

STUDIERSTUBE

wears

an

individually

headtracked

see-through

HMD

providing

a

stereoscopic

real-time

display..

The

HMDs

we

use

-Virtual

I-O

i-glasses

-are

very

lightweight

and

unobtrusive,

but

only

of

limited

resolution

and

small

field

of

view.

As

input

devices

we

use

a

pen

with

multiple

buttons

and

the

Personal

Interaction

Panel

(PIP)

The

PIP

allows

both

intuitive

three-dimensional

manipulations

such

as

placement

of

objects

and

the

input

of

numerical

data

or

commands

via

traditional

sliders

and

buttons

(figure

-historically

one

of

the

weak

points

of

VEs.

The

pen,

the

PIP

and

the

HMDs

are

tracked

by

an

ASCENSION

flock

of

birds

magnetic

tracker..

Integration

of

the

necessary

extensions

for

occlusion

into

Studierstube

proved

to

be

relatively

straightforward.

The

whole

system

is

based

on

the

OpenInventor

Rendering/Simulation

library,

which

reduces

the

addition

of

new

functionality

to

the

Studierstube

environment

to

the

simple

process

of

adding

a

new

dynamically

loadable

module.

The

necessary

tracker

data

is

delivered

by

the

Studierstube

tracker

interface,

which

delivers

positions

and

orientations

in

OpenInventor

fields,

a

high-level

interface

element,

which

allows

easy

connections

to

the

separately

developed

simulation

library

B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Checkpoint evolution for volatile correlation computing Abstract Given a set of data objects, the problem of correlation computing is concerned with efficient identification of strongly-related ones. Existing studies have been mainly focused on static data. However, as observed in many real-world scenarios, input data are often dynamic and analytical results have to be continually updated. Therefore, there is the critical need to develop a dynamic solution for volatile correlation computing. To this end, we develop a checkpoint scheme, which can help us capture dynamic correlation values by establishing an evolving computation buffer. In this paper, we first provide a theoretical analysis of the properties of the volatile correlation, and derive a tight upper bound. Such tight and evolving upper bound is used to identify a small list of candidate pairs, which are maintained as new transactions are added into the database. Once the total number of new transactions goes beyond the buffer size, the upper bound is re-computed according to the next checkpoint, and a new list of candidate pairs is identified. Based on such a scheme, a new algorithm named CHECK-POINT+ has been designed. Experimental results on realworld data sets show that CHECK-POINT+ can significantly reduce the computation cost in dynamic data environments, and has the advantage of compacting the use of memory space. Recent years have witnessed increased interest in computing strongly related data objects (e.g., strongly correlated item pairs, as measured by Pearson's correlation coefficient). Many important applications in science and business (e.g. Alexander 2001; Cohen et al. 2002; Kuo et al. 2002) depend on efficient and effective correlation computing techniques to discover relationships within large collections of data.. Despite the development of correlation computing techniques (e.g. Brin et al. 1997; Jermaine 2001; DuMouchel and Pregibon 2001; Jermaine 2003; Ilyas et al. 2004; Xiong et al. 2004 Xiong et al. , 2006 , researchers and practitioners are still facing increasing challenges to measure associations among data objects produced by emerging data-intensive applications, particularly when the data are dynamic and analytical results need to be continually updated. Indeed, with such large and growing data sets, research efforts are needed to develop a dynamic solution for volatile correlation computing. To that end, in this paper we provide a pilot study of dynamically finding all strongly related item pairs, whose correlation values are above a user-specified minimum threshold, as new data are constantly being collected.. As motivating examples, let us consider the following potential application scenarios. First, consider an e-commerce Web site that would like to promote sales by making recommendations to customers. In order to automate this process, the computerized system can recommend items most highly correlated to those being purchased, according to past transactions. As new orders are being placed, the recommendations should be updated automatically, and reflect recent interests in a timely manner. In this case, the underlying computation would involve finding the strongly correlated item pairs in a real-time fashion. A second example can be found in automatic stock picking. In order to monitor stocks with correlated price movements, a portfolio manager might be interested in knowing highly correlated stocks, whose prices tend to move in the same direction (either going up or going down). Despite the large number of stocks on the market, and the number of days with price quotes, the portfolio manager may want to maintain the up-to-date list of strong pairs as his decision support. Both the above application scenarios require efficient computation of strongly correlated pairs in a dynamic fashion.. A straightforward solution is to recompute the correlations of all item pairs every time when new data become available. However, for large data sets, this approach is not practical, particularly if the application needs the results in a timely fashion. An alternative method is to use more space to save time. Along this line, we present a SAVE-ALL algorithm, which saves the intermediate results for all item pairs. When new transactions are added into the database, SAVE-ALL only updates the stored values corresponding to each item pair, and computes the correlation query results with the intermediate values. Obviously, the SAVE-ALL method compromises space for time. If the number of items in the data set becomes considerably large, the number of pairs grow even larger, to the extent that it is impossible to save the intermediate computing results of all item pairs in the memory space. This motivates our interest in volatile correlation computing.. In our preliminary work , we proposed a CHECK-POINT algorithm that makes a time-space tradeoff and can efficiently incorporate new transactions for correlation computing as they become available. In CHECK-POINT, we set a checkpoint to establish a computation buffer, which can help us determine a correlation upper bound. This checkpoint bound can be exploited to identify a list of candidate pairs, whose frequencies are maintained and correlations are computed, as new transactions are being added into the database. However, if the total number of new transactions exceeds the buffer size, a new upper bound is computed according to the next checkpoint and a new list of candidate pairs is identified.. The rationale behind CHECK-POINT is that, if the number of new transactions is much smaller than the total number of transactions in the database, the correlation coefficients of most item pairs will not change substantially. In other words, we only need to establish a very short list of candidate pairs at the checkpoint and maintain this candidate list in the memory as new transactions are added into the database. Unlike SAVE-ALL, CHECK-POINT only maintains the intermediate computing results of a very small portion of the item pairs. This can greatly compact the use of the memory space, using slightly more time.. In this paper, we derive a tight upper bound for the evolving correlation. The tight bound is exploited to identify a more compact candidate list than CHECK-POINT, so that better correlation computing performances can be achieved. Also, we identify the local monotone property of this new upper bound. Such property can be used for searching the optimal points effectively. In addition, based on this new upper bound, we exploit the local monotone property and design a CHECK-POINT+ algorithm for volatile correlation computing.. As demonstrated by our experimental results on several real-world data sets, CHECK-POINT+ has a much better computational performance than CHECK-POINT. Both CHECK-POINT and CHECK-POINT+ can significantly reduce the computational cost compared to existing correlation computing benchmark algorithms, e.g. TAPER, in dynamic data environments. Also, compared to CHECK-POINT, CHECK-POINT+ is less sensitive to the change of parameters. Moreover, we observe that there is a trade-off between the use of time and space by setting different checkpoint values. Indeed, the size of the candidate list decreases with the increase of the checkpoint density. In contrast, the average computational savings is reduced with the increase of the checkpoint density. Finally, our experimental results show that CHECK-POINT+, as compared to SAVE-ALL and CHECK-POINT, can greatly reduce the use of memory space.. The rest of this paper is organized as follows. Section 2 presents some basic concepts and formulates the problem. In Sect. 3, we discuss the related work. Section 4 provides a checkpoint view of dynamic correlation computing. In Sect. 5, we derive an evolving upper bound of φ correlation coefficient. Section 6 describes the CHECK-POINT+ algorithm. In Sect. 7, we show the experimental results on real-world data. Finally, Sect. 8 concludes the work. In this paper, we studied the problem of correlation computing in large and dynamically growing data sets. Specifically, we designed a CHECK-POINT+ algorithm for dynamically searching all the item pairs with correlations above a user-specified threshold. The key idea is to establish a computation buffer by setting a checkpoint for dynamic input data. This checkpoint can be exploited to identify a list of candidate pairs, which are maintained and whose correlations are computed, as new transactions are added into the database. However, if the total number of new transactions is beyond the checkpoint, a new candidate list is generated by the new checkpoint. All the checkpoints have been established by a tight evolving correlation upper bound, which shows a local monotonicity property.. Moreover, we carried out a number of experiments to evaluate the performance of the CHECK-POINT+ algorithm. Experimental results on real-world data sets show that CHECK-POINT+ can compact the use of memory space by maintaining a reduced candidate pair list, which is only a very small portion of all the item pairs. Also, CHECK-POINT+ can significantly reduce the correlation computing cost in dynamic data sets with a large number of transactions and has a much better computational performance than state-of-theart benchmark methods. Finally, the experimental results also show that CHECK-POINT+ is less sensitive to the change of parameters compared to benchmark algorithms. . where the numerator reduces to . with c 2 = N(na +n b ) 2 − n a n b . Since f (t) = w(t)m(t), the first derivative of f (t) will be Association analysis has been a major topic in the field of data mining. Traditional association-rule mining algorithms (Agrawal et al. 1993; Bayardo and Agrawal 1999) are found to be yielding many spurious patterns (Brin et al. 1997; Xiong et al. 2006 Xiong et al. , 2008 . As a result, in recent years, many statistical correlation measures, such as χ 2 statistics (Brin et al. 1997; DuMouchel and Pregibon 2001; Jermaine 2001 Jermaine , 2003 ), Pearson's correlation coefficients (Xiong et al. 2004 (Xiong et al. , 2006 Zhou and Xiong 2008) , rank-based correlation coefficients (Melucci 2007; Yilmaz et al. 2008) , and mutual information (Ke et al. 2006 (Ke et al. , 2007 have been considered in the setting of large-scale association analysis.. Specifically, due to its flexibility in measuring the association among multiple items, the χ 2 statistic has been proposed to analyze market-basket data (Brin et al. 1997 ). However, χ 2 does not possess an upward closure property for efficient computation (DuMouchel and Pregibon 2001). Also, Jermaine (2001) investigated the implication of incorporating χ 2 -based queries into data cube computations. He showed that finding subcubes that satisfy statistical tests such as χ 2 are inherently NP-hard, but may be made more tractable using approximation schemes. Finally, Jermaine (2003) presented an iterative procedure for correlation analysis by dismissing part of the database based on human feedback.. On the other hand, the upper bound of φ, which is the binary form of Pearson's correlation coefficient (Pearson 1920; Reynolds 1977) , has been found to have an anti-monotone property (Xiong et al. 2004 (Xiong et al. , 2006 . Such property has helped finding all strongly related pairs efficiently, especially for data with Zipf rank-support distributions.. Pruning techniques using upper bounds have been effectively used on both φ (Xiong et al. 2004 (Xiong et al. , 2006 and χ 2 (Morishita and Sese 2000) . However, previous work does not consider dynamic data environments. In our preliminary work , we proposed a CHECK-POINT algorithm that makes a time-space tradeoff for dynamic correlation computation. The idea was to use an upper bound of the evolving φ correlation by setting a checkpoint for future transactions. In this paper, we improve the CHECK-POINT algorithm by deriving a tighter upper bound, which provides better pruning effect and can lead to better computational performances.

B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Point-free

Program

Transformation

Abstract.

The

subject

of

this

paper

is

functional

program

transformation

in

the

so-called

point-free

style.

By

this

we

mean

first

translating

programs

to

a

form

consisting

only

of

categorically-inspired

combinators,

algebraic

data

types

defined

as

fixed

points

of

functors,

and

implicit

recursion

through

the

use

of

type-parameterized

recursion

patterns.

This

form

is

appropriate

for

reasoning

about

programs

equationally,

but

difficult

to

actually

use

in

practice

for

programming.

In

this

paper

we

present

a

collection

of

libraries

and

tools

developed

at

Minho

with

the

aim

of

supporting

the

automatic

conversion

of

programs

to

point-free

(embedded

in

Haskell),

their

manipulation

and

rule-driven

simplification,

and

the

(limited)

automatic

application

of

fusion

for

program

transformation.

Functional

Programming

has

always

been

known

to

be

appropriate

for

activities

involving

manipulation

of

programs,

such

as

program

transformation.

This

is

due

to

the

strong

theoretical

basis

that

underlies

the

programming

languages:

the

semantics

of

functional

programs

are

easier

to

formalize..

As

with

any

programming

paradigm,

different

functional

programmers

use

different

styles

of

programming;

it

is

however

true

that

most

advanced

programmers

resort

to

some

concise

form

where

functions

are

written

as

combinations

of

other

functions,

rather

than

programming

by

explicit

manipulation

of

the

arguments

and

explicit

recursion.

For

instance

a

function

that

sums

the

squares

of

the

elements

in

a

list

can

be

written

in

Haskell

as

(foldr

(map

sq)

where

sq

x

x*x

A

radical

style

of

programming

is

the

so-called

point-free

style,

which

totally

dispenses

with

variables.

For

instance

the

function

sq

above

can

be

written

as

sq

mult

(id

id),

where

the

infix

operator

corresponds

to

the

split

combinator

that

applies

two

functions

to

an

argument,

producing

a

pair,

and

mult

is

the

uncurried

product..

The

origins

of

the

point-free

style

can

be

traced

back

to

the

ACM

Turing

Award

Lecture

given

by

John

Backus

in

Instead

of

explicitly

referring

arguments,

Backus

recommended

the

use

of

functional

forms

(combinators)

to

build

functions

by

combining

simpler

ones.

The

particular

choice

of

combinators

should

be

driven

by

the

associated

algebraic

laws..

In

the

modern

incarnation

of

these

ideas,

the

combinators

correspond

to

morphisms

in

a

category

(where

the

denotational

semantics

of

the

language

are

constructed)

and

the

desired

laws

follow

directly

from

universal

properties

of

this

category.

What

is

more,

this

approach

extends

smoothly

to

the

treatment

of

recursion

in

what

is

known

as

the

data

type-generic

approach

to

programming

This

allows

one

to

reason

equationally

about

functions

obtained

by

applying

standard

recursion

patterns,

thus

replacing

the

use

of

fixpoint

induction..

The

generic

aspect

of

this

approach

comes

from

the

fact

that

all

the

constructions

are

parameterized

by

the

recursive

data

types

involved

in

the

computations.

It

is

widely

accepted

that

this

style

is

a

good

choice

for

reasoning

about

programs

equationally

and

generically.

It

has

also

proved

to

be

fruitful

in

the

field

of

program

transformation

where

well-known

concepts

like

folding

or

fusion

over

lists

were

first

introduced

by

Bird

to

derive

accumulator-based

implementations

from

inefficient

specifications

As

a

simple

example

of

the

kind

of

transformation

we

mean,

in

the

function

given

above,

the

fold

can

be

equationally

fused

with

the

map

to

give

the

following

one-pass

function,

where

plus

is

uncurried

sum.

The

drawback

of

using

this

radical

point-free

style

is

that,

as

the

examples

in

this

paper

show,

programs

written

without

variables

are

not

always

easy

to

write

or

understand.

In

fact,

it

is

virtually

impossible

to

program

without

using

variables

here

and

there.

Pointwise

vs.

point-free

is

a

lively

discussion

subject

in

Haskell

forums;

the

goal

of

the

present

paper

is

to

present

a

set

of

libraries

and

tools

that

support

point-free

program

transformation,

but

this

includes

the

automatic

translation

of

code

to

point-free

form,

so

that

programmers

may

apply

point-free

techniques

to

their

code

with

variables..

Specifically,

we

present

here

the

following

components,

which

are

all

freely

available

as

part

of

the

UMinho

Haskell

Software

distribution..

Pointless:

a

library

for

point-free

programming,

allowing

programmers

to

typecheck

and

execute

point-free

code

with

recursion

patterns,

parameterized

by

data

types.

With

the

help

of

extensions

to

the

Haskell

type

system,

we

have

implemented

an

implicit

coercion

mechanism

that

provides

a

limited

form

of

structural

equivalence

between

types.

This

has

allowed

us

to

embed

in

Haskell

a

syntax

almost

identical

to

the

one

used

at

the

theoretical

level.

DrHylo:

a

tool

that

allows

programmers

to

automatically

convert

Haskell

code

to

point-free

form

with

recursion

patterns.

In

particular,

we

employ

the

wellknown

equivalence

between

simply

typed

λ-calculi

and

cartesian

closed

categories

suggested

by

Lambek

This

serves

as

the

basis

for

the

translation

of

a

core

functional

language

to

categorical

combinators,

extended

by

the

first

author

to

cover

sum

types.

A

second

component

here

is

the

application

Fig.

Typing

rules

of

a

standard

algorithm

that

converts

recursive

functions

to

hylomorphisms

of

adequate

regular

data-types,

thus

removing

explicit

recursion.

SimpliFree:

a

tool

for

manipulating

point-free

code.

Its

use

will

be

exemplified

here:

(i)

for

the

simplification

of

the

very

verbose

terms

produced

by

DrHylo;

and

(ii)

for

program

transformation

by

applying

fold

fusion..

Organization

of

the

Paper.

Section

introduces

the

languages

used

in

the

paper,

and

Sect.

reviews

notions

of

point-free

equational

reasoning,

with

the

help

of

an

example.

Pointless,

DrHylo,

and

SimpliFree

are

described

in

Sects.

and

Finally

Sect.

concludes

the

paper.

We

have

focused

on

the

most

important

aspects

of

each

component

of

the

framework;

more

documentation

can

be

found

at

the

UMinho

Haskell

Software

pages:.

While

Pointless

has

reached

a

stable

stage

of

development,

there

are

still

many

points

for

improvement

in

the

other

components.

In

DrHylo,

the

translation

of

recursive

functions

must

be

improved

with

the

automatic

translation

to

other

standard

recursion

patterns

such

as

folds,

unfolds,

and

paramorphisms,

rather

than

always

resorting

to

the

all-encompassing

hylomorphisms..

In

SimpliFree,

we

plan

to

incorporate

other

laws

for

recursive

functions,

such

as

unfold-fusion.

An

immediate

goal

is

to

make

the

fusion

mechanism

more

powerful,

to

cover

at

least

all

the

transformations

that

can

be

done

in

state-ofthe-art

tools

such

as

MAG.

O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Integrating Bulk-Data Transfer into the Aurora Distributed Shared Data System The Aurora distributed shared data system implements a shared-data abstraction on distributed-memory platforms, such as clusters, using abstract data types. Aurora programs are written in C++ and instantiate shared-data objects whose data-sharing behaviour can be optimized using a novel technique called scoped behaviour. Each object and each phase of the computation (i.e., use-context) can be independently optimized with per-object and per-context flexibility. Within the scoped behaviour framework, optimizations such as bulk-data transfer can be implemented and made available to the application programmer. Scoped behaviour carries semantic information regarding the specific data-sharing pattern through various layers of software. We describe how the optimizations are integrated from the uppermost application-programmer layers down to the lowest UDP-based layers of the Aurora system. A bulk-data transfer network protocol bypasses some bottlenecks associated with TCP/IP and achieves higher performance on an ATM network than either TreadMarks (distributed shared memory) or MPICH (message passing) for matrix multiplication and parallel sorting. Distributed-memory platforms, such as networks of workstations and clusters, are attractive because of their ubiquitousness and good price-performance, but they suffer from high communication overheads. Sharing data between distributed memories is more expensive than sharing data using hardware-based shared memory. Also, existing network protocols, such as TCP/IP, were not originally designed for communication-intensive clusters and may not be the best choice for performance.. Flexibility is important because different loops or phases of a computation can have different access patterns for the same data structure. For example, in a chain or pipeline of computational phases, the output of one phase becomes the input of the next phase. A data structure may have a read-only access pattern during one phase and a read-modify-write access pattern during a later phase. If the same data structure is large and has to be communicated to all processes, a third data-sharing policy may be required to efficiently handle the bulk-data transfer. Therefore, it is desirable to be able to select a different data-sharing policy for each computational phase.. To address the flexibility issue, the Aurora DSD system uses scoped behaviour. As the programmer's interface for specifying a data-sharing optimization, scoped behaviour allows each shared-data object and each portion of the source code (i.e., context), to be optimized independently of other objects and contexts. Once the programmer has selected an optimization, the high-level semantics of the optimization are also carried across various software (and perhaps hardware) layers. For example, in an all-to-all data exchange, knowledge about the senders, the receivers, and the size and layout of the data to be communicated, can be important in the implementation of the optimization.. We begin by illustrating and summarizing Aurora's programming model. Then we illustrate how two multi-phase applications, a two-stage matrix multiplication and parallel sorting, require a bulk-data transfer and can be optimized using scoped behaviour. Finally, we demonstrate how the Aurora programs can outperform comparable implementations using a DSM system and a message-passing system on a cluster of workstations with an ATM network. When developing applications for distributed-memory platforms, such as a network of workstations, shared-data systems are often preferred for their ease-of-use. Therefore, researchers have experimented with a number of DSM and DSD systems. A system that provides the benefits of a shared-data model and that can achieve performance comparable with a message-passing model is desirable.. The Aurora DSD system takes an abstract data type approach to a shared-data model. Aurora achieves good performance through flexible data-sharing policies and by optimizing specific data-sharing patterns. What distinguishes Aurora is its use of scoped behaviour to provide per-context and per-object flexibility in applying data-sharing optimizations.. Scoped behaviour is both an API to a set of system-provided data-sharing optimizations and an implementation framework for the optimizations. As a framework, one advantage of scoped behaviour is how it carries semantic information about specific data-sharing patterns across software layers and enables specialized per-object and per-context protocols. We have described how, when loading a read cache in matrix multiplication, the scoped behaviour is specified by the application programmer. Furthermore, the knowledge that all processes must participate in the bulk-data transfer and all of the data must be transferred is passed down to Aurora's run-time layer and exploited using a bulk-data transfer protocol. A similar protocol is used when exchanging keys in a parallel sort.. In a comparison of the two applications implemented using Aurora, TreadMarks, and MPICH, the performance of Aurora is comparable to or better than the other systems. MPICH appears to suffer performance problems due to its reliance on TCP/IP for bulk-data transfers, even in a high-contention all-to-all pattern. TreadMarks suffers from its reliance on a request-response data movement protocol. In contrast to MPICH, Aurora uses UDP/IP for bulk-data transfers to avoid the protocol bottlenecks of TCP/IP. In contrast to TreadMarks, the scoped behaviour pipelines the data transfer to avoid the message and latency overheads of TreadMarks's request-response protocol. The pipelining is possible because the scoped behaviour encapsulates the fact that all of the data must be transferred and the system can be more proactive instead of reactive. Consequently, on a network of workstations connected by an ATM network, Aurora generally outperforms the other systems in the situations where a bulk-data transfer protocol is beneficial.

O B I O O O O B I O O O O B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O B O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O B B I O O O O O O O O O O B O O O O O O B O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O B I O O O


Identifying program, test, and environmental changes that affect behaviour Developers evolve a software system by changing the program source code, by modifying its context by updating libraries or changing its configuration, and by improving its test suite. Any of these changes can cause differences in program behaviour. In general, program paths may appear or disappear between executions of two subsequent versions of a system. Some of these behavioural differences are expected by a developer; for example, executing new program paths is often precisely what is intended when adding a new test. Other behavioural differences may or may not be expected or benign. For example, changing an XML configuration file may cause a previously-executed path to disappear, which may or may not be expected and could be problematic. Furthermore, the degree to which a behavioural change might be problematic may only become apparent over time as the new behaviour interacts with other changes.We present an approach to identify specific program call dependencies where the programmer's changes to the program source code, its tests, or its environment are not apparent in the system's behaviour, or vice versa. Using a static and a dynamic call graph from each of two program versions, we partition dependencies based on their presence in each of the four graphs. Particular partitions contain dependencies that help a programmer develop insights about often subtle behavioural changes. When developers make a change they focus on two primary questions: "Did I successfully implement the intended change?" and "Did I break anything else?" Most variations in behaviours between the original and modified executables provide evidence useful for answering these questions. Confidence that a change performs as expected may, for example, be increased when tests for the newly added feature pass. Concern that a change may have inadvertently broken something is raised, for example, when a regression test fails.. Some behavioural variations, however, may not be easily isolated or identified as pertinent to either question. Such a variation may be benign, may represent a subtle bug, or may only manifest as benign or buggy over time as it interacts with other behaviours. Our approach intends to help developers more easily identify specific variations of this kind, allowing them to thoughtfully decide whether the variations of immediate or long-term concern.. A developer can evolve a software system's behaviour in at least three ways: changing the program source code, modifying the test suite, and altering the environment. Given a set of changes in one or more of these dimensions, we extract both a static and a dynamic call graph from each of two program versions, and partition the calls based on their presence in each of the four graphs. Some partitions contain dependencies that are unlikely to concern the developer; for example, if new tests are added alongside new methods in the program source, these will be captured in the partition that contains pairs that are not observed in the original version but are observed both statically and dynamically in the new version. Other partitions isolate pairs that likely deserve more scrutiny; for example, a call that appears dynamically without any corresponding change to the program source may occur because of a change to a configuration file that is not apparent in the source code.. Our contributions include:. • Defining and implementing a simple but novel mechanism for partitioning program changes in terms of static and dynamic method calls.. • Identifying which of these partitions represent consistent, inconsistent, unchanged, unlikely and unexecuted changes to the program's dependencies; the inconsistent category represents those changes that likely deserve deeper developer attention. By focusing on a developer's current change, rather than the aggregation of all past changes, we are able to return a small subset of pertinent dependencies for the developer to examine.. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. • Evaluating our approach over 10 versions of three different open source systems, demonstrating various quantitative and qualitative properties of our approach. For example, we show that fewer than 1% of the extracted pairs are of likely interest to the programmer with respect to behavioural changes. • Applying our approach to an industrial code base for one complete development sprint. The limited feedback we received suggests that the behavioural information can provide a useful alternative view for tracking system evolution. Our industrial partner valued the information that our approach provided enough to request that it be incorporated into their existing nightly build system. Section 2 details our approach for defining these partitions and categorizations; it also provides a concrete scenario to illustrate our partitioning approach. Section 3 presents the static and dynamic extraction tools we use, and how we reconcile static and dynamic calls with one another. Section 4 reports on application of our approach to three open source systems, describes our initial industrial application, and discusses threats to validity. Section 5 covers key related work, and Section 6 concludes. Software derives enormous power from its malleability. Experience shows that this power is often harder to harness in practice than in theory. One particular way in which this flexibility is often compromised arises when the behaviours of the executable program are hard to understand from the program's source. Even when this association is clear in a program's initial design and implementation, this relationship tends to becoming increasingly opaque as successive changes are made to the system.. We have presented an approach intended to concisely identify specific dependencies that suggest to programmers when a program change and the subsequent program behaviours may be less consistent than they may have intended. The approach relies on existing techniques and tools to extract static and dynamic dependence graphs from pairs of versions, along with set-based manipulations that partition these dependencies based on their absence or presence in the four graphs. We have argued, theoretically, that this partitioning provides an opportunity for finer-grained and more concise results than classes of existing approaches; and we have argued empirically that the partitioning identifies a small set of apparently useful dependencies that can be difficult to succinctly identify using current analysis approaches. Software changes. Literature arguing that changes to behaviours are especially daunting includes: Brooks' observation that defects tend to arise from changes that have nonobvious system-wide ramifications [7, p. 123 ]; Ko et al.'s evidence that feedback about the fidelity of their changes proved to be the developers' most-sought piece of information [9] ; and Sillito et al.'s study documenting that developers are keenly interested in the impact of their changes [20] . By providing developers feedback as they modify their system, we aim to help them identify dependencies that are otherwise difficult to isolate and that are likely to help build confidence in their understanding of these questions.. Unintended consequences of changes to programs are widely documented. Since 1985, the Risks Digest has documented thousands of computer-related risks to the public; 6 many of these, with effects from trivial to catastrophic, have been traced to unintended consequences of changes to programs. The observation that fixing errors is itself an erroneous process -often called imperfect debugging -has been modelled as part of software reliability engineering since the mid-1970's [12] .. Belady and Lehman [10] have argued empirically that program change is inevitable (in part) because the needs of users evolve. Their Law of Increasing Complexity asserts that as programs change they become increasingly less structured (unless such entropy is consciously counteracted). Repeated behavioural changes alongside structural degradation naturally lead to a progressively opaque relationship between the static and the dynamic structures.. Exceptions in logical structural differencing [8] identify static changes that may have not been made consistently and completely. A study of refactoring argues that in practice some flurries of refactorings lead to an increase in bugs, while other flurries do not [24] .. Not all program changes cause problems. Purushothaman & Perry [14] , based on an extensive study of a major commercial system, have shown that about 10% of changes altered a single line of code and of these about 4% resulted in additional faults. They also showed that approximately 40% of all changes resulted in additional faults. Although rarely observed in the literature, these data mean that (at least in this study) roughly 60% of all changes and 96% of one-line changes improve, or at least do not harm to, the program.. Nonetheless, few, if any, would argue that programmers make changes with absolute certainty about their possible consequences when the program is executed. Our approach, like all other approaches, does not and never will lead to a situation in which programmers can attain such certainty. Rather, our approach is meant to provide one way among many in which programmers can increase their confidence that a change improves a program in intended ways. Impact analysis. Another very broad area of related work is impact analysis, which is generally concerned with identifying possible consequences of program changes [2] . A number of impact analysis approaches can be contrasted to our approach based on our partitions.. One common approach to impact analysis for regression testing relies on comparing two static dependence graphs: "most techniques select tests based on information about the code of the program and the modified version" [18, p. 529] . A classic example is safe regression test algorithms that eliminate all tests from an original program's test suite that cannot (under specified conditions) expose a fault in the modified program; Rothermel & Harrold have analyzed many variants [18] , and a meta-analysis of empirical results is available as well [6] . Another example is test prioritization, which orders a test suite to increase the likelihood that newly introduced paths in the modified program are tested before unchanged paths. An example is Echelon [21] , which exploits binary differencing of versions to identify paths and tests to exercise. We believe that our technique augments existing regression testing approaches by recording more behavioural data that can be further analyzed in the future, regardless of whether an explicit assertion in a regression test failed.. Person et al. apply differential symbolic execution to characterize the behavioural aspects of a code change more precisely than traditional diff-based mechanisms [13] . While the focus of their work is proving whether a change affected the behaviour of a system our approach focuses on partitioning and implicitly ranking behavioural changes, albeit with less precision, in a lightweight way that developers can easily reason about.. Any analysis based on comparing static dependence graphs across two versions can distinguish precisely three combinations of partitions: those with a label including s -that is, s, sd, sd + and sd − -as well as those with a label including s + and those including s − . Regardless of the kind of static dependence graphs that are extracted and regardless of the algorithm used to compare the two static call graphs, other distinctions cannot be made: for example, a dynamic dependence that appears or disappears cannot be determined using this approach.. Another collection of impact analysis approaches execute two distinct test suites across a single program. Software reconnaissance uses this approach to identify parts of a program that implement a particular feature: the feature is exercised by the first test suite, but not the second [25] . Eisenberg and de Volder have extended this approach to relax the explicit requirement of exhibiting and non-exhibiting test suites [5] . Reps et al. [17] used a similar approach to identify programs that might be susceptible to problems such as Y2K. The Tripoli system [19] can be used to compare two arbitrary executions of a system and determine their coverage differences, assuming that the source code remains unchanged. This class of approaches, based on two dynamic graphs and one static graph, can exploit up to eight partitions. Some distinctions available in our approach, however, cannot be made using these partitions. As an example, s cannot be distinguished from s − , as only one static dependence graph is present. The degree to which this matters is empirical: if distinctions like this one arise in practice and represent useful information for the programmer, then our approach can provide additional information over these approaches. In addition, while traditional coverage metrics would report the same coverage percentage if one statement was removed and another added to the same method, our approach would capture both of these changes for the developer to consider.. Ren, Chesley, and Ryder present an approach to help perform root cause analysis (RCA) to identify the change that caused a test case to fail [16] . The AVA technique [3] further extends RCA to include rationale that can differentiate between successful and passing tests to differentiate passing behaviour from failing behaviour. Our approach aims to detect behavioural changes whether the test suite succeeds or fails but does not try to determine the root cause of any behavioural change. Raghavan et al.'s Dex tool extracts a variety of metrics about C patches using syntactic and semantic differencing; in contrast to our approach, Dex enables defect classification to enhance test prioritization [15] .. One form of impact analysis that is incomparable using our partitioning is that based on historical information [27, 22] . These approaches generally look for patterns in source code repositories that likely represent co-dependences among, for instance, checked-in files; a programmer could be alerted upon attempting to commit a set of files that omit a resource that has usually been edited and checked-in alongside those files. These approaches are complementary, exploiting historical information not directly related to the sourcebehaviour relation.. Several previous approaches have used mixed analysis, using both static and dynamic dependence analysis in concert. They generally, and very reasonably, apply static and dynamic dependences to distinct parts of the problem, exploiting the strengths and weaknesses of each style of analysis. For example, Chen et al. [26] perform selective retesting of a system by using static analysis to determine which parts of a system changed and then comparing this to dynamicallyderived coverage data. Rohatgi et al. [1] perform a software reconnaissance-like approach to extract features by comparing dynamic traces, following up by ordering the returned components based on their static relationships to one another. In contrast, our approach treats and manipulates the static and dynamic dependence graphs as peers, instead leveraging them by labelling specific partitions in terms of the underlying program source and its behaviour. We selected the 10 most recent change sets from each project's source repository that involved committing a modification to a source code file; a few changes to documentationonly files were ignored. To analyze the effects of each commit we compared all consecutive pairs of versions for each of the systems; that is, we considered 27 changes across three systems. Ten of these changes involved fixing a bug and updating a test, three involved adding a feature and updating a test, two involved fixing a bug without a test, three involved adding a feature without a test, two changed source code documentation, and the rest were simple refactorings or code cleanups.. We extracted the static dependence graph from each version and collected the dynamic dependence graph by running each project's entire test suite (using the tools described in Section 3). The external environment -libraries and runtime environment -for every version of the same system was held constant. The unlikely partitions were always empty as expected (and thus we do not discuss them further).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Performance driven reliable link design for networks on chips Abstract-With decreasing feature size of transistors, the interconnect wire delay is becoming a major bottleneck in current Systems on Chips (SoCs). Another effect of shrinking feature size is that the wires are becoming unreliable as they are increasingly susceptible to various noise sources such as cross-talk, coupling noise, soft errors etc. Increasing importance of wire delay and reliability has lead to a communication centric design approach, Networks on Chip (NoC), for building complex SoCs.Current NoC communication design methodologies are based on conservative design approaches and consider worst case operating conditions for link design, resulting in large latency penalty for data transmission. In order to substantially decrease the link delay and thereby increase system performance an aggressive design approach is needed. In this work we present Terror, timing error tolerant communication system, for aggressively designing the links of NoCs. In our methodology, instead of avoiding timing errors by a worst-case design, we do aggressive design by tolerating timing errors. Simulation results show large latency savings (up to 35%) for the Terror based system compared to traditional design methodology. With continued scaling of transistors the wire delay as a fraction of the total delay is increasing [2] . The delay in crossing a chip diagonally for 50nm technology is around 6 to 10 cycles, with only a small fraction of chip area (0.6% to 1.4 %) being reachable in a single clock cycle [2] . Scaling is accompanied with a decrease in supply voltage and an increase in clock rate. This makes wires unreliable as the effect of various noise sources such as cross-talk, coupling noise, soft errors increase [3] , [4] , [5] . To effectively design future SoCs, Networks on Chips (NoCs), a communication centric design approach that considers the delay and reliability issues of the wires has been proposed in [1] , [6] .. As the total system performance increasingly depends upon the communication system, the NoCs need to support high throughput, low latency communication with low power requirements. To effectively tackle the delay of NoC links, the links can be pipelined by adding buffers (i.e. flip-flops) that segment the links into stages (Figure 1 ) [7] . Link pipelining increases the link throughput and reduces the average packet latency for data transmission. The number of cycles needed for a data bit to traverse a pipelined link is equal to the number of pipeline stages in the link. The number of pipeline stages in the link, in turn, depends upon the distance a data bit can travel on the link in one cycle and the length of the link. As the links are becoming increasingly susceptible to various noise sources such as cross-talk, coupling noise, local temperature variations, process variations, etc., the data bit delay on the link in a cycle is becoming unpredictable as the various noise sources introduce significant delay variations in the transmission of a bit. Link design in current NoC design methodolo- ... gies are based on a worst-case design approach that consider all the delay variations that can possibly occur due to the various noise sources and environmental effects and target a safe operation of the system under all conditions. Such a conservative design approach targets timing error-free operation of the system. In such a scheme, the distance traveled by a data bit is significantly lesser than the ideal case when no timing variations occur due to noise sources. Thus, a worst-case design methodology leads to poor system performance as the number of pipeline stages in the link and hence the link latency is much higher compared to the ideal case when no timing errors occur. An aggressive design approach (in which the data bit is assumed to travel the maximum distance possible in one cycle) can result in large reduction in latency for communication.. However, in such a system, timing errors can occur when the noise sources introduce delay variations in data communication.. As an example, let us consider an on-chip link that connects all the communicating blocks in an SoC designed in 100nm technology. Assuming a conservative design approach, the number of pipeline stages (and buffers) on the link is 6 (detailed explanation is given in section VIII). On the other hand, if the link is designed aggressively, the distance between successive pipeline buffers can be increased up to 50%, resulting in a 4-stage pipeline. Thus, a 33% reduction in latency is obtained by the aggressive design approach. But, in this aggressive design, timing errors can occur due to delay variations introduced by the environment and wire characteristics. Traditionally, such timing errors are detected by adding redundant bits to the data that is transmitted. Once an error is detected in the receiver, the receiver requests retransmission of the data. The latency and power overheads for retransmission can be quite high. As an example, with 5% error rate, the latency incurred in sending 1000 bits on a 4-stage pipeline link is 1553 cycles, while ideally (when there are no errors) it should take 1003 cycles. Here we have assumed that only the data bits that 749 8B-2 0-7803-8736-8/05/$20.00 ©2005 IEEE.

B O O B O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O