Gamification for smarter learning: tales from the trenches Gamification has been explored recently as a way to promote content delivery in education, yielding promising results. However, little is known regarding how it helps different students experience learning and acquire knowledge. In this paper we study and analyze data from a gamified engineering course, to search for distinct behavior patterns. We examined data collected from two gamified years, between which game changes took place. By clustering students according to their performance, we identified three distinct student types, common to both years: Achievers, Disheartened, and Underachievers. Interestingly, in the second year a new type of student emerged: the Late Awakeners. In this paper we carefully describe each student type, and explain how gamification can provide for smarter learning by catering to students with different profiles. Furthermore, we discuss how our findings, both in gamification and cluster analysis can be used to develop adaptive and smart learning environments. Good games are natural learning machines that, unlike traditional educational materials, can deliver information both on demand and within context (Gee, 2003) . By design, they prevent players from becoming either bored or frustrated, while allowing them to experience flow (Chen, 2007; Csikszentmihalyi, 1991) . Thus, it is not surprising that videogames have long been used to transmit knowledge (de Aguilera and Mendiz, 2003; Squire, 2003) , given the remarkable educational possibilities they unveil (Bennett et al., 2008; O'Neil et al. 2005; Prensky, 2001) .. Gamification draws on the motivational qualities of games to entice users to adopt specific behaviors. To this end, it includes game design elements in non-game processes (Deterding et al., 2011a; Deterding et al., 2011b) , to make them both more fun and engaging (Reeves and Read, 2009; Shneiderman, 2004) . Gamification has been used for many different purposes, ranging from marketing (Zichermann and Cunningham, 2011) , to fitness and health (Brauner et al., 2013) , helping in driving instruction (Fitz-Walter et al., 2013) , improving productivity (Sheth et al., 2011) and promoting eco-friendly driving (Inbar et al., 2011) .. Gamification in education is a growing topic and a variety of studies already show that gamification has the potential to significantly increase student activity (Denny, 2013) and performance (Domínguez et al., 2013) . However, little is known about how different students experience gamified learning and what different strategies and traits they present. Therefore, understanding how different students learn in gamified environments may provide a stepping-stone to develop new and adaptive gamified courses, capable of adjusting to different students' particular needs and thus improving their learning experience.. In a previous work (Barata et al., 2013) we described a two-year study to gamify a college course, Multimedia Content Production (MCP), in order to engage students and encourage them to be more participative. We found that students were both more participative and proactive as compared to previous non-gamified editions of the course. Furthermore, they also considered it as being more motivating and interesting than other "regular" courses. Yet, we knew nothing about how different students performed under our gamified experience, and how it could be designed to adapt to their different traits, needs and wants.. In this paper we describe how we used clustering algorithms to analyze student progression on two consecutive years and identify different student profiles. While we found three distinct student types on the first year, a fourth type emerged in the second year. We will present a detailed description of each student type and propose cluster analysis in our gamified approach as a tool to support adaptive gamified learning environments. In a previous experiment we have gamified a college course on Multimedia Content Production. This included adding game elements such as experience points and levels, badges and a leaderboard to the learning experience. Additionally, course activities were shaped into meaningful endeavors, such as achievements and time-limited tasks, namely Challenges and Online Quests. Results show that students participate more in activities with gamification and also report the course as being both more motivating and interesting than non-gamified courses.. In this paper we presented an exploratory study where we tried to understand how different students experienced our gamified learning experience. We analyzed how they acquired XP over the term, during two years of gamified delivery, and adopted a clustering algorithm to group them according to different XP acquisition patterns. We identified three clusters common to both years: 1) Achievers, which participated the most and strived to be the best, 2) Underachievers, which had the lowest performance and participation levels and seemed to lack interest for the course, and 3) Disheartened students, whose initial performance was closer to that of Achievers but soon dropped to average levels. A fourth cluster, which we named Late Awakeners, emerged in the second year, where a few changes to improve game rewards and to create more participation opportunities took place. These students were characterized by an XP accumulation pattern opposite to that of Disheartened students: their performance dropped during the first weeks, but then recovered to levels closer to Achievers'. Still, our learning experience has yet to appeal to all students. Indeed, Disheartened students and Underachievers seemingly are not taking as much advantage of the gamified experience as the others do, and might require specific help. We believe that our clustering mechanism based on student XP acquisition patterns can be used to train a classifier and develop an adaptive gamified learning experience, which might prove useful in improving performance by dynamically tailoring student learning experiences and courseware delivery in real-time. Our analysis suggests that our learning environment can cater to the different student groups observed in our course. This effect seems more predominant in the second year, with a new student cluster emerging, while we observed a significant reduction of the Underachiever population from 40 % down to 25 %. In this section we describe each cluster, analyze how they evolved from one year to the next, and discuss how the improvements made in the second year seemed to affect learner attitudes and behavior. We finalize by exploring how the results of this study could be used to further improve our gamified learning approach, making it smarter and more adaptive to students. Several studies reported that using serious games to teach resulted in significant improvements in student understanding, diligence and motivation, at different academic levels, from grade school (Lee et al., 2004) , through high school (Kebritchi et al., 2008) , to college (Coller and Shernoff, 2009) , and in diverse fields of knowledge, such as biology (Mcclean et al., 2001) , programing (Moreno, 2012) , numerical methods (Coller and Shernoff, 2009) or electromagnetism (Squire et al., 2004) . This suggests that games can indeed provide smart ways to improve people's learning experience and outcomes.

B O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Code injection attacks on harvard-architecture devices Harvard architecture CPU design is common in the embedded world. Examples of Harvard-based architecture devices are the Mica family of wireless sensors. Mica motes have limited memory and can process only very small packets. Stack-based buffer overflow techniques that inject code into the stack and then execute it are therefore not applicable. It has been a common belief that code injection is impossible on Harvard architectures. This paper presents a remote code injection attack for Mica sensors. We show how to exploit program vulnerabilities to permanently inject any piece of code into the program memory of an Atmel AVR-based sensor. To our knowledge, this is the first result that presents a code injection technique for such devices. Previous work only succeeded in injecting data or performing transient attacks. Injecting permanent code is more powerful since the attacker can gain full control of the target sensor. We also show that this attack can be used to inject a worm that can propagate through the wireless sensor network and possibly create a sensor botnet. Our attack combines different techniques such as return oriented programming and fake stack injection. We present implementation details and suggest some counter-measures. Worm attacks exploiting memory-related vulnerabilities are very common on the Internet. They are often used to create botnets, by compromising and gaining control of a large number of hosts.. It is widely believed that these types of attacks are difficult, if not impossible, to perform on Wireless Sensor Networks (WSN) that use Mica motes [21, 11] . For example, Mica sensors use a Harvard architecture, that physically separates data and program memories. Standard memoryrelated attacks [1] that execute code injected in the stack are therefore impossible.. As opposed to sensor network defense (code attestation, detection of malware infections, intrusion detection [22, 4] ) that has been a very active area of research, there has been very little research on node-compromising techniques. The only previous work in this area either focused on Von Neumann architecture-based sensors [10] or only succeeded to perform transient attacks that can only execute sequences of instructions already present in the sensor program memory [12] . Permanent code injection attacks are much more powerful: an attacker can inject malicious code in order to take full control of a node, change and/or disclose its security parameters. As a result, an attacker can hijack a network or monitor it. As such, they create a real threat, especially if the attacked WSN is connected to the Internet [20] . This paper presents the design of the first worm for Harvard architecture-based WSNs. We show how to inject arbitrary malware into a sensor. This malware can be converted into a worm by including a self-propagating module. Our attack combines several techniques. Several special packets are sent to inject a fake stack in the victim's data memory. This fake stack is injected using sequences of instructions, called gadgets [23] , already present in the sensor's program memory. Once the fake stack is injected another speciallycrafted packet is sent to execute the final gadget chain. This gadget uses the fake stack and copies the malware (contained in the fake stack) from data memory to program memory. Finally, the malware is executed. The malware is injected in program memory. It is therefore persistent, i.e., it remains even if the node is reset.. Our attack was implemented and tested on Micaz sensors. We present implementation details and explain how this type of attacks can be prevented.. The paper is structured as follows: Section 2 introduces the platform hardware and software. The major difficulties to overcome are detailed in Section 3. Section 4 presents the related work. Section 5 gives an overview of the attack, whose details are provided in Section 6. Protection measures are introduced in Section 7. Finally, Section 8 concludes the paper and presents some future work. This paper describes how an attacker can take control of a wireless sensor network. This attack can be used to silently eavesdrop on the data that is being sent by a sensor, to modify its configuration, or to turn a network into a botnet.. The main contribution of our work is to prove the feasibility of permanent code injection into Harvard architecturebased sensors. Our attack combines several techniques, such as fake frame injection and return-oriented programming, in order to overcome all the barriers resulting from sensor's architecture and hardware. We also describe how to transform our attack into a worm, i.e., how to make the injected code self-replicating.. Even though packet authentication, and cryptography in general, can make code injection more difficult, it does not prevent it completely. If the exploited vulnerability is located before the authentication phase, the attack can proceed simply as described in this paper. Otherwise, the attacker has to corrupt one of the network nodes and use its keys to propagate the malware to its neighbors. Once the neighbors are infected they will infect their own neighbors. After few rounds the whole network will be compromised.. Future work consists of evaluating how the worm propagates on a large scale deployment. We are, for example, interested in evaluating the potential damage when infection packets are lost, as this could lead to the injection of an incomplete image of the malware. Future work will also explore code injection optimizations and efficient countermeasures.

B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A fully resolved active musculo-mechanical model for esophageal transport Esophageal transport is a physiological process that mechanically transports an ingested food bolus from the pharynx to the stomach via the esophagus, a multilayered muscular tube. This process involves interactions between the bolus, the esophagus, and the neurally coordinated activation of the esophageal muscles. In this work, we use an immersed boundary (IB) approach to simulate peristaltic transport arXiv:1501.02010v1 [physics.comp-ph] 9 Jan 2015 in the esophagus. The bolus is treated as a viscous fluid that is actively transported by the muscular esophagus, which is modeled as an actively contracting, fiberreinforced tube. A simplified version of our model is verified by comparison to an analytic solution to the tube dilation problem. Three different complex models of the multi-layered esophagus, which differ in their activation patterns and the layouts of the mucosal layers, are then extensively tested. To our knowledge, these simulations are the first of their kind to incorporate the bolus, the multi-layered esophagus tube, and muscle activation into an integrated model. Consistent with experimental observations, our simulations capture the pressure peak generated by the muscle activation pulse that travels along the bolus tail. These fully resolved simulations provide new insights into roles of the mucosal layers during bolus transport. In addition, the information on pressure and the kinematics of the esophageal wall due to the coordination of muscle activation is provided, which may help relate clinical data from manometry and ultrasound images to the underlying esophageal motor function. Interactions between fluids and deformable structures are widespread in biological systems, and such interactions often involve complex moving interfaces and large structural deformations [1] . Esophageal transport is one such process, whereby the food bolus is transported to the stomach via the esophagus. The esophagus is a flexible, multi-layered tube that consists of mucosal, interfacial, circumferential, and longitudinal muscle layers. The pumping force required to produce this peristaltic transport process is generated by neurally coordinated muscle activation along the esophagus [2, 3] , and accounting for the full physiological details of the transport process is challenging. Simplified analytical models can provide some insights into this biophysical process [4] but are often limited in their scope. More complete models are needed to investigate esophageal pathophysiology, such as motility disorders, and hold the potential to advance diagnoses and patient treatment.. Current studies on the modeling of esophageal transport have focused on specific, albeit important, subproblems, such as characterizing the material properties of each layer of the esophagus tube [5, 6, 7, 8, 9, 10] , investigating the flow of bolus with specified time-dependent bolus geometry or known lumen pressure [4, 11, 12] , and estimating the muscle active tension based on known time-dependent pressure distribution [13, 14] . To the best of our knowledge, however, there is presently no computational model of esophageal transport that couples models of the bolus, the esophageal structure, and the muscle activation within an integrative numerical model. This work presents one such integrative model of esophagael transport that is based on the immersed boundary (IB) method [15] . The IB method is an approach to modeling fluid-structure interaction that was introduced to simulate the fluid dynamics of heart valves [16, 17] , and which has subsequently been applied to a broad range of problems in biology [15] . The IB method uses an Eulerian description of the momentum and incompressibility of the coupled fluid-structure system along with a Lagrangian description of the structural forces produced by the elasticity or active tension generation of the structure. The primary advantage of this formulation is that it avoids the need to employ body-fitted grids, and thereby eliminates the need to develop complex remeshing strategies as the immersed structure deforms [15, 18] . In this paper, we present a fully resolved fluid-structure intreaction model of esophageal transport, which includes detailed descriptions of the esophageal wall, the bolus, and their interaction. The model is fully resolved in the sense that it does not assume simplified fluid dynamics or structural deformations.. In our model, the majority of the computational domain is occupied by the immersed body, with a fluid (i.e., the bolus) confined in a narrow lumen. Thus, it is important to describe the mechanical response of the esophagus tube. To that end, we discretize the continuous fibers of the esophagus into springs and beams and associate a volumetric patch with each such spring and beam. This allows us to compute the spring and beam parameters from the material properties of the esophagus (e.g., its Young's modulus). To test the fiber-based esophagus tube model, we simulate the problem of dilation of a three-dimensional tube and compare the numerically obtained inner fluid pressure with an analytically derived solution.. To simulate bolus transport in a physiologically realistic manner, an esophagus model is constructed as a four-layered structure. Muscle activation that results in the peristaltic motion of the esophagus is modeled via springs with dynamic rest lengths. To handle the numerical challenge arising from large deformations of the mucosal layer, we employ a locally refined structural discretization that ensures that the Lagrangian structure does not "leak" even under very large deformations [15] . We consider three cases with different muscle activation models and mucosal layer fiber arrangements, and we discuss the key features related to muscle cross-section area and pressure peaks during bolus transport. In this work, we introduce a method based on the volumetric patch to characterize the elasticity of the immersed fiber-based structure employed in the typical IB method [15, 19, 21] . Model verification is performed via comparisons between the computational results and an analytic solution to an idealized tube dilation problem. Low relative errors (5%) are obtained. To study the esophageal transport, we develop a fully resolved active musculo-mechanical model that is able to incorporate the liquid bolus, multi-layered esophageal wall, and muscle activation into a unified model. We present three cases of the esophageal transport that differ in the choice of muscle activation model and mucosal fiber arrangement, thereby demonstrating the capabilities and generality of the model. The key feature of bolus transport observed experimentally is a "tear-drop" bolus driven by a muscle contraction wave. This is captured in our simulations. Moreover, new insights are also provided by fully resolved simulations. The simulations show that perfect synchrony between LM shortening and CM contraction leads to an overlap of the high pressure region and an increased muscle CSA. This helps to relate clinical test data from manometry and ultrasound image to the underlying neurally-controlled activity, such as the coordination between CM contraction and LM shortening. Detailed information on kinematics elucidates the role of the mucosal layer in shaping the bolus. Specifically, the mucosal layer provides distensibility to the esophageal lumen, and lubricates the running bolus. We remark that this is the first study that directly looks at the interaction between the bolus and the mucosal layer. Future work should include a parametric study of the effect of changing mucosal property on bolus transport. This will help to understand certain esophageal diseases that are related to the inflammation of the mucosa.. and viscous terms vanish. Thus, eq. (26) becomes. For the fiber-based structure, let R and r denote the initial and deformed radial coordinates, respectively, at each material point. At the boundaries of the tube, let R i and R o denote the initial inner and outer radius, and r i and r o denote the current inner and outer radius, respectively. Then we can obtain the relationship: R = R(r), based on volume conservation and plain-strain deformation:. The elasticity of the fiber-based tube is represented by three families of springs: circumferential, radial, and axial springs. In the (r, θ) plane, only deviatoric stresses along the radial and circumferential orientations are nonzero,. Here, S r and S θ are the moduli of radial and circumferential fibers, respectively. At steady state, the inertial term in the momentum equations of fluid and structure vanishes, and we have ∇·σ e = 0. Therefore, along the r-direction we get. This gives us the equation for p e as dp e dr = dσ rr dr. We impose zero-traction boundary condition on the four lateral surfaces of the fluid domain. At steady state, this implies that the exterior fluid pressure is zero, so that traction continuity at the exterior fluid-solid interface implies,. The solution to eq. (32) for p e with the boundary condition given by eq. (33) reads as. The inner pressure, P inner is obtained from eq. (27) as. Under dilation and in the absence of shearing motions, the radial fibers will become compressed. For compression-resistant fibers such as those used in this model, this is an unstable configuration, and the minimum energy configuration is obtained when the layers of the tube rotate to release the energy of compression. In this configuration, the radial fibers do not contribute to the elastic stress. Therefore, by letting S θ = S and S r = 0, the fluid pressure at the inner surface r i becomes. Substituting eq. (28) into eq. (36), we obtain the explicit form of P inner as. Here C = r . In (r, θ) plane, the patch of a radial spring connected by nodes (r, θ, z) and (r+∆r, θ, z) is (r, r + ∆r) × (θ − 0.5∆θ, θ + 0.5∆θ). The patch of a circumferential spring connected by the nodes (r, θ, z) and (r, θ + ∆θ, z) is (r − ∆ 1 r , r + ∆ 2 r ) × (θ, θ + ∆θ) for a < r < b; (r − ∆ 1 r , r) × (θ, θ + ∆θ) for r = b and (r, r + ∆ 2 r ) × (θ, θ + ∆θ) for r = a. Here ∆ 1 r = r − r(r − ∆r) and ∆ 2 r = −r + r(r + ∆r) are such that the circumferential patch is partitioned into two equal volumes. (b) Schematic of patches (hatched areas) associated with radial springs (blue lines) and axial springs (magenta lines) in (r, z) plane. The red curves denote the boundaries of the tube: the inner (r = a), outer (r = b), lower (z = 0) and the upper surface (z = l). In (r, z) plane, the patch of a radial spring connected by nodes (r, θ, z) and (r + ∆r, θ, z) is (r, r + ∆r) × (z − 0.5∆z, z + 0.5∆z) for 0 < z < l; (r, r + ∆r) × (z − 0.5∆z, z) for z = l and (r, r + ∆r) × (z, z + 0.5∆z) for z = 0. The patch of an axial spring connected by nodes (r, θ, z) and (r, θ, z + ∆z) is (r − ∆ 1 r , r + ∆ 2 r ) × (z, z + ∆z) for a < r < b; (r − ∆ 1 r , r) × (z, z + ∆z) for r = b and (r, r + ∆ 2 r ) × (z, z + ∆z) for r = a.

O O O O O O O B I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Resource-Aware Compiler Prefetching for Fine-Grained Many-Cores Abstract Super-scalar, out-of-order processors that can have tens of read and write requests in the execution window place significant demands on Memory Level Parallelism (MLP). Multi-and many-cores with shared parallel caches further increase MLP demand. Current cache hierarchies however have been unable to keep up with this trend, with modern designs allowing only 4-16 concurrent cache misses. This disconnect is exacerbated by recent highly parallel architectures (e.g. GPUs) where power and area per-core budget favor numerous lighter cores with less resources, further reducing support for MLP on a per-core basis. Support for hardware and software prefetch increases MLP pressure since these techniques overlap multiple memory requests with existing computation. In this paper, we propose and evaluate a novel Resource-Aware Prefetching (RAP) compiler algorithm that is aware of the number of simultaneous prefetches supported, and optimized for the same. We implemented our algorithm in a GCC-derived compiler and evaluated its performance using an emerging fine-grained many-core architecture. Our results show that the RAP algorithm outperforms a wellknown loop prefetching algorithm by up to 40.15% in run-time on average across benchmarks and the state-of-the art GCC implementation by up to 34.79%, depending upon hardware configuration. Moreover, we compare the RAP algorithm with a simple hardware prefetching mechanism, and show run-time improvements of up to 24.61%. To demonstrate the robustness of our approach, we conduct a design-space exploration (DSE) for the considered target architecture by varying (i) the amount of chip resources designated for per-core prefetch storage and (ii) off-chip bandwidth. We show that the RAP algorithm is robust in that it improves performance across all design points considered. We also identify the Pareto-optimal hardware-software configuration which delivers 53.66% run-time improvement on average while using only 5.47% more chip area than the bare-bones design. Memory systems have been under a lot of pressure to keep up with the increasing demand for parallelism coming from every new generation of microprocessors. Superscalar, out-of-order processors can have a large number of memory operations in flight in the execution window at one time. In simultaneous multi-threading (SMT) architectures, as well as multicores and manycores, the demand for Memory-Level Parallelism (MLP) has further increased. This has put additional pressure for memory systems to support numerous concurrent memory requests.. Current cache hierarchy designs support only limited amounts of MLP due to the high cost in terms of chip area and energy use. Existing architectures employ lock-up free caches (e.g. [30] ) to avoid stalling the CPU and allow the cache miss to be serviced in the background. Figure 1 depicts a cache system and its attached Miss Handling Architectures (MHA). This consists of several Miss Information/Status Holding Register (MSHR) files, and is responsible for keeping track of the outstanding concurrent misses. To meet the demand for high bandwidth and low latency, each MSHR has its own comparator, and is a small fully associative cache. The maximum number of outstanding cache misses the system supports is limited by the number of MSHR entries.. When a new request is received, all the comparators must be activated in parallel in order to retrieve the corresponding entry in one clock cycle, leading to high power consumption and area requirements. This severely limits the size of the MSHR file that can be included, even for today's large transistor budgets. For example, the L1 cache of an Intel Pentium 4 processor supports only 8 outstanding misses [4] . For more recent AMD Opteron and Intel Core i7 architectures, an empirical study showed that single thread performance does not improve past 7 concurrent memory requests, suggesting that the same limitation holds [25] .. Currently, the two predominant paradigms of how single-chip processors are built are: 1. Limited-scale multi-cores that replicate the single-processor model on one die and strive to maintain backwards compatibility. They generally target applications with low degrees of parallelism, programmed to take advantage of local caches and limit expensive inter-core communication. Presently such systems have 2-12 cores, each supporting coarse grained threads and they are not expected to exceed a few tens of cores in the foreseeable future. 2. Many-cores that are not typically confined to traditional architectures and programming models, and use tens to hundreds of lightweight cores in order to provide stronger speedups. The best known representatives of this class are GPU architectures from NVIDIA and AMD, which are now increasingly used for general-purpose computation that offer abundant parallelism and can be effectively mapped to these GPUs. Other examples include the Sun Niagara 2 [24] and the upcoming Intel Many Integrated Core (MIC) architecture. Research machines such as UT Austin's TRIPS [9] , MIT's RAW [28] (available commercially as the TILE architecture from Tilera [1] ), and UMD's XMT [33, 36] are also examples of single-chip many-cores.. Traditional multi-cores in the first category use several methods to hide memory latency: out-of-order execution, advanced hardware prefetching mechanisms, large per-core coherent caches, simultaneous multi-threading (SMT) etc. These often mitigate the need for advanced software prefetching. Hence we do not target traditional multicores. In contrast, in many-core architectures, these techniques do not scale to the hundreds (and possibly thousands) of cores because of the limited area and power resources available per core, leading to the urgent need for alternative latencyhiding tools. In our work, we are focusing on this challenging, unsolved problem for a proposed high-risk, high-payoff research many-core machine.. Many-cores are motivated by a variety of factors such as high scalability to a large number of lightweight cores in a single chip, very low inter-core communication latencies, high on-chip communication bandwidth, overcoming wire-length restrictions at high clock speeds, and fast hardware-assisted inter-core synchronization. Some manycore designs have demonstrated great results, far exceeding the performance of traditional multi-cores for the same silicon area. Such designs are an exciting new frontier in computer architecture, as evidenced by commercial and research interest.. Regardless of the type or motivation, in all many-cores each lightweight core has much smaller area than a traditional core. In terms of prefetching resources, this has two repercussions: (i) the size (number of entries) of MSHR files is much more constrained in a many-core due to area constraints; and (ii) the total energy consumption of all MSHRs across cores in a processor is much higher than in a traditional processor due to the larger number of cores, which further limits the size of MSHRs. As a consequence it becomes crucial to carefully manage the use of scarce MSHR resources for these architectures.. We present a new prefetching method called Resource-Aware Prefetching (RAP) to manage MSHR resources carefully. It is mainly beneficial in many-cores because of their limited MSHR file capacity. It can apply to traditional multi-cores as well, but given their larger number of MSHR entries and their much larger area and power budget for other latency tolerating tecniques mentioned above, the run-time gains from RAP are likely to be very small, hence we do not discuss those further.. To evaluate the RAP algorithm, we are using XMT 1 -a general-purpose manycore architecture [23] . A recent study showed that when configured to use the same chip area, XMT can outperform both an Intel Core 2 (speedups up to 13.83× [6] ), AMD Opteron (speedups up to 8.56× [36] ) and also an NVIDIA GTX280 GPU (speedups of up to 8.10× [5] on irregular workloads). The XMT hardware design is simple and scalable, allowing researchers to easily understand and tune architectural parameters. A highly configurable cycle-accurate simulator and an open-source compiler are available, enabling us to implement and evaluate our optimizations on different hardware configurations. An overview of the XMT platform is presented in Sect. 4. An alternate evaluation platform would have been GPUs from NVIDIA or AMD; however they are not used because in the current generation there is no true support for software prefetching in these platforms. Moreover, the vendors do not disclose architecture or compiler details, making it difficult to implement and evaluate hardware or software improvements.. As an extension of our evaluation, we investigate the ability of the RAP algorithm to adapt to the available amount of prefetching hardware resources for a particular target platform. To accomplish that, we conducted a design-space exploration (DSE) of the XMT architecture in which we varied the size of the MSHR file in order to accommodate more simultaneous prefetch instructions. For a more complete DSE, we also varied the resources allocated for off-chip bandwidth (i.e. memory controllers and DRAM channels), which is the other resource closely related to memory-level parallelism. We present our extensive case study in Sect. 6 and show that the RAP algorithm robustly improves performance on all evaluated design space points and across a heterogeneous set of benchmarks. In addition, we show that a non-obvious choice for allocating the available chip area between the two resources provides the best performance, and we make recommendations to the hardware designers for future improvements.. Our main contributions are:. 1. Resource-aware compiler loop prefetching algorithm. We propose an enhanced compiler loop prefetching algorithm and compare it with existing solutions. We show that by providing the compiler with accurate information about the hardware, we can improve the prefetch performance on average by up to 36.7% over a hardware-oblivious one, depending on the hardware configuration. 2. Empirical design space exploration We scale two parameters of the architecture:. (i) off-chip DRAM bandwidth and (ii) capacity of MSHR files, and combine performance results with area information to determine the Pareto-optimal configurations. These are design points such that no other configurations have better performance for a lower area. Such configurations represent meaningful trade-offs for the hardware designer; all others can be eliminated. We maintain our focus on the memory sub-system and pay special attention to both software and hardware support for data prefetching. For example, our findings show that a 64-processor configuration with 2 DRAM channels and 8-word per-TCU prefetch buffer unit capacity outperforms a similar configuration with 8 DRAM channels and 4-word prefetch buffer units by 4% on average, while actually using 4.3% less chip area.. The remainder of this paper is organized as follows. In Sect. 2 we review existing compiler prefetching approaches and identify their shortcomings, and in Sect. 3 we present our enhanced Resource-Aware Prefetching (RAP) algorithm. Section 4 introduces XMT, our experimental platform, which is then used in Sect. 5 to evaluate the performance improvements of our RAP implementation. In Sect. 6 we present the findings of the Design-Space Exploration study, followed by a related work survey in Sect. 7 and our conclusions in Sect. 8. We presented RAP-an improved compiler loop prefetching algorithm targeted at many-core architectures, and showed that under resource constrained scenarios it outperforms Mowry's well known loop prefetching algorithm by up to 40.15%, the GCC improved implementation by up to 34.79% and a simple hardware prefetching scheme by up to 24.61%. The RAP algorithm is robust, providing considerable improvements and never falling behind significantly on any of the hardware configurations tested, making it a timely and necessary addition to compilers targeting fine-grained manycore architectures.. In addition, we conducted a Design-Space Exploration focused on resources directly affecting support for Memory-Level Parallelism on an emerging many-core architecture, the XMT Paraleap. We identified the Pareto-optimal hardware-software configuration which delivered 53.66% performance improvement on average while using only 5.47% more chip area than the bare-bones design. The methodology of evaluating each design point by adapting the RAP algorithms to the prefetch resources available is also of interest. Prefetching is a widely studied technique used to hide the increasingly high latencies (in terms of clock cycles) of memory accesses in modern architectures. Software prefetching [7, 19, 21] relies on the existence of non-blocking prefetch instructions and is usually enabled by the compiler. In hardware prefetching (e.g. [14, 18, 27] ) a specialized hardware unit infers prefetching opportunities by monitoring run-time behavior. Prefetching schemes for parallel architectures in both software [19, 20, 31] and hardware (e.g. [8] ) build upon uni-processor prefetching by taking into consideration issues caused by sharing of data and resources, such as coherence traffic and overheads.. Several studies have considered the interaction of the architectural parameters with the performance of software prefetching algorithms. In his comprehensive work on software data prefetching, Mowry [22] explores the effect on execution time of varying the number of outstanding prefetch requests that can be handled simultaneously by the hardware. The author also compares two versions of the prefetch issue buffer hardware-one in which the processor stalls when the buffer is full, and one where additional requests are simply dropped. Their results were mixed, with the former performing slightly better when using the prefetching algorithm described in Sect. 2. In follow-up work, Mowry [20] , as well as McIntosh [19] , settle for a fixed-size prefetch issue buffer of 16 locations. Several other papers study the effects of changing the size of the prefetch destination (either cache or dedicated prefetch buffers) for systems with software [7, 15] or hardware prefetching [14] . However, unlike our approach, in all of these existing schemes the prefetch algorithm is unaware of the prefetch hardware configuration, and does not adapt its behavior.. GCC is the only attempt to consider the amount of prefetch resources available as part of the loop prefetching algorithm. As described in Sect. 1, the GCC algorithm limits the number of memory references prefetched to meet a fixed upper bound. However, no guidance is given on how to chose this upper bound, as it is not clear what the underlying hardware limitation is accounted for. Yang et. al [37] empirically set the maximum number of prefetch instructions issued by the GCC compiler for the IA64 platform to 12. However, their study does not address the underlying limitations of the GCC algorithm discussed above. In our approach, we identify the hardware resource dictating the maximum number of prefetch requests allowed (the MSHR file), and provide an original scheduling algorithm which limits the prefetch distance instead of the number of references prefetched, and show that it outperforms both Mowry's and GCC's implementations.. In recent work, Tuck et al. [30] propose an alternate MHA organization that would provide increased MSHR capacity at the expense of slightly higher latencies. However, it is not clear how the performance of the scheme would change when scaled to the degree of parallelism required by the emerging manycore architectures. Different schemes for improving the functionality of existing MHAs, by either dynamically adjusting the MHA capacity to reflect the memory bus load [13] or by devising a MHA-aware cache replacement policy to reduce the number of cache misses [26] have been proposed. Our compiler algorithm is orthogonal to these techniques and can function alongside these hardware enhancements.. In the area of design-space exploration, recent years have seen a burst of studies targeted at understanding the interactions between performance, energy, thermal efficiency and area in the design of chip multi-processors (CMPs), or multi-cores. Huh et al. [12] explore the design space of CMPs as the available transistor budget grows with the fabrication technology. They consider in-order vs. out-of-order cores, amount of cache per processor and availability of off-chip bandwidth, while varying the area constraints. Li et al. [16] extend the study by including pipeline depth and width, operating voltage and frequency and cooling mechanisms as design space dimensions, as well as adding thermal constraints. Methods for automatically or semi-automatically design space exploration for specialized architectures such as System-On-Chip and signal processing have also been proposed [10, 17] . These studies are all focused on share-nothing workloads, optimizing architectures designed for throughput rather than single-task completion time.. The research presented in this paper differs from prior work in that it focuses on MLP resources-MSHR capacity and DRAM channels-for design space exploration. It also does so in conjunction with a resource-aware compiler algorithm that efficiently uses the limited resources at each design point. This study is targeted at the needs of lightweight manycore architectures which aim to optimize single-task completion time, an emerging architectural paradigm.

O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O B O O O O O B O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O


Relevance judgments between TREC and Non-TREC assessors This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment. Results show that 63% of documents judged relevant by our users matched official TREC judgments. Several factors contributed to differences in the agreements: the number of retrieved relevant documents; the number of relevant documents judged; system effectiveness per topic and the ranking of relevant documents. Relevance assessments are of critical importance to the evaluation of information retrieval systems. The Text REtrieval Conference 1 (TREC) has established an evaluation practice where a binary relevance scale is combined with liberal relevance criteria. However, the low threshold for relevance criteria followed in TREC has been criticized in affecting the ability to identify and develop IR methods capable at retrieving highly relevant documents [1] . Voorhees [5] studied the effect of variations in relevance assessments on the measured retrieval effectiveness. She found 32.8% agreement between official TREC judges and a set of new assessors. Despite this apparently low number, she concluded that different relevance assessments did not affect the ranking of systems in the TREC evaluation. Recent studies have examined graded relevance: Sormunen [2] used a four-point scale to compare agreement between relevance judgments and to reassess document pools for 38 TREC topics. He found 39% of documents rated relevant by TREC assessors were similarly rated relevant by a set of new assessors. Vakkari & Sormunen [4] explored the consistency with which users could identify relevant documents. They found that users could identify 45% of documents previously judged relevant by TREC assessors. Similarly, a recent study by Turpin, & Scholer [3] showed 45% agreement between TREC assessment and a group of student users. All of these previous studies [2] [4] [5] are based on re-assessing documents pools of TREC topics.. In this study, we examine the overlap in agreement between official TREC assessments and the relevance judgments created by new users within an Interactive IR experiment. The overlap is defined as the size of intersection between the sets of documents judged relevant by TREC assessors and our users, divided by the size of set of documents users judged as relevant. In this paper, we also explore circumstances in which users strongly disagree with official TREC assessments. Results show that 63% of documents judged relevant by our users matched official TREC judgments. One explanation for this agreement could be that TREC topics used in this experiment (and associated relevance) were clear and lack ambiguity. This high agreement might indicate that when a retrieval system believe documents are relevant, human are also likely to agree on relevance. These findings help us understand the potential impact of using interactive studies to generate a test collection. Of the 2,262 documents judged relevant by our users (R(U)), 1,428 (63%) were also assessed relevant in TREC (we also label it as consistent judgment); while 834 (37%) were assessed irrelevant (NR(T)) in TREC (we also label it as inconsistent judgment) as shown in Table 1 . Of the 1,428 consistent judgments, users rated 37% of the documents as partially relevant and 63% as highly relevant; while of the 834 documents they rated 61% as partially relevant and 39% as highly relevant. Out of the 2,133 irrelevant judgments (NR(U)) made by our users, 308 (14%) documents were regarded as relevant in TREC. 3 The document directly addresses the core issue of the topic. 4 The document only points to the topic: it does a not discus the themes of the topic thoroughly.. Copyright is held by the author/owner(s). SIGIR'08, July 20-24, 2008, Singapore. ACM 978-1-60558-164-4/08/07. lower level of agreement for documents ranked lower. The highest agreement was at Rank 1 and it decreased as the rank increased. Relevance judgments where assessments differed between TREC assessors and our users were not evenly distributed over the 56 topics and the 56 users. 82% 79% 76% 75% 75% 75% Eighty percent of documents that our users had rated irrelevant contrary to TREC came from 25 topics and 24 users, while 80% of documents that our users had rated relevant contrary to TREC came from 29 topics and 33 users. We have examined whether the same users who judged documents relevant (contrary to TREC assessment) were the same who judged documents irrelevant (contrary to TREC assessment). This was to investigate whether the same users constantly disagreed with TREC judgments, however, the resulting correlation did not support this assumption (r=-0.16, p<.000). Similarly, we investigated the same effect based on individual topics (whether users judged documents relevant and irrelevant contrary to TREC assessment occurred in the same set of topics). According to the resulting correlation (r=-0.13, p<0.000) topics did not have an effect on users' disagreeing with TREC assessment.. System effectiveness (as measured by MAP), per topic, did effect users' agreement with TREC assessors. It was found that users consistent judgements with TREC assessors correlated (r=0.4) significantly more (p=0.00) with system effectiveness than their inconsistent judgements (r=-0.06). The low correlation between inconsistent judgements and system effectiveness indicated that users' disagreement with TREC judgment increased as the system effectiveness decreased. For example topic 302 with MAP=0.55, total documents judged relevant by our users were 11, 10 of them were judged consistent with TREC assessment (relevant) whereas only one document was judged inconsistent with TREC. On the contrary, topic 420 with MAP=0.11, total documents judged relevant by the users were 12, 9 of them were regarded irrelevant by TREC assessors and only three documents were judged consistently relevant with TREC assessment. This meant that people disagreed more with TREC judgements on topics with lower system effectiveness, and consequently agreed more on the topics with higher system effectiveness.. We further analyzed possible causes for disagreements between our users and the TREC assessors. We found that as the total number of retrieved relevant documents for a particular topic decreased users tended to select proportionally more documents inconsistent with TREC judgments (r=0.02, p<.000), whereas users tended to rate documents more consistently with TREC assessments (r=0.65) as the total number of retrieved relevant documents increased. For example, topic 410, the total number of retrieved relevant documents was 9, users judged 4 documents inconsistently relevant with TREC while only 1 document was judged consistent with TREC and the rest were not identified by the users. On the other hand, topic 415, the total number of retrieved relevant documents was 34, users judged 13 documents consistently relevant with TREC while only 3 documents were judged inconsistent with TREC. Thus, a small number of potentially relevant documents in the retrieved sets lead users to accept more documents inconsistently relevant with TREC assessment. We also found that as the number of relevant documents judged increased (consistent and inconsistent with TREC), users tended to rate documents more consistently with TREC assessments (r=0.83) as compared to the inconsistent ratings (r=0.63). For example topic 415 mentioned above, users judged in total 16 documents, the majority of them were consistent with TREC (13). This suggested that if users judged few relevant documents, they may have felt obliged to continue saving further documents (relevant or irrelevant with regards to TREC assessments) due to the fear of "failure" to complete the set task. For example, topic 356, the total number of relevant documents judged by the users was 5; only 1 document was consistently relevant with TREC judgments while 4 were inconsistent.. The lack of inconsistency in relevance assessments between our users and TREC assessors stemmed from the following conditions: users who retrieved few documents and judged few relevant documents appeared to relax their criteria for relevance, accepting more documents as relevant contradicting TREC assessments. Moreover, some users had difficulty identifying relevant documents because they simply found some TREC topics harder than others. This signifies that we are likely to get lower agreement in relevance judgments for topics that relevant documents are difficult to find. Therefore, in interactive IR studies which make use of TREC test collections, when these conditions occur, care should be taken when comparing user effectiveness with system effectiveness.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A stable and convergent scheme for viscoelastic flow in contraction channels We present a new algorithm to simulate unsteady viscoelastic flows in abrupt contraction channels. In our approach we split the viscoelastic terms of the Oldroyd-B constitutive equation using DuhamelÕs formula and discretize the resulting PDEs using a semi-implicit finite difference method based on a Lax-Wendroff method for hyperbolic terms. In particular, we leave a small residual elastic term in the viscous limit by design to make the hyperbolic piece well-posed. A projection method is used to impose the incompressibility constraint. We are able to compute the full range of unsteady elastic flows in an abrupt contraction channel -from the viscous limit to the elastic limit -in a stable and convergent manner. We demonstrate the range of our method for unsteady flow of a Maxwell fluid with and without viscosity in planar contraction channels. We also demonstrate stable and convergent results for benchmark high Weissenberg number problems at We = 1 and We = 10. Published by Elsevier Inc.PACS: 65N06; 76D05 In this paper, we present a numerical method for solving the equations describing incompressible viscoelastic fluids such as dilute polymer solutions . Here, l s is the solvent viscosity, and the non-Newtonian contribution of the stress is given by the Oldroyd-B model. where s r is the upper-convected Maxwell (UCM) derivative (see Eq. (24) below) and D is the rate of deformation. If l s vanishes, then this system is often referred to as the Maxwell model. We choose the Oldroyd-B model as it is sufficiently well-understood to serve as a test bed for numerical methods. Our approach is based on the method of Bell, Colella and Glaz (BCG) [4] , a second-order accurate version of ChorinÕs projection method [6] . We use a continuous splitting of the relaxation terms so that, in the limit of vanishing k, we recover a BCG-type algorithm for a Newtonian fluid with an augmented viscosity. In particular, the time step is controlled only by the advective CFL number in that limit. In the elastic limit k ! 1, l p /k finite, we obtain a method whose time step is controlled only by the CFL condition for the elastic shear waves. Kupferman employed a BCG-type projection method with staggered centering to study stability of viscoelastic fluids in Taylor-Couette flow [15] . Our method differs from the one presented there in that (1) in general, we consider high Weissenberg number flows in the presence of geometric singularities; (2) our method uses a much less restrictive time step in the viscous limit; and (3) we use fixed cell centering which is more amenable to the approaches taken in [2] for adaptive mesh refinement and [11] for embedded boundaries.. Numerical simulation of viscoelastic flow in contraction channels ( Fig. 1 ) is a problem that has been given much attention over the past 30 years because of its importance to polymer processing [8, 7] . Much has been published on the inability to compute strongly-elastic, ''high Weissenberg number'' flows -ones for which the ratio of the elastic relaxation time scale to the advective time scale is large -particularly in the presence of geometric singularities such as re-entrant corners. These limits have been observed primarily in the context of numerical simulations of steady flows. Keunings [14] used a finite-element method to compute steady 2D flows of Maxwell fluids through 4:1 planar sudden contractions. The observed critical values of We were less than one (0.112-0.873, depending on the mesh). Lipscomb et al. [18] repeated the Keunings calculations with the single change -the relaxation time was set to zero in the reentrant corner element to control the stress singularity -increasing the critical We from 0.1 to 0.6 with no noticeable difference in flow outside of the corner element. Many attempts followed, of which a detailed literature review and definition of the benchmark for viscoelastic flow are given by Phillips and Williams [20, 21] who used a semi-Lagrangian finite volume method to observe steady-state solutions for both creeping and inertial flows in planar contractions, obtaining results up to We = 2.5. They also clarified definitions of Weissenberg and Deborah (De) numbers, useful for comparing high We flow to previous studies in high De as in [19] . Alves et al. [3] present similar results for benchmark solutions of Oldroyd-B inertialess flows up to De = 3. Xue et al. [28] presented a finite volume method for both 2D and 3D problems and observed a critical value of We = 4.4. Recently, Aboubacar et al. [1] employed a hybrid finite volume/finite element scheme for Oldroyd-B creeping flows in rounded and abrupt contractions, and observed a critical value of We = 4.6.. The high Weissenberg number problem is actually a frustratingly low Weissenberg number problem for steady flows beyond which stable and convergent calculations have not been achievable. There appears to be no analytic expression for the limiting value of the Weissenberg number, and considerable indication that the critical value depends on the mesh resolution. One of the principal conclusions of this paper is that the difficulties that have been observed in the prior work may be an artifact of using steady-state or implicit methods, rather than an appropriately-designed unsteady flow method that can resolve the hyperbolic wave behavior of the problem. This is consistent with the point of view taken in [27, 13] , in which the elastic Mach number -the ratio of the fluid velocity to the elastic wave velocityis identified as the critical parameter in understanding viscoelastic flows. In particular, we find that our method is stable and accurate for any value of the Weissenberg number, provided that the elastic Mach number is less than one. This restriction on the elastic Mach number may be a consequence of the use of a Lax-Wendroff discretization for the hyperbolic terms, rather than a well-designed upwind method suitable for use in situations when discontinuities might spontaneously appear. We will explore this possibility in future work.. A preliminary version of our results appears in [25] . We demonstrate a stable and convergent numerical method for viscoelastic flow in abrupt contraction channels. The method is different from prior steady-state or implicit methods in that it is an appropriately designed unsteady flow method which can capture wave motion in strongly elastic flows for any value of the Weissenberg number and is only restricted by subcritical elastic Mach number. The method requires a single CFL condition for the range of elastic flows, from the viscous limit to the elastic limit. This is achieved through a variation of the BCG projection method which obeys the desired viscous and elastic limits of the equations of motion. In steady-state problems where the elastic Mach number is supercritical hyperbolicity is maintained by imposing a slightly smaller timestep than that computed by the stability condition. For Maxwell fluids we demonstrate unsteady results for arbitrarily high We and Ma < 1 where elastic shear waves propagate in an incompressible fluid and are captured by the method in a stable and convergent manner. The method has also been shown to be stable and convergent for unsteady high Weissenberg number flows of Maxwell fluids with viscosity in abrupt contractions. More experimentation is needed to quantify the extent of the behavior (vortex enhancement, for example) in these flows. The method also calculates the viscous limit of the Oldroyd-B equation, with a time step restricted only slightly more than that governed by the advective velocity alone. The method compares very well to known exact solutions and preliminary data from experimental flows in microfluidic devices [25] .. Finally, we show a stable and convergent calculation of high Weissenberg number flows for We = 1 and We = 10. We demonstrate our method for the benchmark problem and show that the algorithm is capable of resolving non-singular structure in the stress along the contraction wall a short distance (O(1) relative to h) downstream of the re-entrant corner. We note that in the many previous works on the benchmark problem, summarized in [20, 21, 3] , the methods employed break down around We = 2 -that is, they fail to achieve steady-state, and they do not converge numerically. This observed critical Weissenberg number with Re = 1 corresponds to the critical elastic Mach number, Ma = 1, where waves are no longer suppressed at supercritical values. We notice this wave behavior gradually with increasing We. For supercritical flows steady-state solutions are not necessarily attainable, but stable and numerically convergent results are. We are able to capture this behavior, as demonstrated above with the results for Maxwell flows with and without viscosity, and also with higher Weissenberg number flows in the benchmark parameter regime. Our calculation of a higher Weissenberg number, We = 10, Ma = 2.3, is a first for this regime of flows. We do not observe a limiting value of We in benchmark problems. In the future we will modify the scheme in several ways. To improve the accuracy of viscous flows to second-order, we will implement the implicit Runge-Kutta scheme of Twizell, Gumel and Arigu in [26] for the viscous stresses as it is known to be L 0 -stable in the presence of boundaries, unlike Crank-Nicolson [11] . We will also take advantage of the ability to cast the PDEs (5) and (7) in a hybrid conservation form which exploits hyperbolicity while maintaining the viscous and elastic limits. This means we will naturally extend the method to make use of the higher-order Godunov method of the BCG formulation, allowing us to eliminate any subcritical Mach number restriction. Higher-order Godunov methods are more robust than LW for hyperbolic systems; and the CFL time step is an order of magnitude larger than that of the LW method used here. Therefore, for certain applications such as particle-laden flow in microfluidic devices and other complex microscale geometries where sharp gradients (or even waves) exist it would be advantageous to develop BCG for viscoelasticity. Finally, though biological flows in the planar geometries of microfluidic devices are the application of interest, a more robust test of the algorithm would be to simulate vortex enhancement seen in experimental flows through axisymmetric contractions. In this we would also like to extend the scheme to other viscoelastic constitutive equations because the Oldroyd-B model does not provide a realistic description of polymeric fluids, though it serves well as a test bed for the numerical method. FENE-type equations, for example, are more physical as they model nonlinear elastic effects in the fluid. We apply the method to unsteady flow of Oldroyd-B fluids in abrupt planar contraction channels (Fig. 1) . First, we simulate the Oldroyd-B fluid without viscosity (l s = 0) in sudden contraction channels to demonstrate stable and convergent calculation of the flow of an unsteady Maxwell fluid, which allows finite elastic wave propagation. We then add some viscosity to the Maxwell fluid and demonstrate stable and convergent simulations at high We in abrupt contractions for unsteady flow with hybrid viscous and elastic behavior. Next we present results for the viscous limit of an Oldroyd-B fluid in abrupt contraction channels. Finally, we show stable and convergent results for benchmark high Weissenberg number problems at We = 1 and We = 10. We characterize our results with the elastic Mach number and Weissenberg number where the characteristic velocity, U, is the average velocity in the contraction channel, or the average inflow velocity times the contraction ratio. We demonstrate accuracy of the method for each case. For viscous flows convergence results are obtained away from the inlet of the channel in order to isolate the re-entrant corner and to omit known singularities at the inlet due to prescribed constant velocity boundary conditions. The method is valid for both 2D and 3D flows, though only 2D results are shown here for simplicity in demonstrating the robustness of the algorithm. For 3D results we refer to our previous work in [25] .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O