Online social networks and offline protest Large-scale protests occur frequently and sometimes overthrow entire political systems. Meanwhile, online social networks have become an increasingly common component of people's lives. We present a large-scale longitudinal study that connects online social media behaviors to offline protest. Using almost 14 million geolocated tweets and data on protests from 16 countries during the Arab Spring, we show that increased coordination of messages on Twitter using specific hashtags is associated with increased protests the following day. The results also show that traditional actors like the media and elites are not driving the results. These results indicate social media activity correlates with subsequent large-scale decentralized coordination of protests, with important implications for the future balance of power between citizens and their states. In the last two decades, large-scale protests have increased in frequency; these protests often lead to mass casualties, change countries' political systems, and sometimes lead to war, as is the case in Iran in  or Libya and Syria in . Here we present results from a large-scale longitudinal study that connects these two worlds, showing that coordination on social networks correlates with protest offline. Online social networks have become an increasingly common component of people's lives and are used to predict box office returns [] , stock market fluctuations [] , social collective phenomena [-], and even private traits such as sexual orientation and political ideology [] . They can also be harnessed for crowd sourced searching [] and get-out-the-vote campaigns [] . Using . million geolocated tweets and machine-coded data on protests from  countries during the Arab Spring, we show that coordination of messages on Twitter is associated with increased protests the following day. The large-scale, systematic study we present here provides extensive evidence that social media may help decentralized groups coordinate online to organize protests offline. These results suggest that individuals in a wide range of countries use social media to organize large-scale protests and not only at the height of protest events; coordination is a perpetual, systematic activity. The results therefore extends existing work which focuses on protest events in one country [-] .. The Arab Spring protests started in Tunisia on December th, , and eventually led to the resignation of that country's president on January th, . They spread to neighboring countries over the following weeks, inspiring massive turnout in Egypt that caused President Hosni Mubarak to resign on February th, . In the aftermath, many other countries experienced protest, with some, such as Libya and Syria, quickly turning into civil wars.. Accounts of the Arab Spring have frequently credited Twitter, Facebook, and other social network sites with helping the protests self-organize [] . Indeed, there are many reasons to expect that social media did play a role. When deciding whether or not to protest, individuals have to estimate the risk of arrest or violent state repression, and they have to weigh that cost against the potential benefits of any change in policy that may result. They are primarily interested in how many other individuals are going to protest and whether those participants are first-time protesters or not [] . Protesting en masse decreases the chance an individual will personally experience state violence and increases the probability of achieving policy change. Individuals wanting to protest are therefore strongly incentivized to coordinate their protest action with others. We have used a large dataset that covers thousands of protests in many countries to reveal a relationship between people coordinating on social media and increased protests the following day. Past work suggests that individuals are more likely to protest when their close friends or neighbors are protesting [] and when they have prior experience with protest [] , but the results here suggest that weak ties can also facilitate mobilization [] . They do so by exposing individuals to information about participation from outside of their local, strong-tie social network, allowing those who are on the fence about protesting to approximate how widespread, and therefore safe, the protests are. In this sense, social media helps people not only to learn about protests, but also to see that others are learning about them, aiding in coordination [] .. These results do not mean that there will always exist a relationship between coordination and subsequent events. For example, in democracies, where the cost of protesting is lower, individuals may not need to coordinate with each other to protest. Or in countries with free media, coordinating may be more likely to come from media sources than individuals. Future work should investigate these boundaries.. A growing literature suggests that new technology lowers violence [] and decreases a state's control of information [] . But as states learn how to control information and communication technology [] , they have also learned how to use it to track individual participants in protest [] and activities the state has defined to be illegal [] . By making coordination visible to a broad audience, activists can facilitate protest, but they also make their coordination visible to the state, enabling targeted repression. Thus, online social networks may help individuals protest, but they also help states repress protesting individuals. Future research should try to discern which of these effects prevails. Figure  reveals a robust statistical relationship between yesterday's level of coordination and today's number of protests. To reach this conclusion, we use a negative binomial regression model where the dependent variable is the count of protests at time t and the primary independent variables is the coordination at time t -. The coordination measure has a p-value less than . (coefficient of . and standard error of .), and a one standard deviation increase in the coordination measure is associated with a .% increase in the number of protests on the following day. Figure  also shows that the result holds when a number of potential other methods of coordination are modeled. The percent of tweets that contain a hashtag indicates the extent to which people are contributing to a tagged conversation, in some cases about protests. When a high percent of tweets are retweeted, a smaller number of original tweets drive the conversation. The percent of tweets that contain links indicates that more people are referencing an important blog or news item. And the percent of tweets that mention other users indicates that more direct communication is happening between people on Twitter. Retweets and hashtags are especially conducive to spreading information that may have a coordinating effect. None of these measures correlate as strongly with protests on the following day as coordination, and none of them are significantly associated with protests after controlling for coordination. A more detailed presentation of the model and controls is reported in the Supplementary Materials. We also show in the Supplementary Materials that these results are robust to model type (linear vs. negative binomial regression), protest data source (a handcoded dataset and a different machine-coded one deliver the same results), and serial correlation in errors (multiple tests suggest that including a lagged dependent variable is sufficient to deal with the problem, and including several extra lags does not change the results). All models account for unobserved between-country differences with country fixed effects (accounting for stable characteristics like country size and political history), and they control for unobserved sources of variation over time with day fixed effects (helping to control for day-of-week variation in Twitter activity or special events like Ramadan that affect all countries in the study). Moreover, a model that drops high protest days also shows a significant relationship, suggesting that the association is not driven by a few large events. Finally, to ensure that the results are not driven by individuals trying to draw international attention to the protests [], we drop all English tweets; the results remain the same.. An important question about the potential effect of social media coordination is its source -is it decentralized or does it come from traditional sources like news media and political activists? In the Supplementary Materials, we add to our models a number of measures of Twitter activity from these traditional sources [] , including the percent of tweets on a given day that were sent by media organizations, media personalities, digital activists, and the top % most active Twitter users. However, these models with controls continue to show a similar association with hashtag coordination, with an effect for media organizations and no effect for digital activists. Coordination therefore appears to come through decentralized activity of many individuals, not the frequent tweeting of a few specialized actors. Figure  shows the prevalence of the hashtags in Bahrain, Egypt, Morocco, and Qatar most often used for coordination during the study period. They were chosen by observing the most common hashtag in a country on a given day, counting how many times that hashtag was the most common during the study period, and keeping the  most common. In Egypt, three features stand out. First, there is little coordination on any one hashtag before January th. The hashtag '#egypt' is barely used, the first appearance of '#jan' is not until January th (and then accounts for only .% of tweets), and '#tahrir' does not appear until January th (and it does not appear in large numbers until just before the resignation of Mubarak). Second, which hashtag is most prevalent depends on the type of upcoming event. During the  days of initial protest, '#jan' dominates, as this was the focal date for the protests. Though the largest protests while Mubarak was in power took place in Tahrir Square, they were contentious, which is why more general hashtags such as '#jan' and '#egypt' dominate. Moreover, '#jan' consistently declines in usage after the  days of protest. By the middle of March, it will never be the most common of the three hashtags again, and it ceases to correlate with the other two. Third, overall levels of coordination decrease after the first  days. They first decrease sharply after President Mubarak's resignation, and their average prevalence gradually continues to decline. '#tahrir' might be an exception, as it is used much more narrowly to coordinate specific, Tahrir Square-centric events, but the frequency of its usage declines as well.. In Bahrain, there is a dramatic spike in coordination almost immediately after the start of protests on February th. This coordination is then sustained throughout the year, with increases and spikes around important events. We include in Figure  the prevalence of the hashtag #lulu -though it is never the most common hashtag, it is the one used to Hashtags were chosen by determining which were most often the most common hashtag on a given day across all days in the period of study and tracking the 4 most common overall. discuss events concerning the Pearl Roundabout, the main focus of protestor activity in Bahrain. #lulu is not in our dataset before February th, and its use varies over time in more specific patterns than #Bahrain. Finally, note the focus on events in Egypt at the end of the Egyptian protests and starting again in mid-October.. Morocco experienced much less protest, as shown in Figure  , and Figure  shows they also experienced less coordination. The initial protests in Morocco occurred on February th, when coordination peaked, but otherwise the level of coordination is not on the same scale seen in more contentious countries. Note also that Moroccans did not appear to take as much interest in events occurring elsewhere in the Middle East and North Africa, suggesting that they may have felt less affinity towards protestors in other countries. Similarly, Qatar exhibits low levels of coordination and no attempts at organizing protests. The day with the highest level of hashtag coordination is December nd, , when Qatar learned it was chosen to host the  World Cup. However, people in Qatar paid close attention to the events in #Libya, the third most common hashtag (and, not shown, hashtags about the Egypt protests ranked th and th).

O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B O O O O O B I O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O O O B O O O O B O O O O O O O O O O O O O O B I O O O O O O O O B O O O O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O


A Virtual Laboratory Model for Encouraging Undergraduate Research Undergraduate students who perform research benefit greatly from the experience, yet achieving high levels of voluntary participation remains an elusive goal. This paper describes the implementation of an innovative laboratory model designed to encourage computer science undergraduates to actively pursue collaborative research with faculty and other students. As an extension of earlier research into team-based software engineering education, the Applied Computing Technology Laboratory was formed to provide an authentic and engaging experience in realworld computer science research. The model, which relies on a web site as its focus, is presented along with the results of one year of active research. Preparing undergraduates for life beyond graduation motivates many computer science programs to structure software engineering and capstone courses so that students are immersed in an industry-like experience. The exposure to common industry practices, business methods, and many of the real-world complexities inherent in an ambitious, team-based software project is valuable [4, 5, 9, 10] . Providing students with authentic research experiences within the context of a course is difficult; research can involve literature reviews, planning, design, prototype system implementation, experimentation, and writing, all of which require a significant investment in effort over a relatively long period of time. Educators can limit the scope of research projects in order to introduce students to aspects of research, typically as part of courses in software engineering and capstone projects, an approach that has provided good results [6, 8] .. Offering research opportunities outside of the classroom is an attractive idea, although student participation hinges on making the experience more worthwhile than other activities on which students may spend their free time. Designing engaging research problems of an appropriate scope, providing independent study credit, and otherwise capturing the imagination of students is critical.. A number of successful research programs have demonstrated that meaningful undergraduate research is an achievable goal [2, 3, 7] , with a heavy emphasis placed on teamstructure, formal research approaches and highly selective criteria for participation. This paper presents an ambitious research laboratory model that builds upon the successes of these previous approaches. The result of this model is the Applied Computing Technology Laboratory (ACT Lab) which expands the idea of undergraduate research to create an environment inspired by such exciting and dynamic places as the MIT Media Lab (media.mit.edu) and Xerox PARC (parc.xerox.com). Upon joining the ACT Lab team, undergraduate students are encouraged to find innovative ways to use computing technology to solve any problem that catches their fancy. The ACT Lab is not a part of any class, and members need not be upper division or even computer science students, most of the current members came through software engineering and capstone projects classes taught with a company-based approach [10] that primed them for participation in research. The model is that of a "virtual laboratory" in that it requires no laboratory space of its own (yet), existing solely as a web site; when students need access to equipment, guidance or information, they invent ways to acquire what they need. Faculty members work with student researchers, when needed, offering advice, mentoring and handson collaboration. Students learn to do research by doing research, and they do it because it is their own.. Formed nearly one year ago, the ACT Lab is already producing intriguing software tools and a number of publishable results in a wide variety of areas. The model for the ACT Lab, including its organization, motivation and benefits, and a summary of a number of current projects, are the subjects of the remainder of this paper.. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. By creating a virtual laboratory, such as the ACT Lab, undergraduate students can be drawn into research by providing a personal stake in all aspects of a project. While faculty-directed research is likely to always be the hallmark of graduate-level research, this student-directed, faculty-guided model of research may prove to be an attractive tool for encouraging otherwise reluctant undergraduates to participate enthusiastically in research.. The formation of the ACT Lab has increased student interest in undergraduate research, contributing to our department's efforts to boost student participation through partial research scholarships, research seminars and inclusion of undergraduates in funded research projects. Positive feedback from recent graduates who participated in a classroom implementation of this applied approach to research [10] have been encouraging.. A plan for long-term assessment of the impact of the ACT Lab on those involved is under development.. In addition to overseeing the ongoing evolution of the ACT Lab within its home department and university, plans are being made to broaden its scope to other departments and universities, and to actively pursue collaboration with industry.

O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Estimating domestic US airline cost of delay based on European model Abstract-Researchers are applying more holistic approaches to the feedback control of the air transportation system. Many of these approaches rely on economic feedback, including the cost of delays to the airlines. Establishing an accurate mechanism for estimating the cost of a delays for each portion of a flight (gate costs, taxiing in and out costs, and en-route costs) is useful for many aspects of modeling airline behavior and for better understanding the likely impact of regulations.EuroControl (2004) developed a rigorous methodology and collected data for estimating the components of airline delay costs for various segments of a scheduled flight. The model, based on confidential information from European airlines for twelve types of aircraft circa 2003, was not transparent with regards to how each of the major components of cost (crew costs, fuel costs, maintenance, depreciation, etc) impacted that total. This paper describes the development of airline cost model, based on the Eurocontrol model. The airline cost model explicitly identifies the components of airline costs, is based on U.S. airline cost data, and includes 111 aircraft types. The new model is designed to allow costs to be updated whenever any of the factors (e.g. crew, fuel, maintenance, and ground costs) change. It considers the type of the aircraft when making calculations, both from the perspective of fuel burn and passenger costs. A case-study analysis of airline costs of operation at 12 major U.S. airports is provided.Keywords-component; airline delay costs; airline delays; economic modeling of airlinesThe cost factors from the EC report and costs as reported by US carriers in BTS P52 database follow similar trends. The appropriate multipliers for crew and maintenance costs are determined that, when combined with the other factors produced multipliers close to those reported in the EC report. Airborne delays, when incurred, dominate ground delay costs, so airlines are economically encouraged to maximize ground delay costs. Newer more fuel efficient aircraft provide airlines with the least delay costs.The cost of delay is not proportional to the flights flown. The airline industry moves millions of passengers and tons of cargo annually. Recent studies have estimated the cost of delays to the U.S. economy in 2007 ranging from $32.9 billion [NEXTOR, 2010] to $41 billion [JEC 2008] . Researchers have proposed holistic approaches to incentivize the development of increased capacity and improved productivity [Donohue et. al., 2008; Ball et. al., 2007] and feedback control of the air transportation system [NextGen, 2008; Xiong, 2010; Rupp, 2005] . These approaches rely on economic feedback, including the cost of delays to the airlines. An accurate model of the cost of a delay is not only of interest to the airlines that incur these costs, but is essential for air transportation policy, management, and control. Direct costs are accrued by airlines when flights are delayed. There are two main causes of flight delays:. (1) the flight does not depart due to aircraft or flight specific reasons (e.g. mechanical problems, misaligned crew or aircraft, crew work rules), or (2) mismatch between demand and capacity. At several highly utilized airports, systemic over-scheduling and reductions in capacity of both the airspace and the runways due to weather result in delayed flights. Based on weather forecasts and schedules, air traffic management estimates the resulting reduction in capacity within various segments of the airspace and at a variety of airports. It announces Ground Delay Programs (GDPs) that hold aircraft at the departing airport, in order to have the flying aircraft better match the capacity of the system. For capacity reduction in air, Air Flow Programs (AFPs) are employed that suggest/announce alternative routes for the flights. Since holding aircraft at a gate is both cheaper and safer than airborne holds, most delays are gate holds. Delays often propagate through the system, causing future delays, because the aircraft or crews may not arrive at their next assignment in time to allow the next flight to leave on time.. The Performance Review Unit, EuroControl published a report [EuroControl, 2004] describing a methodology for evaluating true cost of flight delays. The methodology presents results detailing the cost to airlines of delays during various segments of a scheduled flight. The costs are divided into short delays (less than 15 minutes) and long delays (greater than 65 minutes). The report provides a cost factor (Euros per minute) for each flight segment. The types of delays considered include gate delay, access to runway delay (both taxi in and out delays), en-routes delays, and landing delays (circling or longer flight paths to overcome congestion while approaching the airport). The data used in the study consisted of data collected from European airlines, air traffic management as well as interviews and surveys conducted by the research team. Although each of the factors making up the overall cost factors are explained, the individual factors are not provided because the information was considered proprietary. In the absence of this transparency, the factors provided prohibit the separation of fuel costs from crew or maintenance costs and prohibit an update of the summary factors when any of these costs change or when alternative aircraft need to be considered. Furthermore, the model is based on data from EU airlines for 12 aircraft types.. The motivation of this paper is therefore to: identify coefficients for the cost factors model each of the individual coefficients and cost factors update model with publicly available costs of U.S. airlines extended fleet mix to over 100 aircraft types structure the model to enable update of the data over various time periods This paper is organized as follows. Section II describes the EC report, Section III provides the methodology for determining the cost components and multipliers that make up the final multipliers used in the EuroControl report and describe the validation of the new model on European data from the period of the EC report. In Section IV and V, delay costs are examined for US airlines departures from 12 major airports (EWR, JFK, LGA, DCA, BWI, IAD, SFO, OAK, SFO, BOS, PHL, DFW) for one of the busiest months in US aviation history (July, 2007) . Delays by segment of flight, by aircraft type, by airline and by hour of day are examined in this case study. Section VI provides conclusions and Section VII points out the future research. From the analysis, the following conclusions are made:. The cost factors from the EC report and costs as reported by US carriers in BTS P52 database follow similar trends. Thus, the general approach taken by EuroControl can be applied, with minor modifications, to compute the cost of delays for US flights The appropriate multipliers for crew and maintenance costs are determined that, when combined with the other factors produced multipliers close to those reported in the EC report. Airborne delays, when incurred, dominate ground delay costs, so airlines are economically encouraged to maximize ground delay costs. Newer more fuel efficient aircraft provide airlines with the least delay costs. The cost of delay is not proportional to the flights flown. One reason for this non-intuitive result is that when a flight is cancelled, it is recorded as having zero delay. Future research will address how to cost cancelled flights.. The calculations of the cost of delayed flights (ignoring all cancelled flights) total $63.8M for July 2007.. Many economic modeling and analysis efforts require a good understanding of the costs that an airline will incur when it experiences delays at the gate, while taxiing or while en-route. This paper has presented a relatively straightforward mechanism for calculating such costs and for predicting how such costs are likely to increase when there is a change in fuel costs, aircraft type, or when some other cost might be added to the overall cost structure. It is informative in explaining why airlines are currently down-gauging the aircraft size: the newer regional jets are more fuel efficient and airborne fuel costs dominate the overall cost. Fuel costs, coupled with the fact that the airlines can offer increased frequency and observe higher load factors, encourage airlines to down-guage. Although such policies are favored by the industry, they result in less efficient use of both the airspace and airport runways. Future analysis will both expand and apply this model in a variety of efforts currently underway:. A mechanism for including the costs of cancellations in the overall cost calculations needs to be developed. The research of Xiong [2010] , Wang, et al. [2006] , Rupp [2005] , Sherry {2010] and Bratu & Barnhart [2005] will assist in this effort. Sensitivity analysis needs to be done on the model to determine how robust it is to significant cost changes in fuel or crew, and/or changes in aircraft usage. Having separated the cost factors into their component parts, alternative cost factors can be applied to a variety of aircraft types not studied in the EC model. Initial work in this direction is reported in Kara et al. [2010] .. Analysis, based on these costs, needs to be done to predict which flights are most likely to be cancelled or delayed when weather conditions result in the initiation of a Ground Delay Program.. The delay costs as provided in the above study are needed to evaluate savings to airlines of possible changes to ground delay program rules that use market-based mechanisms to determine departure order. See Gao et. al. [2010] for more on this effort. The delay costs as provided in the above study need to be included as part of a larger equilibrium model that predicts the actions of airlines under various policy decisions. See Ferguson et. al. [2010] for more on this effort. These delay costs will be used as a tool in a congestion-pricing model to determine the flights that are most likely to be cancelled first when capacity at an airport is reduced. An understanding of airline behavior (based on their cost structure and network configuration) is necessary when attempting to determine the prices that a regulator would need to charge in order to have supply approximately equal demand when congestion pricing are imposed at some airport.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Statistical

atlas

based

exudate

segmentation

Diabetic

macular

edema

(DME)

is

characterized

by

hard

exudates.

In

this

article,

we

propose

a

novel

statistical

atlas

based

method

for

segmentation

of

such

exudates.

Any

test

fundus

image

is

first

warped

on

the

atlas

co-ordinate

and

then

a

distance

map

is

obtained

with

the

mean

atlas

image.

This

leaves

behind

the

candidate

lesions.

Post-processing

schemes

are

introduced

for

final

segmentation

of

the

exudate.

Experiments

with

the

publicly

available

HEI-MED

data-set

shows

good

performance

of

the

method.

A

lesion

localization

fraction

of

at

of

non-lesion

localization

fraction

on

the

FROC

curve

is

obtained.

The

method

is

also

compared

to

few

most

recent

reference

methods.

In

this

article,

we

have

presented

a

novel

method

of

image

registration,

retinal

atlas

building

and

atlas

based

segmentation

of

the

exudates

for

retinal

image

analysis.

An

automatic

landmark

generation

method

has

been

presented

which

removes

manual

annotation

of

the

landmarks

for

vessel

arch

estimation

and

warping.

A

gold

standard

is

built

with

distinct

retinal

landmarks

including

optic

disc,

macula,

superior

and

inferior

vessel

arches

and

the

pigmentation

of

the

retinal

epithelium.

We

showed

that

unlike

other

methods

in

literature,

we

can

segment

the

bright

lesion

onto

the

atlas

space

without

doing

much

pre-processing

steps

like

image

normalization,

optic

disc

and

vessel

removal

etc.

The

lesions

are

more

apparent

relative

to

the

background

vasculatures

in

the

distance

image

on

the

atlas

space.

This

method

can

also

be

used

for

retinal

image

grading

and

for

potential

follow-up

of

the

lesion

growth..

We

have

evaluated

the

global

segmentation

performance

of

the

method

on

a

publicly

available

dataset

are

found

our

method

to

be

simpler

and

equally

comparable

with

the

other

best

methods

in

literature.

In

this

section

we

present

different

methods

that

have

been

used

to

evaluate

the

segmentation

of

DME

in

retinal

fundus

images

using

the

proposed

atlas

based

approach.

The

method

was

evaluated

on

publicly

available

HEI-MED

dataset

discussed

in

the

previous

section

We

have

evaluated

our

method

using

the

African

American

images

from

the

dataset.

FROC

curves

are

used

to

analyze

the

accuracy

of

the

methods.

The

performance

accuracy

of

the

diagnosis

by

an

algorithm

has

to

be

evaluated

based

on

some

criterion

which

takes

into

account

the

output

of

the

algorithm

versus

a

ground

truth.

The

possibility

of

disease

detection

by

an

algorithm

is

characterized

by

plotting

ROC

curve

and

FROC

curve.

The

area

under

the

curve

gives

the

accuracy

of

the

algorithm

in

use.

FROC

is

a

region

based

analysis

unlike

the

pixel-based

ROC-method

and

is

widely

being

used

in

the

evaluation

of

computer

aided

diagnosis.

For

our

evaluation

purpose

with

the

ground

truth

provided

in

the

HEI-MED

dataset,

a

true

positive

is

considered

when

at

least

a

part

of

the

lesion

overlaps

with

the

ground

truth.

The

detection

is

a

false

positive

when

the

exudate

is

found

outside

the

region

of

the

manually

annotated

ground

truth

and

a

false

negative

if

no

lesion

is

found

in

the

image

while

a

lesion

exists.

as

lesion

localization

fraction).

The

results

obtained

for

the

different

segmentation

methods

(depending

on

the

type

of

post-processing

scheme)

discussed

below

is

shown

in

Fig.

The

FROCanalysis

curve

is

shown

in

Fig.

Thresholding.

The

bright

lesions

have

high

distance

with

the

atlas

mean

image.

So,

these

lesions

can

be

directly

segmented

using

simple

thresholding

method

without

using

any

post-processing

approaches.

We

obtained

a

descent

accuracy

of

on

FROC-curve

for

DME

segmentation..

Steerable

wavelet.

We

have

used

transform

which

is

multi-directional

and

multi

scale

wavelet.

This

post-processing

is

done

to

improve

the

result

compared

to

simple

thresholding

method.

However,

the

improvement

is

small

giving

us

an

accuracy

of

on

the

same

dataset..

Kirsch's

operator.

A

edge

detector

is

used

to

boost

the

edges

of

the

exudate

regions.

We

do

so

because

these

lesion

have

well-defined

edges

which

help

us

to

remove

false

positives

and

increase

true

positives

detection.

The

accuracy

of

the

segmentation

increases

a

little

bit

compared

with

simple

thresholding,

Kirsch

with

Steerable

wavelet.

Again,

as

a

post

processing

scheme,

we

first

used

the

steerable

multi-directional

filters

to

enhance

the

edges

then

we

used

Kirsch's

operator

to

detect

the

edges.

This

revealed

few

lesions

which

were

suppressed

while

using

steerable

or

Kirsch's

operator

alone.

Experiments

show

that

the

combination

of

these

two

directional

edge

enhancement

methods

increases

the

true

positive

detection.

The

accuracy

rate

increases

to

From

our

experiments,

we

can

see

the

direct

effect

of

using

an

atlas

in

exudate

segmentation.

A

simple

thresholding

result

in

atlas

space

is

equally

comparable

to

other

post-processing

as

shown

in

Table

This

can

be

useful

for

necessary

feature

extraction

in

automatic

lesion

detection

methods

without

using

complex

processing

steps.

Some

detection

examples

are

shown

in

Fig.

The

thresholds

used

to

evaluate

the

algorithms

are

th

thres

The

interval

is

chosen

so

as

to

trace

the

correct

sensitivity

parameter

at

a

given

threshold.

From

the

FROC

curve

shown

in

Fig.

it

is

clear

that

using

filter

with

Kirsch

edge

operator

gives

higher

sensitivity

which

is

at

FP's

per

image

which

is

an

improvement

of

while

using

the

methods

separately.

As

explained

in

previous

section,

Riesz

filters

here

increases

TP's

of

the

exudate

detection

but

at

the

same

time

also

increases

the

FP

detection

per

image

due

to

its

wider

band.

Kirsch

operator

has

a

narrow

band

comparatively

and

thus

when

it

is

used

with

Kirsch

operator,

it

limits

the

FP

detection

rate

thereby

increasing

the

sensitivity

of

the

exudates

detection.

As

we

can

see

in

the

curve

that

at

lower

false

positive

rates

the

sensitivity

is

almost

similar

in

all

the

cases

but

when

the

FP's

increases

the

combined

post-processing

scheme

has

comparatively

higher

sensitivity..

Our

experiments

show

that

using

the

steerable

filters

for

the

enhancement

of

the

edge-like

bright

lesions

from

the

suppressed

distance

image

worked

well

after

combining

with

the

Kirsch

operator.

The

area

under

the

curve

for

all

these

methods

are

shown

in

Table

Several

ideas

have

been

exploited

in

the

past

for

the

detection

of

both

dark

(mostly

microaneurysms)

and

bright

lesions.

The

methods

of

bright

lesion

segmentation

can

be

divided

into

adaptive

grey

level

based

thresholding

region

growing

method

morphology

based

techniques

and

classification

methods

Philip

et

al.

used

a

two

step

strategy

to

segment

the

bright

lesions.

The

retina

images

were

first

shade

corrected

in

order

to

eliminate

the

non-uniformities

and

to

improve

the

contrast

of

the

bright

lesions.

Then

to

segment

these

lesions

a

global

and

local

threshold

values

were

used.

The

method

was

reported

to

have

sensitivity

between

and

However,

artefacts

due

to

noise

in

image

acquisition

can

be

detected

as

exudate

with

this

technique..

Ege

et

al..

used

template

matching,

region

growing

and

thresholding

methods

to

detect

the

bright

lesions.

They

used

a

Bayesian

classifier

to

identify

the

bright

lesions

as

exudates

and

cotton

wool

spots.

They

achieved

sensitivity

for

exudates

detection

and

a

relatively

low

sensitivity

of

for

the

cottonwool

spots.

Sinthanayothin

et

al..

used

recursive

region

growing

on

pre-processed

color

images.

They

classified

the

features

into

haemorrhages

and

microaneurysms

as

one

group

and

hard

exudates

as

another

in

non-proliferative

diabetic

retina

images.

The

dataset

with

images

containing

exudates

and

without

pathology

were

considered.

A

sensitivity

of

at

specificity

was

reported.

Both

of

these

methods

have

used

computationally

expensive

region

growing

algorithm.

Sopharak

et

al.

proposed

a

method

to

automatically

extract

the

exudates

from

the

images

of

diabetic

patients

with

non-dilated

pupil.

In

their

approach

they

used

fuzzy

C-mean

clustering

followed

by

morphological

closing

reconstruction.

Standard

deviation,

hue,

intensity

and

number

of

edge

pixels

are

selected

as

input

features

based

on

exudate

characteristics.

The

algorithm

was

evaluated

on

images

against

their

ground

truth

manually

obtained

by

ophthalmologists.

They

present

an

overall

sensitivity

of

at

specificity

of

B.

Dupas

et

al.

used

mathematical

morphological

transformation

along

with

black

top-hat

step

for

the

candidate

selection.

Then

a

k-neareset

neighbours

(k-NN)

classifier

was

used.

The

article

reported

of

sensitivity

on

dataset

of

images.

However,

the

same

method

fo

diabetic

retinopahty

(DR)

was

reported

to

have

accuracy.

Feroui

et

al.

combined

the

mathematical

morphology

and

k-means

clustering

for

hard

exudate

detection

in

the

retinal

images.

The

method

was

tested

on

opthalmologic

manually

segmented

by

an

expert.

The

drawback

of

both

these

methods

is

that

first

morphological

closing

reconstruction

technique

can

give

distance

information

between

the

detected

exudates

and

the

optic

disc

but

it

may

fail

to

detect

tiny

exudates.

The

algorithm

also

will

fail

to

separate

the

hard

exudates

with

the

other

bright

lesions

like

appearing

structures

like

imaging

artifacts,

drusen

etc.

Niemeijer

et

al.

applied

a

machine

learning

approach

for

the

automatic

detection

of

exudates

and

cotton-wool

spots

in

color

fundus

images.

They

presented

a

method

for

distinguishing

among

drusen,

exudate

and

cotton

wool

spots.

retinal

images

of

patients

with

diabetes

were

chosen

form

tele-diagnosis

database.

The

gold

standard

was

built

on

this

dataset

by

two

retinal

specialists.

They

achieved

sensitivity

at

specificity

for

the

bright

lesions.

The

sensitivity/specificity

pairs

were

and

for

drusen,

exudates

and

cotton

wool

spots

detection

respectively..

Giancardo

et

al.

introduced

a

new

methodology

for

automatic

diagnosis

of

diabetic

macular

edema

using

a

set

of

features

that

includes

color,

wavelet

decomposition

and

automatic

lesion

segmentation.

These

features

were

used

to

train

a

classifier.

The

method

is

evaluated

on

the

publicly

available

HEI-MED

dataset

and

MESSIDOR

dataset

The

employed

methodology

in

the

article

shows

an

area

under

the

curve

between

and

All

of

these

classification

methods

require

an

accurate

manual

annotations

for

the

training

dataset.

In

addition

they

are

also

highly

dependent

on

the

type

of

features

selected

for

classification..

In

literature,

a

common

drawback

of

the

methods

for

exudate

segmentation

is

the

capability

to

separate

highly

correlated

lesion

pixels

and

the

optic

disc

pixels

effectively

without

missing

true

positives

and/or

introducing

false

positives.

Optic

disc

removal

in

is

done

by

manually

cropping

the

region

whereas

used

snake

based

segmentation

method

with

an

initial

fitted

circle

to

locate

the

optic

disk

area.

While

the

former

is

a

manual

method,

the

later

one

is

time

consuming

as

it

is

based

on

an

energy

minimization

technique.

Both

techniques

are

inefficient

and

prone

to

missing

true

positives

of

the

probable

lesions.

We

present

a

novel

method

of

exudates

segmentation

by

building

a

retinal

statistical

atlas

based

on

ethnicity.

Ethnic

background

plays

a

significant

role

in

retinal

luminance

in

the

fundus

images

Our

idea

is

to

exploit

the

chromatic

informations

present

in

a

particular

ethnic

group

for

efficient

segmentation

of

lesion

like

pixels.

The

method

first

creates

a

statistical

atlas

based

on

ethnicity

then

warps

any

test

image

of

the

same

ethnic

group

patient

to

that

atlas

co-ordinate

frame.

A

simple

distance

map

between

the

atlas

and

the

test

image

suppresses

the

anatomical

locations

like

optic

disc,

vessels,

macula

and

leads

to

a

good

segmentation

of

the

pixels

belonging

to

the

abnormality.

This

pipeline

avoids

any

pre-processing,

image

normalization

or

any

other

complex

image

processing

schemes

presented

as

discussed

above.

Two

post-processing

schemes

are

also

proposed

for

separating

these

segmented

pixels

as

exudates

from

other

abnormalities

that

might

be

present

in

the

distance

map

thresholding.

This

technique

also

ensures

the

follow

up

of

the

lesion

in

the

patients

much

more

precisely

because

all

the

test

images

are

warped

to

one

reference

co-ordinate

frame.

The

rest

of

the

paper

is

organized

as

follows.

Section

gives

a

brief

description

of

the

datasets

used

for

the

building

statistical

atlas

and

for

the

evaluation

of

the

proposed

method.

In

Section

we

present

a

novel

method

for

retinal

atlas

building.

Section

deals

with

the

subsequent

segmentation

of

the

exudates

using

the

mean

atlas

image

and

the

warped

test

image.

The

post

processing

schemes

for

separating

the

artifacts

from

the

lesion

are

also

discussed

here.

The

results

are

presented

and

analyzed

in

Section

Finally,

the

paper

ends

with

concluding

remarks

in

Section

The

purpose

of

building

the

ethnicity

based

statistical

atlas

is

to

represent

the

chromatic

information

of

the

retinal

pigmentation

of

the

vasculatures.

Another

goal

is

to

know

the

locations

of

major

retinal

landmarks

such

as

the

superior

and

inferior

major

vessel

arches,

the

optic

disc

and

the

macula.

These

information

can

be

used

to

register

any

test

image

of

the

patient

to

a

built

reference

co-ordinate

system

and

subsequently

help

to

suppress

these

anatomical

structures

of

an

eye

seen

in

fundus

images

with

the

known

reference

locations

and

chromatic

information

for

efficient

segmentation

of

abnormalities

(like

lesions

and

artifacts).

Further,

postprocessing

schemes

are

applied

to

segregate

the

exudates

from

other

similar

pixels.

The

overall

method

is

shown

in

the

Fig.

In

this

figure,

we

can

see

that

a

test

image

is

first

warped

to

the

atlas

co-ordinate

system

(atlas

space)

and

then

a

distance

image

with

the

atlas

mean

image

gives

a

good

approximation

of

the

lesion

pixels

and

further

processing

enables

the

segmentation

of

the

exudates.

The

methods

has

been

explained

in

the

following

sub-sections.

O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Numerical evaluation of multi-loop integrals for arbitrary kinematics with SecDec 2.0 We present the program SecDec 2.0 which contains various new features: First, it allows the numerical evaluation of multi-loop integrals with no restriction on the kinematics. Dimensionally regulated ultraviolet and infrared singularities are isolated via sector decomposition, while threshold singularities are handled by a deformation of the integration contour in the complex plane. As an application we present numerical results for various massive two-loop four-point diagrams. SecDec 2.0 also contains new useful features for the calculation of more general parameter integrals, related e.g. to phase space integrals. PACSExtraction of ultraviolet and infrared singularities from parametric integrals appearing in higher order perturbative calculations in gauge theories. Numerical integration in the presence of integrable singularities (e.g. kinematic thresholds).Algebraic extraction of singularities in dimensional regularisation using iterated sector decomposition. This leads to a Laurent series in the dimensional regularisation parameter ǫ, where the coefficients are finite integrals over the unit-hypercube. Those integrals are evaluated numerically by Monte Carlo integration. The integrable singularities are handled by choosing a suitable integration contour in the complex plane, in an automated way.Restrictions: Depending on the complexity of the problem, limited by memory and CPU time. The restriction that multi-scale integrals could only be evaluated at Euclidean points is superseded in version 2.0.Between a few minutes and several days, depending on the complexity of the problem. Currently we are in the fortunate situation of being confronted with a wealth of high energy collider physics data, enabling us to test our present understanding of fundamental interactions and to explore physics at the TeV scale. However, the accuracy which has been or will be reached by the experiments has to be matched by comparable precision in the theory predictions, and in most cases this means that calculations beyond the leading order in perturbation theory are necessary.. It is well known that in the calculation of higher order corrections, various types of singularities can arise at intermediate stages of the calculation. For example, loop integrals can contain ultraviolet (UV) as well as infrared (IR) singularites, phase space integrals over unresolved massless particles lead to infrared singularities, and there can be integrable singularities due to kinematic thresholds. The UV and IR singularities can be regularised by dimensional regularisation, such that they appear as poles in 1/ǫ, which cancel when the different parts of the calculation are combined to a physical observable. However, before such cancellations are possible, the 1/ǫ poles have to be extracted. In the calculation of multi-loop integrals or real radiation at higher orders, this usually leads to the task of factorising the poles from complicated multiparameter integrals. Sector decomposition [1, 2, 3] is a method to achieve such a factorisation. The program SecDec 1.0, presented in [4] , performs this task in an automated way. Other public implementations of sector decomposition can be found in [5, 6, 7, 8] , see also [9] . The method already has been applied in various calculations, listing all of them is beyond the scope of this paper, for a review see [10] . Here we just mention that there are also fruitful combinations of sector decomposition with other techniques, e.g. non-linear transformations [11] , Mellin-Barnes and differential equation techniques [7, 12, 13] , high-energy expansions [14, 15] , or in the context of subtraction for unresolved double real radiation at NNLO [16, 17, 18, 19, 20, 21, 22, 23, 24] . A method developed over many years [25, 26, 27, 28, 29, 30 ] to calculate one-and two-loop integrals numerically in the physical region also partly uses sector decomposition, in combination with a careful analysis of the singularity structure of certain classes of integrals and the use of functional relations between loop integrands.. A limitation of the program SecDec 1.0 was the fact that the numerical integration of multi-scale integrals was only possible for Euclidean points, or, more precisely, values of the Mandelstam invariants and masses for which the denominator of the integrand is guaranteed to be of definite sign. For physical applications which go beyond one-scale problems, it is however crucial to be able to deal with integrable singularities, usually related to kinematic thresholds, in addition to the singularities in ǫ. The program SecDec 2.0 is able to achieve this task, by an automated deformation of the integration contour into the complex plane. This allows the numerical calculation of multiscale integrals in the physical region in an automated way. Non-planarity of the considered integral does not add any extra complications. Adding more mass scales also does not necessarily increase the complexity of the calculation with this method, as additional masses usually lead to a simpler IR singularity structure and therefore lead to less functions in the iterated decomposition. Therefore one can for instance obtain numerical results for two-loop integrals involving several mass scales where analytic methods reach their limit.. The method of contour deformation in a multi-dimensional parameter space in the context of perturbative calculations has been pioneered in [31] and later has been refined in various ways to be applied to calculations at one loop [32, 33, 34, 35, 36, 37, 38, 39] and at two loops [40, 41, 42, 43] . Another purely numerical method uses an extrapolation from large to small values of the (analytically infinitesimal) parameter moving the integration contour away from poles on the real axis [44, 45] . Numerical methods using dispersion relations, differential equations and/or numerical integration of MellinBarnes representations also have been worked out, see e.g. [46, 47, 48, 49, 50, 51, 52] . However, most numerical methods to calculate multi-scale integrals beyond one loop so far are either limited to specific types of integrals, or the parameters for the numerical integration have been carefully adapted to the individual integrals by the authors.. The aim of the work presented here is to provide a public program where the user can calculate multi-scale integrals without worrying too much about the details of the integrand. The singularity structure does not have to be known beforehand (but certainly the user has to make sure that, after the extraction of the poles regulated by dimensional regularisation, only integrable singularities remain). The program contains a sophisticated procedure to check and adjust the contour deformation parameters to optimize the convergence. For complicated integrals, the convergence can nonetheless depend critically on the settings for the numerical integration; therefore we also offer the possibility for the user to choose various parameters at the input level to tune the deformation. We should note that, even though the functions which are produced after the factorisation of the singularites in ǫ are available in algebraic form, they are usually too complicated to be integrated analytically. Therefore the final integration is done by the Monte Carlo methods, meaning that the precision which can be achieved is limited, but in favour of a gain in general applicability. We also should remark that the method is applicable to any number of loops in principle, however memory problems in the algebraic part where the functions are generated, or bad numerical convergence can be expected if the complexity of the integral is very high.. The structure of this paper is as follows. In Section 2, we briefly describe the general framework. Section 3 gives an overview of the structure of the program and the new features introduced in SecDec version 2.0. Section 4 contains installation and usage instructions, while examples and results are presented in Section 5. A brief user manual is given in the Appendix. Detailed documentation is also coming with the code which is available at http://secdec.hepforge.org. We have presented the program SecDec 2.0, which can be used to factorise dimensionally regulated singularities and numerically calculate multi-loop integrals in an automated way. As a new feature of the program, it now can deal with fully physical kinematics, i.e. is not restricted to the Euclidean region anymore. A new construction of the integrand, based entirely on topological rules, is also included. The new features are demonstrated by several examples, among them a massive two-loop four-point function which is not yet known analytically. In addition, the program can produce numerical results for more general parameter integrals, as they occur for example in phase space integrals for multi-particle production with several unresolved massless particles. The program also offers the possibility to include symbolic functions which can be used for instance to define measurement functions like jet algorithms in a flexible way. The program setup is such that the evaluation of several functions in parallel can lead to a major speed-up. To calculate full two-loop amplitudes involving several mass scales, the timings still leave room for improvement, but considering the fact that the method is very suitable for intense parallelisation, we are convinced that the program will be a very useful tool for a multitude of applications to higher order corrections in quantum field theories. (LHCPhenoNet).

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Nonintrusive tracing in the Internet Intruders that log in through a series of machines when conducting an attack are hard to trace because of the complex architecture of the Internet. The thumbprinting method provides an efficient way of tracing such intruders by determining whether two connections are part of the same connection chain. Because many connections are transient and therefore short in length, choosing the best time interval to thumbprint over can be an issue. In this paper, we provide a way to shorten the time interval used for thumbprinting. We then study some special properties of the thumbprinting function. We also study another mechanism for tracing intruders in the Internet based on a timestamping approach, which passively monitors flows between source and destination pairs. Given a potentially suspicious source, we identify its true destination. We compute the error probability of our algorithm and show that its value decreases exponentially as the observation time increases. Our simulation results show that our approach performs well. Constant change is perhaps a guiding principle in the Internet. Recent advances in technology have lead to significant growth of the Internet by factors of 10 3 and 10 6 in backbone speed and the number of hosts, respectively [1] . Since the public expansion of the Internet in 1990, many new challenges have surfaced; among them, operations between untrusted end points, more demanding applications, and less sophisticated users have caused severe stress on the Internet requirements. Furthermore, the number of attacks on networked computer systems has been growing exponentially from year to year. When considering the task of tracing intruders in the Internet, three main challenges must be taken into account. First, attackers hide their origin by making use of the Internet's architecture. By using different hosts that belong to different countries and administrative domains to route malicious acts, intruders' actions become extremely difficult to trace. Second, the data collected from an Internet trace is usually incomplete or has missing values. For example, different domains of Internet service providers may not share data because of access issues, such as when they are in different counties. Finally, routes change frequently, packets are lost, and the network latency and time to convergence can be significantly increased because of routing instability.. To deal with such problems, two tracing mechanisms exist [2] : (i) methods of keeping track of all individuals and accounting for all activities and (ii) reactive tracing, in which no global accounting is attempted until a problem arises. In our view, the first mechanism is most related to network accountability, and the second mechanism is most related to network forensics. Network-based tracing and host-based tracing are two common approaches for reactive tracing in connection chains. Host-based tracing involves one tracing system per network host [2] , and a chain of connection hosts can be known via (i) host communications [3] or (ii) reversing the attack chain by breaking the hosts in reverse order [4] . Host-based tracing schemes suffer when an extended connection crosses a host not running the system [2] . Network-based tracing has the advantage of not relying on hosts that can be untrusty or requiring host participation. Instead, network-based tracing uses the invariance of connections at higher protocol layers (such as the transport layer) to establish whether two connections are part of the same connection chain.. We study two approaches for network-based tracing. The first approach is based on the idea of a connection thumbprint. A thumbprint is very similar to a checksum or the computed summary of the content of a connection [2, 5] . The thumbprint summarizes the content of a connection by using only a small amount of data while simultaneously preserving the uniqueness of the connection. Therefore, thumbprints of related connections are similar and can lead to the construction of a connection chain. However, the thumbprinting method is dependent on the duration of the connection, so the time used for thumbprinting is very important especially in the case of transient flows. In this paper, we provide a way to shorten the time interval required for thumbprinting. We study the tradeoff introduced by using a smaller time interval and its effect on basic thumbprint properties such as sensitivity and robustness.. Because thumbprinting relies on the content of the packages and therefore can be counterattacked by encryption, we study a second mechanism for tracing Internet intruders that involves passively monitoring flows between source and destination pairs in the Internet, a process similar to [6] . This approach is based on monitoring the transmission activities of nodes and does not interfere with network operations. A one-hop communication graph can be constructed by matching transmission timestamps and acknowledgements. On the basis of this graph, the nodes that are part of a connection chain are those which communicate at a sufficiently high rate. Our approach is different from the approach in [6] in that we make fewer statistical assumptions and we use clearer, simpler derivations. In addition, because we are working with the Internet, we face other challenges, such as the availability of only partial information and the route instability mentioned earlier. To account for this, we assume that the information on every link has a certain probability of availability.. Our motivations in this paper can be derived from the previous discussions as follows:. (1) To find a way to improve current thumbprinting method for transient traffic flows by shortening the time interval required for thumbprinting. (2) To study the tradeoff introduced by using a smaller time interval and its effect on basic thumbprint properties such as sensitivity and robustness.. According to these motivations, relevant contributions are made as follows:. (1) To find out the equation for sampling traffic in any time interval and the properties that can be used for identifying related flows so that we can shorten the time interval required for thumbprinting. (2) To obtain the error probability of our algorithm and show that its value decreases exponentially as the observation time increases. The remainder of the paper is organized as follows. Section 2 presents significant work related to our problem, Section 3 presents our results concerning minimizing the thumbprinting interval, Section 4 contains a detailed introduction to the tracing algorithm followed by an evaluation of its consistency in terms of error probability, Section 5 presents some graphical interpretations of the derivations, Section 6 provides simulations, and Section 7 is the paper's conclusion. We provided a way to shorten the time interval used for thumbprinting and tune it suitably depending on network conditions. We have found interesting properties of the thumbprinting function by using the mean value and provided a general method to compute the value of the function. The method works for any function that satisfies certain properties. We have also studied another mechanism for tracing intruders in the Internet by passively monitoring flows between source and destination pairs. We computed the error probability of our algorithm and showed that its value decreases exponentially as the observation time increases. Our simulation shows that our algorithm is an alternative way to trace targets in anonymous networks and that it is better than the algorithm in [6] . Our tracing algorithm is a variation of the one from Section 4.1, in which we account for the lack of information in the observations. We model the lack of information by considering that the information on each link is available with a certain probability. We denote this probability by p k , k 2 {1, 2, . . ., K}, where K is the number of links in the considered path. In this modified algorithm, only flows that support rates of at least r k /p k ≥ t are selected from the initial graph.. It is now useful to study how well our variation of the algorithm from Section 4.1 converges. We thus study the probability P e with which the algorithm finds the destination erroneously (called error probability) and analyze its asymptotic behavior over time T. We would like to mention that even though we assume Poisson distribution of packets on a link for our specific derivation, the algorithm and analysis here can be applied to different types of traffic.. Our error probability derivation is also inspired from [6] . Assume that the transmission activities on each link follow a Poisson process of rate R < 1. It is well known that the Poisson distribution of the number of events in a time interval (t, t + t 0 ] is given by the following relation:. where m tþt 0 À m t represents the number of events in time interval (t, t + t 0 ] (see for example [44] ). Let O be the source whose destination we are trying to find, and let θ be the true destination of O. We denote the destination rendered by our algorithm withθ . We then derive the probability thatθ is not the true destination of O. Let n k be the number of timestamps taken in time interval (m k , m k + 1 ), corresponding to link (k, k + 1). We denote the total number of timestamps taken for one flow by N 1 .. By definition, P e :¼ P e TD ð Þ ¼ P θ 6 ¼θ À Á ¼ P θ is not correctly detected by algorithm TD ð Þ According to [6] , Theorem 4.4, the fact that the algorithm does not find the true destination is due to one of three possibilities:. Nonintrusive tracing in the Internet A. Olteanu et al.. (1) θ is not detected because the empirical rate along the path from O to θ is less than t; we will denote the probability of this event by P(A). (2) θ is mistaken for a relay node because there is some node j, which is a successor of θ, for which the empirical rate from O to j is greater than t. We denote the probability of this event by P(B). (3) θ is incorrectly detected as being the destination of a flow originating in some predecessor of O. We denote the probability of this event by P(C).. In cases 2 and 3, all the empirical probabilities are greater than or equal to t, so. In the following, we establish an upper bound for P(B). The empirical rate on the path from O to j is given by N 1 (T)/T. Paper [2] presents an Internet Protocol traceback method based on the thumbprint idea. Often, attacks are conducted by logging through a chain of host machines. In this way, intruders make use of the Internet architecture to hide their true origin. The thumbprint technology, which is the summary of a connection, is used to compare connections and therefore trace the source of such an attack. A thumbprint is computed for each time interval, typically a minute, of each connection [2] . Thumbprinting has the advantage of preserving the characteristics of a connection while being cheap to compute and requiring little storage. Similar methods such as checksum and message digest have two main disadvantages. Firstly, an error in the content leads to a different value being computed, and thus these methods are not robust. Secondly, they are not additive (i.e., two successive values cannot be combined to provide a new value for a longer interval). Other compression schemes require far more storage space than thumbprinting.. As the thumbprinting method is dependent on the duration of the connection, it is of importance to shorten the time interval used for thumbprinting, especially in the case of transient flows. This is the purpose of our derivation in Section 3.. However, fingerprinting techniques are vulnerable and may not be of interest when the attacker uses encryption at every hop in the chain. To deal with such problems, techniques based on packet timing information are more suitable.. Paper [6] presents a mechanism for tracing intruders in anonymous mobile ad hoc networks (MANETs) by passively monitoring flows between source and destination pairs. The algorithm is concerned with tracing the destination of a certain source that is considered to be suspicious according to a higher layer intrusion detection protocol. Because anonymity of nodes in the MANET is assumed and the monitoring is performed in a nonintrusive way, this research only considers the timestamps of packets on adjacent links and uses these to establish causal relationships between packets. A graph is used to model the communications among nodes. On the basis of traffic analysis and information carried in the packets, the graph is partitioned into two parts, one containing the set of possible destinations and one with which the source does not communicate. The following assumptions are made about the distribution of traffic: the transmission activities on each link are assumed to follow a Poisson process, the rate of the flow to be traced is sufficiently high, and the duration of the observation is long enough. Therefore, the method will not work for low-rate or transient flows.. Much like [6] , we focus on tracing flows between pairs (source and destination), but we do so with the Internet. Unlike [6] , we assume differences in data due to propagation delays, clocks being skewed during observation at multiple nodes at the same time, and the data that is either missing from observations or observed erroneously. We account for this by setting a probability at which information is available on every link. We also make fewer assumptions about the rate of the Poisson process, and the calculations are significantly simplified. Some other related work includes .

B B O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O B O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O