Information technology innovation diffusion: an information requirements paradigm Keywords: adoption, diffusion, information processing view, innovationThe internet has changed the way firms conduct business, enabling new value creating activities and supporting enhanced collaboration with trading partners. However, the use of the internet and its related impacts vary widely across industries. In the personal computer (PC) manufacturing industry, for example, internet-based technologies have had wide-ranging doi:10.1111/j. 1365-2575.2007.00260.x Info Systems J (2008 (Kraemer et al., 2000) . In contrast, diffusion of these technologies has been modest in other industries such as textiles (Forza et al., 2000) . According to the dominant diffusion of innovation paradigm, technology characteristics, including trialability, observability, relative advantage, complexity and compatibility might provide an explanation for such differences (Rogers, 1983) . However, focusing on technology characteristics provides only a partial explanation for industry variation because similar types of information technologies (IT) are available to all firms and are readily implemented by a burgeoning IT consultancy industry. Another explanation provided by the dominant paradigm is technology adopter characteristics, such as firm size and scope, technology leadership, technology competency, readiness and expected benefits (Rogers, 1983) . Indeed, empirical research shows the influence of such organizational characteristics on innovation adoption (Mustonen-Ollila & Lyytinen, 2003) . Given substantial intraindustry firm variation along these dimensions (Chwelos et al., 2001; Macher et al., 2002; Zhu et al., 2004) , this explanation is also incomplete. Moving beyond the dominant paradigm, the competitive environment, including competitive pressure and industry standards, may also play a role in technology diffusion and adoption (Tornatzky & Fleischer, 1990) . In sum, although the dominant paradigm and extensions may provide some insight into observed variation in internet adoption across industries, it is not clear to what extent these or other factors may explain such differences.Reviewing the IT innovation literature, Fichman (2004) concludes that most studies have used economic-rationalistic models, an organizational level of analysis, and a logic that organizations possessing a greater amount of the right stuff will display a greater amount of innovation. Such a focus has yielded substantial insight, but has also constrained modalities of inquiry, and by extension, understanding of core phenomena. According to Fichman (2004) , 'IT innovation researchers must also occasionally question fundamental assumptions and seek out more radical concepts and methods to reinvigorate the field.' In this vein, studying industry-level phenomena holds great potential to provide new insight into IT innovation adoption (King et al., 1994; Crowston & Myers, 2004) .We move beyond the dominant paradigm of IT innovation literature by examining IT innovation from an information processing perspective. The production process, supply chain and competitive environment vary substantially across industries, creating differing information processing needs. According to the information processing view, enhancement of a firm's information processing capabilities is a viable response to such information challenges and can be achieved through investments in technology-based process improvements. It would thus seem reasonable to posit that industry-level characteristics are associated with variation in IT-based innovation adoption. For example, the home building industry has not widely adopted IT and internet-enabled business practices. It is a fragmented industry, highly labour intensive, and thus not amenable to mass production methods. However, the emerging modular home segment uses mass-production processes to manufacture walls, roofs, etc., which are then trucked to home sites and assembled onsite. Relative to mainstream home building, this segment of the industry is a considerable user of IT and the internet. One possible explanation is that alternative methods of production and supply chain management require The dominant paradigm, with its focus on technological and organizational characteristics, would be limited in explanatory power in this context. As a first step contributing to understanding in this area, we undertake an analysis within the manufacturing sector. Our fundamental research question is: How do information processing requirements determine the adoption and diffusion of IT innovation across industries?The structure of the paper is as follows. We begin by describing the theoretical antecedents of the information requirements paradigm of IT innovation diffusion, including diffusion theory and the information processing view of the firm. Next, we develop a conceptual framework linking information processing requirements to information processing capabilities. We then apply the framework as a tool to interpret empirical data on US internet-based innovation adoption and alternative innovation profiles of two contrasting industries: wood products and beverage manufacture. In doing so, we show the validity of the framework and extend knowledge of how information processing requirements shape IT innovation adoption and diffusion. The last section describes implications for research and practice and identifies limitations of the study.T H E O R E T I C A L B A C K G R O U N DThree stages of diffusion theory can be identified in the literature: dominant, technologyorganization-environment and emergent. According to the dominant paradigm, the rate and pattern of the adoption and diffusion of ideas, practices or objects through populations of potential adopters is affected by the characteristics of both the innovation and the adopter (Rogers, 1983) . The multistage adoption process is affected by the actions of 'key adopters', the resulting profile being characterized by an S-shaped curve. Information systems (IS) researchers have applied the dominant paradigm to examine a variety of IT-based innovations, including electronic scanners in supermarket chains (Zmud & Apple, 1992) , spreadsheet software (Brancheau & Wetherbe, 1990) , IT outsourcing (Loh & Venkatraman, 1992), and internet computing (Lyytinen & Rose, 2003) . Examining this literature, Swanson (1994) concludes that firm-level effects are salient in explaining the adoption and diffusion of IS innovations. A drawback of the dominant paradigm, however, is its narrow focus on only two innovation drivers, namely, technology and organizational factors.The dominant paradigm was extended by Tornatzky & Fleischer (1990) , who added the external environment as another factor driving adoption and diffusion of innovations. In the context of internet-based innovations, this extension suggests that organization-level factors, technology factors, as well as the firm's external environment all impact the diffusion process. Zhu et al. (2004) test this framework by analysing innovations associated with the electronic pre-processing, negotiation, performance and post processing of business transactions among firms via the internet. The authors find that internal factors -technology competence and firm (2004) suggests seven 'promising opportunities' to extend the dominant paradigm: contagion effects, management fashion, innovation mindfulness, innovation configurations, technology destiny, quality of innovation, and the performance impacts of innovative IT. Several of these approaches offer the potential of extending and refining our understanding of IT innovation diffusion from an industry perspective. As an example, social contagion may be an important factor among firms within a single industry. Firms may be more likely to be affected by other firms in their industry who have already adopted an IT innovation.Although the dominant paradigm, technology-organization-environment extension and emergent stages of diffusion theory have improved understanding of the antecedents of IT adoption and diffusion, they do not explicitly address the role of information processing requirements. We thus extend our focus to move beyond diffusion theory, consistent with the suggestion of Cooper & Zmud (1992, p. 137) that studies will be improved if they 'adequately account for the "fit" between the technology being examined and the work context within which the technology is being introduced.' We adopt the information processing view of the firm to shed light on the connection between information processing requirements across industries and the adoption and diffusion of IT innovations.The information processing view provides an alternative rationale as to why firms adopt IT innovations: to respond to information processing requirements. According to the information processing view, organizations can be viewed as information processing systems that develop information processing mechanisms to deal with environmental sources of uncertainty (March & Simon, 1958; Galbraith, 1977; Tushman & Nadler, 1978; Van de Ven & Ferry, 1980) . Given imperfect knowledge and imprecise decision-making, firms develop patterns and routines of behaviour to complete individual organizational tasks. When changes in the environment occur, they give rise to uncertainty, as established routines can no longer be used. Firms face new information requirements as additional information must now be collected, processed and distributed in order for the firm to make decisions and complete its activities. To continue to operate efficiently, a firm must align its information processing capabilities with its new information requirements. The firm responds to complexity by either increasing its information processing capabilities or by reducing its information processing needs (Galbraith, 1977) . System effectiveness is contingent upon the degree of fit between system and requirements, and organization scholars have found empirical associations between complexity and information processing modes (Tushman & Nadler, 1978; Van de Ven & Ferry, 1980) . IS scholars have used the information processing view to explore the relationship between IS and information requirements. Anandarajan & Arinze (1998) examine the extent to which a client/server IS architecture matches contextual information needs. Using survey data from 89 client/server implementations, the authors find that efficiency is a function of the fit between task characteristics -task analysability and task interdependence -to the client/server architecture -client-vs. server-centric. Indeed, the overall performance of a firm is associated with how well a firm matches the information processing requirements it faces with the degree of alignment between the role of IS, centralization of IS and the formalization of IS (Wang, 2003) . The information processing view has also informed analysis of environmental uncertainty and task characteristics in the context of user satisfaction with data (Karimi et al., 2004) .Existing research shows that information processing capabilities adopted in response to information processing requirements can vary. An examination of the US and Japanese automobile industries uncovers five configurations of interorganizational relationships between buyers and suppliers (Bensaou & Venkatraman, 1995) . In one configuration -electronic interdependence -the authors found that in the case of highly customized and complex products, firms tend to be significant users of electronic communications tools as well as traditional means as a response to substantial environmental uncertainty. The authors conclude that there may be several ways to design interorganizational relationships that are equally effective. Similarly, variation in response has been identified in an investigation of production maintenance operations. Specifically, maintenance operations respond to complexity by using various combinations of computerized maintenance management systems, preventive and predictive maintenance systems, coordination and workforce size (Swanson, 2003) .The information processing view in the IS context is also supported by studies of organizational response to industry clock speed (Mendelson & Pillai, 1998; 1999) . In a study of manufacturers of computers and related products, for example, Mendelson & Pillai (1998) adopt the view that information processing requirements underlie the adoption and diffusion of communication technologies. In highly dynamic business environments, firms must adopt new ways of processing increased volumes of information, with IT being a major enabler of such processing. Results support the information processing view of the firm in that higher clock speed environments are associated with greater use of IT by firms.In sum, the information processing view is useful in the context of IT-based innovation as it provides an alternative perspective of why firms adopt such innovations. Internal and environmental uncertainty, whether based in production methods, supply chains or the larger competitive landscape, gives rise to new information requirements -a key driver of firm adoption of information technologies. We now develop a conceptual framework of IT-based innovation based on the information requirements paradigm.The conceptual framework is developed to shed light on our primary research question, which asks how information processing requirements shape the adoption and diffusion of IT across industries. The basic thesis is that within an industry, common means of production, supply chains, and a shared competitive landscape provide common information requirements for firms within that industry, leading to incentives to adopt IT innovations. As an example, firms in industries with short product life cycles and rapid product innovation are characterized by a high degree of uncertainty, leading to substantial information processing requirements. Firms may turn to various forms of IT as a means of providing the information processing capacity necessary for rapid product development and market introduction. In contrast, all else being equal, we might expect industries with longer product life cycles to be subject to less stringent information processing requirements, resulting in lower IT adoption. We build on prior research by developing a conceptual framework that includes information processing requirements (process complexity, clock speed and supply chain complexity) and information processing capabilities (production systems and interorganizational systems) (Galbraith, 1977; Tushman & Nadler, 1978; Bensaou & Venkatraman, 1995; Mendelson & Pillai, 1998) . Information processing requirements arise from the gap between information required and information available, i.e. uncertainty. Three sources of uncertainty are (1) task complexity and subunit task interdependence; (2) task environment; and (3) interunit task interdependence (Tushman & Nadler, 1978) . The framework is intended to represent interdependencies between the three task characteristics: uncertainty, fit and information processing requirements. We adapt the three sources of uncertainty used in the original framework as follows. To match our approach of analysing IT innovations across industries, we include process complexity to reflect the focus of modern firms on business processes and provide a contemporary construct encompassing the original task construct. The second dimension -clock speedencompasses the rate of environmental change that was included in the original framework and is adopted from prior IS research (Mendelson & Pillai, 1998) . The third dimension is supply chain complexity, which extends the idea of external interdependence of tasks to the contemporary business environment of complex, global supply chains. Regarding information processing capabilities, we focus on salient categories of IS used in the manufacturing environment: IT-based production control and supply chain management systems. (As our focus is not on other information processing mechanisms, such as meetings and site visits, we do not include them in our framework.) We model the five measures as formative indicators of their respective constructs, as the direction of causality flows from measure to construct and the measures do not necessarily covary (Jarvis et al., 2003) (Figure 1 ).Information processing requirementsProcess complexity includes a variety of factors associated with the production process that give rise to uncertainty (Table 1) . First is product variety, which denotes the number of parts, models, brands, etc. High product variety raises complexity and uncertainty due to multiple interrelated demand signals (Prasad, 1998) . Related to the type of product being produced, the second process complexity factor -production method -includes job shop, flow, continuous processing and assembly, each showing alternative levels of process complexity (Woodward, 1994) . In some industries products are made to stock, whereas in others, production is driven by demand pull. The latter requires a greater degree of information processing capabilities in order to capture and act upon dynamic demand signals. The third process complexity dimen- sion is information intensity of the production process, including the volume of information required for quality control steps, physical tolerances, etc.Clock speed denotes the speed of industry evolution, including the rate at which new products, processes and organizational structures are introduced (Fine, 2000) . Resulting from endogenous industry factors such as competition and technological change, specific measures include the duration of product life cycles and the rate of decline in the prices of input materials (Mendelson & Pillai, 1998) . Firms in high-clock speed industries face extremely short design and production horizons, as well as severe pressure on inventory (Fine, 1998). Many well-known cases of internet-based innovation appear in high-clock speed industries such as microprocessors (Phan, 2003) and data routers (Kraemer & Dedrick, 2002) . In an industry such as PC manufacturing with rapidly declining prices, controlling inventory costs is a key to success, and internet-based technologies are used to manage supply chains using build-to-order and online sales (Kraemer et al., 2000) . In the IT products industry, increased clock speed is positively associated with the use of various communication technologies such as electronic data interchange (EDI), further supporting its inclusion in the framework (Mendelson & Pillai, 1998) .Supply chain complexity denotes the degree of coordination required across firm boundaries. This includes the informational and product span across physical and logical spaces as well as the nature of interactions between upstream suppliers and downstream buyers. For example, the apparel industry is characterized by a global end-to-end supply chain of design, production and distribution (Teng & Jaramillo, 2005) . In contrast, wood products tend to be processed near the source of raw materials (forests), and then trucked or shipped to plants that use processed timber as an input (e.g. home builders). Such a difference in the geographic dispersion of information and production associated with supply chain processes presents alternative needs for the use of interorganizational IS, including both traditional EDI as well as web-based interorganizational IS. Also salient is the institutional structure of the demand channel, which, if highly concentrated, may afford bargaining power to pressure upstream firms to adopt IT innovations.Information processing capabilitiesComputers and networks are an integral part of the manufacturing production process. Mainframe, minicomputer and PC-based systems enable computer-aided design and computeraided manufacturing (CAD/CAM). System functionality includes analytic applications used in forecasting, scheduling and other operations management functions as well as digital sche- matic representation and manipulation. An auto manufacturing plant, for example, may use programmable logic controllers and distributed control systems to collect sensory data, check it against control parameters, and send feedback signals to production machine tools. A PC-based production system may also use analytic software for forecasting and scheduling. Given variation in demand dynamics and process complexity across industries, we expect that different forms of production control are used across industries.EDI is an industry standard method of communicating electronic documents, such as purchase orders and invoices, via proprietary value added networks. In recent years, the internet and its open standards has emerged as a platform of choice for communication and exchange across the supply chain, subsuming EDI functionality and adding an array of real-time collaboration and exchange functions (Johnson & Whang, 2002) . Exchange includes procurement and selling between supply chain partners (auctions, electronic catalogues, etc.) as well as associated supply chain processes, such as sending an invoice electronically to a customer or online customer support. Collaboration involves nonexchange related interfirm cooperation such as online design spaces, online meeting spaces, etc., between supply chain partners. Internet-based exchange and collaboration also support global supply chains such as those found in the electronics manufacturing industry. Extranets, for example, enable trading partners to access portions of internal data, thereby increasing information processing capabilities (Fowler et al., 2000) . Another example is online vendor-managed inventory, which improves supply chain efficiencies by placing the responsibility of managing inventory in the hands of the stakeholder that is in control of logistics, typically the supplier. Real-time customer inventory data are communicated back to the supplier so that accurate decisions can be made regarding supplier-related manufacturing schedules and finished goods stock levels. There is wide variation in such internet-based innovation across industries, with empirical evidence suggesting an informational basis for such variation (Cagliano et al., 2003) .As reviewed above, prior research has not focused on examining how information processing requirements shape the adoption and diffusion of IT innovations across industries. We are thus faced with several research hurdles, including a lack of applicable prior research on which to build, as well as an emergent and inherently complex phenomenon. Moreover, the majority of studies analysing internet-based innovations have been conducted in high-tech industries, leaving uncertainty as to what is happening in other industries. Based on these considerations, we have chosen to apply our framework using descriptive industry data analysis and focused industry studies. We conduct descriptive data analysis of 21 North American Industry Classification System (NAICS) industries and qualitative analysis of two industries: wood products and beverage manufacturing. While the former grounds our perspective in aggregate empirical observation, the latter provides a rich tableau of the variation in information processing requirements and information processing capabilities across two varying industries. Cross-industry analysis is important for uncovering phenomena that may be localized to a particular industry and have not been exposed in prior industry research (Chiasson & Davidson, 2005) . Indeed, our selection of industries provides such an opportunity as a majority of industry-related research examines manufacturing, high-technology, and finance, real estate, and insurance industries. (In their examination of research published in MIS Quarterly and Information Systems Research between 1997 , Chiasson & Davidson (2005 find that of all articles that draw data from a particular industry, 71% do so from these three industries.) Industry-level analysis is also appropriate in situations where the industry can be affected by variations in how firms adopt and implement new technologies (Wigand et al., 2005) . Finally, industry analysis in which the analysis unit is the industrial production environment is appropriate for analysing emergent phenomena, especially when the boundaries between phenomena and context are not clearly evident (Mendelson & Pillai, 1998; Cagliano et al., 2003) . Such is the case in Wigand et al.'s (2005) study of the effect of new XML-based standards for interorganizational coordination in the US home mortgage industry. In sum, the advantage of our analysis approach is that it provides richness and scope, while the key limitation is the lack of formal statistical analysis.We examine industry-level data to provide insight into how internet-based innovation adoption and diffusion vary across industries due to variation in information processing requirements. According to our thesis, we would expect a wide variation in such processes across industries. Although few data sources and little prior research has been carried out, there is a database collected by the US Census Bureau at the NAICS industry level. The data do not allow for empirical analysis at the firm level as published data comprise tabulations of NAICS industries. Further, they do not enable formal statistical analysis, as there are only 21 three-digit NAICS industries. However, they do provide a unique lens into the use of internet business processes across roughly 30 000 US manufacturing firms in 21 industries for the year 2000.Five technologies that underlie internet-based innovation are intranets, the internet, local area networks (LAN), EDI and extranets. Based on our developed conceptual framework, we might expect a greater percentage of firms in industries characterized by a high degree of process complexity, rapid clock speed, and high supply chain complexity to use these technologies. To examine whether this might be the case, we tally the number of instances in which a given industry appears in the top five or bottom five in terms of percentage of plants using these five technologies (Table 2) .Electrical equipment and transportation equipment are in the top five in every technology category in terms of rank ordered percentage of using plants in the industry. Next are computer and electronic products (four times in top five), paper and primary metals (three times), machinery and chemicals (twice), and apparel (one time). Electrical equipment and transportation production are characterized by high degrees of process complexity and supply chain complexity, as global supply chains are used to manage parts procurement and demand fulfilment. This is also the case for computer products, with the added factor of a very high clock speed. Thus, at least based on these descriptives, the use of internet-based innovation by these three industries appears to be consistent with the developed conceptual framework. On the other end of the scale are wood products, apparel and petroleum and coal products. Both wood and petroleum are not characterized by rapid product evolution, and thus have lower clock speeds. Moreover, process steps are fewer and global supply chains are not as complex as other industries. Descriptive statistics are thus consistent with the information requirements paradigm. An anomaly is apparel, which is in the top five one time (EDI) and in the bottom five for the remaining four categories. This is difficult to explain without more detailed and extensive analysis.Another perspective on internet-based innovation is to move beyond technologies and examine functionality of innovations. Figure 2 displays five basic processes -email with trading partners, internet buying, EDI buying, internet selling, EDI selling -according to the percentage of using plants (lowest industry, median, highest industry). Internet use and EDI use are included to provide a baseline comparison. The data indicate that three industries account for the highest percentage of plants using these processes: computer and electronics, transportation, and printing and related. The first two are consistent with data on technology usage presented in Table 2 . Industries with the lowest percentage of adopting plants include petroleum and coal, wood, non-metallic metals, textile mills, and printing and related. Once again, we see clear consistency with these industries and those that adopt technologies slowly. An anomaly is printing, which is highest in internet buying and lowest in internet selling.

O O B B O O O O O B B B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O B O O O O O O O O O O O O O O O O O O O O O O O B B O B O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O B O B O O O B I I O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Combined feedforward and feedback control of a redundant, nonlinear, dynamic musculoskeletal system Abstract A functional electrical stimulation controller is presented that uses a combination of feedforward and feedback for arm control in high-level injury. The feedforward controller generates the muscle activations nominally required for desired movements, and the feedback controller corrects for errors caused by muscle fatigue and external disturbances. The feedforward controller is an artificial neural network (ANN) which approximates the inverse dynamics of the arm. The feedback loop includes a PID controller in series with a second ANN representing the nonlinear properties and biomechanical interactions of muscles and joints. The controller was designed and tested using a two-joint musculoskeletal model of the arm that includes four mono-articular and two bi-articular muscles. Its performance during goal-oriented movements of varying amplitudes and durations showed a tracking error of less than 4°in ideal conditions, and less than 10°even in the case of considerable fatigue and external disturbances. Functional electrical stimulation (FES) systems restore function after spinal cord injury by stimulating paralyzed muscles to contract in appropriately coordinated patterns. Various FES systems have been developed that benefit different populations, such as standing systems for people with paraplegia [9] , and hand grasp systems for people with mid-cervical level spinal cord injuries [15] . We are interested in expanding the FES benefits to individuals with high-level tetraplegia, whose entire upper extremity is essentially paralyzed. This FES system aims to restore shoulder and arm function to this population, and allow them to move their arm throughout a functional workspace.. Current FES systems include highly developed hardware (e.g., implanted stimulators and numerous electrodes), but very basic control algorithms to calculate the stimulation patterns needed for the desired function, namely, predefined patterns, fixed for every task [15] . This limits the functional benefit of the FES system to a set number of tasks, and does not guarantee its performance in the presence of muscle fatigue or unexpected perturbations. A more sophisticated controller is needed for our upperextremity FES system, because unlike the cyclic movements of the lower extremity, upper-extremity tasks are goal directed [8] . This means that the amplitude, speed and direction of the motions change continuously, so the controller needs to continuously calculate the stimulation patterns for a wide range of motions commanded by the user.. There are different control strategies used in existing FES systems. Feedforward control is commonly used in clinical practice [13, 15] . The output of this type of controller depends only on the user command, and not on the system performance. The advantage of this design is that it does not need sensors to measure the system output, but the disadvantage is that it is unable to make corrections if the actual movement deviates from the desired movement. Feedback control uses sensors to monitor the system output, so it can correct for errors in the system trajectory [8] . Feedback is necessary in order to maintain good tracking performance in the presence of fatigue and any external disturbances encountered. However, inherent delays in the response of the system can cause problems to the feedback controller [1] , especially in the case of fast movements [23] . Therefore, a combination of feedforward and feedback control has the best results, and is the preferred control method in several FES system designs [1, 6, 20] , including the one presented in this paper.. The feedforward controller is typically an inversedynamic model of the controlled system [19] . Since an actual inverse-dynamic model is usually not available, data collected from the system itself are used to train an artificial neural network (ANN) to behave like the inversedynamic model. An ANN that includes time-delayed inputs can approximate any dynamic system based solely on the knowledge of inputs and outputs [19] . In order to train the network to represent the inverse dynamics of the system, inputs and outputs are recorded from the system and used as the outputs and inputs, respectively, of the ANN. This has been used in many FES controllers of single-joint, single-muscle systems [2, 3, 6, 20, 26] . However, in the case of a system with multiple muscles crossing one joint, the inverse of the system is not unique, and the ANN will train on the average of possible solutions, which is not always a correct solution [14] .. For this reason, we have used the actual inversedynamic model of the system to train the ANN in the feedforward component of our controller. Optimization was used to ensure uniqueness of the solution. In [10] , the inverse-dynamic model itself was used as a feedforward controller, but a relatively simple model had to be designed so that it was fast enough to run in real-time. An ANN, however, is a system of simple processing elements, connected into a network by a set of weights [19] , so it is computationally light. Off-line training of the ANN according to the inverse-dynamic model allows us to include all the complexities of the musculoskeletal system in a real-time controller.. For the feedback part of the controller, the feasibility of PID controllers for use in FES applications has been examined [1, 6, 25] . However, the performance of these controllers has been limited. As discussed in [20] , PID controllers are linear and have been incapable of providing good control over highly nonlinear musculoskeletal systems [4] . Moreover, the PID controllers mentioned above were all tested in single-joint, single-muscle systems, but the complexity of musculoskeletal systems increases substantially when they include more degrees of freedom and more muscles, some of them bi-articular, that cause mechanical coupling between joints [3] . A multivariable feedback controller that uses PI control is described by Lan [17] , but its performance was only evaluated under isometric conditions. These authors suggest that nonlinear control methods may be necessary for control of movements of a multi-joint system.. In order to address these issues, we have created a neuro-PID controller for the feedback loop, which combines the attractive features of PID control (robustness and ease of implementation) with the nonlinear nature of ANN. The goal of the ANN is to model the nonlinear relationships among muscles and joints, allowing the linear PID controller to deal solely with the dynamic response.. The feedforward-feedback FES controller presented in this paper was designed and tested in simulation. Modelbased evaluations were used to explore various control strategies before implementing invasive, expensive FES systems in human subjects. The model-based approach is intended to allow initial development of the control system to a point where a human implementation can be justified. The model used includes two joints, and six muscles, two of them bi-articular. This system is sufficiently complex to include both the nonlinear properties of the muscles themselves and the nonlinearities and complicated interactions between multiple muscles and joints. The controller was evaluated for a large set of goal-directed movements that cover the range of both joints. Muscle fatigue and external disturbances were also simulated to evaluate the performance of the controller for realistic conditions. A feedforward-feedback controller has been developed for a two-joint, six-muscle arm model. An inversedynamic model of the arm was used to train the ANNbased feedforward component so that it accounted for the complex interactions between muscles and joints, while solving the muscle redundancy problem by distributing the muscle forces according to a minimum energy consumption criterion. The inverse-dynamic model in the steady state was also used to train a separate static ANN that was used in conjunction with a linear PID controller in the feedback loop to create a neuro-PID feedback controller capable of handling the highly nonlinear and dynamic nature of the musculoskeletal system. The controller showed excellent tracking performance during goal-oriented movements, with less than 4°joint error (3-cm endpoint error) in ideal conditions and less than 10°(6 cm) even in the case of considerable fatigue and large external disturbances.. Using the actual inverse-dynamic model for ANN training solved the muscle redundancy problem caused by the greater number of muscles (six) than joints (two) in the model. In addition, the muscle activations calculated by the controller were almost always at low levels (except for the case of extreme fatigue), since the criterion used to select the unique solution of the muscle force distribution minimized energy consumption. As shown in Figs. 3 Fig. 4 The mean RMS errors for shoulder angles (filled bars) and elbow angles (open bars) across all forty 60-s trials tested using the three controllers: FF, FB, and FF ? FB. The paired t test showed significant differences among the three controllers, for both the shoulder and the elbow all muscle activations remained below 0.5 for almost all conditions until fatigue increased above 40-50%.. In the comparison among FF, FB and FF ? FB controllers, FF had the poorest performance, even for conditions where there was no fatigue and no external perturbations. This occurred for two reasons. First, the inverse-dynamic model used to train the ANN was not the exact inverse of the forward model used in the controller evaluations because of numerical considerations. Specifically, the presence of numerical approximations and the complexity of the model used for the individual muscles prevent it from being perfectly inverted [11] . Also, practical solution of the redundancy problem by optimization requires the use of discrete time steps in the inversedynamic model, meaning the time-history dependence of muscle force was not captured as it was in the forwarddynamic model [26] . Second, even after training, the ANN was still only an approximation of the inverse-dynamic model. This single, rather simple ANN was nonetheless capable of predicting the needed muscle activations with decent accuracy for a wide range of movements. In practice, we anticipate that the feedforward control will not be used without feedback for controlling arm movements, since it is very difficult to measure all the parameters needed for a perfect inverse-dynamic model of the real human arm [24] . However, as shown in Figs. 3 and 4 , even this imperfect feedforward component substantially improved the performance of the overall FF ? FB controller.. The inclusion of the FF component was beneficial in one more important way. The FB component had a tracking performance almost as good as FF ? FB, but it produced muscle activations that are much larger and rapidly varying than the FF controller. For example, around the 4th second of the movement shown in Fig. 3 , the activation of the anterior deltoid calculated by the FB-only controller The fatigue model used here was fairly simplistic, since it did not include activation level and stimulation frequency dependence, or a recovery with rest component [22] . However, our goal was to test the controller in the case of reduced muscle force output, and the fatigue model we chose created the right conditions for that. As shown in Figs. 5 and 6, under moderate fatigue conditions the tracking performance of the controller was as good as the non-fatigued case. During normal functional use (i.e., occasional goal-directed movements), it is very unlikely that all the muscles will fatigue below 50% without the chance for recovery.. The perturbations added to the wrist simulated obstacles or objects that are picked up and held during the movement. They had a much smaller effect on the controller performance than fatigue, with an average error of less than 10°for forces up to 15 N. Figure 9 shows that the generally small joint angle errors translate into small endpoint errors: in ideal conditions the endpoint error is less than 3 cm, and even in the presence of the largest perturbations tested, the error does not exceed 6 cm. An error at that level is functionally trivial, since it would certainly allow the FES user to eat, or (i) Fig. 7 Example of the FF ? FB controller performance during a goal-oriented movement without perturbations (left panels), and in the presence of two sets of random external forces at the wrist: small forces (middle panels) and large forces (right panels). The forces are shown in panels a, d and g. Panels b, e and h illustrate the shoulder angles, and panels c, f and i illustrate the elbow angles reach for various objects. The endpoint error approaches 20 cm only during extreme fatigue, when there is insufficient muscle force for trajectory tracking.. The controller evaluated here was tested using a twojoint, six-muscle system, but it could easily be extended for use with a full arm model. If the inverse-dynamic model is available, the ANN could be trained with more inputs and outputs, although a larger number of neurons in the hidden layer may be needed. One important practical benefit of using PID in series with an ANN instead of a purely PID feedback loop is that the number of gains that need to be tuned is reduced to 6 in the case of our model (proportional, integral and derivative for each angle) instead of 36 (the previous 6 PID gains for each of the 6 muscles). This will become a major advantage when the number of muscles increases.. For the controller to be implemented in the real FES system, the first step will be customizing the arm model to reflect the FES user's arm as closely as possible, by including information about possible voluntary muscle forces, denervation, as well as which muscles are the targets for FES stimulation. A controller for this customized arm model will be built, but because of the differences between the model and the FES user's arm, it is unlikely that the model-based ANN parameters will result in optimal performance. This controller will instead be used as a starting point for adaptation, using data collected from the FES-driven arm, as shown previously in [1] . While the FES system is in use, the stimulation levels and resulting movements will be recorded for either online or off-line adaptation of the ANN parameters. As both the feedforward and feedback ANN are adapted to the arm dynamics of the specific user, the controller performance will continue to improve, produce more accurate movements, and provide more functional benefits to the user.

O O O B I O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Realising social justice in the water sector: 1 The present work is directed to the analysis, design and construction of web-based systems capable of catalysing processes of stakeholder participation in such ways as to realise changes that are judged by these stakeholders as positive in the water sectors of both the human and the natural economies. The thesis is advanced that the judgement that is so passed is a recognition on the part of the stakeholders of a realisation of social justice. This paper is thus given over in its first part to an analysis of the origins and the nature of social justice in this sector. It is emphasised that, although the resources of hydroinformatics in this endeavour are largely based upon quantitative assessments, the ways in which these translate through stakeholder participation into experiences of social justice are essentially qualitative. Thus, although we are currently concerned for the most part with a hydroinformatics of the quantities, this now has to be extended into a hydroinformatics of the qualities.Environment (Abbott 1991), even as, at the same time, the limitations of this response were also presented. Hydroinformatics at that time was taken up almost entirely with numbers and their calculations, making use of computational hydraulics and drawing upon computational fluid dynamics (the first an established technology and the second a putative science) to provide numerical models; so that it was using numbers in ways that were essentially new, as expressed in the aphorism that "the numbers are beginning to function in another way". At the same time, of course, these models were themselves being applied within a new environment, being one in which they were becoming the central elements in a much more far-ranging activity employing measuring equipment, its radio links and SCADA systems, weather radar, meteorological forecasts and early warning systems generally, remote sensing equipment and its interpretation facilities, and many other such accessories as are introduced later here. It was then this whole complex of equipment and activities that was brought together under the one rubric of hydroinformatics (Abbott 1991).Already within this founding document, however, the limitations of a world that was being reduced only to collections of numbers was criticised as something undesirable in itself and dangerous in its potentialities. It accordingly drew heavily upon the insights and analyses of the philosopher Martin Heidegger, and then very specifically upon his Die Technik und die Kehre, ofConcerning Technology in 1977. However, its author could not stop with Heidegger, but was obliged to deepen his analysis by placing it within the more general environment of a rapidly spreading and specifically European nihilism, following such authors as Kierkegaard, Nietzsche and Barth. This, however, might be said to have been the book's undoing, for it was launched for the most part into a Western world which was rapidly losing its foundations in

O B I O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Efficient

classification

using

parallel

and

scalable

compressed

model

and

Its

application

on

intrusion

detection

Abstract.

In

order

to

achieve

high

efficiency

of

classification

in

intrusion

detection,

a

compressed

model

is

proposed

in

this

paper

which

combines

horizontal

compression

with

vertical

compression.

OneR

is

utilized

as

horizontal

compression

for

attribute

reduction,

and

affinity

propagation

is

employed

as

vertical

compression

to

select

small

representative

exemplars

from

large

training

data.

As

to

be

able

to

computationally

compress

the

larger

volume

of

training

data

with

scalability,

MapReduce

based

parallelization

approach

is

then

implemented

and

evaluated

for

each

step

of

the

model

compression

process

abovementioned,

on

which

common

but

efficient

classification

methods

can

be

directly

used.

Experimental

application

study

on

two

publicly

available

datasets

of

intrusion

detection,

and

demonstrates

that

the

classification

using

the

compressed

model

proposed

can

effectively

speed

up

the

detection

procedure

at

up

to

times,

most

importantly

at

the

cost

of

a

minimal

accuracy

difference

with

less

than

on

average.

With

the

larger

and

larger

amount

of

network

communication

data

generated,

the

design

of

Intrusion

Detection

System

(IDS)

with

high

efficiency

has

become

much

more

challenging.

It

is

very

important

to

discover

abnormal

behaviors

at

early

stage,

therefore,

compared

to

the

traditional

signature-based

detection,

research

on

anomaly

detection

has

been

more

popular

in

academia,

as

it

has

the

potential

power

to

detect

unknown

attacks

by

kinds

of

heuristic

learning

on

the

historical

training

data..

Anomaly

detection

generally

includes

two

steps,

building

a

model

on

training

data

and

using

the

model

for

detection.

However,

training

data

are

usually

in

a

large

scale,

which

can

severely

impede

the

detection

since

many

detection

models

may

need

to

scan

all

of

them

in

certain

cases.

Intuitionally,

an

effective

and

direct

way

to

reduce

time

cost

for

detection

is

to

minimize

the

volume

of

a

model

that

is

used

in

the

detection

process,

but

building

a

systematic

and

scalable

solution

on

generating

such

minimizing

training

data

model

for

efficient

intrusion

detection

is

still

in

challenge..

To

address

this

problem,

our

work

will

pay

attention

largely

to

the

building

of

data

compression

instead

of

the

detection

phase,

striving

to

boost

the

detection

efficiency

based

on

a

proposed

compressed

model

of

training

data.

Therefore,

the

solution

presented

in

this

paper

is

applicable

for

those

model-based

anomaly

detection

approaches,

especially

for

the

classification

based

system

because

extracting

the

classification

model

directly

from

the

huge

volume

of

training

data

instances

inevitably

needs

intense

computation..

As

for

how

to

build

compressed

model,

our

proposal

is

made

through

inspecting

into

the

following

common

natures

of

the

training

data,

that

is

to

say

the

motivation

and

inspiration

of

our

works

are

generated

from

the

following

observations:

a.

By

analyzing

the

attributes

of

the

training

data,

we

can

easily

find

that

the

values

of

some

attributes

(features)

in

the

whole

training

data

only

range

in

a

small

scale,

which

may

have

less

impact

on

the

detection

accuracy.

b.

Some

training

instances

are

similar,

because

they

are

only

different

from

each

other

on

several

attributes

and

the

values

of

these

attributes

are

slightly

different,

which

may

have

redundancy

for

detection

model

building.

c.

For

high

dimensional

training

data,

computing

the

similarity

or

some

akin

metric

between

each

pair

of

instance

is

time-consuming.

That

means,

for

some

novel

but

promising

data

processing

algorithms

such

like

Affinity

Propagation

the

general

computing

memory

would

likely

explode

when

the

dimensionality

of

training

data

matrix

increases

to

some

large

extent..

For

the

purpose

of

effectively

and

efficiently

handling

these

problems,

we

will

propose

a

new

framework

of

compressed

model

on

training

data.

The

model

compression

procedure

mainly

includes

horizontal

compression

and

vertical

compression.

The

overall

idea

is

presented

in

Fig.

where

the

first

step

is

to

normalize

the

original

data

followed

by

the

horizontal

compression

and

the

vertical

compression

sequentially.

Based

on

the

compressed

model,

efficient

classification

can

be

directly

built

to

detect

new

data

without

losing

accuracy.

The

main

idea

of

compressed

model

for

data

classification

Furthermore,

to

computationally

compress

as

large-scale

as

the

training

dataset

could

be,

a

cloud-based

computing

framework

will

be

employed

to

parallelize

the

compression

procedure,

which

realizes

the

scalability

on

training

data

compressing..

To

summarize,

we

mainly

make

the

following

contributions

in

this

paper:.

a.

We

propose

a

compressed

detection

model,

which

is

a

compact

version

from

the

original

training

dataset

with

regard

to

reducing

both

feature

dimension

and

instance

volume.

b.

We

implement

a

MapReduce

based

parallel

computing

solution

for

the

abovementioned

model

compression,

which

can

compress

larger

scale

of

training

data

if

only

scaling

up

the

involving

distributed

nodes.

c.

We

finalize

our

compressed

model-based

classification

approaches,

and

demonstrate

the

performance

of

our

method

on

detection

efficiency

and

accuracy

on

two

publicly

available

intrusion

detection

datasets,

and

The

remainder

of

this

paper

is

organized

as

follows.

Literature

works

are

firstly

studied

in

section

Section

introduces

the

methodology

of

our

parallel

and

scalable

compressed

model,

and

explains

its

implementation

procedure

in

detail.

Section

describes

the

efficient

classification

deployments

using

the

compressed

model.

Application

experiments

and

its

performance

analysis

on

intrusion

detection

are

presented

in

Section

while

concluding

remark

and

future

work

are

discussed

in

last

section.

is

the

rd

Cybersecurity

Data

Mining

Competition

satellited

with

ICONIP

where

a

novel

intrusion

detection

dataset

is

published

as

task

Note

that

two

graduate

students

supervised

by

the

first

author

have

ever

attended

this

competition

and

finally

won

the

First

place

on

task

with

the

highest

detection

accuracy

You

may

please

refer

to

if

interested.

We

have

proposed

a

compressed

model

for

intrusion

detection,

including

horizontal

compression

and

vertical

compression

on

training

data.

To

be

specific,

the

first

endeavor

in

this

paper

is

to

horizontally

select

useful

attributes

of

the

training

data

through

efficient

method

of

OneR,

and

then

vertically

extract

the

representative

data

as

exemplars

from

a

larger

dataset

through

the

novel

clustering

method

of

AP.

Based

on

the

compressed

model

on

training

dataset,

we

finally

studied

KNN

and

SVM

as

two

detection

methods

and

empirically

identified

all

parameters

of

AP

for

model

compression

and

SVM

for

classification.

Comprehensive

experiments

on

intrusion

detection

datasets

have

been

conducted

to

demonstrate

the

high

performance

resulting

from

the

compressed

model

we

proposed.

In

the

best

case,

it

runs

times

faster

than

the

traditional

one

without

our

model

compression,

and

neither

the

recall

(detection

rate)

nor

the

false

positive

rate

sacrifices

only

in

a

very

small

range

with

less

than

on

average

which

can

be

tolerable

or

ignorable

compared

to

the

big

efficiency

improvement.

Another

important

contribution

is

that

we

have

implemented

a

MapReduce

based

parallelization

framework

for

each

step

of

the

model

compression

procedure.

Actually,

we

have

made

efforts

to

improve

both

the

performance

and

scalability

of

compression

on

large

volume

of

training

data

by

MapReduce-based

parallelization.

Experimental

analysis

have

been

conducted

to

show

that

our

model

compression

framework

can

handle

more

than

data

points

in

minutes

using

commodity

PC

nodes,

while

the

memory

of

one

general

PC

can

only

process

not

exceeding

instances

using

AP

clustering..

The

main

merits

of

our

works

on

intrusion

detection

are

twofold.

One

is

to

initially

introduce

a

compressed

model

of

training

data

space

for

efficient

intrusion

detection

which

focuses

on

both

efficiency

and

accuracy.

The

other

is,

to

the

best

of

our

knowledge,

the

first

effort

of

MapReduce

based

implementation

to

do

the

parallel

and

scalable

compression

on

large

amount

of

training

data.

However,

there

are

also

two

open

problems

left

to

be

solved

in

our

approach,

the

ability

of

incremental

compression

processing

and

the

performance

refinement

of

MapReduce-based

parallel

computing

framework..

As

the

future

work,

we

will

further

develop

our

approach

in

two

ways.

One

is

to

explore

the

study

on

incremental

clustering

ability

of

AP

as

in

order

to

efficiently

handle

the

incremental

clustering

on

network

intrusion

training

data.

Another,

improvement

on

the

performance

of

MapReduce

based

parallel

computing

is

a

long-term

goal,

especially

to

study

on

the

iteration

mechanism

employing

some

latest

parallelization

frameworks

such

as

SPARK

We

hope

our

methodology

proposed

in

this

paper

can

finally

be

served

as

a

general

reference

model

to

meet

kinds

of

security

mining

challenge

from

big

data

systems

rather

than

intrusion

detection

During

the

past

two

decades,

researchers

in

related

fields

have

paid

much

attention

to

intrusion

detection.

Signature

based

detection

(e.g.

Snort

relies

on

the

knowledge

of

system

vulnerabilities

and

known

attack

patterns,

which

hence

is

unable

to

detect

unknown

attacks.

Correspondingly,

anomaly

detection

is

more

dynamic

and

be

able

to

detect

novel

attacks,

which

has

attracted

a

lot

of

works

worldwide.

Generally,

an

anomaly-based

intrusion

detection

system

includes

following

steps,

data

gathering,

data

preprocessing

model

building

and

model-based

detection

Although

some

common

classification

methods

can

be

well

used

for

the

model-based

detection,

the

way

of

model

building

may

affect

the

detection

results

directly

and

heavily.

Therefore,

academic

research

on

the

model-based

classification

approach

for

intrusion

detection

is

one

unceasing

focused

topic.

At

beginning,

detection

accuracy,

usually

known

as

detection

rate(recall)

and

false

alarm

rate(false

positive),

is

widely

concerned

for

the

real-world

application

purpose.

Recently,

detection

efficiency

is

more

considered

rather

than

accuracy

to

practical

significance,

especially

on

the

potential

abnormal

behavior

detection

for

high-speed

and

real-time

network

traffics.

Nevertheless,

both

O B O O O O B I O O O O B I O O O O O O O O B O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O B I O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Armada: a parallel I/O framework for computational grids High-performance computing increasingly occurs on "computational grids" composed of heterogeneous and geographically distributed systems of computers, networks, and storage devices that collectively act as a single "virtual" computer. One of the great challenges for this environment is to provide efficient access to data that is distributed across remote data servers in a grid. In this paper, we describe our solution, a framework we call armada. The framework allows applications and dataset providers to flexibly compose graphs of processing modules that describe the distribution, application interfaces, and processing required of the dataset before computation. The armada runtime system then restructures the graph, and places the processing modules at appropriate hosts to reduce network traffic. A recent trend in high-performance computing is to build "computational grids" that tie together heterogeneous, geographically distributed computers and storage systems. Applications for these environments use high-speed networks to logically assemble resources into a virtual supercomputer. The trend toward grid computing derives from many factors [1] . Among them are the technological advances in networking, computing, and distributed computing software; the increase in the number of demanddriven applications-applications that have infrequent requirements for large computing resources (e.g., a medical diagnosis system during surgery or a seismic simulation after an earthquake); the utilization of idle compute resources; and the increased need to share data that was computed or gathered at geographically distributed locations, e.g., weather prediction simulations that use data gathered from many satellites and stored at locations around the world.. A particularly challenging class of grid applications is the class of data-intensive grid applications. A data-intensive grid application typically requires access to large (terabyte-petabyte size) remote datasets, and has significant computational requirements that may require high-performance supercomputers. In addition, the data is often stored in "raw" formats and requires significant preprocessing or filtering before the computation can take place. For example, seismic data, used to extract images of the subsurface, requires a variety of processing steps to filter and transform data before computation [2] . Data-intensive applications also exist in climate modeling [3, 4] physics and astronomy [5] , biology and chemistry [6, 7] , visualization [8] [9] [10] , and many others.. In this paper, we present the armada framework for building I/O-access paths for data-intensive grid applications. Armada's goal is to allow grid applications to access datasets that are distributed across a computational grid, and in particular to allow the application programmer and the dataset provider to design and deploy a flexible network of application-specific and dataset-specific functionality across the grid. In this paper, we describe the design of a framework that supports data-intensive parallel applications in a wide-area computational network, or "grid". Our framework, armada, allows applications and data providers to create datasets that span the grid. Applications combine flexible modules called "ships" into series-parallel graphs called "armadas". These armadas provide a path for the application to access the data. Typically, the armada is a combination of a graph designed by the dataset owner and extensions provided by the application. The armada thus encodes the structure of the data, the application's interface, and intermediate processing and filtering.. Our framework includes a rich hierarchy of ship classes, a "blueprint" mechanism to specify the structure of an armada graph, execution environments called "harbors" that securely host ships, and algorithms that optimize blueprints and deploy ships onto the network of harbors.. The preliminary results presented here demonstrate the value of the armada approach. The ability to flexibly combine modular ships into an access graph leads directly to the ability to adjust the placement of functionality within the grid. It is well known that placement can improve performance by reducing network load. Our placement and graph-optimization schemes enable applications-specific code to migrate deep into potentially complex data distributions to filter data close to the source. Since data-intensive grid applications are typically limited by the network, we expect armada's approach will lead to better overall application performance.. Our work on the armada system is far from complete. Our priority is to complete the implementation by including capability-based mechanisms on the harbors; developing efficient placement algorithms that consider memory and CPU rates, as well as network capacity; and performance tuning. Our plans also include a further study of algorithms for optimizing the graphs described by blueprints, and protocols that can improve the communication efficiency of data-flowing through an armada. Various groups within the research community as well as the commercial sector are investigating issues related to I/O for computational grids. Here, we discuss the systems most related to armada. In particular we discuss I/O systems designed to be flexible, systems that optimize data paths to improve I/O performance, and systems that support remote execution of application code. In this section, we discuss the implementation of armada. A quick overview: armada ships are Java objects instantiated on hosts in the grid. Each host has a persistent server to host ships, called a harbor. Each harbor monitors and manages the resources used by each ship, and controls access to system resources through a capability-based mechanism [40, 41] .. We chose Java for several reasons: it provides a "sandbox" [42] for executing untrusted client code, it is reasonably efficient now that just-in-time compilers are available, it is increasingly popular among HPC programmers, it has convenient mechanisms for remote execution and communication (RMI), and it can interface to application code in other languages through the Java native interface (JNI). Only the ships and harbors need to be written in Java; client code could be in C, C++, or any other language that interfaces with Java.. In the remainder of this section, we detail the ship and harbor implementations, and discuss how we use the extensible markup language (XML) for armada blueprints.

O O B I B O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O