Query: Towards Machine Learning-Based Auto-tuning of MapReduce

Relevant passages:
Starfish: A Self-tuning System for Big Data Analytics <=SEP=> Hadoop is a MAD system for data analytics  Magnetism: attracts all sources of data  Agility: adapts in sync with rapid data evolution  Depth: supports complex analytics needs Starfish makes Hadoop MADDER and Self-Tuning  Data-lifecycle-awareness: achieves good performance throughout data lifecycle  Elasticity: adjusts resources and operational costs  Robustness: provides availability and predictability Just-in-Time Job Optimization

Approximation capabilities of multilayer feedforward networks <=SEP=> 

The HiBench benchmark suite: Characterization of the MapReduce-based data analysis <=SEP=> The MapReduce model is becoming prominent for the large-scale data analysis in the cloud. In this paper, we present the benchmarking, evaluation and characterization of Hadoop, an open-source implementation of MapReduce. We first introduce HiBench, a new benchmark suite for Hadoop. It consists of a set of Hadoop programs, including both synthetic micro-benchmarks and real-world Hadoop applications. We then evaluate and characterize the Hadoop framework using HiBench, in terms of speed (i.e., job running time), throughput (i.e., the number of tasks completed per minute), HDFS bandwidth, system resource (e.g., CPU, memory and I/O) utilizations, and data access patterns.

MapReduce: Simplified Data Processing on Large Clusters <=SEP=> MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks. Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google's clusters every day, processing a total of more than twenty petabytes of data per day.

The Google file system <=SEP=> We have designed and implemented the Google File System, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. While sharing many of the same goals as previous distributed file systems, our design has been driven by observations of our application workloads and technological environment, both current and anticipated, that reflect a marked departure from some earlier file system assumptions. This has led us to reexamine traditional choices and explore radically different design points. The file system has successfully met our storage needs. It is widely deployed within Google as the storage platform for the generation and processing of data used by our service as well as research and development efforts that require large data sets. The largest cluster to date provides hundreds of terabytes of storage across thousands of disks on over a thousand machines, and it is concurrently accessed by hundreds of clients. In this paper, we present file system interface extensions designed to support distributed applications, discuss many aspects of our design, and report measurements from both micro-benchmarks and real world use.

A Constrained, Weighted-L1 Minimization Approach for Joint Discovery of Heterogeneous Neural Connectivity Graphs <=SEP=> Determining functional brain connectivity is crucial to understanding the brain and neural differences underlying disorders such as autism. Recent studies have used Gaussian graphical models to learn brain connectivity via statistical dependencies across brain regions from neuroimaging. However, previous studies often fail to properly incorporate priors tailored to neuroscience, such as preferring shorter connections. To remedy this problem, the paper here introduces a novel, weighted-`1, multi-task graphical model (W-SIMULE). This model elegantly incorporates a flexible prior, along with a parallelizable formulation. Additionally, W-SIMULE extends the often-used Gaussian assumption, leading to considerable performance increases. Here, applications to fMRI data show that W-SIMULE succeeds in determining functional connectivity in terms of (1) log-likelihood, (2) finding edges that differentiate groups, and (3) classifying different groups based on their connectivity, achieving 58.6% accuracy on the ABIDE dataset. Having established W-SIMULE’s effectiveness, it links four key areas to autism, all of which are consistent with the literature. Due to its elegant domain adaptivity, W-SIMULE can be readily applied to various data types to effectively estimate connectivity.

An ontology for clinical questions about the contents of patient notes <=SEP=> OBJECTIVE
Many studies have been completed on question classification in the open domain, however only limited work focuses on the medical domain. As well, to the best of our knowledge, most of these medical question classifications were designed for literature based question and answering systems. This paper focuses on a new direction, which is to design a novel question processing and classification model for answering clinical questions applied to electronic patient notes.


METHODS
There are four main steps in the work. Firstly, a relatively large set of clinical questions was collected from staff in an Intensive Care Unit. Then, a clinical question taxonomy was designed for question and answering purposes. Subsequently an annotation guideline was created and used to annotate the question set. Finally, a multilayer classification model was built to classify the clinical questions.


RESULTS
Through the initial classification experiments, we realized that the general features cannot contribute to high performance of a minimum classifier (a small data set with multiple classes). Thus, an automatic knowledge discovery and knowledge reuse process was designed to boost the performance by extracting and expanding the specific features of the questions. In the evaluation, the results show around 90% accuracy can be achieved in the answerable subclass classification and generic question templates classification. On the other hand, the machine learning method does not perform well at identifying the category of unanswerable questions, due to the asymmetric distribution.


CONCLUSIONS
In this paper, a comprehensive study on clinical questions has been completed. A major outcome of this work is the multilayer classification model. It serves as a major component of a patient records based clinical question and answering system as our studies continue. As well, the question collections can be reused by the research community to improve the efficiency of their own question and answering systems.

Taming Subgraph Isomorphism for RDF Query Processing <=SEP=> RDF data are used to model knowledge in various areas such as l ife sciences, Semantic Web, bioinformatics, and social graphs . The size of real RDF data reaches billions of triples. This calls for a framework for efficiently processing RDF data. The core func tion of processing RDF data is subgraph pattern matching. There h av been two completely different directions for supporting ef ficient subgraph pattern matching. One direction is to develop spec ialized RDF query processing engines exploiting the properties of R DF data for the last decade, while the other direction is to deve lop fficient subgraph isomorphism algorithms for general, labele d graphs for over 30 years. Although both directions have a similar go al (i.e., finding subgraphs in data graphs for a given query grap h), they have been independently researched without clear reas on. We argue that a subgraph isomorphism algorithm can be easily mo dified to handle the graph homomorphism, which is the RDF patter n matching semantics, by just removing the injectivity const raint. In this paper, based on the state-of-the-art subgraph isomorp hism algorithm, we propose an in-memory solution, TurboHOM++, which is tamed for the RDF processing, and we compare it with the rep resentative RDF processing engines for several RDF benchmark s in a server machine where billions of triples can be loaded in mem ory. In order to speed upTurboHOM++, we also provide a simple yet effective transformation and a series of optimization tech niques. Extensive experiments using several RDF benchmarks show th at TurboHOM++ consistently and significantly outperforms the representative RDF engines. Specifically, TurboHOM++ outperforms its competitors by up tofiveorders of magnitude.

Identifying and Accounting for Task-Dependent Bias in Crowdsourcing <=SEP=> Models for aggregating contributions by crowd workers have been shown to be challenged by the rise of taskspecific biases or errors. Task-dependent errors in assessment may shift the majority opinion of even large numbers of workers to an incorrect answer. We introduce and evaluate probabilistic models that can detect and correct task-dependent bias automatically. First, we show how to build and use probabilistic graphical models for jointly modeling task features, workers’ biases, worker contributions and ground truth answers of tasks so that task-dependent bias can be corrected. Second, we show how the approach can perform a type of transfer learning among workers to address the issue of annotation sparsity. We evaluate the models with varying complexity on a large data set collected from a citizen science project and show that the models are effective at correcting the task-dependent worker bias. Finally, we investigate the use of active learning to guide the acquisition of expert assessments to enable automatic detection and correction of worker bias.

A soft robotic approach to robust and dexterous grasping <=SEP=> In this paper, we present a compliant robotic gripper, Edgy-2, with 4-DOF dexterity, enabling four grasping modes: parallel grasping, power grasping, finger-tip pinch and fully-actuated grasping. The roboticfinger is based on soft-rigid-hybrid structures, combining fiber-reinforced soft pneumatic actuators with rigid joints, which exhibit reliable structural rigidity and grasping robustness while maintaining excellent adaptability and inherent compliance. With both grasping dexterity and grasping robustness, the Edgy-2 achieves excellent grasping reliability with various daily objects, from a fragile cherry to a 2 kg water bottled water. The relationship of design parameters and grasping strength is presented with analytical models. The performance of a prototype Edgy-2 is verified by dedicated experiments. The proposed hybrid dexterous grasping approach can be easily extended into different end-effector designs according to application requirements. The proposed mechanism for grasping provides excellent human-robot interaction safety and reliability.

Cachexia: a new definition. <=SEP=> On December 13th and 14th a group of scientists and clinicians met in Washington, DC, for the cachexia consensus conference. At the present time, there is no widely agreed upon operational definition of cachexia. The lack of a definition accepted by clinician and researchers has limited identification and treatment of cachectic patient as well as the development and approval of potential therapeutic agents. The definition that emerged is: "cachexia, is a complex metabolic syndrome associated with underlying illness and characterized by loss of muscle with or without loss of fat mass. The prominent clinical feature of cachexia is weight loss in adults (corrected for fluid retention) or growth failure in children (excluding endocrine disorders). Anorexia, inflammation, insulin resistance and increased muscle protein breakdown are frequently associated with cachexia. Cachexia is distinct from starvation, age-related loss of muscle mass, primary depression, malabsorption and hyperthyroidism and is associated with increased morbidity. While this definition has not been tested in epidemiological or intervention studies, a consensus operational definition provides an opportunity for increased research.

Which NoSQL Database? A Performance Overview <=SEP=> NoSQL data stores are widely used to store and retrieve possibly large amounts of data, typically in a key-value format. There are many NoSQL types with different performances, and thus it is important to compare them in terms of performance and verify how the performance is related to the database type. In this paper, we evaluate five most popular NoSQL databases: Cassandra, HBase, MongoDB, OrientDB and Redis. We compare those databases in terms of query performance, based on reads and updates, taking into consideration the typical workloads, as represented by the Yahoo! Cloud Serving Benchmark. This comparison allows users to choose the most appropriate database according to the specific mechanisms and application needs. TYPE OF PAPER AND

Pattern Recognition <=SEP=> In this paper a novel reconstruction method is presented that uses the topological relationship of detected image features to create a highly abstract but semantically rich 3D model of the reconstructed scenes. In the first step, a combined image-based reconstruction of points and lines is performed based on the current state of art structure from motion methods. Subsequently, connected planar threedimensional structures are reconstructed by a novel method that uses the topological relationships between the detected image features. The reconstructed 3D models enable a simple extraction of geometric shapes, such as rectangles, in the scene.

SD-Map - A Fast Algorithm for Exhaustive Subgroup Discovery <=SEP=> In this paper we present the novel SD-Map algorithm for exhaustive but efficient subgroup discovery. SD-Map guarantees to identify all interesting subgroup patterns contained in a data set, in contrast to heuristic or samplingbased methods. The SD-Map algorithm utilizes the well-known FP-growth method for mining association rules with adaptations for the subgroup discovery task. We show how SD-Map can handle missing values, and provide an experimental evaluation of the performance of the algorithm using synthetic data.

Mindfulness practice leads to increases in regional brain gray matter density <=SEP=> Therapeutic interventions that incorporate training in mindfulness meditation have become increasingly popular, but to date little is known about neural mechanisms associated with these interventions. Mindfulness-Based Stress Reduction (MBSR), one of the most widely used mindfulness training programs, has been reported to produce positive effects on psychological well-being and to ameliorate symptoms of a number of disorders. Here, we report a controlled longitudinal study to investigate pre-post changes in brain gray matter concentration attributable to participation in an MBSR program. Anatomical magnetic resonance (MR) images from 16 healthy, meditation-naïve participants were obtained before and after they underwent the 8-week program. Changes in gray matter concentration were investigated using voxel-based morphometry, and compared with a waiting list control group of 17 individuals. Analyses in a priori regions of interest confirmed increases in gray matter concentration within the left hippocampus. Whole brain analyses identified increases in the posterior cingulate cortex, the temporo-parietal junction, and the cerebellum in the MBSR group compared with the controls. The results suggest that participation in MBSR is associated with changes in gray matter concentration in brain regions involved in learning and memory processes, emotion regulation, self-referential processing, and perspective taking.

Demystifying Learning Analytics in Personalised Learning <=SEP=> This paper presents learning analytics as a mean to improve students’ learning. Most learning analytics tools are developed by in-house individual educational institutions to meet the specific needs of their students. Learning analytics is defined as a way to measure, collect, analyse and report data about learners and their context, for the purpose of understanding and optimizing learning. The paper concludes by highlighting framework of learning analytics in order to improve personalised learning. In addition, it is an endeavour to define the characterising features that represents the relationship between learning analytics and personalised learning environment. The paper proposes that learning analytics is dependent on personalised approach for both educators and students. From a learning perspective, students can be supported with specific learning process and reflection visualisation that compares their respective performances to the overall performance of a course. Furthermore, the learners may be provided with personalised recommendations for suitable learning resources, learning paths, or peer students through recommending system. The paper’s contribution to knowledge is in considering personalised learning within the context framework of learning analytics.

Credit card incidents and control systems <=SEP=> Credit and debit cards have spread and skyrocketed all around the world to become the most popular means of payments in many countries. Despite their enormous popularity, cards are not free of risk. Technology development and e-commerce have exponentially increased internal credit card incidents. This paper identifies and quantifies the different types of credit card fraud and puts into question the effectiveness of the role assigned to cardholders in its detection. © 2012 Elsevier Ltd. All rights reserved.

A unified view of gradient-based attribution methods for Deep Neural Networks <=SEP=> Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years. While several methods have been proposed to explain network predictions, only few attempts to analyze them from a theoretical perspective have been made in the past. In this work we analyze various state-of-the-art attribution methods and prove unexplored connections between them. We also show how some methods can be reformulated and more conveniently implemented. Finally, we perform an empirical evaluation with six attribution methods on a variety of tasks and architectures and discuss their strengths and limitations.

Cyclic deadlock prediction and avoidance for zone-controlled AGV system <=SEP=> Automated guided vehicles (AGVs) are now becoming popular in automated materials handling systems, flexible manufacturing systems and even containers handling applications at seaports. Its performance affects the efficiency of the entire system. Deadlock formation is a serious problem as it stalls the AGVS. The objective of this paper is to develop an efficient AGV deadlock prediction and avoidance algorithm for container port operations. Deadlock in a broad sense is a situation in which at least a part of the system stalls. There are a lot of situations in which the system may stall and most of these situations can be avoided through the control and navigation system. This paper discusses the development of an efficient strategy for predicting and avoiding the deadlocks in a large scale AGVS A variety of deadlock-detecting algorithms are available, but these methods work mainly for manufacturing system where the network layout is simple and uses only a small number of AGVs. The AGVS under study has complex layout and involves close to 80 AGVs. The proposed solution is implemented using the Automod simulation software, and the performance of the technique is presented. Simulation shows that all the potential deadlock situations can be detected and avoided via this methodology. Also, the proposed strategy is computationally efficient and is able to provide the movement decision to the vehicles within the 1.5-s sampling time. r 2002 Published by Elsevier Science B.V.

The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems <=SEP=> We propose a deep learning based method, the Deep Ritz Method, for numerically solving variational problems, particularly the ones that arise from partial differential equations. The Deep Ritz method is naturally nonlinear, naturally adaptive and has the potential to work in rather high dimensions. The framework is quite simple and fits well with the stochastic gradient descent method used in deep learning. We illustrate the method on several problems including some eigenvalue problems.

The effect of music background on the emotional appraisal of film sequences <=SEP=> In this study the effects of musical background on the emotional appraisal of film sequences was investigated. Four pairs of polar emotions defined in Plutchik’s model were used as basic emotional qualities: joy-sadness, anticipation-surprise, fear-anger, and trustdisgust. In the preliminary study eight film sequences and eight music themes were selected as the best representatives of all eight Plutchik’s emotions. In the main experiment the participant judged the emotional qualities of film-music combinations on eight seven-point scales. Half of the combinations were congruent (e.g. joyful film joyful music), and half were incongruent (e.g. joyful film sad music). Results have shown that visual information (film) had greater effects on the emotion appraisal than auditory information (music). The modulation effects of music background depend on emotional qualities. In some incongruent combinations (joysadness) the modulations in the expected directions were obtained (e.g. joyful music reduces the sadness of a sad film), in some cases (anger-fear) no modulation effects were obtained, and in some cases (trust-disgust, anticipation-surprise) the modulation effects were in an unexpected direction (e.g. trustful music increased the appraisal of disgust of a disgusting film). These results suggest that the appraisals of conjoint effects of emotions depend on the medium (film masks the music) and emotional quality (three types of modulation effects).

Dynamic surface self-reconstruction is the key of highly active perovskite nano-electrocatalysts for water splitting. <=SEP=> The growing need to store increasing amounts of renewable energy has recently triggered substantial R&D efforts towards efficient and stable water electrolysis technologies. The oxygen evolution reaction (OER) occurring at the electrolyser anode is central to the development of a clean, reliable and emission-free hydrogen economy. The development of robust and highly active anode materials for OER is therefore a great challenge and has been the main focus of research. Among potential candidates, perovskites have emerged as promising OER electrocatalysts. In this study, by combining a scalable cutting-edge synthesis method with time-resolved X-ray absorption spectroscopy measurements, we were able to capture the dynamic local electronic and geometric structure during realistic operando conditions for highly active OER perovskite nanocatalysts. Ba0.5Sr0.5Co0.8Fe0.2O3-δ as nano-powder displays unique features that allow a dynamic self-reconstruction of the material's surface during OER, that is, the growth of a self-assembled metal oxy(hydroxide) active layer. Therefore, besides showing outstanding performance at both the laboratory and industrial scale, we provide a fundamental understanding of the operando OER mechanism for highly active perovskite catalysts. This understanding significantly differs from design principles based on ex situ characterization techniques.

Applying multi-agent technique in multi-section flexible manufacturing system <=SEP=> In the highly competitive market, cooperative multi-agent transaction and negotiation mechanism have become an important research topic. This paper uses multi-agent technology to construct a multi-section flexible manufacturing system (FMS) model, and utilizes simulation to build a manufacturing environment based on JADE framework for multi-agent to combine with dispatching rules, such as shortest imminent processing time (SIPT), first come first serve (FCFS) earliest due date (EDD), and Buffer Sequence. This paper finds that using multi-agent technique for multi-section FMS model can enhance the production efficiency in practice. Meanwhile, in this study, multi-agent systems combined with dynamic dispatching can be used to identify the best dispatching rules combination for achieving largest throughput, and thus it can provide the reference for production scheduling in advance. 2010 Elsevier Ltd. All rights reserved.

Providing Research Graph Data in JSON-LD Using Schema.org <=SEP=> In this position paper, we describe a pilot project that provides Research Graph records to external web services using JSON-LD. The Research Graph database contains a largescale graph that links research datasets (i.e., data used to support research) to funding records (i.e. grants), publications and researcher records such as ORCID profiles. This database was derived from the work of the Research Data Alliance Working Group on Data Description Registry Interoperability (DDRI), and curated using the Research Data Switchboard open source software. By being available in Linked Data format, the Research Graph database is more accessible to third-party web services over the Internet, which thus opens the opportunity to connect to the rest of the world in the semantic format. The primary purpose of this pilot project is to evaluate the feasibility of converting registry objects in Research Graph to JSON-LD by accessing widely used vocabularies published at Schema.org. In this paper, we provide examples of publications, datasets and grants from international research institutions such as CERN INSPIREHEP, National Computational Infrastructure (NCI) in Australia, and Australian Research Council (ARC). Furthermore, we show how these Research Graph records are made semantically available as Linked Data through using Schema.org. The mapping between Research Graph schema and Schema.org is available on GitHub repository. We also discuss the po©c 2017 International World Wide Web Conference Committee (IW3C2), published under Creative Commons CC BY 4.0 License. WWW 2017 Companion, April 3–7, 2017, Perth, Australia. ACM 978-1-4503-4914-7/17/04. http://dx.doi.org/10.1145/3038912.3038914 . tential need for an extension to Schema.org vocabulary for scholarly communication.

Technology adoption: A conjoint analysis of consumers' preference on future online banking services <=SEP=> The importance of service delivery technology and online service adoption and usage in the banking industry has received an increased discussion in the literature in recent years. Owing to the fact that Strong online banking services are important drivers for bank performance and customer service delivery; several studies have been carried out on online banking service adoption or acceptance where services are already deployed and on the factors that influence customers' adoption and use or intention to use those services. However, despite the increasing discussion in the literatures, no attempt has been made to look at consumers' preference in terms of future online banking service adoption. This study used conjoint analysis and stated preference methods with discrete choice model to analyze the technology adoption pattern regarding consumers' preference for potential future online banking services in the Nigerian banking industry. The result revealed that to increase efficiency and strengthen competitiveness, banks need to promote smart and practical branded services especially self-services at the same time promote a universal adoption of e-banking system services that add entertainment or extra convenience to customers such as ease of usage including digital wallet, real-time interaction (video banking), ATMs integrated with smart phones, website customization, biometric services, and digital currency. These services can contribute to an increasing adoption of online services. & 2015 Elsevier Ltd. All rights reserved.

An Improved Speech Enhancement Method based on Teager Energy Operator and Perceptual Wavelet Packet Decomposition <=SEP=> According to the distribution characteristic of noise and clean speech signal in the frequency domain, a new speech enhancement method based on teager energy operator (TEO) and perceptual wavelet packet decomposition (PWPD) is proposed. Firstly, a modified Mask construction method is made to protect the acoustic cues at the low frequencies. Then a level-dependent parameter is introduced to further adjust the thresholds in light of the noise distribution feature. At last the sub-bands which have very little influence are set directly 0 to improve the signal-to-noise ratio (SNR) and reduce the computation load. Simulation results show that, under different kinds of noise environments, this new method not only enhances the signal-to-noise ratio (SNR) and perceptual evaluation of speech quality (PESQ), but also reduces the computation load, which is very advantageous for real-time realizing.

Unikernels: library operating systems for the cloud <=SEP=> We present unikernels, a new approach to deploying cloud services via applications written in high-level source code. Unikernels are single-purpose appliances that are compile-time specialised into standalone kernels, and sealed against modification when deployed to a cloud platform. In return they offer significant reduction in image sizes, improved efficiency and security, and should reduce operational costs. Our Mirage prototype compiles OCaml code into unikernels that run on commodity clouds and offer an order of magnitude reduction in code size without significant performance penalty. The architecture combines static type-safety with a single address-space layout that can be made immutable via a hypervisor extension. Mirage contributes a suite of type-safe protocol libraries, and our results demonstrate that the hypervisor is a platform that overcomes the hardware compatibility issues that have made past library operating systems impractical to deploy in the real-world.

A Taxonomy of Attacks in RPL-based Internet of Things <=SEP=> The growing interest for the Internet of Things is contributing to the large-scale deployment of Low power and Lossy Networks (LLN). These networks support communications amongst objects from the real world, such as home automation devices and embedded sensors, and their interconnection to the Internet. An open standard routing protocol, called RPL, has been specified by the IETF in order to address the specific properties and constraints of these networks. However, this protocol is exposed to a large variety of attacks. Their consequences can be quite significant in terms of network performance and resources. In this paper, we propose to establish a taxonomy of the attacks against this protocol, considering three main categories including attacks targeting network resources, attacks modifying the network topology and attacks related to network traffic. We describe these attacks, analyze and compare their properties, discuss existing counter-measures and their usage from a risk management perspective.

Geostatistical Space-Time Models , Stationarity , Separability , and Full Symmetry <=SEP=> 4.

Who Invented the Reverse Mode of Differentiation ? <=SEP=> Nick Trefethen [13] listed automatic differentiation as one of the 30 great numerical algorithms of the last century. He kindly credited the present author with facilitating the rebirth of the key idea, namely the reverse mode. In fact, there have been many incarnations of this reversal technique, which has been suggested by several people from various fields since the late 1960s, if not earlier. Seppo Linnainmaa (Lin76) of Helsinki says the idea came to him on a sunny afternoon in a Copenhagen park in 1970. He used it as a tool for estimating the effects of arithmetic rounding errors on the results of complex expressions. Gerardi Ostrowski (OVB71) discovered and used it some five years earlier in the context of certain process models in chemical engineering. Here and throughout references that are not listed in the present bibliography are noted in parentheses and can be found in the book [7]. Also in the sixties Hachtel et al. [6] considered the optimization of electronic circuits using the costate equation of initial value problems and its discretizations to compute gradients in the reverse mode for explicitly time-dependent problems. Here we see, possibly for the first time, the close connection between the reverse mode of discrete evaluation procedures and continuous adjoints of differential equations. In the 1970s Iri analyzed the properties of dual and adjoint networks. In the 1980s he became one of the key researchers on the reverse mode. From a memory and numerical stability point of view the most difficult aspect of the reverse mode is the reversal of a program. This problem was discussed in the context of Turing Machines by Benett (Ben73), who foreshadowed the use of checkpointing as a tradeoff between numerical computational effort and memory requirement.