Query: A Strategy for Ranking Optimization Methods using Multiple Criteria

Relevant passages:
Input Warping for Bayesian Optimization of Non-stationary Functions <=SEP=> Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions. The ability to accurately model distributions over functions is critical to the effectiveness of Bayesian optimization. Although Gaussian processes provide a flexible prior over functions, there are various classes of functions that remain difficult to model. One of the most frequently occurring of these is the class of non-stationary functions. The optimization of the hyperparameters of machine learning algorithms is a problem domain in which parameters are often manually transformed a priori, for example by optimizing in “log-space,” to mitigate the effects of spatially-varying length scale. We develop a methodology for automatically learning a wide family of bijective transformations or warpings of the input space using the Beta cumulative distribution function. We further extend the warping framework to multi-task Bayesian optimization so that multiple tasks can be warped into a jointly stationary space. On a set of challenging benchmark optimization tasks, we observe that the inclusion of warping greatly improves on the state-of-the-art, producing better results faster and more reliably.

Practical Bayesian Optimization of Machine Learning Algorithms <=SEP=> The use of machine learning algorithms frequently involves careful tuning of learning parameters and model hyperparameters. Unfortunately, this tuning is often a “black art” requiring expert experience, rules of thumb, or sometimes bruteforce search. There is therefore great appeal for automatic approaches that can optimize the performance of any given learning algorithm to the problem at hand. In this work, we consider this problem through the framework of Bayesian optimization, in which a learning algorithm’s generalization performance is modeled as a sample from a Gaussian process (GP). We show that certain choices for the nature of the GP, such as the type of kernel and the treatment of its hyperparameters, can play a crucial role in obtaining a good optimizer that can achieve expertlevel performance. We describe new algorithms that take into account the variable cost (duration) of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.

Statistical Comparisons of Classifiers over Multiple Data Sets <=SEP=> While methods for comparing two learning algorithms on a single data set have been scrutinized for quite some time already, the issue of statistical tests for comparisons of more algorithms on multiple data sets, which is even more essential to typical machine learning studies, has been all but ignored. This article reviews the current practice and then theoretically and empirically examines several suitable tests. Based on that, we recommend a set of simple, yet safe and robust non-parametric tests for statistical comparisons of classifiers: the Wilcoxon signed ranks test for comparison of two classifiers and the Friedman test with the corresponding post-hoc tests for comparison of more classifiers over multiple data sets. Results of the latter can also be neatly presented with the newly introduced CD (critical difference) diagrams.

Rank aggregation methods for the Web <=SEP=> We consider the problem of combining ranking results from various sources. In the context of the Web, the main applications include building meta-search engines, combining ranking functions, selecting documents based on multiple criteria, and improving search precision through word associations. We develop a set of techniques for the rank aggregation problem and compare their performance to that of well-known methods. A primary goal of our work is to design rank aggregation techniques that can e ectively combat \spam," a serious problem in Web searches. Experiments show that our methods are simple, e cient, and e ective.

BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits <=SEP=> BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by building a posterior distribution to capture the evidence and prior knowledge for the target function. Built in standard C++, the library is extremely efficient while being portable and flexible. It includes a common interface for C, C++, Python, Matlab and Octave.

A taxonomy of argumentation models used for knowledge representation <=SEP=> Understanding argumentation and its role in human reasoning has been a continuous subject of investigation for scholars from the ancient Greek philosophers to current researchers in philosophy, logic and artificial intelligence. In recent years, argumentation models have been used in different areas such as knowledge representation, explanation, proof elaboration, commonsense reasoning, logic programming, legal reasoning, decision making, and negotiation. However, these models address quite specific needs and there is need for a conceptual framework that would organize and compare existing argumentation-based models and methods. Such a framework would be very useful especially for researchers and practitioners who want to select appropriate argumentation models or techniques to be incorporated in new software systems with argumentation capabilities. In this paper, we propose such a conceptual framework, based on taxonomy of the most important argumentation models, approaches and systems found in the literature. This framework highlights the similarities and differences between these argumentation models. As an illustration of the practical use of this framework, we present a case study which shows how we used this framework to select and enrich an argumentation model in a knowledge acquisition project which aimed at representing argumentative knowledge contained in texts critiquing military courses of action.

Bandwidth Modeling in Large Distributed Systems for Big Data Applications <=SEP=> The emergence of Big Data applications provides new challenges in data management such as processing and movement of masses of data. Volunteer computing has proven itself as a distributed paradigm that can fully support Big Data generation. This paradigm uses a large number of heterogeneous and unreliable Internet-connected hosts to provide Peta-scale computing power for scientific projects. With the increase in data size and number of devices that can potentially join a volunteer computing project, the host bandwidth can become a main hindrance to the analysis of the data generated by these projects, especially if the analysis is a concurrent approach based on either in-situ or in-transit processing. In this paper, we propose a bandwidth model for volunteer computing projects based on the real trace data taken from the Docking@Home project with more than 280,000 hosts over a 5-year period. We validate the proposed statistical model using model-based and simulation-based techniques. Our modeling provides us with valuable insights on the concurrent integration of data generation with in-situ and in-transit analysis in the volunteer computing paradigm.

Instant messaging and the future of language <=SEP=> The writing style commonly used in IMing, texting, and other forms of computer-mediated communication need not spell the end of normative language.

A Survey of Artificial Cognitive Systems: Implications for the Autonomous Development of Mental Capabilities in Computational Agents <=SEP=> This survey presents an overview of the autonomous development of mental capabilities in computational agents. It does so based on a characterization of cognitive systems as systems which exhibit adaptive, anticipatory, and purposive goal-directed behavior. We present a broad survey of the various paradigms of cognition, addressing cognitivist (physical symbol systems) approaches, emergent systems approaches, encompassing connectionist, dynamical, and enactive systems, and also efforts to combine the two in hybrid systems. We then review several cognitive architectures drawn from these paradigms. In each of these areas, we highlight the implications and attendant problems of adopting a developmental approach, both from phylogenetic and ontogenetic points of view. We conclude with a summary of the key architectural features that systems capable of autonomous development of mental capabilities should exhibit

Which robot am I thinking about? The impact of action and appearance on people's evaluations of a moral robot <=SEP=> In three studies we found further evidence for a previously discovered Human-Robot (HR) asymmetry in moral judgments: that people blame robots more for inaction than action in a moral dilemma but blame humans more for action than inaction in the identical dilemma (where inaction allows four persons to die and action sacrifices one to save the four). Importantly, we found that people's representation of the “robot” making these moral decisions appears to be one of a mechanical robot. For when we manipulated the pictorial display of a verbally described robot, people showed the HR asymmetry only when making judgments about a mechanical-looking robot, not a humanoid robot. This is the first demonstration that robot appearance affects people's moral judgments about robots.

Satellite alerts track deforestation in real time <=SEP=> A satellite-based alert system could prove a potent weapon in the fight against deforestation. As few as eight hours after it detects that trees are being cut down, the system will send out e-mails warning that an area is endangered. That rapid response could enable environmental managers to catch illegal loggers before they damage large swathes of forest. " It's going to be very, very helpful, " says Brian Zutta Salazar, a remote-sensing scientist at the Peruvian Ministry of the Environment in Lima. Satellites are already valuable tools for monitoring deforestation; in recent decades, they have delivered consistent data on forest change over large and often remote areas. One such effort, the Real Time System for Detection of Deforestation, or DETER, has helped Brazil's government to reduce its deforestation rate by almost 80% since 2004, by alerting the country's environmental police to large-scale forest clearing. But DETER and other existing alert systems can be relatively slow to yield useful information. They use data from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA's Terra satellite, which at its top resolution produces images with pixels covering an area 250 metres on each side, roughly equivalent to 10 football pitches. This is too big to spot small changes in land cover, so it can take computer programs that process MODIS data weeks or even months to detect that a forest is being cleared. " By the time MODIS picks up on it, it's almost too late, " says Peter Ellis, a forest-carbon scientist at the Nature Conservancy, a conservation group in Arlington, Virginia. Seeking to provide a sharper view, geographer Matthew Hansen of the University of Maryland in College Park and his colleagues published maps showing year-to-year changes in global forest cover from 2000 to 2012 (M. C. Hansen et al. Science 342, 850–853; 2013). The researchers relied on data from NASA's two active Landsat satellites, which together photograph every spot on Earth every eight days. Each pixel in a Landsat image is 30 metres on each side — roughly the size of a baseball diamond. This means that an area covered by just one MODIS pixel is captured in roughly 70 smaller Landsat pixels. Hansen and his team wrote data-processing software that can use these higher-resolution images to recognize a disturbance as small as a road snaking its way through a previously untouched forest, something that often appears before clear-cutting begins. " …

OpenSense: open community driven sensing of environment <=SEP=> This paper outlines a vision for community-driven sensing of our environment. At its core, community sensing is a dynamic new form of mobile geosensor network. We believe that community sensing networks, in order to be widely deployable and sustainable, need to follow utilitarian approaches towards sensing and data management. Current projects exploring community sensing have paid less attention to these underlying fundamental principles. We illustrate this vision through OpenSense -- a large project that aims to explore community sensing driven by air pollution monitoring.

Automated Essay Scoring 1 Automated Essay Scoring: Writing Assessment and Instruction <=SEP=> Automated Essay Scoring 2 Introduction This chapter documents the advent and rise of automated essay scoring (AES) as a means of both assessment and instruction. The first section discusses what AES is, how it works, and who the major purveyors of the technology are. The second section describes outgrowths of the technology as it applies to ongoing projects in measurement and education. In 1973, the late Ellis Page and colleagues at the University of Connecticut programmed the first successful automated essay scoring engine, " Project Essay Grade (PEG) " (1973). The technology was foretold some six years earlier in a landmark Phi Delta Kappan article entitled, " The Imminence of Grading Essays by Computer " (Page, 1966). At the time the article was provocative and a bit outrageous, though in hindsight, it can only be deemed prophetic. As a former high school English teacher , Page was convinced that students would benefit greatly by having access to technology that would provide quick feedback on their writing. He also realized that the greatest hindrance to having secondary students write more was the requirement that, ultimately, a teacher had to review stacks of papers. While PEG produced impressive results, the technology of the time was too primitive to make it a practical application. Text had to be typed on IBM 80-column punched cards and read into a mainframe computer before it could be evaluated. As a consequence, the technology sat dormant until the early 1990s and was revitalized with the confluence of two technological developments: microcomputers and the Internet. Microcomputers permitted the generation of electronic text from a regular keyboard and the internet provided a universal platform to submit text for review Automated Essay Scoring 3 Automated essay scoring is a measurement technology in which computers evaluate written work (Shermis & Burstein, 2003). Most of the initial applications have 2002). Computers do not " understand " the written text being evaluated. So, for example, the computer would not " get " the following joke. Q-When is a door not a door? A-When it is ajar. Unlike humans, a computer cannot interpret the play on words, and infer that the predicate in the answer (i.e., " ajar ") is being cleverly used as a noun (i.e., " a jar "). What the computer does in an AES context is to analyze the written text into its observable components. Different AES systems evaluate different …

SSD-Assisted Backup and Recovery for Database Systems <=SEP=> Backup and recovery is an important feature of database systems since it protects data against unexpected hardware and software failures. Database systems can provide data safety and reliability by creating a backup and restoring the backup from a failure. Database administrators can use backup/recovery tools that are provided with database systems or backup/recovery methods with operating systems. However, the existing tools perform time-consuming jobs and the existing methods may negatively affect run-time performance during normal operation even though high-performance SSDs are used. In this paper, we present an SSD-assisted backup/recovery scheme for database systems. In our scheme, we extend the out-of-place update characteristics of flash-based SSDs for backup/recovery operations. To this end, we exploit the resources (e.g., flash translation layer and DRAM cache with supercapacitors) inside SSDs, and we call our SSD with new backup/recovery features BR-SSD. We design and implement the backup/recovery functionality in the Samsung enterprise-class SSD (i.e., SM843Tn) for more realistic systems. Furthermore, we conduct a case study of BR-SSDs in replicated database systems and modify MySQL with replication to integrate BR-SSDs. The experimental result demonstrates that our scheme provides fast recovery while it does not negatively affect the run-time performance during normal operation.

A micro-video recommendation system based on big data <=SEP=> With the development of the Internet and social networking service, the micro-video is becoming more popular, especially for youngers. However, for many users, they spend a lot of time to get their favorite micro-videos from amounts videos on the Internet; for the micro-video producers, they do not know what kinds of viewers like their products. Therefore, this paper proposes a micro-video recommendation system. The recommendation algorithms are the core of this system. Traditional recommendation algorithms include content-based recommendation, collaboration recommendation algorithms, and so on. At the Bid Data times, the challenges what we meet are data scale, performance of computing, and other aspects. Thus, this paper improves the traditional recommendation algorithms, using the popular parallel computing framework to process the Big Data. Slope one recommendation algorithm is a parallel computing algorithm based on MapReduce and Hadoop framework which is a high performance parallel computing platform. The other aspect of this system is data visualization. Only an intuitive, accurate visualization interface, the viewers and producers can find what they need through the micro-video recommendation system.

SUIT: A Supervised User-Item Based Topic Model for Sentiment Analysis <=SEP=> Probabilistic topic models have been widely used for sentiment analysis. However, most of existing topic methods only model the sentiment text, but do not consider the user, who expresses the sentiment, and the item, which the sentiment is expressed on. Since different users may use different sentiment expressions for different items, we argue that it is better to incorporate the user and item information into the topic model for sentiment analysis. In this paper, we propose a new Supervised User-Item based Topic model, called SUIT model, for sentiment analysis. It can simultaneously utilize the textual topic and latent user-item factors. Our proposed method uses the tensor outer product of text topic proportion vector, user latent factor and item latent factor to model the sentiment label generalization. Extensive experiments are conducted on two datasets: review dataset and microblog dataset. The results demonstrate the advantages of our model. It shows significant improvement compared with supervised topic models and collaborative filtering methods.

Developing learner corpus annotation for Chinese grammatical errors <=SEP=> This study describes the construction of the TOCFL (Test Of Chinese as a Foreign Language) learner corpus, including the collection and grammatical error annotation of 2,837 essays written by Chinese language learners originating from a total of 46 different mother-tongue languages. We propose hierarchical tagging sets to manually annotate grammatical errors, resulting in 33,835 inappropriate usages. Our built corpus has been provided for the shared tasks on Chinese grammatical error diagnosis. These demonstrate the usability of our learner corpus annotation.

Information, Utility & Bounded Rationality <=SEP=> Perfectly rational decision-makers maximize expected utility, but crucially ignore the resource costs incurred when determining optimal actions. Here we employ an axiomatic framework for bounded rational decision-making based on a thermodynamic interpretation of resource costs as information costs. This leads to a variational “free utility” principle akin to thermodynamical free energy that trades off utility and information costs. We show that bounded optimal control solutions can be derived from this variational principle, which leads in general to stochastic policies. Furthermore, we show that risk-sensitive and robust (minimax) control schemes fall out naturally from this framework if the environment is considered as a bounded rational and perfectly rational opponent, respectively. When resource costs are ignored, the maximum expected utility principle is recovered.

Resilient 3D hierarchical architected metamaterials. <=SEP=> Hierarchically designed structures with architectural features that span across multiple length scales are found in numerous hard biomaterials, like bone, wood, and glass sponge skeletons, as well as manmade structures, like the Eiffel Tower. It has been hypothesized that their mechanical robustness and damage tolerance stem from sophisticated ordering within the constituents, but the specific role of hierarchy remains to be fully described and understood. We apply the principles of hierarchical design to create structural metamaterials from three material systems: (i) polymer, (ii) hollow ceramic, and (iii) ceramic-polymer composites that are patterned into self-similar unit cells in a fractal-like geometry. In situ nanomechanical experiments revealed (i) a nearly theoretical scaling of structural strength and stiffness with relative density, which outperforms existing nonhierarchical nanolattices; (ii) recoverability, with hollow alumina samples recovering up to 98% of their original height after compression to ≥ 50% strain; (iii) suppression of brittle failure and structural instabilities in hollow ceramic hierarchical nanolattices; and (iv) a range of deformation mechanisms that can be tuned by changing the slenderness ratios of the beams. Additional levels of hierarchy beyond a second order did not increase the strength or stiffness, which suggests the existence of an optimal degree of hierarchy to amplify resilience. We developed a computational model that captures local stress distributions within the nanolattices under compression and explains some of the underlying deformation mechanisms as well as validates the measured effective stiffness to be interpreted as a metamaterial property.

Handwritten Digit Recognition with a Back-Propagation Network <=SEP=> We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1 % error rate and about a 9% reject rate on zipcode digits provided by the U.S. Postal Service.

Common mode noise cancellation for electrically non-contact ECG measurement system on a chair. <=SEP=> Electrically non-contact ECG measurement system on a chair can be applied to a number of various fields for continuous health monitoring in daily life. However, the body is floated electrically for this system due to the capacitive electrodes and the floated body is very sensitive to the external noises or motion artifacts which affect the measurement system as the common mode noise. In this paper, the Driven-Seat-Ground circuit similar to the Driven-Right-Leg circuit is proposed to reduce the common mode noise. The analysis of this equivalent circuit is performed and the output signal waveforms are compared between with Driven-Seat-Ground and with capacitive ground. As the results, the Driven-Seat-Ground circuit improves significantly the properties of the fully capacitive ECG measurement system as the negative feedback.

Minimum Steiner Tree Construction <=SEP=> In optimizing the area of Very Large Scale Integrated (VLSI) layouts, circuit interconnections should generally be realized with minimum total interconnect. This chapter addresses several variations of the corresponding fundamental Steiner minimal tree (SMT) problem, where a given set of pins is to be connected using minimum total wirelength. Steiner trees are important in global routing and wirelength estimation [15], as well as in various non-VLSI applications such as phylogenetic tree reconstruction in biology [48], network routing [61], and civil engineering, among many other areas [21, 25, 26, 29, 51, 74].

Dynamic path planning and replanning for mobile robots using RRT <=SEP=> It is necessary for a mobile robot to be able to efficiently plan a path from its starting, or current location to a desired goal location. This is a trivial task when the environment is static. However, the operational environment of the robot is rarely static, and it often has many moving obstacles. The robot may encounter one, or many of these unknown and unpredictable moving obstacles. The robot will need to decide how to proceed when one of these obstacles is obstructing it's path. A method of dynamic replanning using RRT∗ is presented. The robot will modify it's current plan when an unknown random moving obstacle obstructs the path. Various experimental results show the effectiveness of the proposed method.

Statistical Pattern Recognition <=SEP=> This course will provide an introduction to statistical pattern recognition. The lectures will focus on different techniques including methods for feature extraction, dimensionality reduction, data clustering and pattern classification. State-of-art approaches such as ensemble learning and sparse modelling will be introduced. Selected real-world applications will illustrate how the techniques are applied in practice.

Impacts of IoT and big data to automotive industry <=SEP=> With the recent advancement in technologies such as embedded system, wireless distributed sensor, light weight material, smart cognitive radio networks, cloud computing, higher efficiency and ultra-low emission internal combustion engines, intelligent converter, high performance battery and fuel cell technology, the production of smarter, safer, energy efficient and zero emission vehicles is possible in near future. Apart from vehicle technologies, other factors such as road users', well maintained road infrastructure, well maintained vehicles, drivers' attitudes and law and enforcement are also important to be considered and they should work together in order to make our world natural resources can be preserved and maintain cleaner environment and produce sustainable mobility. This paper will discuss the impacts of IoT and Big Data and other emerging technologies mentioned above to the automotive industry. It will include discussion on education, economy, advanced technology, environment, safety and energy.

Multilingual Models for Compositional Distributed Semantics <=SEP=> We present a novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings. Our models leverage parallel data and learn to strongly align the embeddings of semantically equivalent sentences, while maintaining sufficient distance between those of dissimilar sentences. The models do not rely on word alignments or any syntactic information and are successfully applied to a number of diverse languages. We extend our approach to learn semantic representations at the document level, too. We evaluate these models on two cross-lingual document classification tasks, outperforming the prior state of the art. Through qualitative analysis and the study of pivoting effects we demonstrate that our representations are semantically plausible and can capture semantic relationships across languages without parallel data.

Efficient GPU Screen-Space Ray Tracing <=SEP=> We present an efficient GPU solution for screen-space 3D ray tracing against a depth buffer by adapting the perspective-correct DDA line rasterization algorithm. Compared to linear ray marching, this ensures sampling at a contiguous set of pixels and no oversampling. This paper provides for the first time full implementation details of a method that has been proven in production of recent major game titles. After explaining the optimizations, we then extend the method to support multiple depth layers for robustness. We include GLSL code and examples of pixel-shader ray tracing for several applications.

Compressive phase retrieval via generalized approximate message passing <=SEP=> In this paper, we propose a novel approach to compressive phase retrieval based on loopy belief propagation and, in particular, on the generalized approximate message passing (GAMP) algorithm. Numerical results show that the proposed PR-GAMP algorithm has excellent phase-transition behavior, noise robustness, and runtime. In particular, for successful recovery of synthetic Bernoulli-circular-Gaussian signals, PR-GAMP requires ≈4 times the number of measurements as a phase-oracle version of GAMP and, at moderate to large SNR, the NMSE of PR-GAMP is only ≈3 dB worse than that of phase-oracle GAMP. A comparison to the recently proposed convex-relation approach known as “CPRL” reveals PR-GAMP's superior phase transition and orders-of-magnitude faster runtimes, especially as the problem dimensions increase. When applied to the recovery of a 65k-pixel grayscale image from 32k randomly masked magnitude measurements, numerical results show a median PR-GAMP runtime of only 13.4 seconds.

A Rule-Based System to Extract Financial Information <=SEP=> Extracting up-to-date information from financial documents can be important in making investment decisions. However, the unstructured nature and enormity of the volume of such data makes manual analysis tedious and time consuming. Information extraction technology can be applied to automatically extract the most relevant and precise financial information. This paper introduces a rule-based information extraction methodology for the extraction of highly accurate financial information to aid investment decisions. The methodology includes a rule-based symbolic learning model trained by the Greedy Search algorithm and a similar model trained by the Tabu Search algorithm. The methodology has been found very effective in extracting financial information from NASDAQ-listed companies. Also, the Tabu Search based model performed better than some well-known systems. The simple rule structure makes the system portable and it should make parallel processing implementations easier.

Autonomous Intersection Management For Semi-Autonomous Vehicles <=SEP=> Recent advances in autonomous vehicle technology will open the door to highly efficient transportation systems in the future, as demonstrated by Autonomous Intersection Management, an intersection control protocol designed for fully autonomous vehicles. We, however, anticipate there will be a long transition period during which most vehicles have some but not all capabilities of fully autonomous vehicles. This chapter introduces a new protocol called Semi-Autonomous Intersection Management, which allows vehicles with partiallyautonomous features such as adaptive cruise control to enter an intersection from different directions simultaneously. Our experiments show that this protocol can greatly decrease traffic delay when most vehicles are semi-autonomous. Our incremental deployment study reveals that traffic delay keeps decreasing as more and more vehicles employ features of autonomy.