nils-herrmann/clean_scirepeval huggingface dataset:
Automatic Derivation of the Multinomial PCA Algorithm

Machine learning has reached a point where probabilistic methods can be understood as variations, extensions, and combinations of a small set of abstract themes, e.g., as different instances of the EM algorithm, or as exponential family methods. This allows the automatic derivation of algorithms customized for different models. One interesting new model is a multinomial version of PCA which has received attention due to its ability to better model documents as having multiple topics. Here we explain how AutoBayes, an automatic synthesis system for machine learning programs, derives from a high-level statistical specification of this model code similar to the original non-negative matrix factorization algorithm of Lee and Seung. The derivation combines multiple statistical schemes to solve a new problem not originally envisaged. This demonstrates that the approach can be scaled up from text-book problems to research-level algorithm design.
Variational Extensions to EM and Multinomial PCA

Several authors in recent years have proposed discrete analogues to principle component analysis intended to handle discrete or positive only data, for instance suited to analyzing sets of documents. Methods include non-negative matrix factorization, probabilistic latent semantic analysis, and latent Dirichlet allocation. This paper begins with a review of the basic theory of the variational extension to the expectation-maximization algorithm, and then presents discrete component finding algorithms in that light. Experiments are conducted on both bigram word data and document bag-of-word to expose some of the subtleties of this new class of algorithms.
Building and Maintaining Web Taxonomies

A recognized problem for internet commerce is the task of building a product taxonomy from web pages, without access to corporate databases, and then populating a database with link information about service, spare parts, reviews, product specifications, product family, etc. A key precursor for this task is the ability to build classification hierarchies in an unsupervised manner of web pages potentially useful. However, a nasty aspect of the real world is that most web pages have multiple facets. A web page might contain information about cameras and computers, as well as having both specification and sale data. We are interested in methods for unsupervised learning of multiple facet models which automatically allow these multiple facets to be extracted or predicted from examples. We will present some examples using the Reuters 2000 XML Corpus containing 800,000 Reuters newswires from the year beginning 20th August 1996. Our system, called Ydin, inputs XML/XHTML pages and displays results using a custom web interface. It extracts different facets and provides a navigation tool for investigating the results.
Automatic Derivation of Statistical Algorithms: The EM Family and Beyond

Machine learning has reached a point where many probabilistic methods can be understood as variations, extensions and combinations of a much smaller set of abstract themes, e.g., as different instances of the EM algorithm. This enables the systematic derivation of algorithms customized for different models. Here, we describe the AUTOBAYES system which takes a high-level statistical model specification, uses powerful symbolic techniques based on schema-based program synthesis and computer algebra to derive an efficient specialized algorithm for learning that model, and generates executable code implementing that algorithm. This capability is far beyond that of code collections such as Matlab toolboxes or even tools for model-independent optimization such as BUGS for Gibbs sampling: complex new algorithms can be generated without new programming, algorithms can be highly specialized and tightly crafted for the exact structure of the model and data, and efficient and commented code can be generated for different languages or systems. We present automatically-derived algorithms ranging from closed-form solutions of Bayesian textbook problems to recently-proposed EM algorithms for clustering, regression, and a multinomial form of PCA.
Learning as applied to stochastic optimization for standard-cellplacement

Stochastic combinatorial optimization techniques, such as simulated annealing and genetic algorithms, have become increasingly important in design automation as the size of design problems have grown and the design objectives have become increasingly complex. However, stochastic algorithms are, often slow since a large number of random design perturbations are required to achieve an acceptable result-they have no built-in "intelligence". In this paper, it is shown that statistical learning techniques can improve the quality of results and reduce the number of expensive cost-function evaluations for stochastic optimization for a particular solution quality. In particular, simulated annealing was selected as a representative stochastic optimization approach and a two-dimensional cell-based layout placement problem was used to evaluate the utility of such a learning-based approach. In this paper, we used regression to learn the properties of the solution space and tested the trained algorithm on a number of examples to demonstrate the improvement gained. A general response model is constructed by learning from the annealing of benchmark circuits. This model is then used in the trained simulated annealing, which returns significantly better annealing quality than the untrained algorithm for the same number of moves in the solution space. The annealing quality improvement was 15%-43% for the set of examples used in training and 7%-21% when the trained algorithm was applied to new examples.
Autonomous Science Decisions for Mars Sample Return

In the near future NASA intends to explore Mars in preparation for a sample return mission using robotic devices such as landers rovers, orbiters, airplanes, and/or balloons. Such platforms will likely carry imaging devices to characterize the surface morphology, and a variety of analytical instruments intended to evaluated the chemical and mineralogical nature of the environment(s) that they encounter. Historically, mission operations have involved the following sequence of activities: (1) return of scientific data from the vehicle; (2) evaluation of the data by space scientists; (3) recommendations of the scientists regarding future mission activity; (4) transmission of commands to the vehicle to achieve this activity; and (5) new activity by the vehicle in response to those commands.
Towards automated synthesis of data mining programs

discovery community. Code synthesis is routinely used in industry to generate GUIs and for database support. In this paper we consider whether code synthesis could also be applied as a rapid prototyping method to the data mining phase of knowledge discovery. Rapid prototyping of statistical data analysis algorithms would allow experienced analysts to experiment with diierent statistical models before choosing one, but without requiring prohibitively expensive programming efforts. It would also smooth the steep learning curve often faced by novice users of data mining tools and libraries. Finally, it would accelerate dissemination of essential research results. For the synthesis task, we use a specification language that generalizes Bayesian networks, a dependency model on variables. Using decomposition methods and algorithm templates, our system transforms the network through several levels of representation into pseudo-code which can be translated into the implementation language of choice. Here, we explain the framework on a mixture of Gaussians model used in some commercial clustering tools. We show the effectiveness of our framework by generating pseudo-code for some more sophisticated algorithms from recent literature.
Autonomous Science Decision Making for Mars Sample Return

In the near future NASA intends to explore Mars in preparation for a sample return mission using robotic devices such as landers, rovers, orbiters, airplanes, and/or balloons. Such platforms will likely carry imaging devices to characterize the surface morphology, and a variety of analytical instruments intended to evaluate the chemical and mineralogical nature of the environment(s) that they encounter. Historically, mission operations have involved the following sequence of activities: (1) return of scientific data from the vehicle; (2) evaluation of the data by space scientists; (3) recommendations of the scientists regarding future mission activity; (4) transmission of commands to the vehicle to achieve this activity; and (5) new activity by the vehicle in response to those commands. This is repeated for the duration of the mission, with command opportunities once or perhaps twice per day. In a rapidly changing environment, such as might be encountered by a rover traversing hundreds of meters a day or an airplane soaring over several hundred of kilometers, this traditional cycle of data evaluation and commands is not amenable to rapid long range traverses, discovery of novelty, or rapid response to any unanticipated situations. In addition, to issues of response time, the nature of imaging and/or spectroscopic devices are such that tremendous data volumes can be acquired, for example during a traverse. These data volumes can rapidly exceed on-board memory capabilities prior to an opportunity to transmit it to Earth.
Transformation Systems at NASA Ames

In this paper, we describe the experiences of the Automated Software Engineering Group at the NASA Ames Research Center in the development and application of three different transformation systems. The systems span the entire technology range, from deductive synthesis, to logic-based transformation, to almost compiler-like source-to-source transformation. These systems also span a range of NASA applications, including solving solar system geometry problems, generating data analysis software, and analyzing multithreaded Java code.
On-board Science Understanding: NASA Ames' Efforts

In the near future NASA intends to explore various regions of our solar system using robotic devices such as rovers, spacecraft, airplanes, and/or balloons. Such platforms will likely carry imaging devices, and a variety of analytical instruments intended to evaluate the chemical and mineralogical nature of the environment(s) that they encounter. Historically, mission operations have involved: (1) return of scientific data from the craft; (2) evaluation of the data by space scientists; (3) recommendations of the scientists regarding future mission activity; (4) commands for achieving these activities being transmitted to the craft; and (5) the activity being undertaken. This cycle is then repeated for the duration of the mission with command opportunities once or perhaps twice per day. In a rapidly changing environment, such as might be encountered by a rover traversing hundreds of meters a day or a spacecraft encountering an asteroid, this historical cycle is not amenable to rapid long range traverses, discovery of novelty, or rapid response to any unexpected situations. In addition to real-time response issues, the nature of imaging and/or spectroscopic devices are such that tremendous data volumes can be acquired, for example during a traverse. However, such data volumes can rapidly exceed on-board memory capabilities prior to the ability to transmit it to Earth. Additionally, the necessary communication band-widths are restrictive enough so that only a small portion of these data can actually be returned to Earth. Such scenarios clearly require the enabling of some crucial decisions to be made on-board by these robotic explorers. These decisions transcend the electromechanical control, health, and navigation issues associated with robotic operations. Instead they focus upon a long term goal of automating scientific discovery based upon data returned by sensors of the robot craft. Such an approach would eventually enable it to understand what is interesting because the data deviates from expectations generated by current theories/models of planetary processes that could have resulted in the observed data. Such interesting data and/or conclusions can then be selectively transmitted to Earth thus reducing memory and communications demands.
Analysing Rock Samples for the Mars Lander

In the near future NASA intends to explore various regions of our solar system using robotic devices such as rovers, spacecraft, airplanes, and/or balloons. Such platforms will carry imaging devices, and a variety of analytical instruments intended to evaluate the chemical and mineralogical nature of the environment(s) that they encounter. The imaging and/or spectroscopic devices will acquire tremendous volumes of data. The communication band-widths are restrictive enough so that only a small portion of these data can actually be sent to Earth. 
 
The aim of this research was to develop a system which analyses rock spectra to automatically determine which spectra are interesting, and to compress the spectral data for communication to Earth. In the research we report here we classify laboratory data using clustering techniques (ACPro an enhanced version of Auto-class) and provide the planetary scientists with a rapid, visually oriented method of evaluating the underlying chemical and mineralogical information contained within the clusters. We show how clustering can be used to identify interesting rock samples and estimate the compression that using such a system can achieve.
Data Understanding Applied to Optimization

The goal of this research is to explore and develop software for supporting visualization and data analysis of search and optimization. Optimization is an ever-present problem in science. The theory of NP-completeness implies that the problems can only be resolved by increasingly smarter problem specific knowledge, possibly for use in some general purpose algorithms. Visualization and data analysis offers an opportunity to accelerate our understanding of key computational bottlenecks in optimization and to automatically tune aspects of the computation for specific problems. We will prototype systems to demonstrate how data understanding can be successfully applied to problems characteristic of NASA's key science optimization tasks, such as central tasks for parallel processing, spacecraft scheduling, and data transmission from a remote satellite.
Learning as applied to stochastic optimization for standard cell placement

Although becoming increasingly important, stochastic algorithms are often slow since a large number of random design perturbations are required to achieve an acceptable result-they have no built-in "intelligence". In this work, we used regression to learn the swap evaluation function while simulated annealing is applied to 2D standard-cell placement problem. The learned evaluation function is then applied to the trained simulated annealing algorithm (TSA). The annealing quality improvement of TSA was 15%/spl sim/43% for the set of examples used in learning and 7%/spl sim/21% for new examples. With the same amount of CPU time, TSA could improve the annealing quality by up to 28% for some benchmark circuits we tested. In addition the use of the evaluation function successfully predicted the effect of the windowed sampling technique and derived the informally accepted advantages of windowing from the test set automatically.
Trends & Controversies: Will Domain-Specific Code Synthesis Become a Silver Bullet?

In the commercial world, the biggest hurdles to delivering embedded intelligence in software are the software-engineering costs. We members of the intelligent-systems community have brilliant techniques available, and no end of applications is in sight, but the gap between intelligent-systems techniques and delivered software is too great. Standard programming languages and tools are too general-purpose, and the standard means of delivering a technology-as code libraries or new-fangled componentware-is simply too restricting for some specialties. One way would be through domain-specific code synthesis. Support for the development of domain-specific extensions to programming languages and of adaptive software-where code synthesis is in the inner-loop of a self-modifying programming system-would have a profound effect in this regard. This is the future of intelligent systems development.
Adaptive methods for netlist partitioning

An algorithm that remains in use at the core of many partitioning systems is the Kemighan-Lin algorithm and a variant the Fidducia-Matheysses (FM) algorithm. To understand the FM algorithm we applied principles of data engineering where visualization and statistical analysis are used to analyze the run-time behavior. We identified two improvements to the algorithm which, without clustering or an improved heuristic function, bring the performance of the algorithm near that of more sophisticated algorithms. One improvement is based on the observation, explored empirically, that the full passes in the FM algorithm appear comparable to a stochastic local restart in the search. We motivate this observation with a discussion of recent improvements in Monte Carlo Markov Chain methods in statistics. The other improvement is based on the observation that when an FM-like algorithm is run 20 times and the best run chosen, the performance trace of the algorithm on earlier runs is useful data for learning when to abort a later run. These improvements, implemented with a simple adaptive scheme, are orthogonal to techniques used in state-of-the-art implementations, and therefore should be applicable to other VLSI optimization algorithms.
Bayesian in principle, but not always in practice

Before considering the whys and wherefores of Bayesian reasoning in practice, the author considers the different uses of the term and the role that Bayesian reasoning should play for the scientific community. Probabilistic reasoning or modeling, together with its partner, decision theory, is a key theoretical tool for addressing uncertainty in intelligent systems of all kinds. Probabilistic reasoning comes in many shapes and sizes, and does not necessarily imply the use of Bayesian methods proper, as the author discusses.
Adaptive methods for netlist partitioning

An algorithm that remains in use at the core of many partitioning systems is the Kernighan-Lin algorithm and a variant the Fidducia-Matheysses (FM) algorithm. To understand the FM algorithm we applied principles of data engineering where visualization and statistical analysis are used to analyze the run-time behavior. We identified two improvements to the algorithm which, without clustering or an improved heuristic function, bring the performance of the algorithm near that of more sophisticated algorithms. One improvement is based on the observation, explored empirically, that the full passes in the FM algorithm appear comparable to a stochastic local restart in the search. We motivate this observation with a discussion of recent improvements in Monte Carlo Markov Chain methods in statistics. The other improvement is based on the observation that when an FM-like algorithm is run 20 times and the best run chosen, the performance trace of the algorithm on earlier runs is useful data for learning when to abort a later run. These improvements, implemented with a simple adaptive scheme, are orthogonal to techniques used in state-of-the-art implementations, and should therefore be applicable to other VLSI optimization algorithms.
A Guide to the Literature on Learning Probabilistic Networks from Data

The literature review presented discusses different methods under the general rubric of learning Bayesian networks from data, and includes some overlapping work on more general probabilistic networks. Connections are drawn between the statistical, neural network, and uncertainty communities, and between the different methodological communities, such as Bayesian, description length, and classical statistics. Basic concepts for learning and Bayesian networks are introduced and methods are then reviewed. Methods are discussed for learning parameters of a probabilistic network, for learning the structure, and for learning hidden variables. The article avoids formal definitions and theorems, as these are plentiful in the literature, and instead illustrates key concepts with simplified examples.
Graphical models for discovering knowledge

There are many diierent ways of representing knowledge, and for each of these ways there are many diierent discovery algorithms. How can we compare diierent representations? How can we mix, match and merge representations and algorithms on new problems with their own unique requirements? This chapter introduces probabilistic modeling as a philosophy for addressing these questions and presents graphical models for representing probabilistic models. Probabilis-tic graphical models are a uniied qualitative and quantitative framework for representing and reasoning with probabilities and independencies. 4.1  Perhaps one common element of the discovery systems described in this and previous books on knowledge discovery is that they are all diierent. Since the class of discovery problems is a challenging one, we cannot write a single program to address all of knowledge discovery. The KEFIR discovery system applied to health care by Matheus, Piatetsky-Shapiro, and McNeill (1995), for instance, is carefully tailored for a particular class of situations and could not have been easily used on the SKICAT application (Fayyad, Djorgovski, and Weir 1995). I do not know of a universal learning or discovery algorithm (Buntine 1990), and a universal problem description for discovery is arguably too broad to be used as a program speciication. As a consequence, the power to perform in an application lies in the way knowledge about the application is obtained, used, represented and modiied. Unfortunately with
Predicting engine parameters using the optic spectrum of the space shuttle main engine exhaust plume

The Optical Plume Anomaly Detection (OPAD) system is under development to predict engine anomalies and engine parameters of the Space Shuttle's Main Engine (SSME). The anomaly detection is based on abnormal metal concentrations in the optical spectrum of the rocket plume. Such abnormalities could be indicative of engine corrosion or other malfunctions. Here, we focus on the second task of the OPAD system, namely the prediction of engine parameters such as rated power level (RPL) and mixture ratio (MR). Because of the high dimensionality of the spectrum, we developed a linear algorithm to resolve the optical spectrum of the exhaust plume into a number of separate components, each with a different physical interpretation. These components are used to predict the metal concentrations and engine parameters for online support of ground-level testing of the SSME. Currently, these predictions are labor intensive and cannot be done online. We predict RPL using neural networks and give preliminary results.
Chain graphs for learning

Chain graphs combine directed and undi­ rected graphs and their underlying mathe­ matics combines properties of the two. This paper gives a simplified definition of chain graphs based on a hierarchical combination of Bayesian (directed) and Markov (undirected) networks. Examples of a chain graph are multivariate feed-forward networks, cluster­ ing with conditional interaction between vari­ ables, and forms of Bayes classifiers. Chain graphs are then extended using the notation of plates so that samples and data analysis problems can be represented in a graphical model as well. Implications for learning are discussed in the conclusion.