nils-herrmann/clean_scirepeval huggingface dataset:
Unstructured adaptive mesh computations of rotorcraft high-speed impulsive noise

A new method is developed for modeling helicopter high-speed impulsive (HSI) noise. The aerodynamics and acoustics near the rotor blade tip are computed by solving the Euler equations on an unstructured grid. A stationary Kirchhoff surface integral is then used to propagate these acoustic signals to the far field. The near-field Euler solver uses a solution-adaptive grid scheme to improve the resolution of the acoustic signal. Grid points are locally added and/or deleted from the mesh at each adaptive step. An important part of this procedure is the choice of an appropriate error indicator. The error indicator is computed from the flow field solution and determines the regions for mesh coarsening and refinement. Computed results for HSI noise compare favorably with experimental data for three different hovering rotor cases.
Helicopter Noise Predictions Using Kirchhoff Methods

This paper presents two methods for predicting the noise from helicopter rotors in forward flight. Aerodynamic and acoustic solutions in the near field are computed with a finite-difference solver for the Euler equations. Two different Kirchhoff acoustics methods are then used to propagate the acoustic signals to the far field in a computationally-efficient manner. One of the methods uses a Kirchhoff surface that rotates with the rotor blades. The other uses a nonrotating Kirchhoff surface. Results from both methods are compared to experimental data for both high-speed impulsive noise and blade-vortex interaction noise. Agreement between experimental data and computational results is excellent for both cases. The rotating and nonrotating Kirchhoff methods are also compared for accuracy and efficiency. Both offer high accuracy with reasonable computer resource requirements. The Kirchhoff integrations efficiently extend the near-field finite-difference results to predict the far field helicopter noise.
A Solution Adaptive Structured/Unstructured Overset Grid Flow Solver with Applications to Helicopter Rotor Flows

This paper summarizes a method that solves both the three dimensional thin-layer Navier-Stokes equations and the Euler equations using overset structured and solution adaptive unstructured grids with applications to helicopter rotor flowfields. The overset structured grids use an implicit finite-difference method to solve the thin-layer Navier-Stokes/Euler equations while the unstructured grid uses an explicit finite-volume method to solve the Euler equations. Solutions on a helicopter rotor in hover show the ability to accurately convect the rotor wake. However, isotropic subdivision of the tetrahedral mesh rapidly increases the overall problem size.
Coarticulation: an approach for generating concurrent plans in Markov decision processes

We study an approach for performing concurrent activities in Markov decision processes (MDPs) based on the coarticulation framework. We assume that the agent has multiple degrees of freedom (DOF) in the action space which enables it to perform activities simultaneously. We demonstrate that one natural way for generating concurrency in the system is by coarticulating among the set of learned activities available to the agent. In general due to the multiple DOF in the system, often there exists a redundant set of admissible sub-optimal policies associated with each learned activity. Such flexibility enables the agent to concurrently commit to several subgoals according to their priority levels, given a new task defined in terms of a set of prioritized subgoals. We present efficient approximate algorithms for computing such policies and for generating concurrent plans. We also evaluate our approach in a simulated domain.
Samuel Meets Amarel: Automating Value Function Approximation Using Global State Space Analysis

Most work on value function approximation adheres to Samuel's original design: agents learn a task-specific value function using parameter estimation, where the approximation architecture (e.g, polynomials) is specified by a human designer. This paper proposes a novel framework generalizing Samuel's paradigm using a coordinate-free approach to value function approximation. Agents learn both representations and value functions by constructing geometrically customized task-independent basis functions that form an orthonormal set for the Hilbert space of smooth functions on the underlying state space manifold. The approach rests on a technical result showing that the space of smooth functions on a (compact) Riemanian manifold has a discrete spectrum associated with the Laplace-Beltrami operator. In the discrete setting, spectral analysis of the graph Laplacian yields a set of geometrically customized basis functions for approximating and decomposing value functions. The proposed framework generalizes Samuel's value function approximation paradigm by combining it with a formalization of Saul Amarel's paradigm of representation learning through global state space analysis.
A Variational Learning Algorithm for the Abstract Hidden Markov Model

We present a fast algorithm for learning the parameters of the abstract hidden Markov model, a type of hierarchical activity recognition model. Learning using exact inference scales poorly as the number of levels in the hierarchy increases; therefore, an approximation is required for large models. We demonstrate that variational inference is well suited to solve this problem. Not only does this technique scale. but it also offers a natural way to leverage the context specific independence properties inherent in the model via the fixed point equations. Experiments confirm that the variational approximation significantly reduces the time necessary for learning while estimating parameter values that can be used to make reliable predictions.
Hierarchical reinforcement learning in continuous state and multi-agent environments

This dissertation investigates the use of hierarchy and abstraction as a means of solving complex sequential decision making problems such as those with continuous state and/or continuous action spaces, and domains with multiple cooperative agents. This thesis develops several novel extensions to hierarchical reinforcement learning (HRL), and designs algorithms that are appropriate for such problems. 
It has been shown that the average reward optimality criterion is more natural than the more commonly used discounted criterion for continuing tasks. This thesis investigates two formulations of HRL based on the average reward semi-Markov decision process (SMDP) model, both for discrete-time and continuous-time. These formulations correspond to two notions of optimality that have been explored in previous work on HRL: hierarchical optimality and recursive optimality. Novel discrete-time and continuous-time algorithms, termed hierarchically optimal average reward RL (HAR) and recursively optimal average reward RL (RAR) are presented, which learn to find hierarchically and recursively optimal average reward policies. Two automated guided vehicle (AGV) scheduling problems are used as experimental testbeds to empirically study the performance of the proposed algorithms. 
Policy gradient reinforcement learning (PGRL) methods have several advantages over the more traditional value function RL algorithms in solving problems with continuous state spaces. However, they suffer from slow convergence. This thesis defines a family of hierarchical policy gradient RL (HPGRL) algorithms for scaling PGRL methods to high-dimensional domains. 
This thesis also examines the use of HRL to accelerate policy learning in cooperative multi-agent tasks. The use of hierarchy speeds up learning in multi-agent domains by making it possible to learn coordination skills at the level of subtasks instead of primitive actions. Subtask-level coordination allows for increased cooperation skills as agents do not get confused by low-level details. A framework for hierarchical multi-agent RL is developed and an algorithm called Cooperative HRL is presented that solves cooperative multi-agent problems more efficiently.
Value Function Approximation with Diffusion Wavelets and

We investigate the problem of automatically constructing efficient representations or basis functions for approximating value functions based on analyzing the structure and topology of the state space. In particular, two novel approaches to value function approximation are explored based on automatically constructing basis functions on state spaces that can be represented as graphs or manifolds: one approach uses the eigenfunctions of the Laplacian, in effect performing a global Fourier analysis on the graph; the second approach is based on diffusion wavelets, which generalize classical wavelets to graphs using multiscale dilations induced by powers of a diffusion operator or random walk on the graph. Together, these approaches form the foundation of a new generation of methods for solving large Markov decision processes, in which the underlying representation and policies are simultaneously learned.
Representation Policy Iteration

This paper addresses a fundamental issue central to approximation methods for solving large Markov decision processes (MDPs): how to automatically learn the underlying representation for value function approximation? A novel theoretically rigorous framework is proposed that automatically generates geometrically customized orthonormal sets of basis functions, which can be used with any approximate MDP solver like least-squares policy iteration (LSPI). The key innovation is a coordinate-free representation of value functions, using the theory of smooth functions on a Riemannian manifold. Hodge theory yields a constructive method for generating basis functions for approximating value functions based on the eigenfunctions of the self-adjoint (Laplace-Beltrami) operator on manifolds. In effect, this approach performs a global Fourier analysis on the state space graph to approximate value functions, where the basis functions reflect the large-scale topology of the underlying state space. A new class of algorithms called Representation Policy Iteration (RPI) are presented that automatically learn both basis functions and approximately optimal policies. Illustrative experiments compare the performance of RPI with that of LSPI using two handcoded basis functions (RBF and polynomial state encodings).
Switching kalman filters for prediction and tracking in an adaptive meteorological sensing network

We consider the problem of configuring sen- sors in an adaptive sensor network being used to monitor meteorological features. One way to decide future sensor configurations is to base them on information currently being collected. For instance, if a meteorological sensor network is being used to monitor storms in Oklahoma, then the sensors could be dynamically configured based on the predicted storm locations. While Kalman filters and their extensions are commonly used for prediction and tracking, they have been primarily applied to objects with known or fixed dynamics such as missiles or people. We explore the advantages and limitations of using Kalman filters to track objects with nonstationary dynamics (e.g., a storm can grow in size). In particular, we focus on tracking meteorological features over time with the objective of using this information to determine where radars should focus their sensing. We present results for tracking storm cells comparing least-squares regression with Kalman filter and switching Kalman filter methods. Our results show that on average the Kalman filter methods better predict the future location of a storm centroid than does a least-squares regression algorithm currently in use for meteorological storm tracking.
Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results

This paper presents a detailed study of average reward reinforcement learning, an undiscounted optimality framework that is more appropriate for cyclical tasks than the much better studied discounted framework. A wide spectrum of average reward algorithms are described, ranging from synchronous dynamic programming methods to several (provably convergent) asynchronous algorithms from optimal control and learning automata. A general sensitive discount optimality metric called n-discount-optimality is introduced, and used to compare the various algorithms. The overview identifies a key similarity across several asynchronous algorithms that is crucial to their convergence, namely independent estimation of the average reward and the relative values. The overview also uncovers a surprising limitation shared by the different algorithms: while several algorithms can provably generate gain-optimal policies that maximize average reward, none of them can reliably filter these to produce bias-optimal (or T-optimal) policies that also maximize the finite reward to absorbing goal states. This paper also presents a detailed empirical study of R-learning, an average reward reinforcement learning method, using two empirical testbeds: a stochastic grid world domain and a simulated robot environment. A detailed sensitivity analysis of R-learning is carried out to test its dependence on learning rates and exploration levels. The results suggest that R-learning is quite sensitive to exploration strategies, and can fall into sub-optimal limit cycles. The performance of R-learning is also compared with that of Q-learning, the best studied discounted RL method. Here, the results suggest that R-learning can be fine-tuned to give better performance than Q-learning in both domains.
Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions

We investigate the problem of automatically constructing efficient representations or basis functions for approximating value functions based on analyzing the structure and topology of the state space. In particular, two novel approaches to value function approximation are explored based on automatically constructing basis functions on state spaces that can be represented as graphs or manifolds: one approach uses the eigenfunctions of the Laplacian, in effect performing a global Fourier analysis on the graph; the second approach is based on diffusion wavelets, which generalize classical wavelets to graphs using multiscale dilations induced by powers of a diffusion operator or random walk on the graph. Together, these approaches form the foundation of a new generation of methods for solving large Markov decision processes, in which the underlying representation and policies are simultaneously learned.
Spatial and Temporal Abstractions in POMDPs Applied to Robot Navigation

Partially observable Markov decision processes (POMDPs) are a well studied paradigm for programming autonomous robots, where the robot sequentially chooses actions to achieve long term goals efficiently. Unfortunately, for real world robots and other similar domains, the uncertain outcomes of the actions and the fact that the true world state may not be completely observable make learning of models of the world extremely difficult, and using them algorithmically infeasible. In this paper we show that learning POMDP models and planning with them can become signifcantly easier when we incorporate into our algorithms the notions of spatial and temporal abstraction. We demonstrate the superiority of our algorithms by comparing them with previous flat approaches for large scale robot navigation.
Proto-value functions: developmental reinforcement learning

This paper presents a novel framework called proto-reinforcement learning (PRL), based on a mathematical model of a proto-value function: these are task-independent basis functions that form the building blocks of all value functions on a given state space manifold. Proto-value functions are learned not from rewards, but instead from analyzing the topology of the state space. Formally, proto-value functions are Fourier eigenfunctions of the Laplace-Beltrami diffusion operator on the state space manifold. Proto-value functions facilitate structural decomposition of large state spaces, and form geodesically smooth orthonormal basis functions for approximating any value function. The theoretical basis for proto-value functions combines insights from spectral graph theory, harmonic analysis, and Riemannian manifolds. Proto-value functions enable a novel generation of algorithms called representation policy iteration, unifying the learning of representation and behavior.
Reconfigurable and Adaptable Micro-Robots

This report documents the mechanical structure, kinematic models and control implementation of two micro-robots, called Flipper and Crawler, that can traverse both on walls and floors. They are battery powered and use specially designed feet using suction cups. The design of these feet is described in Section 2. The mechanical design was arrived at through extensive analysis. It has a biped structure with revolute and prismatic hip joints. The underactuated mechanical structure was chosen to reduce weight and power consumption by decreasing the number of actuators. Micro-controllers from Motorola, Inc. and Texas Instruments, Inc. were used as embedded controllers with the ability to implement motion planning and trajectory generation algorithms to control these robots. The details are given in Sections 3 and 4. The report concludes with the summary and suggested improvements.
Hierarchical Multiagent Reinforcement Learning

In this paper, we investigate the use of hierarchical reinforcement learning (HRL) to speed up the acquisition of cooperative multiagent tasks. We introduce a hierarchical multiagent reinforcement learning (RL) framework and propose a hierarchical multiagent RL algorithm called Cooperative HRL. In our approach, agents are cooperative and homogeneous (use the same task decomposition). Learning is decentralized, with each agent learning three interrelated skills: how to perform subtasks, which order to do them in, and how to coordinate with other agents. We define cooperative subtasks to be those subtasks in which coordination among agents significantly improves the performance of the overall task. Those levels of the hierarchy which include cooperative subtasks are called cooperation levels. Since coordination at high levels allows for increased cooperation skills as agents do not get confused by low-level details, we usually define cooperative subtasks at the high levels of the hierarchy.
Quantifying prior determination knowledge using the PAC learning model

Prior knowledge, or bias, regarding a concept can reduce the number of examples needed to learn it. Probably Approximately Correct (PAC) learning is a mathematical model of concept learning that can be used to quantify the reduction in the number of examples due to different forms of bias. Thus far, PAC learning has mostly been used to analyzesyntactic bias, such as limiting concepts to conjunctions of boolean prepositions. This paper demonstrates that PAC learning can also be used to analyzesemantic bias, such as a domain theory about the concept being learned. The key idea is to view the hypothesis space in PAC learning as that consistent withall prior knowledge, syntactic and semantic. In particular, the paper presents an analysis ofdeterminations, a type of relevance knowledge. The results of the analysis reveal crisp distinctions and relations among different determinations, and illustrate the usefulness of an analysis based on the PAC learning model.
Probabilistic Plan Recognition in Multiagent Systems

We present a theoretical framework for online probabilistic plan recognition in cooperative multiagent systems. Our model extends the Abstract Hidden Markov Model (AHMM) (Bui, Venkatesh, & West 2002), and consists of a hierarchical dynamic Bayes network that allows reasoning about the interaction among multiple cooperating agents. We provide an in-depth analysis of two different policy termination schemes, Tall and Tany for concurrent action introduced in (Rohanimanesh & Mahadevan 2003). In the Tall scheme, a joint policy terminates only when all agents have terminated executing their individual policies. In the Tany scheme, a joint policy terminates as soon as any of the agents terminates executing its individual policy. Since exact inference is intractable, we describe an approximate algorithm using Rao-Blackwellized particle filtering. Our approximate inference procedure reduces the complexity from exponential time in N, the number of agents and K, the number of levels, to time linear in both N and K^ ≤ K (the lowest-level of plan coordination) for the Tall termination scheme and O(N log N) and linear in K^ for the Tany termination scheme.
Rapid Concept Learning for Mobile Robots

Concept learning in robotics is an extremely challenging problem: sensory data is often high-dimensional, and noisy due to specularities and other irregularities. In this paper, we investigate two general strategies to speed up learning, based on spatial decomposition of the sensory representation, and simultaneous learning of multiple classes using a shared structure. We study two concept learning scenarios: a hallway navigation problem, where the robot has to induce features such as “opening” or “wall”. The second task is recycling, where the robot has to learn to recognize objects, such as a “trash can”. We use a common underlying function approximator in both studies in the form of a feedforward neural network, with several hundred input units and multiple output units. Despite the high degree of freedom afforded by such an approximator, we show the two strategies provide sufficient bias to achieve rapid learning. We provide detailed experimental studies on an actual mobile robot called PAVLOV to illustrate the effectiveness of this approach.
An Expert System for Assigning Patients into Clinical Trials Based on Bayesian Networks

Assigning patients into clinical trials is a knowledge and data intensive task. Eligibility determination for entry into a clinical trial is based upon specific inclusion and exclusion criteria. This paper investigates the use of an excerpt system to assist the physician through this task. This expert system uses Bayesian networks, a probabilistic method that can take advantage of pre-existing statistical knowledge. The paper also describes the feasibility of such a system by presenting the implementation of three clinical protocols. The experimental results reveal that the approach is feasible. The system gives correct eligibility scores when all evidence is available but also predicts eligibility when there is missing evidence. The system directs the physician to the protocols the patient is most eligible for, according to the current evidence. The system has the ability of learning its prior and conditional probabilities (expert knowledge) from training examples.