nils-herrmann/clean_scirepeval huggingface dataset:
A Dozen Tricks with Multitask Learning

Multitask Learning is an inductive transfer method that improves generalization accuracy on a main task by using the information contained in the training signals of other related tasks. It does this by learning the extra tasks in parallel with the main task while using a shared representation; what is learned for each task can help other tasks be learned better. This chapter describes a dozen opportunities for applying multitask learning in real problems. At the end of the chapter we also make several suggestions for how to get the most our of multitask learning on real-world problems.
Using the Future to Sort Out the Present: Rankprop and Multitask Learning for Medical Risk Evaluation

A patient visits the doctor; the doctor reviews the patient's history, asks questions, makes basic measurements (blood pressure, ...), and prescribes tests or treatment. The prescribed course of action is based on an assessment of patient risk--patients at higher risk are given more and faster attention. It is also sequential--it is too expensive to immediately order all tests which might later be of value. This paper presents two methods that together improve the accuracy of backprop nets on a pneumonia risk assessment problem by 10-50%. Rankprop improves on backpropagation with sum of squares error in ranking patients by risk. Multitask learning takes advantage of future lab tests available in the training set, but not available in practice when predictions must be made. Both methods are broadly applicable.
Identifying Temporal Patterns and Key Players in Document Collections

This paper considers the problem of analyzing the development of a document collection over time without requiring meaningful citation data. Given a collection of timestamped documents, we formulate and explore the following two questions. First, what are the main topics and how do these topics develop over time? Second, to gain insight into the dynamics driving this development, what are the documents and who are the authors that are most influential in this process? Unlike prior work in citation analysis, we propose methods addressing these questions without requiring the availability of citation data. The methods use only the text of the documents as input. Consequentially, they are applicable to a much wider range of document collections (email, blogs, etc.), most of which lack meaningful citation data. We evaluate our methods on the proceedings of the Neural Information Processing Systems (NIPS) conference. Even with the preliminary methods that we implemented, the results show that the methods are effective and that addressing the questions based on the text alone is feasible. In fact, the text-based methods sometimes even identify influential papers that are missed by citation analysis.
Autonomous Detection and Control of Human Tools for Robot Manipulation

Robots that can manipulate everyday objects in unstructured, human settings could more easily work with people and perform tasks that are important to people. Ideally, a robot would be able to detect and control task-relevant visual features of an object it has not previously encountered, since the set of objects found within human environments is large and diverse. For a significant set of tasks and tool-like objects, detection and control of the distal end of the object is sufficient for its use. The tips of these objects, such as the end of a screwdriver or the mouth of a bottle, have an approximately convex projection into the visual scene. In this paper we address the problem of visually detecting and controlling the tip of an unknown tool-like object that is rigidly grasped by a robot. We present a multi-scale motionbased feature detector that visually detects a tool tip and we show results in using a series of these 2D detections to produce a 3D estimate of the tool tip’s position in the hand’s frame. We also describe and evaluate a method for controlling the tool tip’s position and orientation in the image.
Visual Tool Tip Detection and Position Estimation for Robotic Manipulation of Unknown Human Tools

Robots that use human tools could more easily work with people, perform tasks that are important to people, and bene t from human strategies for accomplishing these tasks. For a wide variety of tools and tasks, control of the tool's endpoint is su cient for its use. In this paper we present a straight-forward method for rapidly detecting the endpoint of an unmodeled tool and estimating its position with respect to the robot's hand. The robot rotates the tool while using optical ow to detect the most rapidly moving image points, and then nds the 3D position with respect to its hand that best explains these noisy 2D detections. The resulting 3D position estimate allows the robot to control the position of the tool endpoint and predict its visual location. We show successful results for this method using a humanoid robot with a variety of traditional tools, including a pen, a hammer, and pliers, as well as more general tools such as a bottle and the robot's own
A wearable system that learns a kinematic model and finds structure in everyday manipulation by using absolute orientation sensors and a camera

This thesis presents Duo, the first wearable system to autonomously learn a kinematic, model of the wearer via body-mounted absolute orientation sensors and a head-mounted camera. With Duo, we demonstrate the significant benefits of endowing a wearable system with the ability to sense the kinematic configuration of the wearer's body. We also show that a kinematic model can be autonomously estimated offline from less than an hour of recorded video and orientation data from a wearer performing unconstrained, unscripted, household activities within a real, unaltered, home environment. We demonstrate that our system for autonomously estimating this kinematic model places very few constraints on the wearer's body, the placement of the sensors, and the appearance of the hand, which, for example, allows it to automatically discover a left-handed kinematic model for a left-handed wearer, and to automatically compensate for distinct camera mounts, and sensor configurations. Furthermore, we show that this learned kinematic model efficiently and robustly predicts the location of the dominant hand within video from the head-mounted camera even in situations where vision-based hand detectors would be likely to fail. Additionally, we show ways in which the learned kinematic model can facilitate highly efficient processing of large databases of first person experience. Finally, we show that the kinematic model can efficiently direct visual processing so as to acquire a large number of high quality segments of the wearer's hand and the manipulated objects. 
Within the course of justifying these claims, we present methods for estimating global image motion, segmenting foreground motion, segmenting manipulation events, finding and representing significant hand postures, segmenting visual regions, and detecting visual points of interest with associated shape descriptors. We also describe our architecture and user-level application for machine augmented annotation and browsing of first person video and absolute orientations. Additionally, we present a real-time application in which the human and wearable cooperate through tightly integrated behaviors coordinated by the wearable's kinematic perception, and together acquire high-quality visual segments of manipulable objects that interest the wearable. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.
Duo: a wearable system that learns about everyday objects and actions

Duo is a wearable system designed to learn about everyday actions and the objects to which they are applied. Duo uses first person video and kinematic sensing of the head, torso, and dominant arm. Duo fundamentally relies on general methods for segmenting both kinematic and visual data into meaningful units. We describe methods for segmenting the wearer's arm, the objects with which the wearer interacts, and the actions the wearer applies to these objects. Both real-time interactive methods and offline annotation tools are used to help Duo create meaningful segmentations from which to learn.
Duo: A Human/Wearable Hybrid for Learning About Everyday Human Activities

We are investigatingwearable computing as away formachines to learn about everyday human activities from a first-person perspective  . Our wearable system, Duo, uses body-mounted orientation sensors and a head-mounted camera to sense the kinematic configuration of the wearer’s body while watching the world from the wearer’s point of view. Duo both passively monitors the wearer’s behavior and actively requests help from the wearer in order to better learn from the wearer’s activities.
The Whole World in Your Hand: Active and Interactive Segmentation

Object segmentation is a fundamental problem
in computer vision and a powerful resource for
development. This paper presents three embodied approaches to the visual segmentation of objects. Each approach to segmentation is aided
by the presence of a hand or arm in the proximity of the object to be segmented. The first
approach is suitable for a robotic system, where
the robot can use its arm to evoke object motion. The second method operates on a wearable system, viewing the world from a human's
perspective, with instrumentation to help detect
and segment objects that are held in the wearer's
hand. The third method operates when observing
a human teacher, locating periodic motion (finger/arm/object waving or tapping) and using it
as a seed for segmentation. We show that object segmentation can serve as a key resource for
development by demonstrating methods that exploit high-quality object segmentations to develop
both low-level vision capabilities (specialized feature detectors) and high-level vision capabilities
(object recognition and localization).
Shoes as a platform for vision

We explore the use of a shoe-mounted camera as a sensorysystem for wearable computing. We demonstrate toolsuseful for gait analysis, obstacle detection, and contextrecognition. Using only visual information, we detect periodsof stability and motion during walking. In the stablephase, the foot can be assumed to be parallel to the groundplane. In this condition, the floor dominates the lower partof the camera's view, and we show that it can be segmentedout from the remainder of the scene, leaving walls and obstacles.We also demonstrate floor surface recognition forcontext awareness.
Duo: A Human/Wearable Hybrid for Learning About Common Manipulate Objects

Humanoid robots would benefit from a better understanding of common manipulable objects and the human behaviors associated with them. Duo is a human/wearable hybrid that is designed to learn about this important domain of human intelligence by interacting with natural manipulable objects in unconstrained environments. Duo's wearable AI system measures the kinematic configuration of the human's head, torso and dominate arm, while watching the workspace of the human's hand through a head-mounted camera. Duo also requests helpful actions from the human through speech via headphones. This paper presents results on an initial set of behaviors for Duo which lead to high-quality segmentations of common manipulable objects in unconstrained human environments. In Duo, the wearable AI system essentially subsumes the abilities of its cooperative human partner by sharing the human's sensor input and directing a portion of the human's actions. Together, the cooperative human and the wearable AI system can be thought of as constituting a new kind of humanoid robot that complements more traditional, wholly synthetic humanoid robots by allowing researchers to circumvent some of the currently unsolved problems in the field, from dextrous object manipulation to unrestricted mobility.
Think Like a Rat

Since 1971 when the results of single electrode recordings of neurons with “place fields” were first documented, scientists have produced an enormous body of research in an attempt to understand these small but significant components of the rat brain. This research has confirmed that after a rat has gained familiarity with a simple test environment, a neuron will respond strongly and selectively to the rat’s presence within a small part of that environment. But despite 30 years of research, remarkably little is known about the computations that produce and use the output of these “place cells”.
Alternative Essences of Intelligence

We present a novel methodology for building humanlike artificially intelligent systems. We take as a model the only existing systems which are universally accepted as intelligent: humans. We emphasize building intelligent systems which are not masters of a single domain, but, like humans, are adept at performing a variety of complex tasks in the real world. Using evidence from cognitive science and neuroscience, we suggest four alternative essences of intelligence to those held by classical AI. These are the parallel themes of development, social interaction, embodiment, and integration. Following a methodology based on these themes, we have built a physical humanoid robot. In this paper we present our methodology and the insights it affords for facilitating learning, simplifying the computation underlying rich behavior, and building systems that can scale to more complex tasks in more challenging environments.
An Ontological Perspective to Scaling Sensorimotor Intelligence

Much of the work in autonomous agent design has been heavily inspired and influenced by biological systems. Researchers strive to build artificial systems that emulate the robust, flexible, and adaptive behavior of animals. In particular, the behavior-based approach (Brooks 1991) has adopted many of its key models, metaphors, and concepts from the fields of ethology, neuroscience, and evolutionary theory to name a few. Looking to animals to glean insight into how to build intelligent autonomous systems, the field has brought the issues of embodiment, situatedness, emergence, task-based decomposition, and environmental complexity to the foreground (Brooks & Steels 1995). Indeed, the treatment of these issues characterizes various biologically-inspired approaches, several of which have been successfully applied to the control of autonomous robots--behavior based systems (Brooks 1986), evolution-based systems (Cliff et al. 1993), and connectionist systems (Beer 1990), for instance. date, the behavior of these systems mirrors that of simpler organisms, such as insects, situated in relatively complex environments. Wishing to move beyond the behavior of insects and other simple animals, researchers question the scalability of these approaches and heavily debate how far systems using these approaches can be pushed. Often the question of scaling is posed in the context of increasing either environmental complexity or task complexity. Implicit in this question is that of cognitive scaling, i.e., can the intelligence of these physically embodied systems approach that of higher-level animals such as dogs, dolphins, primates, and humans? If this is the case, what roles could embodiment, situatedness, emergence, environmental complexity, or having a task-based view play in scaling cognitive and behavioral capabilities of artificial and natural systems? In contrast to many autonomous agent programs that are shaped by an evolutionary or ethological perspective, we propose a research program that pursues an ontogenetic path. We argue that the field of developmental psychology gives fresh insight into the issues of cognitive and behavioral scaling. It offers a viable alternate viewpoint from which to study embodiment, situatedness, emergence, environmental complexity, and goal-oriented action in the world, illuminating the critical roles they may play in bootstrapping a child to more sophisticated levels of mental competence. Our contention is that a developmental perspective to autonomous agent design profoundly affects the implemented control architecture, mechanisms, subsystems, and internal structures in ways which distinguish an ontogenetic system from other learning-based systems.
An Ontogenetic Perspective to Scaling Sensorimotor Intelligence

Much of the work in autonomous agent design has been heavily inspired and in uenced by biological systems. Researchers strive to build arti cial systems that emulate the robust, exible, and adaptive behavior of animals. In particular, the behavior-based approach (Brooks 1991) has adopted many of its key models, metaphors, and concepts from the elds of ethology, neuroscience, and evolutionary theory to name a few. Looking to animals to glean insight into how to build intelligent autonomous systems, the eld has brought the issues of embodiment, situatedness, emergence, task-based decomposition, and environmental complexity to the foreground (Brooks & Steels 1995). Indeed, the treatment of these issues characterizes various biologically-inspired approaches, several of which have been successfully applied to the control of autonomous robots|behavior based systems (Brooks 1986), evolution-based systems (Cli et al. 1993), and connectionist systems (Beer 1990), for instance. To date, the behavior of these systems mirrors that of simpler organisms, such as insects, situated in relatively complex environments. Wishing to move beyond the behavior of insects and other simple animals, researchers question the scalability of these approaches and heavily debate how far systems using these approaches can be pushed. Often the question of scaling is posed in the context of increasing either environmental complexity or task complexity. Implicit in this question is that of cognitive scaling, i.e., can the intelligence of these physically embodied systems approach that of higher-level animals such as dogs, dolphins, primates, and humans? If this is the case, what roles could embodiment, situatedness, emergence, environmental complexity, or having a task-based view play in scaling cognitive and behavioral capabilities of arti cial and natural systems? In contrast to many autonomous agent programs that are shaped by an evolutionary or ethological perspective, we propose a research program that pursues an ontogenetic path. We argue that the eld of developmental psychology gives fresh insight into the issues of cognitive and behavioral scaling. It o ers a viable alternate viewpoint from which to study embodiment, situatedness, emergence, environmental complexity, and goal-oriented action in the world, illuminating the critical roles they may play in bootstrapping a child to more sophisticated levels of mental competence. Our contention is that a developmental perspective to autonomous agent design profoundly a ects the implemented control architecture, mechanisms, subsystems, and internal structures in ways which distinguish an ontogenetic system from other learning-based systems.
Exploration and apprenticeship learning in reinforcement learning

We consider reinforcement learning in systems with unknown dynamics. Algorithms such as E3 (Kearns and Singh, 2002) learn near-optimal policies by using "exploration policies" to drive the system towards poorly modeled states, so as to encourage exploration. But this makes these algorithms impractical for many systems; for example, on an autonomous helicopter, overly aggressive exploration may well result in a crash. In this paper, we consider the apprenticeship learning setting in which a teacher demonstration of the task is available. We show that, given the initial demonstration, no explicit exploration is necessary, and we can attain near-optimal performance (compared to the teacher) simply by repeatedly executing "exploitation policies" that try to maximize rewards. In finite-state MDPs, our algorithm scales polynomially in the number of states; in continuous-state linear dynamical systems, it scales polynomially in the dimension of the state. These results are proved using a martingale construction over relative losses.
Discriminative Training of Kalman Filters

Kalman filters are a workhorse of robotics and are routinely used in state-estimation problems. However, their performance critically depends on a large number of modeling parameters which can be very difficult to obtain, and are often set via significant manual tweaking and at a great cost of engineering time. In this paper, we propose a method for automatically learning the noise parameters of a Kalman filter. We also demonstrate on a commercial wheeled rover that our Kalman filter’s learned noise covariance parameters—obtained quickly and fully automatically—significantly outperform an earlier, carefully and laboriously hand-designed one.
Learning vehicular dynamics, with application to modeling helicopters

We consider the problem of modeling a helicopter's dynamics based on state-action trajectories collected from it. The contribution of this paper is two-fold. First, we consider the linear models such as learned by CIFER (the industry standard in helicopter identification), and show that the linear parameterization makes certain properties of dynamical systems, such as inertia, fundamentally difficult to capture. We propose an alternative, acceleration based, parameterization that does not suffer from this deficiency, and that can be learned as efficiently from data. Second, a Markov decision process model of a helicopter's dynamics would explicitly model only the one-step transitions, but we are often interested in a model's predictive performance over longer timescales. In this paper, we present an efficient algorithm for (approximately) minimizing the prediction error over long time scales. We present empirical results on two different helicopters. Although this work was motivated by the problem of modeling helicopters, the ideas presented here are general, and can be applied to modeling large classes of vehicular dynamics.
Learning first-order Markov models for control

First-order Markov models have been successfully applied to many problems, for example in modeling sequential data using Markov chains, and modeling control problems using the Markov decision processes (MDP) formalism. If a first-order Markov model's parameters are estimated from data, the standard maximum likelihood estimator considers only the first-order (single-step) transitions. But for many problems, the first-order conditional independence assumptions are not satisfied, and as a result the higher order transition probabilities may be poorly approximated. Motivated by the problem of learning an MDP's parameters for control, we propose an algorithm for learning a first-order Markov model that explicitly takes into account higher order interactions during training. Our algorithm uses an optimization criterion different from maximum likelihood, and allows us to learn models that capture longer range effects, but without giving up the benefits of using first-order Markov models. Our experimental results also show the new algorithm outperforming conventional maximum likelihood estimation in a number of control problems where the MDP's parameters are estimated from data.