In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
--------
Question: Deep reinforcement learning (RL) can acquire complex behaviors from low-level inputs, such as images. However, real-world applications of such methods require generalizing to the vast variability of the real world. Deep networks are known to achieve remarkable generalization when provided with massive amounts of labeled data, but can we provide this breadth of experience to an RL agent, such as a robot? The robot might continuously learn as it explores the world around it, even while it is deployed and performing useful tasks. However, this learning requires access to a reward function, to tell the agent whether it is succeeding or failing at its task. Such reward functions are often hard to measure in the real world, especially in domains such as robotics and dialog systems, where the reward could depend on the unknown positions of objects or the emotional state of the user. On the other hand, it is often quite practical to provide the agent with reward functions in a limited set of situations, such as when a human supervisor is present, or in a controlled laboratory setting. Can we make use of this limited supervision, and still benefit from the breadth of experience an agent might collect in the unstructured real world? In this paper, we formalize this problem setting as semisupervised reinforcement learning (SSRL), where the reward function can only be evaluated in a set of “labeled” MDPs, and the agent must generalize its behavior to the wide range of states it might encounter in a set of “unlabeled” MDPs, by using experience from both settings. Our proposed method infers the task objective in the unlabeled MDPs through an algorithm that resembles inverse RL, using the agent’s own prior experience in the labeled MDPs as a kind of demonstration of optimal behavior. We evaluate our method on challenging, continuous control tasks that require control directly from images, and show that our approach can improve the generalization of a learned deep neural network policy by using experience for which no reward function is available. We also show that our method outperforms direct supervised learning of the reward.

Answer: GENERALIZING SKILLS WITH SEMI-SUPERVISED REINFORCEMENT LEARNING


Question: Abstract—Spectral clustering is a powerful tool for unsupervised data analysis. In this paper, we propose a contextaware hypergraph similarity measure (CAHSM), which leads to robust spectral clustering in the case of noisy data. We construct three types of hypergraph—the pairwise hypergraph, the k-nearest-neighbor (kNN) hypergraph, and the high-order over-clustering hypergraph. The pairwise hypergraph captures the pairwise similarity of data points; the kNN hypergraph captures the neighborhood of each point; and the clustering hypergraph encodes high-order contexts within the dataset. By combining the affinity information from these three hypergraphs, the CAHSM algorithm is able to explore the intrinsic topological information of the dataset. Therefore, data clustering using CAHSM tends to be more robust. Considering the intracluster compactness and the inter-cluster separability of vertices, we further design a discriminative hypergraph partitioning criterion (DHPC). Using both CAHSM and DHPC, a robust spectral clustering algorithm is developed. Theoretical analysis and experimental evaluation demonstrate the effectiveness and robustness of the proposed algorithm.

Answer: Context-Aware Hypergraph Construction for Robust Spectral Clustering


Question: In chemometrics, data from infrared or near-infrared (NIR) spectroscopy are often used to identify a compound or to analyze the composition of a material. This involves the calibration of models that predict the concentration of material constituents from the measured NIR spectrum. An interesting aspect of multivariate calibration is to achieve a particular accuracy level with a minimum number of training samples, as this reduces the number of laboratory tests and thus the cost of model building. In these chemometric models, the input refers to a proper representation of the spectra and the output to the concentrations of the sample constituents. The search for a most informative new calibration sample thus has to be performed in the output space of the model, rather than in the input space as in conventional modeling problems. In this paper, we propose to solve the corresponding inversion problem by utilizing the disagreements of an ensemble of neural networks to represent the prediction error in the unexplored component space. The next calibration sample is then chosen at a composition where the individual models of the ensemble disagree most. The results obtained for a realistic chemometric calibration example show that the proposed active learning can achieve a given calibration accuracy with less training samples than random sampling.

Answer:
Neural Network-Based Active Learning in Multivariate Calibration