pwc-archive/papers-with-abstracts huggingface dataset:
Using Enhanced Gaussian Cross-Entropy in Imitation Learning to Digging the First Diamond in Minecraft
Authors: Yingjie Cai, Xiao Zhang
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: Imitation Learning, Minecraft, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Although state-ofthe-art reinforcement learning (RL) systems has led to breakthroughs in many difficult tasks, the sample inefficiency of standard reinforcement learning methods still precludes their application to more extremely complex tasks. Such limitation will make many reinforcement learning systems cannot be applied to real-world problem, in which environment samples are expensive. To solve this problem, MineRL (13) provide an ideal develop environment to facilitate the research that leveraging fewer human demonstrations with more efficient reinforcement learning systems. Based on the MineRL environmnet, we proposed an enhanced Gaussian cross entropy (EGCE) loss for imitation learnning problems to achieve ideal performance. In the ObtainDiamond task, our EGCE achieves about 7.7% improvement than a strong baseline imitation learning pipeline. The demo video is available at here.
Reinforcement Learning in 20Q Game with Generic Knowledge Bases
Authors: Chun Hei Lo, Luyang Lin
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

20Q, invented by Robin Burgener in 1988, is a computerized game of twenty questions that began as a test in artificial intelligence. The game asks the player to think of something and the system will then try to guess what they are thinking of with twenty yes-or-no questions. Due to the highly flexible and volatile game setting, designing a best questioning strategy for the system is not trivial. In the proposal, we hope to provide possible research directions, particularly under the formulation of the problem as reinforcement learning. We would also investigate on methods and potential challenges of incorporating the use of a knowledge base for the game.  The presentation can be found at https://drive.google.com/file/d/1lZ7tTvTFfspJ1P_TEpJvG5TPu0lFSVW6/view?us=sharing.
Reinforcement Learning for Predict+Optimize
Authors: Xinyi Hu, Yuansen Cheng
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Traveling Salesman Problem

Predict+Optimize (P+O) is a machine learning framework for optimization problems with unknown parameters. This paper presents a framework to tackle P+O problems using neural networks and reinforcement learning. We focus on the traveling salesman problem and train a recurrent neural network that, given a directed graph, predicts a distribution over different edges permutations. Using negative tour length as the reward signal, we optimize the parameters of the recurrent neural network using a policy gradient method.
Super Reinforcement Bros: Playing Super Mario Bros with Reinforcement Learning
Authors: Nan Zhang, Zixing Song
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We plan to apply and adjust some well-known reinforcement learning (RL) algorithms to train an automatic agent to play the 1985 Nintendo game Super Mario Bros under a speedrun rule. The agent may learn several control policies from raw pixel data by using deep reinforcement learning. By the end of the project, we expect the model to perform comparably to or top human players in a given stage. 
Code and pre-trained models are available at https://github.com/AnthonySong98/Super-Mario-Bros-PPO. 
The video recording is available at https://drive.google.com/file/d/1A9d81LUKM9h8Msw9l9Xu2PrpCXnZGgcl/view.
Optimal Portfolio Liquidation
Authors: Yuheng Cai, Taichen FAN
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Optimal Portfolio Liquidation is one of the common challenges in stock trading whose aim is to minimize a combination of risks and transactions cost arising from permanent and temporary market impact when selling a large number of stocks within a given time frame. Our attempt in this project is to turn this challenge into a reinforcement learning problem and compare the research result with a known baseline method achieved with Almgren and Chriss model. A quick introduction video is here https://youtu.be/dBjzqcKjlBQ
Deploying Reinforcement Learning in Water Transport
Authors: Donald P. H. Wong, Tsz Kui Chow
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this project, we deployed various types of reinforcement learning algorithms to resolves the rewards maximization problem in water sailing by reaching the destination with the highest priority using the smallest number of steps. We have implemented our own environment for this project and trained different agents using policy iteration, value iteration, and Deep-Q Learning (DQN). We comprehensively evaluates our approach on the environment with 8x8 and 16x16 map, our results show that the agents trained by policy iteration and value iteration can reaches the destination by giving certain number of steps, and the agent trained by DQN can finished the sailing using the smaller number of steps.
Virtual Autonomous Driving with Reinforcement Learning
Authors: Ziwen Lu, ZI QI WANG
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: Autonomous Driving, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Autonomous driving is becoming the trend for future transportation. One of its most
significant challenges is to recognize traffic signs and obey traffic rules specified
by the signs. In this paper, the particular topic of optimal vehicle speed control
whenever a vehicle reaches a speed limit sign is studied. This research is conducted
in a longitudinal environment, which only has one dimension, the straight traffic
lane, along which a vehicle will drive through. There will be multiple traffic
signs in the traffic lane, which the vehicle can recognize 150 meters in advance.
Three factors are taken into account during control optimization, that is, energy
consumption of the vehicle, jerk (change of acceleration), and most importantly,
speed of the vehicle. Methods of Q learning with the deep neural network are
implemented in the research for speed limit control. More concisely, three deep
learning methods are implemented, which are DDPG, TD3 and SAC. The last
of the three is the default method for this environment and thus will serve as the
baseline. Experiment results show that TD3 and SAC algorithms both achieved
comparably high performance within five-hour training span. Specifically, TD3
gained larger policy improvement per epoch of training while SAC achieved faster
training efficiency. On the other hand, DDPG achieved worse performance than
the other two algorithms due to its instability during training and slow training
efficiency.
Smoothing Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios
Authors: Wanli Wang
Date: 2020-12-14 00:00:00
Proceeding: CUHK Course IERG5350 2020 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

The spectrum sharing in a cognitive radio system is related with a secondary user
sharing common spectrum with a primary user for power transmit without induc-
ing harmful inference. The deep reinforcement learning has been considered as
an intelligent power control method via an agent continuously interacting with en-
vironment. Traditional deep Q-network in the frame work of deep reinforcement
learning utilizes a deep neural network for learning a nonlinear function which
maps the state or observation to accumulated rewards conditional on current state
and agent action also called Q-value. The state or observation in the radio sys-
tem is collected from wireless network and corrupted by noises. The deep neural
network may therefore yield undesirable result due to the presence of noises and
induced degraded network parameters. Considering that the kernel-based adaptive
filter is beneficial for adaptive filtering, we aim to apply the kernel-based adaptive
filter into traditional deep Q-network for smoothing network outputs. In addi-
tion, a weighting approach on the basis of past Q-values also works together with
the deep neural network for further network output smoothing. The weighting
approach is especially beneficial for alleviating the over-smoothing issue of the
kernel-based adaptive filter. Simulation results have shown the efficiency of the
proposed smoothing deep Q-network in spectrum sharing in cognitive radios in
comparison with traditional deep Q-network.
Turning Software Engineers into Machine Learning Engineers
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9
Tasks: BIG-bench Machine Learning, Deep Learning

A first challenge in teaching machine learning to software engineering and computer science students consists of changing the methodology from a constructive design-first perspective to an empirical one, focusing on proper experimental work. On the other hand, students nowadays can make significant progress using existing scripts and powerful (deep) learning frameworks -- focusing on established use cases such as vision tasks. To tackle problems in novel application domains, a clean methodological style is indispensable. Additionally, for deep learning, familiarity with gradient dynamics is crucial to understand deeper models. Consequently, we present three exercises that build upon each other to achieve these goals. These exercises are validated experimentally in a master's level course for software engineers.
An Interactive Web Application for Decision Tree Learning
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9

Decision tree learning offers an intuitive and straightforward introduction to machine learning techniques, especially when students are used to program imperative code. Most commonly, trees are trained using a greedy algorithm based on information-theoretic criteria. While there are many static resources such as slides or animations out there, interactive visualizations tend to be based on somewhat outdated UI technology and dense in information. We propose a clean and simple web application for decision tree learning that is extensible and open source.
Teaching the Foundations of Machine Learning with Candy
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9
Tasks: BIG-bench Machine Learning, Classification, Decision Making, Ethics

Machine learning is ubiquitous in decision-making processes across society. The presence and development of ML drives a need for improved education in key concepts at the secondary and tertiary levels that not only trains people to become informed citizens but also trains future researchers to be both principled and ethical practitioners. In this vein, we present a structured classroom activity that simultaneously teaches both supervised classification and critically thinking about ML applications and ethics. We use an active, object-based learning approach to teach supervised classification using a variety of candies, and a problem-based scenario to encourage critical questions about ethics in ML applications
”AI is not Just a Technology”
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9

Reporting on our experiences introducing a broad range of staff of academic libraries to AI, we suggest that training in practical applications of AI requires more than learning the technology. AI projects in libraries require contributions by computing specialists, subject specialists, metadata librarians, conservators, and others; each bringing different expertise and competencies to the project. We offer an integrated approach to knowledge building in AI that considers the processes and roles involved. This approach takes a comprehensive view of the entire AI project from understanding the need, translating that need into tasks, identifying and preparing the data, applying methods, and evaluating the results.
Introductory Machine Learning for non STEM students
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9
Tasks: BIG-bench Machine Learning, Management, Marketing

Data Science in general, and Machine Learning i nparticular, is a powerful tool for decision-makers across non-STEM fields like Human Resources Management, Law or Marketing.  Introductory Machine  Learning,  for  non-majors  that  lack  a strong background in Statistics and Computer Science, is a challenge for both teacher and students. The use of similes and games is a soft way to deal with definitions of concepts and procedures that are essential for further advanced courses on these subjects.
Teaching Computational Machine Learning (without Statistics)
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9
Tasks: BIG-bench Machine Learning

This paper presents an undergraduate machine learning course that emphasizes algorithmic understanding and programming skills while assuming no statistical training. Emphasizing the development of good habits of mind, this course trains students to be independent machine learning practitioners through an iterative, cyclical framework for teaching concepts while adding increasing depth and nuance. Beginning with unsupervised learning, this course is sequenced as a series of machine learning ideas and concepts with specific algorithms acting as concrete examples. This paper also details course organization including evaluation practices and logistics.
XploreML: An interactive approach to study and explore machine learning models
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9
Tasks: BIG-bench Machine Learning

Due to its achievements in recent years, Machine Learning (ML) is now used in a wide variety of problem domains. Educating ML has hence become an important factor to enable novel applications. 
To address this challenge, this paper introduces XploreML -- an interactive approach for lecturers to teach and for students or practitioners to study and explore the fundamentals of machine learning. XploreML allows users to experiment with data preparation, data transformation and a wide range of classifiers. The data sets can be visually investigated in order to understand the complexity of the classification problem. The selected classifier can either be autonomously fitted to the training data or the effect of manually altering model hyperparameters can be explored. Additionally source code of configured ML pipelines can be extracted. 
XploreML can be used within a lecture as an interactive demo or by students in a lab session. Both scenarios were evaluated with a user survey, where both variants were assessed as positive with the first yielding more positive feedback.
XploreML can be used online: ml-and-vis.shinyapps.io/XploreML
Artificial Neural Network Code Tutorial
Authors: Anonymous
Date: 2020-07-08 00:00:00
Proceeding: ECMLPKDD Workshop TeachML 2020 9

A small tutorial session featuring a live Ai session at
UWI MONA for ITS 2019, where
a basic Artificial Neural Network will be written from
scratch & discussed, led by Jordan Micah Bennett
Knowledge Distillation from Few Samples
Authors: Tianhong Li, Jianguo Li, Zhuang Liu, ChangShui Zhang
Date: 2018-09-27 00:00:00
Tasks: Knowledge Distillation

Current knowledge distillation methods require full training data to distill knowledge from a large "teacher" network to a compact "student" network by matching certain statistics between "teacher" and "student" such as softmax outputs and feature responses. This is not only time-consuming but also inconsistent with human cognition in which children can learn knowledge from adults with few examples. This paper proposes a novel and simple method for knowledge distillation from few samples. Taking the assumption that both "teacher" and "student" have the same  feature map sizes at each corresponding block, we add a $1\times 1$ conv-layer at the end of each block in the student-net, and align the block-level outputs between "teacher" and "student" by estimating the parameters of the added layer with limited samples. We prove that the added layer can be absorbed/merged into the previous conv-layer \hl{to formulate a new conv-layer with the same size of parameters and computation cost as previous one. Experiments verifies that the proposed method is very efficient and effective to distill knowledge from teacher-net to student-net constructing in different ways on various datasets.
Unsupervised Disentangling Structure and Appearance
Authors: Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy
Date: 2018-09-27 00:00:00
Tasks: Disentanglement

It is challenging to disentangle an object into two orthogonal spaces of structure and appearance since each can influence the visual observation in a different and unpredictable way. It is rare for one to have access to a large number of data to help separate the influences. In this paper, we present a novel framework to learn this disentangled representation in a completely unsupervised manner. We address this problem in a two-branch Variational Autoencoder framework. For the structure branch, we project the latent factor into a soft structured point tensor and constrain it with losses derived from prior knowledge. This encourages the branch to distill geometry information. Another branch learns the complementary appearance information. The two branches form an effective framework that can disentangle object's structure-appearance representation without any human annotation. We evaluate our approach on four image datasets, on which we demonstrate the superior disentanglement and visual analogy quality both in synthesis and real-world data. We are able to generate photo-realistic images with 256*256 resolution that are clearly disentangled in structure and appearance.
(Unconstrained) Beam Search is Sensitive to Large Search Discrepancies
Authors: Eldan Cohen, J. Christopher Beck
Date: 2018-09-27 00:00:00

Beam search is the most popular inference algorithm for decoding neural sequence models. Unlike greedy search, beam search allows for a non-greedy local decisions that can potentially lead to a sequence with a higher overall probability. However, previous work found that the performance of beam search tends to degrade with large beam widths. In this work, we perform an empirical study of the behavior of the beam search algorithm across three sequence synthesis tasks. We find that increasing the beam width leads to sequences that are disproportionately based on early and highly non-greedy decisions. These sequences typically include a very low probability token that is followed by a sequence of tokens with higher (conditional) probability leading to an overall higher probability sequence. However, as beam width increases, such sequences are more likely to have a lower evaluation score. Based on our empirical analysis we propose to constrain the beam search from taking highly non-greedy decisions early in the search. We evaluate two methods to constrain the search and show that constrained beam search effectively eliminates the problem of beam search degradation and in some cases even leads to higher evaluation scores. Our results generalize and improve upon previous observations on copies and training set predictions.
Zero-Resource Multilingual Model Transfer: Learning What to Share
Authors: Xilun Chen, Ahmed Hassan Awadallah, Hany Hassan, Wei Wang, Claire Cardie
Date: 2018-09-27 00:00:00
Tasks: Cross-Lingual Transfer, Mixture-of-Experts, text-classification, Text Classification, Transfer Learning

Modern natural language processing and understanding applications have enjoyed a great boost utilizing neural networks models. However, this is not the case for most languages especially low-resource ones with insufficient annotated training data. Cross-lingual transfer learning methods improve the performance on a low-resource target language by leveraging labeled data from other (source) languages, typically with the help of cross-lingual resources such as parallel corpora. In this work, we propose a zero-resource multilingual transfer learning model that can utilize training data in multiple source languages, while not requiring target language training data nor cross-lingual supervision. Unlike most existing methods that only rely on language-invariant features for cross-lingual transfer, our approach utilizes both language-invariant and language-specific features in a coherent way. Our model leverages adversarial networks to learn language-invariant features and mixture-of-experts models to dynamically exploit the relation between the target language and each individual source language. This enables our model to learn effectively what to share between various languages in the multilingual setup. It results in significant performance gains over prior art, as shown in an extensive set of experiments over multiple text classification and sequence tagging tasks including a large-scale real-world industry dataset.
VECTORIZATION METHODS IN RECOMMENDER SYSTEM
Authors: Qiang Sun, Bin Wang, Zizhou Gu, Yanwei Fu
Date: 2018-09-27 00:00:00
Tasks: Collaborative Filtering, Position, Recommendation Systems

The most used recommendation method is collaborative filtering, and the key part of collaborative filtering is to compute the similarity. The similarity based on co-occurrence of similar event is easy to implement and can be applied to almost all the situation. So when the word2vec model reach the state-of-art at a lower computation cost in NLP. An correspond model in recommender system item2vec is proposed and reach state-of-art in recommender system. It is easy to see that the position of user and item is interchangeable when their count size gap is not too much, we proposed a user2vec model and show its performance. The similarity based on co-occurrence information suffers from cold start, we proposed a content based similarity model based on doc2vec which is another technology in NLP.
Novel positional encodings to enable tree-structured transformers
Authors: Vighnesh Leonardo Shiv, Chris Quirk
Date: 2018-09-27 00:00:00
Tasks: Program Synthesis, Semantic Parsing, Translation

With interest in program synthesis and similarly ﬂavored problems rapidly increasing, neural models optimized for tree-domain problems are of great value. In the sequence domain, transformers can learn relationships across arbitrary pairs of positions with less bias than recurrent models. Under the intuition that a similar property would be beneficial in the tree domain, we propose a method to extend transformers to tree-structured inputs and/or outputs. Our approach abstracts transformer's default sinusoidal positional encodings, allowing us to substitute in a novel custom positional encoding scheme that represents node positions within a tree. We evaluated our model in tree-to-tree program translation and sequence-to-tree semantic parsing settings, achieving superior performance over the vanilla transformer model on several tasks.