pwc-archive/papers-with-abstracts huggingface dataset:
Towards Supervised Learning of Optimal Replenishment Policies
Authors: Anonymous
Date: 2021-11-24 00:00:00
Proceeding: AAAI Workshop ML4OR-22 2022 2

We propose a supervised learning algorithm for the multi-period inventory problem (MPIP) that tackles  shortcomings of existing multi-step, model-based methods on the one and policy-free reinforcement learning algorithms on the other hand. As a model-free end-to-end (E2E) method that takes advantage of auxiliary data, it avoids pitfalls like model misspecification, multi-step error accumulation and computational complexity induced by a repeated optimization step. Furthermore, it manages to leverage domain knowledge about the optimal solution structure. To the best of our knowledge, this is one of the first supervised learning approaches to solve the MPIP and the first one to learn policy parameters. Given the variety of settings in which OR researchers have developed well-performing policies, our approach can serve as a blueprint of how to design E2E methods that leverage that knowledge. We validate our hypotheses on synthetic data and demonstrate the effect of individual model characteristics.
Reversible Action Design for Combinatorial Optimization with ReinforcementLearning
Authors: Anonymous
Date: 2021-11-24 00:00:00
Proceeding: AAAI Workshop ML4OR-22 2022 2
Tasks: Combinatorial Optimization, Q-Learning, Reinforcement Learning (RL), Traveling Salesman Problem

Combinatorial optimization problem (COP) over graphs is a fundamental challenge in optimization. Reinforcement learning (RL) has recently emerged as a new framework to tackle these problems and has demonstrated promising results. However, most RL solutions employ a greedy manner to construct the solution incrementally, thus inevitably pose unnecessary dependency on action sequences and need a lot of problem-specific designs. We propose a general RL framework that not only exhibits state-of-the-art empirical performance but also generalizes to a variety class of COPs. Specifically, we define state as a solution to a problem instance and action as a perturbation to this solution. We utilize graph neural networks (GNN) to extract latent representations for given problem instances for state-action encoding, and then apply deep Q-learning to obtain a policy that gradually refines the solution by flipping or swapping vertex labels. Experiments are conducted on Maximum $k$-Cut and Traveling Salesman Problem and performance improvement is achieved against a set of learning-based and heuristic baselines.
A Review for Deep Reinforcement Learning in Atari: Benchmarks, Challenges, and Solutions
Authors: Anonymous
Date: 2021-11-24 00:00:00
Proceeding: AAAI Workshop ML4OR-22 2022 2
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

The Arcade Learning Environment (ALE) is proposed as an evaluation platform for empirically assessing the generality of agents across dozens of Atari 2600 games. ALE offers various challenging problems and has drawn significant attention from the deep reinforcement learning (RL) community. From Deep Q-Networks (DQN) to Agent57, RL agents seem to achieve superhuman performance in ALE. However, is this the case? In this paper, to explore this problem, we first review the current evaluation metrics in the Atari benchmarks and then reveal that the current evaluation criteria of achieving superhuman performance are inappropriate, which underestimated the human performance relative to what is possible. To handle those problems and promote the development of RL research, we propose a novel Atari benchmark based on human world records (HWR), which puts forward higher requirements for RL agents on both final performance and learning efficiency.
Furthermore, we summarize the state-of-the-art (SOTA) methods in Atari benchmarks and provide benchmark results over new evaluation metrics based on human world records. We concluded that at least four open challenges hinder RL agents from achieving superhuman performance from those new benchmark results. Finally, we also discuss some promising ways to handle those problems.
GDI: Rethinking What Makes Reinforcement Learning Different from Supervised Learning
Authors: Anonymous
Date: 2021-11-24 00:00:00
Proceeding: AAAI Workshop ML4OR-22 2022 2
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep Q Network (DQN) firstly kicked the door of deep reinforcement learning (DRL) via combining deep learning (DL) with reinforcement learning (RL), which has noticed that the distribution of the acquired data would change during the training process. DQN found this property might cause instability for training, so it proposed effective methods to handle the downside of the property. Instead of focusing on the unfavorable aspects, we find it critical for RL to ease the gap between the estimated data distribution and the ground truth data distribution while supervised learning (SL) fails to do so. From this new perspective, we extend the basic paradigm of RL called the Generalized Policy Iteration (GPI) into a more generalized version, which is called the Generalized Data Distribution Iteration (GDI). We see massive RL algorithms and techniques can be unified into the GDI paradigm, which can be considered as one of the special cases of GDI. We provide theoretical proof of why GDI is better than GPI and how it works. Several practical algorithms based on GDI have been proposed to verify its effectiveness and extensiveness. Empirical experiments prove our state-of the-art (SOTA) performance on Arcade Learning Environment (ALE), wherein our algorithm has achieved 9620.98% mean human normalized score (HNS), 1146.39% median HNS and 22 human world record breakthroughs (HWRB) using only 200M training frames. Our work aims to lead the RL research to step into the journey of conquering the human world records and seek real superhuman agents on both performance and efficiency.
Influence of Movement Energy and Affect Priming on the Perception of Virtual Characters Extroversion and Mood
Authors: Tanja Schneeberger, Fatima Ayman Aly, Daksitha Withanage Don, Katharina Gies, Zita Zeimer, Fabrizio Nunnari, Patrick Gebhard
Date: 2021-06-23 00:00:00
Proceeding: ACM ICMI Workshop GENEA 2021 10

Movement Energy – physical activeness in performing actions and Affect Priming – prior exposure to information about someone’s mood and personality might be two crucial factors that influence how we perceive someone. It is unclear if these factors influence the perception of virtual characters in a way that is similar to what is observed during in-person interactions. This paper presents different configurations of Movement Energy for virtual characters and two studies about how these influence the perception of the characters’ personality, extroversion in particular, and mood. Moreover, the studies investigate how Affect Priming (Personality and Mood), as one form of contextual priming, influences this perception. The results indicate that characters with high Movement Energy are perceived as more extrovert and in a better mood, which corroborates existing research. Moreover, the results indicate that Personality and Mood Priming influence perception in different ways. Characters that were primed as being in a positive mood are perceived as more extrovert, whereas characters that were primed as being introverted are perceived in a more positive mood.
Crossmodal clustered contrastive learning: Grounding of spoken language to gesture
Authors: Dong Won Lee, Chaitanya Ahuja, Louis-Philippe Morency
Date: 2021-06-23 00:00:00
Proceeding: ACM ICMI Workshop GENEA 2021 10
Tasks: Contrastive Learning

Crossmodal grounding is a key challenge for the task of generating relevant and well-timed gestures from just spoken language as an input. Often, the same gesture can be accompanied by semantically different spoken language phrases which makes crossmodal grounding especially challenging. For example, a deictic gesture of spanning a region could co-occur with semantically different phrases "entire bottom row" (referring to a physical point) and "molecules expand and decay" (referring to a scientific phenomena). In this paper, we introduce a self-supervised approach to learn such many-to-one grounding relationships between spoken language and gestures. As part of this approach, we propose a new contrastive loss function, Crossmodal Cluster NCE , that guides the model to learn spoken language representations which are consistent with the similarities in the gesture space. By doing so, we impose a greater level of grounding between spoken language and gestures in the model. We demonstrate the effectiveness of our approach on a publicly available dataset through quantitative and qualitative studies. Our proposed methodology significantly outperforms prior approaches for grounding gestures to language. Link to code: https://github.com/dondongwon/CC_NCE_GENEA.
Probabilistic Human-like Gesture Synthesis from Speech using GRU-based WGAN
Authors: Bowen Wu, Chaoran Liu, Carlos Ishi, Hiroshi Ishiguro
Date: 2021-07-05 00:00:00
Proceeding: ACM ICMI Workshop GENEA 2021 10
Tasks: Gesture Generation

Gestures are crucial for increasing the human-likeness of agents and robots to achieve smoother interactions with humans. The realization of an effective system to model human gestures, which are matched with the speech utterances, is necessary to be embedded in these agents. In this work, we propose a GRU-based autoregressive generation model for gesture generation, which is trained with a CNN-based discriminator in an adversarial manner using a WGAN-based learning algorithm. The model is trained to output the rotation angles of the joints in the upper body, and implemented to animate a CG avatar. The motions synthesized by the proposed system are evaluated via an objective measure and a subjective experiment, showing that the proposed model outperforms a baseline model which is trained by a state-of-the-art GAN-based algorithm, using the same dataset. This result reveals that it is essential to develop a stable and robust learning algorithm for training gesture generation models. Our code can be found in https://github.com/wubowen416/gesture-generation.
Towards data-driven sign language interpreting virtual assistant
Authors: Anonymous
Date: 2021-06-23 00:00:00
Proceeding: ACM ICMI Workshop GENEA 2021 10
Tasks: Sign Language Recognition

Sign Languages (SL) are a form of communication in the visual-gestural modality, and are full-fledged natural languages. Recent years have seen the increase in the use of virtual agents as assistants for the sign language users. Research into sign language recognition has demonstrated promising potential for the improvement of the communication with the deaf people. However, the area of sign language synthesis is still in its infancy. This explains the underdevelopment of the virtual intelligent signing systems, which could bridge the communication with the deaf and make it more favorable. In addition, the existing models are often restricted to the manually written rules and require expert knowledge, while data-driven approach could provide a better solution. 

In this paper, we present a user study on the evaluation of the data-driven Virtual Assistant that performs signing gestures for the Kazakh-Russian sign language using sign sequences. The study sets out to answer three research questions concerning the users' perceptions and feedback on the performance of the four signing agents, namely two data-driven avatars, one motion capture animation avatar and a human sign interpreter. The results of the questionnaire suggest that while signing avatars generally perform well, they could not outperform the human agent in terms of naturalness and likeability. Hence, a further study might include the improvements necessary to increase the naturalness of the signing gestures.
Interpreting and Generating Gestures with Embodied Human Computer Interactions
Authors: Anonymous
Date: 2020-09-18 00:00:00
Proceeding: ACM IVA Workshop GENEA 2020 10
Tasks: Gesture Generation

In this paper, we discuss the role that gesture plays for an embodied intelligent virtual agent (IVA)  in the context of multimodal task-oriented dialogues with a human. We have developed a 
 simulation platform, VoxWorld, for modeling and building {\it Embodied Human-Computer Interactions (EHCI)}, where communication is facilitated through language, gesture, action, facial expressions, and gaze tracking. We believe that EHCI is a fruitful approach for studying and enabling robust interaction and communication between humans and intelligent agents and robots. 
Gesture, language, and action are generated and interpreted by an IVA in a  {\it situated meaning context}, which facilitates grounded and contextualized interpretations of communicative expressions in a dialogue. 
The framework enables multiple methods for performing evaluation of gesture generation and recognition.
We discuss four separate scenarios involving the generation of non-verbal behavior in dialogue: (1)
deixis (pointing) gestures, generated to request information regarding an object, a location, or a direction when performing a specific action; (2) iconic action gestures, generated to clarify how (what manner of action) to perform a  specific task;  (3) affordance-denoting gestures, generated to describe how the IVA can interact with an object, even when it does not know what it is or what it might be used for; and (4) direct situated actions, where the IVA responds to a command or request by acting in the environment directly.
This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics
Authors: Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, Benjamin Timmermans
Date: 2021-05-04 00:00:00

In web search on debated topics, algorithmic and cognitive biases strongly influence how users consume and process information. Recent research has shown that this can lead to a search engine manipulation effect (SEME): when search result rankings are biased towards a particular viewpoint, users tend to adopt this favored viewpoint. To better understand the mechanisms underlying SEME, we present a pre-registered, 5 × 3 factorial user study investigating whether order effects (i.e., users adopting the viewpoint pertaining to higher-ranked documents) can cause SEME. For five different debated topics, we evaluated attitude change after exposing participants with mild pre-existing attitudes to search results that were overall viewpoint-balanced but reflected one of three levels of algorithmic ranking bias. We found that attitude change did not differ across levels of ranking bias and did not vary based on individual user differences. Our results thus suggest that order effects may not be an underlying mechanism of SEME. Exploratory analyses lend support to the presence of exposure effects (i.e., users adopting the majority viewpoint among the results they examine) as a contributing factor to users’ attitude change. We discuss how our findings can inform the design of user bias mitigation strategies.
Should I Follow the Crowd? A Probabilistic Analysis of the Effectiveness of Popularity in Recommender Systems
Authors: Rocío Cañamares, Pablo Castells
Date: 2019-02-06 00:00:00
Tasks: Recommendation Systems

The use of IR methodology in the evaluation of recommender systems has become common practice in recent years. IR metrics have been found however to be strongly biased towards rewarding algorithms that recommend popular items –the same bias that state of the art recommendation algorithms display. Recent research has confirmed and measured such biases, and proposed methods to avoid them. The fundamental question remains open though whether popularity is really a bias we should avoid or not; whether it could be a useful and reliable signal in recommendation, or it may be unfairly rewarded by the experimental biases. We address this question at a formal level by identifying and modeling the conditions that can determine the answer, in terms of dependencies between key random variables, involving item rating, discovery and relevance. We find conditions that guarantee popularity to be effective or quite the opposite, and for the measured metric values to reflect a true effectiveness, or qualitatively deviate from it. We exemplify and confirm the theoretical findings with empirical results. We build a crowdsourced dataset devoid of the usual biases displayed by common publicly available data, in which we illustrate contradictions between the accuracy that would be measured in a common biased offline experimental setting, and the actual accuracy that can be measured with unbiased observations.
Informing the Design of Spoken Conversational Search
Authors: Johanne R. Trippas, Damiano Spina, Lawrence Cavedon, Hideo Joho, Mark Sanderson
Date: 2019-01-12 00:00:00
Tasks: Conversational Search

We conducted a laboratory-based observational study where pairs of people performed search tasks communicating verbally. Examination of the discourse allowed commonly used interactions to be identified for Spoken Conversational Search (SCS). We compared the interactions to existing models of search behaviour. We find that SCS is more complex and interactive than traditional search. This work enhances our understanding of different search behaviours and proposes research opportunities for an audio-only search system. Future work will focus on creating models of search behaviour for SCS and evaluating these against actual SCS systems.
Enhancing Topic Modeling for Short Texts with Auxiliary Word Embeddings
Authors: Chenliang Li, Yu Duan, Haoran Wang, Zhiqian Zhang, Aixin Sun, Zongyang Ma
Date: 2018-12-22 00:00:00
Tasks: GPU, text-classification, Text Classification, Topic Models, Word Embeddings

Many applications require semantic understanding of short texts, and inferring discriminative and coherent latent topics is a critical and fundamental task in these applications. Conventional topic models largely rely on word co-occurrences to derive topics from a collection of documents. However, due to the length of each document, short texts are much more sparse in terms of word co-occurrences. Recent studies show that the Dirichlet Multinomial Mixture (DMM) model is effective for topic inference over short texts by assuming that each piece of short text is generated by a single topic. However, DMM has two main limitations. First, even though it seems reasonable to assume that each short text has only one topic because of its shortness, the definition of “shortness” is subjective and the length of the short texts is dataset dependent. That is, the single-topic assumption may be too strong for some datasets. To address this limitation, we propose to model the topic number as a Poisson distribution, allowing each short text to be associated with a small number of topics (e.g., one to three topics). This model is named PDMM. Second, DMM (and also PDMM) does not have access to background knowledge (e.g., semantic relations between words) when modeling short texts. When a human being interprets a piece of short text, the understanding is not solely based on its content words, but also their semantic relations. Recent advances in word embeddings offer effective learning of word semantic relations from a large corpus. Such auxiliary word embeddings enable us to address the second limitation. To this end, we propose to promote the semantically related words under the same topic during the sampling process, by using the generalized Pólya urn (GPU) model. Through the GPU model, background knowledge about word semantic relations learned from millions of external documents can be easily exploited to improve topic modeling for short texts. By directly extending the PDMM model with the GPU model, we propose two more effective topic models for short texts, named GPU-DMM and GPU-PDMM. Through extensive experiments on two real-world short text collections in two languages, we demonstrate that PDMM achieves better topic representations than state-of-the-art models, measured by topic coherence.
The learned topic representation leads to better accuracy in a text classification task, as an indirect evaluation. Both GPU-DMM and GPU-PDMM further improve topic coherence and text classification accuracy. GPU-PDMM outperforms GPU-DMM at the price of higher computational costs.
Empirical Study of Multi-level Convolution Models for IR Based on Representations and Interactions
Authors: Yifan Nie, Yanling Li, Jian-Yun Nie
Date: 2018-12-20 00:00:00

Deep learning models have been employed to perform IR tasks and have shown competitive results. Depending on the structure of the models, previous deep IR models could be roughly divided into: representation-based models and interaction-based models. A number of experiments have been conducted to test these models, but often under different conditions, making it difficult to draw a clear conclusion on their comparison. In order to compare the two learning schemas for ad hoc search under the same condition, we build similar convolution networks to learn either representations or interaction patterns between document and query and test them on the same test collection. In addition, we also propose multi- level matching models to cope with various types of query, rather than the existing single-level matching. Our experiments show that interaction-based approach generally performs better than representation-based approach, and multi-level matching performs better than single-level matching. We will provide some possible explanations to these observations.
Visual log analysis method for designing individual and organizational BIM skill
Authors: Tsukasa Ishizawa, Yasushi Ikeda
Date: 2021-05-14 00:00:00
Proceeding: AIS-J 2021 5
Tasks: Management

Extending the existing Building Information Modeling (BIM) log mining approach, the paper proposes a novel method for visual analytics on clustered logs collected from multiple organizations. Implementing multiple datasets as case study processes demonstrated that the analytics effectively visualized the structured BIM events in command, organization, and user layers. Identified four major cluster groups correspond to logs' commonality and level of contribution significantly helped decipher recorded BIM activities without requiring background knowledge. The proposed technique allows the intercomparison of BIM activity that enables data-driven skill design for individuals and organizations. Such monitoring allows BIM users and teams to respond to transient project situations dynamically, which is expected to mitigate the observed major BIM problems of skill and management. The method contributes to increasing the likelihood that the model will be used throughout the project duration, leading to enhanced BIM's expected value in projects.