Reviewerly/exHarmony huggingface dataset:
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets

Recent progress in offline reinforcement learning (RL) has made it possible to train strong RL agents from previously-collected, static datasets. However, depending on the quality of the trained agents and the application being considered, it is often desirable to improve such offline RL agents with further online interaction. As it turns out, fine-tuning offline RL agents is a non-trivial challenge, due to distribution shift – the agent encounters out-of-distribution samples during online interaction, which may cause bootstrapping error in Q-learning and instability during fine-tuning. In order to address the issue, we present a simple yet effective framework, which incorporates a balanced replay scheme and an ensemble distillation scheme. First, we propose to keep separate offline and online replay buffers, and carefully balance the number of samples from each buffer during updates. By utilizing samples from a wider distribution, i.e., both online and offline samples, we stabilize the Q-learning. Next, we present an ensemble distillation scheme, where we train an ensemble of independent actor-critic agents, then distill the policies into a single policy. In turn, we improve the policy using the Q-ensemble during fine-tuning, which allows the policy updates to be more robust to error in each individual Q-function. We demonstrate the superiority of our method on MuJoCo datasets from the recently proposed D4RL benchmark suite.
Multi-Camera Based Setup for Geometrical Measurement of Free-Falling Molten Glass Gob

High temperatures complicate the direct measurements needed for continuous characterization of the properties of molten materials such as glass. However, the assumption that geometrical changes when the molten material is in free-fall can be correlated with material characteristics such as viscosity opens the door to a highly accurate contactless method characterizing small dynamic changes. This paper proposes multi-camera setup to achieve accuracy close to the segmentation error associated with the resolution of the images. The experimental setup presented shows that the geometrical parameters can be characterized dynamically through the whole free-fall process at a frame rate of 600 frames per second. The results achieved show the proposed multi-camera setup is suitable for estimating the length of free-falling molten objects.
Few-Round Learning for Federated Learning

Federated learning (FL) presents an appealing opportunity for individuals who are willing to make their private data available for building a communal model without revealing their data contents to anyone else. Of central issues that may limit a widespread adoption of FL is the significant communication resources required in the exchange of updated model parameters between the server and individual clients over many communication rounds. In this work, we focus on limiting the number of model exchange rounds in FL to some small fixed number R, to control the communication burden. Following the spirit of meta-learning for few-shot learning, we take a meta-learning strategy to train the model so that once the meta-training phase is over, only R rounds of FL would produce a model that will satisfy the needs of all participating clients. A key advantage of employing meta-training is that the main labeled dataset used in training could differ significantly (e.g., different classes of images) from the actual data sample presented at inference time. Compared to the meta-training approaches to optimize personalized local models at distributed devices, our method better handles the potential lack of data variability at individual nodes. Extensive experimental results indicate that meta-training geared to few-round learning provide large performance improvements compared to various baselines.
Self Sovereign Identity based E-petition Scheme

Electronic petitions are one of the important tools used for democratic participation. Citizens can express their support or dissatisfaction about government investments or regulations. Citizens can also provide sufficient feedback to organizations or government institutions through electronic petitions. In this paper, we have designed an electronic petition scheme using blockchain as a source of trust for protecting e-petition’s users’ privacy and security. With our e-petition system, citizens can anonymously participate an e-petition and express their support or discontent about a social and political issue freely. We have used Sovrin blockchain as source of trust by using registry for Decentralised Identifiers (DIDs) and their associated public keys and communication endpoints. We have also used other Sovrin components such as verifiable credentials, proofs and agents for implementing our privacy and security preserving e-petition scheme. Lastly, we have analyzed security, privacy and performance analysis of our e-petition system.
Discovering Differential Equations from Earth Observation Data

Modeling and understanding the Earth system is a constant and challenging scientific endeavour. When a clear mechanistic model is unavailable, complex or uncertain, learning from data can be an alternative. While machine learning has provided excellent methods for detection and retrieval, understanding the governing equations of the system from observational data seems an elusive problem. In this paper we introduce sparse regression to uncover a set of governing equations in the form of a system of ordinary differential equations (ODEs). The presented method is used to explicitly describe variable relations by identifying the most expressive and simplest ODEs explaining data to model relevant components of the biosphere.
Utilisation of mobile phones in accessing agricultural information by smallholder farmers in Dzindi irrigation scheme in South Africa

ICTs have revolutionised agricultural information services at every level in the agricultural sector, thus impacting rural development and catalysing poverty alleviation strategies. This has largely been the case with small-scale farmers in rural areas in developing countries where mobile technologies have penetrated more than most other ICT tools. However, in some of the farming environments, mobile phone use is largely driven by agricultural extension workers. This paper seeks to examine the way mobile phones are used for information access in situations where agricultural extension workers are a critical intermediary in the agricultural information services. Interviews were conducted with 10 randomly selected farmers who were part of the Dzindi irrigation scheme. The findings were that from the variety of information available to the farmers the most important source was the extension officer. The extension officer and the radio were indicated to be the most reliable  independent sources of information. The other sources, such as the radio, family members, and friends, suppliers of chemicals, books and magazines, were only considered reliable if the information could be verified or vouched for by the extension officer. Increasing the information handling skills of extension officers, training of farmers to use smart features of their phones and promoting the usual face-to-face communication and use of conventional methods, which is what usually gives rise to the mediation of mobile phones, were recommended.Keywords: Mobile phone, Agricultural extension, Agricultural information
What's in the Box? Exploring the Inner Life of Neural Networks with Robust Rules

We propose a novel method for exploring how neurons within a neural network interact. In particular, we consider activation values of a network for given data, and propose to mine noise-robust rules of the form X→Y , where X and Y are sets of neurons in different layers. To ensure we obtain a small and non-redundant set of high quality rules, we formalize the problem in terms of the Minimum Description Length principle, by which we identify the best set of rules as the one that best compresses the activation data. To discover good rule sets, we propose the unsupervised ExplaiNN algorithm. Extensive evaluation shows that our rules give clear insight in how networks perceive the world: they identify shared, resp. class-specific traits, compositionality within the network, as well as locality in convolutional layers. Our rules are easily interpretable, but also super-charge prototyping as they identify which groups of neurons to consider in unison.
Supporting search engines with knowledge and context

Search engines leverage knowledge to improve information access. Such knowledge comes in different forms: unstructured knowledge (e.g., textual documents) and structured knowledge (e.g., relationships between real-world objects and topics). In order to effectively leverage knowledge, search engines should account for context, i.e., information about the user and query. In this thesis, we aim to support search engines in leveraging knowledge while accounting for context. In the first part of this thesis, we study how to make structured knowledge more accessible to the user when the search engine proactively provides such knowledge as context to enrich search results. As a first task, we study how to retrieve descriptions of knowledge facts from a text corpus. Next, we study how to automatically generate knowledge fact descriptions. And finally, we study how to contextualize knowledge facts, that is, to automatically find facts related to a query fact. In the second part of this thesis, we study how to improve interactive knowledge gathering. We focus on conversational search, where the user interacts with the search engine to gather knowledge over large unstructured knowledge repositories. We study multi-turn passage retrieval as an instance of conversational search and focus on query resolution, that is, add missing context from the conversation history to the current turn. We model query resolution as a term classification task and propose a method to address it. In the final part of this thesis, we focus on search engine support for professional writers in the news domain. We study how to support such writers create event-narratives by exploring knowledge from a corpus of news articles. We propose a dataset construction procedure for this task that relies on existing news articles to simulate incomplete narratives and relevant articles. We study the performance of multiple rankers, lexical and semantic, and provide insights into the characteristics of this task. Awarded by : University of Amsterdam, Amsterdam, The Netherlands on 5 February 2021. Supervised by : Maarten de Rijke. Available at : https://hdl.handle.net/11245.1/78187b29-2403-4711-800a-0f92fcb9b15c.
Iterative Empirical Game Solving via Single Policy Best Response

Policy-Space Response Oracles (PSRO) is a general algorithmic framework for learning policies in multiagent systems by interleaving empirical game analysis with deep reinforcement learning (DRL). At each iteration, DRL is invoked to train a best response to a mixture of opponent policies. The repeated application of DRL poses an expensive computational burden as we look to apply this algorithm to more complex domains. We introduce two variations of PSRO designed to reduce the amount of simulation required during DRL training. Both algorithms modify how PSRO adds new policies to the empirical game, based on learned responses to a single opponent policy. The first, Mixed-Oracles, transfers knowledge from previous iterations of DRL, requiring training only against the opponent's newest policy. The second, Mixed-Opponents, constructs a pure-strategy opponent by mixing existing strategy's action-value estimates, instead of their policies. Learning against a single policy mitigates conflicting experiences on behalf of a learner facing an unobserved distribution of opponents. We empirically demonstrate that these algorithms substantially reduce the amount of simulation during training required by PSRO, while producing equivalent or better solutions to the game.
Guiding Representation Learning in Deep Generative Models with Policy Gradients

Variational Auto Encoder (VAE) provide an efficient latent space representation of complex data distributions which is learned in an unsupervised fashion. Using such a representation as input to Reinforcement Learning (RL) approaches may reduce learning time, enable domain transfer or improve interpretability of the model. However, current state-of-the-art approaches that combine VAE with RL fail at learning good performing policies on certain RL domains. Typically, the VAE is pre-trained in isolation and may omit the embedding of task-relevant features due to insufficiencies of its loss. As a result, the RL approach can not successfully maximize the reward on these domains. Therefore, this paper investigates the issues of joint training approaches and explores incorporation of policy gradients from RL into the VAE's latent space to find a task-specific latent space representation. We show that using pre-trained representations can lead to policies being unable to learn any rewarding behaviour in these environments. Subsequently, we introduce two types of models which overcome this deficiency by using policy gradients to learn the representation. Thereby the models are able to embed features into its representation that are crucial for performance on the RL task but would not have been learned with previous methods.
Color Image Watermarking in DCT Domain Using SVR

A robust digital image watermarking scheme in the DCT domain for color images using support vector regression is presented in this paper. This scheme attempts to achieve a fair balance between imperceptibility and robustness using an array of techniques including machine learning. The host image is transformed from RGB to YCbCr color space which gives a luma component used for embedding the watermark. This scheme utilizes Lagrangian Support Vector Regression (LSVR) for learning image features; entropy for selecting the image blocks in which watermark bits are embedded; discrete cosine transform to de-correlate the luminance component pixels. The low-frequency coefficients scanned in a zigzag manner are used for training and testing of LSVR model, and the output acquired from LSVR is used for embedding the watermark. The proposed method is applied to general image processing dataset images and found to yield fairly good results as shown by the comparison with existing methods.
Learning Lagrangian Fluid Dynamics with Graph Neural Networks

We present a data-driven model for fluid simulation under Lagrangian representation. Our model uses graphs to describe the fluid field, where physical quantities are encoded as node and edge features. Instead of directly predicting the acceleration or position correction given the current state, we decompose the simulation scheme into separate parts - advection, collision, and pressure projection. For these different reasoning tasks, we propose two kinds of graph neural network structures, node-focused networks, and edge-focused networks. By introducing physics prior knowledge, our model can be efficient in terms of training and inference. Our tests show that the learned model can produce accurate results and remain stable in scenarios with a large amount of particles and different geometries. Unlike many previous works, further tests demonstrate that our model is able to retain many important physical properties of incompressible fluids, such as minor divergence and reasonable pressure distribution. Additionally, our model can adopt a range of time step sizes different from ones using in the training set, which indicates its robust generalization capability.
Conversational design: the use of chatbot as an electoral information dissemination tool

A steady decline in voter turnout has been recorded in many democratic countries, with young people (i.e Gen Z and Millennials) identified as a significant portion of those who don’t vote. Research has shown that one of the reasons young people do not vote is a lack of information about the electoral process, candidates and policies. Studies have also shown that preferred information dissemination medium is aligned across age groups, with older people trusting old media such as newspapers and young people leaning towards new media such as social media. With the recent advancement in machine learning, natural language interface and popularity of conversational agents such as chatbots, a new subset of new media has emerged. This study aims to investigate if chatbots could be an efficient medium for young people to access electoral information. It does this by comparing it with a website, using the EU electoral page as a case study.Based on the content of the EU electoral webpage, a chatbot was designed, trained and deployed on Telegram. Participants were randomly divided into two experimental groups (chatbot and website) and asked to find a list of crucial information, interact with the medium and then respond to the System usability scale (SUS), and a perception questionnaire. Observed data and interview notes were also taken. Analysis of the experimental data, observed data and surveys demonstrated that chatbot as a tool is inefficient as an electoral information tool because of the limitation of chatbot when out of scope questions are asked and limitation in following conversation contexts which is crucial to conversations. However, it is rated as above average on usability and joy of use, but raises questions on trust and data protection. It is recommended that the study be performed again when there are more advances to natural language processing to see if its efficiency as an electoral information medium will improve. A longitudinal study is also recommended to see if there’s a direct impact of chatbots as a tool, on young people’s motivation to vote.
Unsupervised Real-World Super-resolution Using Variational Auto-encoder and Generative Adversarial Network

Convolutional Neural Networks (CNNs) have shown promising results on Single Image Super-Resolution (SISR) task. A pair of Low-Resolution (LR) and High-Resolution (HR) images are typically used in the CNN models to train them to super-resolve LR images in a fully supervised manner. Owing to non-availability of true LR-HR pairs, the LR images are generally synthesized from HR data by applying synthetic degradation such as bicubic downsampling. Such networks under-perform when used on real-world data where degradation is different from the synthetically generated LR image. As obtaining true LR-HR pair is a tedious and resource (time and effort) consuming task, we propose a new approach and architecture to super-resolve the real-world LR images in an unsupervised manner by using a Generative Adversarial Network (GAN) framework with Variational Auto-Encoder (VAE). Along with a new network architecture, we also introduce a novel loss metric based on no-reference quality scores of SR images to improve the perceptual fidelity of the SR images. Through the experiments on NTIRE-2020 Real-World SR Challenge dataset, we demonstrate the superiority of the proposed approach over the other competing state-of-the-art methods.
Tibetan interrogative sentence recognition and classification based on phrase features

The recognition of Tibetan interrogative sentences is a basic work in natural language processing, which has a wide application value in terms of Tibetan syntactic analysis, semantic analysis, intelligent question answering, search engine and other research fields. Employing interrogative pronouns as a entry point to analyze the phrase features before and after interrogative pronouns, the paper proposes a method for Tibetan interrogative sentence recognition and classification based on phrase features by designing a Tibetan interrogative sentence recognition and classification model based on phrase features. Experimental results show that the recognition accuracy, recall rate and F value of this method are 98.21%, 100.00% and 99.10% respectively, and the average classification accuracy, recall rate and F value are 96.98%, 100.00% and 98.39%, respectively.
Research on Track and Field Attitude Coordination Tracking Correction Method Based on Projection Tracking

Athletics complicated attitude, attitude tracking conventional athletics stadium method applies only to the museum, the presence of gesture recognition rate is lower, the problem of poor posture correction effect, for this purpose, incorporated herein projection pursuit method, the posture effectively enhance athletics coordinate tracking correction. The track and field sports posture is decomposed according to the characteristics of the human gravity center deviation, and the sports posture characteristics of the human bones and muscles are obtained; based on the projection tracking technology, the tracking model is constructed, and the fast tracking method is designed to realize the sports posture correction. The results show that the tracking correction method based on projection tracking is suitable for indoor and outdoor venues. Compared with the traditional method, the posture coordination recognition rate after tracking correction has increased by 9.17% and 8.23%.
Efficiently Troubleshooting Image Segmentation Models with Human-In-The-Loop

Image segmentation lays the foundation for many high-stakes vision applications such as autonomous driving and medical image analysis. It is, therefore, of great importance to not only improve the accuracy of segmentation models on well-established benchmarks, but also enhance their robustness in the real world so as to avoid sparse but fatal failures. In this paper, instead of chasing state-of-the-art performance on existing benchmarks, we turn our attention to a new challenging problem: how to efficiently expose failures of ``top-performing'' segmentation models in the real world and how to leverage such counterexamples to rectify the models. To achieve this with minimal human labelling effort, we first automatically sample a small set of images that are likely to falsify the target model from a large corpus of web images via the maximum discrepancy competition principle. We then propose a weakly labelling strategy to further reduce the number of false positives, before time-consuming pixel-level labelling by humans. Finally, we fine-tune the model to harness the identified failures, and repeat the whole process, resulting in an efficient and progressive framework for troubleshooting segmentation models. We demonstrate the feasibility of our framework using the semantic segmentation task in PASCAL VOC, and find that the fine-tuned model exhibits significantly improved generalization when applied to real-world images with greater content diversity. All experimental codes will be publicly released upon acceptance.
Time-Matters: Temporal Unfolding of Texts

Over the past few years, the amount of information generated, consumed and stored on the Web has grown exponentially, making it impossible for users to keep up to date. Temporal data representation can help in this process by giving documents a sense of organization. Timelines are a natural way to showcase this data, giving users the chance to get familiar with a topic in a shorter amount of time. Despite their importance, little is known about their use in the context of single documents. In this paper, we present Time-Matters, a novel system to automatically explore arbitrary texts through temporal narratives in an interactive fashion that allows users to get insights into the relevant temporal happenings of a story through multiple components, including temporal annotation, storylines or temporal clustering. In contrast to classical timeline multi-document summarization tasks, we focus on performing text summaries of single documents with a temporal lens. This approach may be of interest to a number of providers such as media outlets, for which automatically building a condensed overview of a text is an important issue.