OpenRaiser/Intern-Atlas huggingface dataset:
Exploration in Deep Reinforcement Learning: A Survey
Venue: arXiv cs.LG 2022
Year: 2022

This paper reviews exploration techniques in deep reinforcement learning. Exploration techniques are of primary importance when solving sparse reward problems. In sparse reward problems, the reward is rare, which means that the agent will not find the reward often by acting randomly. In such a scenario, it is challenging for reinforcement learning to learn rewards and actions association. Thus more sophisticated exploration methods need to be devised. This review provides a comprehensive overview of existing exploration approaches, which are categorized based on the key contributions as follows reward novel states, reward diverse behaviours, goal-based methods, probabilistic methods, imitation-based methods, safe exploration and random-based methods. Then, the unsolved challenges are discussed to provide valuable future research directions. Finally, the approaches of different categories are compared in terms of complexity, computational effort and overall performance.
AI-Driven Contextual Advertising: A Technology Report and Implication Analysis
Venue: arXiv cs.AI 2022
Year: 2022

Programmatic advertising consists in automated auctioning of digital ad space. Every time a user requests a web page, placeholders on the page are populated with ads from the highest-bidding advertisers. The bids are typically based on information about the user, and to an increasing extent, on information about the surrounding media context. The growing interest in contextual advertising is in part a counterreaction to the current dependency on personal data, which is problematic from legal and ethical standpoints. The transition is further accelerated by developments in Artificial Intelligence (AI), which allow for a deeper semantic understanding of context and, by extension, more effective ad placement. In this article, we begin by identifying context factors that have been shown in previous research to positively influence how ads are received. We then continue to discuss applications of AI in contextual advertising, where it adds value by, e.g., extracting high-level information about media context and optimising bidding strategies. However, left unchecked, these new practices can lead to unfair ad delivery and manipulative use of context. We summarize these and other concerns for consumers, publishers and advertisers in an implication analysis.
Assessing unconstrained surgical cuttings in VR using CNNs
Venue: arXiv cs.CV 2022
Year: 2022

We present a Convolutional Neural Network (CNN) suitable to assess unconstrained surgical cuttings, trained on a dataset created with a data augmentation technique.
Music Interpretation Analysis. A Multimodal Approach To Score-Informed Resynthesis of Piano Recordings
Venue: arXiv cs.SD 2022
Year: 2022

This Thesis discusses the development of technologies for the automatic resynthesis of music recordings using digital synthesizers. First, the main issue is identified in the understanding of how Music Information Processing (MIP) methods can take into consideration the influence of the acoustic context on the music performance. For this, a novel conceptual and mathematical framework named "Music Interpretation Analysis" (MIA) is presented. In the proposed framework, a distinction is made between the "performance" - the physical action of playing - and the "interpretation" - the action that the performer wishes to achieve. Second, the Thesis describes further works aiming at the democratization of music production tools via automatic resynthesis: 1) it elaborates software and file formats for historical music archiving and multimodal machine-learning datasets; 2) it explores and extends MIP technologies; 3) it presents the mathematical foundations of the MIA framework and shows preliminary evaluations to demonstrate the effectiveness of the approach
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering
Venue: arXiv cs.CV 2022
Year: 2022

We present Answer-Me, a task-aware multi-task framework which unifies a variety of question answering tasks, such as, visual question answering, visual entailment, visual reasoning. In contrast to previous works using contrastive or generative captioning training, we propose a novel and simple recipe to pre-train a vision-language joint model, which is multi-task as well. The pre-training uses only noisy image captioning data, and is formulated to use the entire architecture end-to-end with both a strong language encoder and decoder. Our results show state-of-the-art performance, zero-shot generalization, robustness to forgetting, and competitive single-task results across a variety of question answering tasks. Our multi-task mixture training learns from tasks of various question intents and thus generalizes better, including on zero-shot vision-language tasks. We conduct experiments in the challenging multi-task and open-vocabulary settings and across a variety of datasets and tasks, such as VQA2.0, SNLI-VE, NLVR2, GQA. We observe that the proposed approach is able to generalize to unseen tasks and that more diverse mixtures lead to higher accuracy in both known and novel tasks.
LIDER: An Efficient High-dimensional Learned Index for Large-scale Dense Passage Retrieval
Venue: arXiv cs.IR 2022
Year: 2022

Many recent approaches of passage retrieval are using dense embeddings generated from deep neural models, called "dense passage retrieval". The state-of-the-art end-to-end dense passage retrieval systems normally deploy a deep neural model followed by an approximate nearest neighbor (ANN) search module. The model generates embeddings of the corpus and queries, which are then indexed and searched by the high-performance ANN module. With the increasing data scale, the ANN module unavoidably becomes the bottleneck on efficiency. An alternative is the learned index, which achieves significantly high search efficiency by learning the data distribution and predicting the target data location. But most of the existing learned indexes are designed for low dimensional data, which are not suitable for dense passage retrieval with high-dimensional dense embeddings. In this paper, we propose LIDER, an efficient high-dimensional Learned Index for large-scale DEnse passage Retrieval. LIDER has a clustering-based hierarchical architecture formed by two layers of core models. As the basic unit of LIDER to index and search data, a core model includes an adapted recursive model index (RMI) and a dimension reduction component which consists of an extended SortingKeys-LSH (SK-LSH) and a key re-scaling module. The dimension reduction component reduces the high-dimensional dense embeddings into one-dimensional keys and sorts them in a specific order, which are then used by the RMI to make fast prediction. Experiments show that LIDER has a higher search speed with high retrieval quality comparing to the state-of-the-art ANN indexes on passage retrieval tasks, e.g., on large-scale data it achieves 1.2x search speed and significantly higher retrieval quality than the fastest baseline in our evaluation. Furthermore, LIDER has a better capability of speed-quality trade-off.
Knowledge Graph Contrastive Learning for Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Knowledge Graphs (KGs) have been utilized as useful side information to improve recommendation quality. In those recommender systems, knowledge graph information often contains fruitful facts and inherent semantic relatedness among items. However, the success of such methods relies on the high quality knowledge graphs, and may not learn quality representations with two challenges: i) The long-tail distribution of entities results in sparse supervision signals for KG-enhanced item representation; ii) Real-world knowledge graphs are often noisy and contain topic-irrelevant connections between items and entities. Such KG sparsity and noise make the item-entity dependent relations deviate from reflecting their true characteristics, which significantly amplifies the noise effect and hinders the accurate representation of user's preference. To fill this research gap, we design a general Knowledge Graph Contrastive Learning framework (KGCL) that alleviates the information noise for knowledge graph-enhanced recommender systems. Specifically, we propose a knowledge graph augmentation schema to suppress KG noise in information aggregation, and derive more robust knowledge-aware representations for items. In addition, we exploit additional supervision signals from the KG augmentation process to guide a cross-view contrastive learning paradigm, giving a greater role to unbiased user-item interactions in gradient descent and further suppressing the noise. Extensive experiments on three public datasets demonstrate the consistent superiority of our KGCL over state-of-the-art techniques. KGCL also achieves strong performance in recommendation scenarios with sparse user-item interactions, long-tail and noisy KG entities. Our implementation codes are available at https://github.com/yuh-yang/KGCL-SIGIR22
Auto-scaling HTCondor pools using Kubernetes compute resources
Venue: arXiv cs.DC 2022
Year: 2022

HTCondor has been very successful in managing globally distributed, pleasantly parallel scientific workloads, especially as part of the Open Science Grid. HTCondor system design makes it ideal for integrating compute resources provisioned from anywhere, but it has very limited native support for autonomously provisioning resources managed by other solutions. This work presents a solution that allows for autonomous, demand-driven provisioning of Kubernetes-managed resources. A high-level overview of the employed architectures is presented, paired with the description of the setups used in both on-prem and Cloud deployments in support of several Open Science Grid communities. The experience suggests that the described solution should be generally suitable for contributing Kubernetes-based resources to existing HTCondor pools.
HarmoF0: Logarithmic Scale Dilated Convolution For Pitch Estimation
Venue: arXiv cs.SD 2022
Year: 2022

Sounds, especially music, contain various harmonic components scattered in the frequency dimension. It is difficult for normal convolutional neural networks to observe these overtones. This paper introduces a multiple rates dilated causal convolution (MRDC-Conv) method to capture the harmonic structure in logarithmic scale spectrograms efficiently. The harmonic is helpful for pitch estimation, which is important for many sound processing applications. We propose HarmoF0, a fully convolutional network, to evaluate the MRDC-Conv and other dilated convolutions in pitch estimation. The results show that this model outperforms the DeepF0, yields state-of-the-art performance in three datasets, and simultaneously reduces more than 90% parameters. We also find that it has stronger noise resistance and fewer octave errors. The code and pre-trained model are available at https://github.com/WX-Wei/HarmoF0.
Safety-Critical Manipulation for Collision-Free Food Preparation
Venue: arXiv cs.RO 2022
Year: 2022

Recent advances allow for the automation of food preparation in high-throughput environments, yet the successful deployment of these robots requires the planning and execution of quick, robust, and ultimately collision-free behaviors. In this work, we showcase a novel framework for modifying previously generated trajectories of robotic manipulators in highly detailed and dynamic collision environments using Control Barrier Functions (CBFs). This method dynamically re-plans previously validated behaviors in the presence of changing environments -- and does so in a computationally efficient manner. Moreover, the approach provides rigorous safety guarantees of the resulting trajectories, factoring in the true underlying dynamics of the manipulator. This methodology is extensively validated on a full-scale robotic manipulator in a real-world cooking environment, and has resulted in substantial improvements in computation time and robustness over re-planning.
Data Justice in Practice: A Guide for Developers
Venue: arXiv cs.CY 2022
Year: 2022

The Advancing Data Justice Research and Practice project aims to broaden understanding of the social, historical, cultural, political, and economic forces that contribute to discrimination and inequity in contemporary ecologies of data collection, governance, and use. This is the consultation draft of a guide for developers and organisations, which are producing, procuring, or using data-intensive technologies.In the first section, we introduce the field of data justice, from its early discussions to more recent proposals to relocate understandings of what data justice means. This section includes a description of the six pillars of data justice around which this guidance revolves. Next, to support developers in designing, developing, and deploying responsible and equitable data-intensive and AI/ML systems, we outline the AI/ML project lifecycle through a sociotechnical lens. To support the operationalisation data justice throughout the entirety of the AI/ML lifecycle and within data innovation ecosystems, we then present five overarching principles of responsible, equitable, and trustworthy data research and innovation practices, the SAFE-D principles-Safety, Accountability, Fairness, Explainability, and Data Quality, Integrity, Protection, and Privacy. The final section presents guiding questions that will help developers both address data justice issues throughout the AI/ML lifecycle and engage in reflective innovation practices that ensure the design, development, and deployment of responsible and equitable data-intensive and AI/ML systems.
Causal Discovery on the Effect of Antipsychotic Drugs on Delirium Patients in the ICU using Large EHR Dataset
Venue: arXiv cs.LG 2022
Year: 2022

Delirium occurs in about 80% cases in the Intensive Care Unit (ICU) and is associated with a longer hospital stay, increased mortality and other related issues. Delirium does not have any biomarker-based diagnosis and is commonly treated with antipsychotic drugs (APD). However, multiple studies have shown controversy over the efficacy or safety of APD in treating delirium. Since randomized controlled trials (RCT) are costly and time-expensive, we aim to approach the research question of the efficacy of APD in the treatment of delirium using retrospective cohort analysis. We plan to use the Causal inference framework to look for the underlying causal structure model, leveraging the availability of large observational data on ICU patients. To explore safety outcomes associated with APD, we aim to build a causal model for delirium in the ICU using large observational data sets connecting various covariates correlated with delirium. We utilized the MIMIC III database, an extensive electronic health records (EHR) dataset with 53,423 distinct hospital admissions. Our null hypothesis is: there is no significant difference in outcomes for delirium patients under different drug-group in the ICU. Through our exploratory, machine learning based and causal analysis, we had findings such as: mean length-of-stay and max length-of-stay is higher for patients in Haloperidol drug group, and haloperidol group has a higher rate of death in a year compared to other two-groups. Our generated causal model explicitly shows the functional relationships between different covariates. For future work, we plan to do time-varying analysis on the dataset.
Investigating the impact of COVID-19 on Online Learning-based Web Behavior
Venue: arXiv cs.CY 2022
Year: 2022

COVID-19, a pandemic that the world has not seen in decades, has resulted in presenting a multitude of unprecedented challenges for student learning across the globe. The global surge in COVID-19 cases resulted in several schools, colleges, and universities closing in 2020 in almost all parts of the world and switching to online or remote learning, which has impacted student learning in different ways. This has resulted in both educators and students spending more time on the internet than ever before, which may be broadly summarized as both these groups investigating, learning, and familiarizing themselves with information, tools, applications, and frameworks to adapt to online learning. This paper takes an explorative approach to further investigate and analyze the impact of COVID-19 on such web behavior data related to online learning to interpret the associated interests, challenges, and needs. The study specifically focused on investigating Google Search-based web behavior data as Google is the most popular search engine globally. The impact of COVID-19 related to online learning-based web behavior on Google was studied for the top 20 worst affected countries in terms of the total number of COVID-19 cases, and the findings have been published as an open-access dataset. Furthermore, to interpret the trends in web behavior data related to online learning, the paper discusses a case study in terms of the impact of COVID-19 on the education system of one of these countries.
Designing thermal radiation metamaterials via hybrid adversarial autoencoder and Bayesian optimization
Venue: arXiv cs.LG 2022
Year: 2022

Designing thermal radiation metamaterials is challenging especially for problems with high degrees of freedom and complex objective. In this letter, we have developed a hybrid materials informatics approach which combines the adversarial autoencoder and Bayesian optimization to design narrowband thermal emitters at different target wavelengths. With only several hundreds of training data sets, new structures with optimal properties can be quickly figured out in a compressed 2-dimensional latent space. This enables the optimal design by calculating far less than 0.001\% of the total candidate structures, which greatly decreases the design period and cost. The proposed design framework can be easily extended to other thermal radiation metamaterials design with higher dimensional features.
Five Ps: Leverage Zones Towards Responsible AI
Venue: arXiv cs.CY 2022
Year: 2022

There is a growing debate amongst academics and practitioners on whether interventions made, thus far, towards Responsible AI would have been enough to engage with root causes of AI problems. Failure to effect meaningful changes in this system could see these initiatives to not reach their potential and lead to the concept becoming another buzzword for companies to use in their marketing campaigns. We propose that there is an opportunity to improve the extent to which interventions are understood to be effective in their contribution to the change required for Responsible AI. Using the notions of leverage zones adapted from the 'Systems Thinking' literature, we suggest a novel approach to evaluate the effectiveness of interventions, to focus on those that may bring about the real change that is needed. In this paper we argue that insights from using this perspective demonstrate that the majority of current initiatives taken by various actors in the field, focus on low-order interventions, such as short-term fixes, tweaking algorithms and updating parameters, absent from higher-order interventions, such as redefining the system's foundational structures that govern those parameters, or challenging the underlying purpose upon which those structures are built and developed in the first place(high-leverage). This paper presents a conceptual framework called the Five Ps to identify interventions towards Responsible AI and provides a scaffold for transdisciplinary question asking to improve outcomes towards Responsible AI.
The Equity Framework: Fairness Beyond Equalized Predictive Outcomes
Venue: arXiv cs.CY 2022
Year: 2022

Machine Learning (ML) decision-making algorithms are now widely used in predictive decision-making, for example, to determine who to admit and give a loan. Their wide usage and consequential effects on individuals led the ML community to question and raise concerns on how the algorithms differently affect different people and communities. In this paper, we study fairness issues that arise when decision-makers use models (proxy models) that deviate from the models that depict the physical and social environment in which the decisions are situated (intended models). We also highlight the effect of obstacles on individual access and utilization of the models. To this end, we formulate an Equity Framework that considers equal access to the model, equal outcomes from the model, and equal utilization of the model, and consequentially achieves equity and higher social welfare than current fairness notions that aim for equality. We show how the three main aspects of the framework are connected and provide an equity scoring algorithm and questions to guide decision-makers towards equitable decision-making. We show how failure to consider access, outcome, and utilization would exacerbate proxy gaps leading to an infinite inequity loop that reinforces structural inequities through inaccurate and incomplete ground truth curation. We, therefore, recommend a more critical look at the model design and its effect on equity and a shift towards equity achieving predictive decision-making models.
Ensemble pruning via an integer programming approach with diversity constraints
Venue: arXiv cs.LG 2022
Year: 2022

Ensemble learning combines multiple classifiers in the hope of obtaining better predictive performance. Empirical studies have shown that ensemble pruning, that is, choosing an appropriate subset of the available classifiers, can lead to comparable or better predictions than using all classifiers. In this paper, we consider a binary classification problem and propose an integer programming (IP) approach for selecting optimal classifier subsets. We propose a flexible objective function to adapt to desired criteria of different datasets. We also propose constraints to ensure minimum diversity levels in the ensemble. Despite the general case of IP being NP-Hard, state-of-the-art solvers are able to quickly obtain good solutions for datasets with up to 60000 data points. Our approach yields competitive results when compared to some of the best and most used pruning methods in literature.
ComPhy: Compositional Physical Reasoning of Objects and Events from Videos
Venue: arXiv cs.CV 2022
Year: 2022

Objects' motions in nature are governed by complex interactions and their properties. While some properties, such as shape and material, can be identified via the object's visual appearances, others like mass and electric charge are not directly visible. The compositionality between the visible and hidden properties poses unique challenges for AI models to reason from the physical world, whereas humans can effortlessly infer them with limited observations. Existing studies on video reasoning mainly focus on visually observable elements such as object appearance, movement, and contact interaction. In this paper, we take an initial step to highlight the importance of inferring the hidden physical properties not directly observable from visual appearances, by introducing the Compositional Physical Reasoning (ComPhy) dataset. For a given set of objects, ComPhy includes few videos of them moving and interacting under different initial conditions. The model is evaluated based on its capability to unravel the compositional hidden properties, such as mass and charge, and use this knowledge to answer a set of questions posted on one of the videos. Evaluation results of several state-of-the-art video reasoning models on ComPhy show unsatisfactory performance as they fail to capture these hidden properties. We further propose an oracle neural-symbolic framework named Compositional Physics Learner (CPL), combining visual perception, physical property learning, dynamic prediction, and symbolic execution into a unified framework. CPL can effectively identify objects' physical properties from their interactions and predict their dynamics to answer questions.