OpenRaiser/Intern-Atlas huggingface dataset:
Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs
Venue: arXiv cs.IR 2021
Year: 2021

The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wrong. In this work, we take a step towards a more natural learning paradigm - from noisy and implicit feedback via question reformulations. A reformulation is likely to be triggered by an incorrect system response, whereas a new follow-up question could be a positive signal on the previous turn's answer. We present a reinforcement learning model, termed CONQUER, that can learn from a conversational stream of questions and reformulations. CONQUER models the answering process as multiple agents walking in parallel on the KG, where the walks are determined by actions sampled using a policy network. This policy network takes the question along with the conversational context as inputs and is trained via noisy rewards obtained from the reformulation likelihood. To evaluate CONQUER, we create and release ConvRef, a benchmark with about 11k natural conversations containing around 205k reformulations. Experiments show that CONQUER successfully learns to answer conversational questions from noisy reward signals, significantly improving over a state-of-the-art baseline.
Role of Artificial Intelligence in Detection of Hateful Speech for Hinglish Data on Social Media
Venue: arXiv cs.CL 2021
Year: 2021

Social networking platforms provide a conduit to disseminate our ideas, views and thoughts and proliferate information. This has led to the amalgamation of English with natively spoken languages. Prevalence of Hindi-English code-mixed data (Hinglish) is on the rise with most of the urban population all over the world. Hate speech detection algorithms deployed by most social networking platforms are unable to filter out offensive and abusive content posted in these code-mixed languages. Thus, the worldwide hate speech detection rate of around 44% drops even more considering the content in Indian colloquial languages and slangs. In this paper, we propose a methodology for efficient detection of unstructured code-mix Hinglish language. Fine-tuning based approaches for Hindi-English code-mixed language are employed by utilizing contextual based embeddings such as ELMo (Embeddings for Language Models), FLAIR, and transformer-based BERT (Bidirectional Encoder Representations from Transformers). Our proposed approach is compared against the pre-existing methods and results are compared for various datasets. Our model outperforms the other methods and frameworks.
Agatha: Smart Contract for DNN Computation
Venue: arXiv cs.CR 2021
Year: 2021

Smart contract is one of the core features of Ethereum and has inspired many blockchain descendants. Since its advent, the verification paradigm of smart contract has been improving toward high scalability. It shifts from the expensive on-chain verification to the orchestration of off-chain VM (virtual machine) execution and on-chain arbitration with the pinpoint protocol. The representative projects are TrueBit, Arbitrum, YODA, ACE, and Optimism. Inspired by visionaries in academia and industry, we consider the DNN computation to be promising but on the next level of complexity for the verification paradigm of smart contract. Unfortunately, even for the state-of-the-art verification paradigm, off-chain VM execution of DNN computation has an orders-of-magnitude slowdown compared to the native off-chain execution. To enable the native off-chain execution of verifiable DNN computation, we present Agatha system, which solves the significant challenges of misalignment and inconsistency: (1) Native DNN computation has a graph-based computation paradigm misaligned with previous VM-based execution and arbitration; (2) Native DNN computation may be inconsistent cross platforms which invalidates the verification paradigm. In response, we propose the graph-based pinpoint protocol (GPP) which enables the pinpoint protocol on computational graphs, and bridges the native off-chain execution and the contract arbitration. We also develop a technique named Cross-evaluator Consistent Execution (XCE), which guarantees cross-platform consistency and forms the correctness foundation of GPP. We showcase Agatha for the DNN computation of popular models (MobileNet, ResNet50 and VGG16) on Ethereum. Agatha achieves a negligible on-chain overhead, and an off-chain execution overhead of 3.0%, which represents an off-chain latency reduction of at least 602x compared to the state-of-the-art verification paradigm.
Permissionless and Asynchronous Asset Transfer [Technical Report]
Venue: arXiv cs.DC 2021
Year: 2021

Most modern asset transfer systems use consensus to maintain a totally ordered chain of transactions. It was recently shown that consensus is not always necessary for implementing asset transfer. More efficient, asynchronous solutions can be built using reliable broadcast instead of consensus. This approach has been originally used in the closed (permissioned) setting. In this paper, we extend it to the open (permissionless) environment. We present Pastro, a permissionless and asynchronous asset-transfer implementation, in which quorum systems, traditionally used in reliable broadcast, are replaced with a weighted Proof-of-Stake mechanism. Pastro tolerates a dynamic adversary that is able to adaptively corrupt participants based on the assets owned by them.
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning (FL) is an emerging learning scheme that allows different distributed clients to train deep neural networks together without data sharing. Neural networks have become popular due to their unprecedented success. To the best of our knowledge, the theoretical guarantees of FL concerning neural networks with explicit forms and multi-step updates are unexplored. Nevertheless, training analysis of neural networks in FL is non-trivial for two reasons: first, the objective loss function we are optimizing is non-smooth and non-convex, and second, we are even not updating in the gradient direction. Existing convergence results for gradient descent-based methods heavily rely on the fact that the gradient direction is used for updating. This paper presents a new class of convergence analysis for FL, Federated Learning Neural Tangent Kernel (FL-NTK), which corresponds to overparamterized ReLU neural networks trained by gradient descent in FL and is inspired by the analysis in Neural Tangent Kernel (NTK). Theoretically, FL-NTK converges to a global-optimal solution at a linear rate with properly tuned learning parameters. Furthermore, with proper distributional assumptions, FL-NTK can also achieve good generalization.
Joint Text and Label Generation for Spoken Language Understanding
Venue: arXiv cs.CL 2021
Year: 2021

Generalization is a central problem in machine learning, especially when data is limited. Using prior information to enforce constraints is the principled way of encouraging generalization. In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data. Specifically, we extract prior knowledge from pretrained LM in the form of synthetic data, which encode the prior implicitly. We fine-tune the LM to generate an augmented language, which contains not only text but also encodes both intent labels and slot labels. The generated synthetic data can be used to train a classifier later. Since the generated data may contain noise, we rephrase the learning from generated data as learning with noisy labels. We then utilize the mixout regularization for the classifier and prove its effectiveness to resist label noise in generated data. Empirically, our method demonstrates superior performance and outperforms the baseline by a large margin.
Accelerating Amoebots via Reconfigurable Circuits
Venue: arXiv cs.DC 2021
Year: 2021

We consider an extension to the geometric amoebot model that allows amoebots to form so-called \emph{circuits}. Given a connected amoebot structure, a circuit is a subgraph formed by the amoebots that permits the instant transmission of signals. We show that such an extension allows for significantly faster solutions to a variety of problems related to programmable matter. More specifically, we provide algorithms for leader election, consensus, compass alignment, chirality agreement and shape recognition. Leader election can be solved in $Θ(\log n)$ rounds, w.h.p., consensus in $O(1)$ rounds and both, compass alignment and chirality agreement, can be solved in $O(\log n)$ rounds, w.h.p. For shape recognition, the amoebots have to decide whether the amoebot structure forms a particular shape. We show how the amoebots can detect a parallelogram with linear and polynomial side ratio within $Θ(\log{n})$ rounds, w.h.p. Finally, we show that the amoebots can detect a shape composed of triangles within $O(1)$ rounds, w.h.p.
Towards Using Diachronic Distributed Word Representations as Models of Lexical Development
Venue: arXiv cs.CL 2021
Year: 2021

Recent work has shown that distributed word representations can encode abstract information from child-directed speech. In this paper, we use diachronic distributed word representations to perform temporal modeling and analysis of lexical development in children. Unlike all previous work, we use temporally sliced corpus to learn distributed word representations of child-speech and child-directed speech under a curriculum-learning setting. In our experiments, we perform a lexical categorization task to plot the semantic and syntactic knowledge acquisition trajectories in children. Next, we perform linear mixed-effects modeling over the diachronic representational changes to study the role of input word frequencies in the rate of word acquisition in children. We also perform a fine-grained analysis of lexical knowledge transfer from adults to children using Representational Similarity Analysis. Finally, we perform a qualitative analysis of the diachronic representations from our model, which reveals the grounding and word associations in the mental lexicon of children. Our experiments demonstrate the ease of usage and effectiveness of diachronic distributed word representations in modeling lexical development.
A Reinforcement Learning Environment for Multi-Service UAV-enabled Wireless Systems
Venue: arXiv cs.LG 2021
Year: 2021

We design a multi-purpose environment for autonomous UAVs offering different communication services in a variety of application contexts (e.g., wireless mobile connectivity services, edge computing, data gathering). We develop the environment, based on OpenAI Gym framework, in order to simulate different characteristics of real operational environments and we adopt the Reinforcement Learning to generate policies that maximize some desired performance.The quality of the resulting policies are compared with a simple baseline to evaluate the system and derive guidelines to adopt this technique in different use cases. The main contribution of this paper is a flexible and extensible OpenAI Gym environment, which allows to generate, evaluate, and compare policies for autonomous multi-drone systems in multi-service applications. This environment allows for comparative evaluation and benchmarking of different approaches in a variety of application contexts.
MiSTA: An Age-Optimized Slotted ALOHA Protocol
Venue: arXiv cs.IT 2021
Year: 2021

We introduce Mini Slotted Threshold ALOHA (MiSTA), a slotted ALOHA modification designed to minimize the network-wide time average Age of Information (AoI). In MiSTA, sources whose ages are below a certain threshold stay silent. When a node with age above the threshold has data to send, it becomes active in the next time frame with a certain probability. The active node first transmits a short control sequence in a mini-slot ahead of actual data transmission, and if collision is sensed, it backs off with a certain probability. We derive the steady state distribution of the number of active sources and analyze its limiting behaviour. We show that MiSTA probabilistically converges to a "thinned" slotted ALOHA, where the number of active users at steady state adjusts to optimize age. With an optimal selection of parameters, MiSTA achieves an AoI scaling with the number of sources, n, as 0.9641n, which is an improvement over the Threshold ALOHA policy proposed earlier (for which the lowest possible scaling is 1.4169n). While achieving this reduction in age, MiSTA also increases achievable throughput to approximately 53%, from the 37% achievable by Threshold ALOHA and regular slotted ALOHA.
Formally Verified Simulations of State-Rich Processes using Interaction Trees in Isabelle/HOL
Venue: arXiv cs.LO 2021
Year: 2021

Simulation and formal verification are important complementary techniques necessary in high assurance model-based systems development. In order to support coherent results, it is necessary to provide unifying semantics and automation for both activities. In this paper we apply Interaction Trees in Isabelle/HOL to produce a verification and simulation framework for state-rich process languages. We develop the core theory and verification techniques for Interaction Trees, use them to give a semantics to the CSP and Circus languages, and formally link our new semantics with the failures-divergences semantic model. We also show how the Isabelle code generator can be used to generate verified executable simulations for reactive and concurrent programs.
Two novel feature selection algorithms based on crowding distance
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, two novel algorithms for features selection are proposed. The first one is a filter method while the second is wrapper method. Both the proposed algorithms use the crowding distance used in the multiobjective optimization as a metric in order to sort the features. The less crowded features have great effects on the target attribute (class). The experimental results have shown the effectiveness and the robustness of the proposed algorithms.
Freshness Based Cache Updating in Parallel Relay Networks
Venue: arXiv cs.IT 2021
Year: 2021

We consider a system consisting of a server, which receives updates for $N$ files according to independent Poisson processes. The goal of the server is to deliver the latest version of the files to the user through a parallel network of $K$ caches. We consider an update received by the user successful, if the user receives the same file version that is currently prevailing at the server. We derive an analytical expression for information freshness at the user. We observe that freshness for a file increases with increase in consolidation of rates across caches. To solve the multi-cache problem, we first solve the auxiliary problem of a single-cache system. We then rework this auxiliary solution to our parallel-cache network by consolidating rates to single routes as much as possible. This yields an approximate (sub-optimal) solution for the original problem. We provide an upper bound on the gap between the sub-optimal solution and the optimal solution. Numerical results show that the sub-optimal policy closely approximates the optimal policy.
Addressing "Documentation Debt" in Machine Learning Research: A Retrospective Datasheet for BookCorpus
Venue: arXiv cs.CL 2021
Year: 2021

Recent literature has underscored the importance of dataset documentation work for machine learning, and part of this work involves addressing "documentation debt" for datasets that have been used widely but documented sparsely. This paper aims to help address documentation debt for BookCorpus, a popular text dataset for training large language models. Notably, researchers have used BookCorpus to train OpenAI's GPT-N models and Google's BERT models, even though little to no documentation exists about the dataset's motivation, composition, collection process, etc. We offer a preliminary datasheet that provides key context and information about BookCorpus, highlighting several notable deficiencies. In particular, we find evidence that (1) BookCorpus likely violates copyright restrictions for many books, (2) BookCorpus contains thousands of duplicated books, and (3) BookCorpus exhibits significant skews in genre representation. We also find hints of other potential deficiencies that call for future research, including problematic content, potential skews in religious representation, and lopsided author contributions. While more work remains, this initial effort to provide a datasheet for BookCorpus adds to growing literature that urges more careful and systematic documentation for machine learning datasets.
Intelligent interactive technologies for mental health and well-being
Venue: arXiv cs.HC 2021
Year: 2021

Mental healthcare has seen numerous benefits from interactive technologies and artificial intelligence. Various interventions have successfully used intelligent technologies to automate the assessment and evaluation of psychological treatments and mental well-being and functioning. These technologies include different types of robots, video games, and conversational agents. The paper critically analyzes existing solutions with the outlooks for their future. In particular, we: i)give an overview of the technology for mental health, ii) critically analyze the technology against the proposed criteria, and iii) provide the design outlooks for these technologies.
SmartOil: Blockchain and smart contract-based oil supply chain management
Venue: arXiv cs.CR 2021
Year: 2021

The traditional oil supply chain suffers from various shortcomings regarding crude oil extraction, processing, distribution, environmental pollution, and traceability. It offers an only a forward flow of products with almost no security and tracking process. In time, the system will lag behind due to the limitations in quality inspection, fraudulent information, and monopolistic behavior of supply chain entities. Inclusion of counterfeiting products and opaqueness of the system urge renovation in this sector. The recent evolution of Industry 4.0 leads to the alternation in the supply chain introducing the smart supply chain. Technological advancement can now reshape the infrastructure of the supply chain for the future. In this paper, we suggest a conceptual framework utilizing Blockchain and Smart Contract to monitor the overall oil supply chain. Blockchain is a groundbreaking technology to monitor and support the security building of a decentralized type supply chain over a peer-to-peer network. The use of the Internet of Things (IoT), especially sensors, opens broader window to track the global supply chain in real-time. We construct a methodology to support reverse traceability for each participant of the supply chain. The functions and characteristics of Blockchain and Smart Contract are defined. Implementation of Smart Contracts has also been shown with detailed analysis. We further describe the challenges of implementing such a system and validate our framework's adaptability in the real world. The paper concludes with future research scope to mitigate the restrictions of data management and maintenance with advanced working prototypes and agile systems achieving greater traceability and transparency.
A Statistical Model for Melody Reduction
Venue: arXiv cs.SD 2021
Year: 2021

A commonly-cited reason for the poor performance of automatic chord estimation (ACE) systems within music information retrieval (MIR) is that non-chord tones (i.e., notes outside the supporting harmony) contribute to error during the labeling process. Despite the prevalence of machine learning approaches in MIR, there are cases where alternative approaches provide a simpler alternative while allowing for insights into musicological practices. In this project, we present a statistical model for predicting chord tones based on music theory rules. Our model is currently focused on predicting chord tones in classical music, since composition in this style is highly constrained, theoretically making the placement of chord tones highly predictable. Indeed, music theorists have labeling systems for every variety of non-chord tone, primarily classified by the note's metric position and intervals of approach and departure. Using metric position, duration, and melodic intervals as predictors, we build a statistical model for predicting chord tones using the TAVERN dataset. While our probabilistic approach is similar to other efforts in the domain of automatic harmonic analysis, our focus is on melodic reduction rather than predicting harmony. However, we hope to pursue applications for ACE in the future. Finally, we implement our melody reduction model using an existing symbolic visualization tool, to assist with melody reduction and non-chord tone identification for computational musicology researchers and music theorists.
News Headline Grouping as a Challenging NLU Task
Venue: arXiv cs.CL 2021
Year: 2021

Recent progress in Natural Language Understanding (NLU) has seen the latest models outperform human performance on many standard tasks. These impressive results have led the community to introspect on dataset limitations, and iterate on more nuanced challenges. In this paper, we introduce the task of HeadLine Grouping (HLG) and a corresponding dataset (HLGD) consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group. On HLGD, human annotators achieve high performance of around 0.9 F-1, while current state-of-the art Transformer models only reach 0.75 F-1, opening the path for further improvements. We further propose a novel unsupervised Headline Generator Swap model for the task of HeadLine Grouping that achieves within 3 F-1 of the best supervised model. Finally, we analyze high-performing models with consistency tests, and find that models are not consistent in their predictions, revealing modeling limits of current architectures.
Could you give me a hint? Generating inference graphs for defeasible reasoning
Venue: arXiv cs.CL 2021
Year: 2021

Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence. A commonly used method in cognitive science and logic literature is to handcraft argumentation supporting inference graphs. While humans find inference graphs very useful for reasoning, constructing them at scale is difficult. In this paper, we automatically generate such inference graphs through transfer learning from another NLP task that shares the kind of reasoning that inference graphs support. Through automated metrics and human evaluation, we find that our method generates meaningful graphs for the defeasible inference task. Human accuracy on this task improves by 20% by consulting the generated graphs. Our findings open up exciting new research avenues for cases where machine reasoning can help human reasoning. (A dataset of 230,000 influence graphs for each defeasible query is located at: https://tinyurl.com/defeasiblegraphs.)
A Nearly Instance-optimal Differentially Private Mechanism for Conjunctive Queries
Venue: arXiv cs.DB 2021
Year: 2021

Releasing the result size of conjunctive queries and graph pattern queries under differential privacy (DP) has received considerable attention in the literature, but existing solutions do not offer any optimality guarantees. We provide the first DP mechanism for this problem with a fairly strong notion of optimality, which can be considered as a natural relaxation of instance-optimality to a constant.