OpenRaiser/Intern-Atlas huggingface dataset:
Brain-based control of car infotainment
Venue: arXiv cs.HC 2020
Year: 2020

Nowadays, the possibility to run advanced AI on embedded systems allows natural interaction between humans and machines, especially in the automotive field. We present a custom portable EEG-based Brain-Computer Interface (BCI) that exploits Event-Related Potentials (ERPs) induced with an oddball experimental paradigm to control the infotainment menu of a car. A preliminary evaluation of the system was performed on 10 participants in a standard laboratory setting and while driving on a closed private track. The task consisted of repeated presentations of 6 different menu icons in oddball fashion. Subject-specific models were trained with different machine learning approaches on cerebral data from either only laboratory or driving experiments (in-lab and in-car models) or a combination of the two (hybrid model) to classify EEG responses to target and non-target stimuli. All models were tested on the subjects' last in-car sessions that were not used for the training. Analysis of ERPs amplitude showed statistically significant (p < 0.05) differences between the EEG responses associated with target and non-target icons, both in the laboratory and while driving. Classification Accuracy (CA) was above chance level for all subjects in all training configurations, with a deep CNN trained on the hybrid set achieving the highest scores (mean CA = 53 $\pm$ 12 %, with 16 % chance level for the 6-class discrimination). The ranking of the features importance provided by a classical BCI approach suggests an ERP-based discrimination between target and non-target responses. No statistical differences were observed between the CAs for the in-lab and in-car training sets, nor between the EEG responses in these conditions, indicating that the data collected in the standard laboratory setting could be readily used for a real driving application without a noticeable decrease in performance.
Towards Model Checking Real-World Software-Defined Networks (version with appendix)
Venue: arXiv cs.NI 2020
Year: 2020

In software-defined networks (SDN), a controller program is in charge of deploying diverse network functionality across a large number of switches, but this comes at a great risk: deploying buggy controller code could result in network and service disruption and security loopholes. The automatic detection of bugs or, even better, verification of their absence is thus most desirable, yet the size of the network and the complexity of the controller makes this a challenging undertaking. In this paper we propose MOCS, a highly expressive, optimised SDN model that allows capturing subtle real-world bugs, in a reasonable amount of time. This is achieved by (1) analysing the model for possible partial order reductions, (2) statically pre-computing packet equivalence classes and (3) indexing packets and rules that exist in the model. We demonstrate its superiority compared to the state of the art in terms of expressivity, by providing examples of realistic bugs that a prototype implementation of MOCS in UPPAAL caught, and performance/scalability, by running examples on various sizes of network topologies, highlighting the importance of our abstractions and optimisations.
Jupyter notebooks as discovery mechanisms for open science: Citation practices in the astronomy community
Venue: arXiv cs.DL 2020
Year: 2020

Citing data and software is a means to give scholarly credit and to facilitate access to research objects. Citation principles encourage authors to provide full descriptions of objects, with stable links, in their papers. As Jupyter notebooks aggregate data, software, and other objects, they may facilitate or hinder citation, credit, and access to data and software. We report on a study of references to Jupyter notebooks in astronomy over a 5-year period (2014-2018). References increased rapidly, but fewer than half of the references led to Jupyter notebooks that could be located and opened. Jupyter notebooks appear better suited to supporting the research process than to providing access to research objects. We recommend that authors cite individual data and software objects, and that they stabilize any notebooks cited in publications. Publishers should increase the number of citations allowed in papers and employ descriptive metadata-rich citation styles that facilitate credit and discovery.
Memory-efficient training with streaming dimensionality reduction
Venue: arXiv cs.LG 2020
Year: 2020

The movement of large quantities of data during the training of a Deep Neural Network presents immense challenges for machine learning workloads. To minimize this overhead, especially on the movement and calculation of gradient information, we introduce streaming batch principal component analysis as an update algorithm. Streaming batch principal component analysis uses stochastic power iterations to generate a stochastic k-rank approximation of the network gradient. We demonstrate that the low rank updates produced by streaming batch principal component analysis can effectively train convolutional neural networks on a variety of common datasets, with performance comparable to standard mini batch gradient descent. These results can lead to both improvements in the design of application specific integrated circuits for deep learning and in the speed of synchronization of machine learning models trained with data parallelism.
Towards Accurate Predictions and Causal 'What-if' Analyses for Planning and Policy-making: A Case Study in Emergency Medical Services Demand
Venue: arXiv cs.LG 2020
Year: 2020

Emergency Medical Services (EMS) demand load has become a considerable burden for many government authorities, and EMS demand is often an early indicator for stress in communities, a warning sign of emerging problems. In this paper, we introduce Deep Planning and Policy Making Net (DeepPPMNet), a Long Short-Term Memory network based, global forecasting and inference framework to forecast the EMS demand, analyse causal relationships, and perform `what-if' analyses for policy-making across multiple local government areas. Unless traditional univariate forecasting techniques, the proposed method follows the global forecasting methodology, where a model is trained across all the available EMS demand time series to exploit the potential cross-series information available. DeepPPMNet also uses seasonal decomposition techniques, incorporated in two different training paradigms into the framework, to suit various characteristics of the EMS related time series data. We then explore causal relationships using the notion of Granger Causality, where the global forecasting framework enables us to perform `what-if' analyses that could be used for the national policy-making process. We empirically evaluate our method, using a set of EMS datasets related to alcohol, drug use and self-harm in Australia. The proposed framework is able to outperform many state-of-the-art techniques and achieve competitive results in terms of forecasting accuracy. We finally illustrate its use for policy-making in an example regarding alcohol outlet licenses.
Jointly Trained Transformers models for Spoken Language Translation
Venue: arXiv cs.SD 2020
Year: 2020

Conventional spoken language translation (SLT) systems are pipeline based systems, where we have an Automatic Speech Recognition (ASR) system to convert the modality of source from speech to text and a Machine Translation (MT) systems to translate source text to text in target language. Recent progress in the sequence-sequence architectures have reduced the performance gap between the pipeline based SLT systems (cascaded ASR-MT) and End-to-End approaches. Though End-to-End and cascaded ASR-MT systems are reaching to the comparable levels of performances, we can see a large performance gap using the ASR hypothesis and oracle text w.r.t MT models. This performance gap indicates that the MT systems are prone to large performance degradation due to noisy ASR hypothesis as opposed to oracle text transcript. In this work this degradation in the performance is reduced by creating an end to-end differentiable pipeline between the ASR and MT systems. In this work, we train SLT systems with ASR objective as an auxiliary loss and both the networks are connected through the neural hidden representations. This train ing would have an End-to-End differentiable path w.r.t to the final objective function as well as utilize the ASR objective for better performance of the SLT systems. This architecture has improved from BLEU from 36.8 to 44.5. Due to the Multi-task training the model also generates the ASR hypothesis which are used by a pre-trained MT model. Combining the proposed systems with the MT model has increased the BLEU score by 1. All the experiments are reported on English-Portuguese speech translation task using How2 corpus. The final BLEU score is on-par with the best speech translation system on How2 dataset with no additional training data and language model and much less parameters.
Wiener Filter for Short-Reach Fiber-Optic Links
Venue: arXiv cs.IT 2020
Year: 2020

Analytic expressions are derived for the Wiener filter (WF), also known as the linear minimum mean square error (LMMSE) estimator, for an intensity-modulation/direct-detection (IM/DD) short-haul fiber-optic communication system. The link is purely dispersive and the nonlinear square-law detector (SLD) operates at the thermal noise limit. The achievable rates of geometrically shaped PAM constellations are substantially increased by taking the SLD into account as compared to a WF that ignores the SLD.
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
Venue: arXiv cs.CL 2020
Year: 2020

Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. In recent years, LegalAI has drawn increasing attention rapidly from both AI researchers and legal professionals, as LegalAI is beneficial to the legal system for liberating legal professionals from a maze of paperwork. Legal professionals often think about how to solve tasks from rule-based and symbol-based methods, while NLP researchers concentrate more on data-driven and embedding methods. In this paper, we introduce the history, the current state, and the future directions of research in LegalAI. We illustrate the tasks from the perspectives of legal professionals and NLP researchers and show several representative applications in LegalAI. We conduct experiments and provide an in-depth analysis of the advantages and disadvantages of existing works to explore possible future directions. You can find the implementation of our work from https://github.com/thunlp/CLAIM.
A gender equality paradox in academic publishing: Countries with a higher proportion of female first-authored journal articles have larger first author gender disparities between fields
Venue: arXiv cs.DL 2020
Year: 2020

Current attempts to address the shortfall of female researchers in Science, Technology, Engineering and Mathematics (STEM) have not yet succeeded despite other academic subjects having female majorities. This article investigates the extent to which gender disparities are subject-wide or nation-specific by a first author gender comparison of 30 million articles from all 27 Scopus broad fields within the 31 countries with the most Scopus-indexed articles 2014-18. The results show overall and geocultural patterns as well as individual national differences. Almost half of the subjects were always more male (7; e.g., Mathematics) or always more female (6; e.g., Immunology & Microbiology) than the national average. A strong overall trend (Spearman correlation 0.546) is for countries with a higher proportion of female first-authored research to also have larger differences in gender disparities between fields (correlation 0.314 for gender ratios). This confirms the international gender equality paradox previously found for degree subject choices: increased gender equality overall associates with moderately greater gender differentiation between subjects. This is consistent with previous USA-based claims that gender differences in academic careers are partly due to (socially constrained) gender differences in personal preferences. Radical solutions may therefore be needed for some STEM subjects to overcome gender disparities.
Revisiting Sequence-to-Sequence Video Object Segmentation with Multi-Task Loss and Skip-Memory
Venue: arXiv cs.CV 2020
Year: 2020

Video Object Segmentation (VOS) is an active research area of the visual domain. One of its fundamental sub-tasks is semi-supervised / one-shot learning: given only the segmentation mask for the first frame, the task is to provide pixel-accurate masks for the object over the rest of the sequence. Despite much progress in the last years, we noticed that many of the existing approaches lose objects in longer sequences, especially when the object is small or briefly occluded. In this work, we build upon a sequence-to-sequence approach that employs an encoder-decoder architecture together with a memory module for exploiting the sequential data. We further improve this approach by proposing a model that manipulates multi-scale spatio-temporal information using memory-equipped skip connections. Furthermore, we incorporate an auxiliary task based on distance classification which greatly enhances the quality of edges in segmentation masks. We compare our approach to the state of the art and show considerable improvement in the contour accuracy metric and the overall segmentation accuracy.
Automated Abstraction of Operation Processes from Unstructured Text for Simulation Modeling
Venue: arXiv cs.IR 2020
Year: 2020

Abstraction of operation processes is a fundamental step for simulation modeling. To reliably abstract an operation process, modelers rely on text information to study and understand details of operations. Aiming at reducing modelers' interpretation load and ensuring the reliability of the abstracted information, this research proposes a systematic methodology to automate the abstraction of operation processes. The methodology applies rule-based information extraction to automatically extract operation process-related information from unstructured text and creates graphical representations of operation processes using the extracted information. To demonstrate the applicability and feasibility of the proposed methodology, a text description of an earthmoving operation is used to create its corresponding graphical representation. Overall, this research enhances the state-of-the-art simulation modeling through achieving automated abstraction of operation processes, which largely reduces modelers' interpretation load and ensures the reliability of the abstracted operation processes.
Privacy Preserving Distributed Energy Trading
Venue: arXiv cs.CR 2020
Year: 2020

The smart grid incentivizes distributed agents with local generation (e.g., smart homes, and microgrids) to establish multi-agent systems for enhanced reliability and energy consumption efficiency. Distributed energy trading has emerged as one of the most important multi-agent systems on the power grid by enabling agents to sell their excessive local energy to each other or back to the grid. However, it requests all the agents to disclose their sensitive data (e.g., each agent's fine-grained local generation and demand load). In this paper, to the best of our knowledge, we propose the first privacy preserving distributed energy trading framework, Private Energy Market (PEM), in which all the agents privately compute an optimal price for their trading (ensured by a Nash Equilibrium), and allocate pairwise energy trading amounts without disclosing sensitive data (via novel cryptographic protocols). Specifically, we model the trading problem as a non-cooperative Stackelberg game for all the agents (i.e., buyers and sellers) to determine the optimal price, and then derive the pairwise trading amounts. Our PEM framework can privately perform all the computations among all the agents without a trusted third party. We prove the privacy, individual rationality, and incentive compatibility for the PEM framework. Finally, we conduct experiments on real datasets to validate the effectiveness and efficiency of the PEM.
Online MinCut: Competitive and Regret Analysis
Venue: arXiv cs.DS 2020
Year: 2020

In this paper we study the mincut problem in the online setting. We consider two distinct models: A) competitive analysis and B) regret analysis. In the competitive setting we consider the vertex arrival model; whenever a new vertex arrives it's neighborhood with respect to the set of known vertices is revealed. An online algorithm must make an irrevocable decision to determine the side of the cut that the vertex must belong to in order to minimize the size of the final cut. Various models are considered. 1) For classical and advice models we give tight bounds on the competitive ratio of deterministic algorithms. 2) Next we consider few semi-adversarial inputs: random order of arrival with adversarially generated and sparse graphs. 3) Lastly we derive some structural properties of \mc-type problems with respect to greedy strategies. Finally we consider a non-stationary regret setting with a variational budget $V_T$ and give tights bounds on the regret function. Specifically, we show that if $V_T$ is sublinear in $T$ (number of rounds) then there is a deterministic algorithm achieving a sublinear regret bound ($O(V_T)$). Further, this is optimal, even if randomization is allowed.
Hierarchical Multi Task Learning with Subword Contextual Embeddings for Languages with Rich Morphology
Venue: arXiv cs.CL 2020
Year: 2020

Morphological information is important for many sequence labeling tasks in Natural Language Processing (NLP). Yet, existing approaches rely heavily on manual annotations or external software to capture this information. In this study, we propose using subword contextual embeddings to capture the morphological information for languages with rich morphology. In addition, we incorporate these embeddings in a hierarchical multi-task setting which is not employed before, to the best of our knowledge. Evaluated on Dependency Parsing (DEP) and Named Entity Recognition (NER) tasks, which are shown to benefit greatly from morphological information, our final model outperforms previous state-of-the-art models on both tasks for the Turkish language. Besides, we show a net improvement of 18.86% and 4.61% F-1 over the previously proposed multi-task learner in the same setting for the DEP and the NER tasks, respectively. Empirical results for five different MTL settings show that incorporating subword contextual embeddings brings significant improvements for both tasks. In addition, we observed that multi-task learning consistently improves the performance of the DEP component.
A novel encryption algorithm using multiple semifield S-boxes based on permutation of symmetric group
Venue: arXiv cs.CR 2020
Year: 2020

With the tremendous benefits of internet and advanced communications, there is a serious threat from the data security perspective. There is a need of secure and robust encryption algorithm that can be implemented on each and diverse software and hardware platforms. Also, in block symmetric encryption algorithms, substitution boxes are the most vital part. In this paper, we investigate semifield substitution boxes using permutation of symmetric group on a set of size 8 S_8 and establish an effective procedure for generating S_8 semifield substitution boxes having same algebraic properties. Further, the strength analysis of the generated substitution boxes is carried out using the well-known standards namely bijectivity, nonlinearity, strict avalanche criterion, bit independence criterion, XOR table and differential invariant. Based on the analysis results, it is shown that the cryptographic strength of generated substitution boxes is on par with the best known $8\times 8$ substitution boxes. As application, an encryption algorithm is proposed that can be employed to strengthen any kind of secure communication. The presented algorithm is mainly based on the Shannon idea of (S-P) network where the process of substitution is performed by the proposed S_8 semifield substitution boxes and permutation operation is performed by the binary cyclic shift of substitution box transformed data. In addition, the proposed encryption algorithm utilizes two different chaotic maps. In order to ensure the appropriate utilization of these chaotic maps, we carry out in-depth analyses of their behavior in the context of secure communication and apply the pseudo-random sequences of chaotic maps in the proposed image encryption algorithm accordingly. The statistical and simulation results imply that our encryption scheme is secure against different attacks and can resist linear and differential cryptanalysis.
Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reports
Venue: arXiv cs.CL 2020
Year: 2020

Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. Automatically writing reports for these images can considerably lighten the workload of radiologists for summarizing descriptive findings and conclusive impressions. The complex structures between and within sections of the reports pose a great challenge to the automatic report generation. Specifically, the section Impression is a diagnostic summarization over the section Findings; and the appearance of normality dominates each section over that of abnormality. Existing studies rarely explore and consider this fundamental structure information. In this work, we propose a novel framework that exploits the structure information between and within report sections for generating CXR imaging reports. First, we propose a two-stage strategy that explicitly models the relationship between Findings and Impression. Second, we design a novel cooperative multi-agent system that implicitly captures the imbalanced distribution between abnormality and normality. Experiments on two CXR report datasets show that our method achieves state-of-the-art performance in terms of various evaluation metrics. Our results expose that the proposed approach is able to generate high-quality medical reports through integrating the structure information.
An Extension of LIME with Improvement of Interpretability and Fidelity
Venue: arXiv cs.LG 2020
Year: 2020

While deep learning makes significant achievements in Artificial Intelligence (AI), the lack of transparency has limited its broad application in various vertical domains. Explainability is not only a gateway between AI and real world, but also a powerful feature to detect flaw of the models and bias of the data. Local Interpretable Model-agnostic Explanation (LIME) is a widely-accepted technique that explains the prediction of any classifier faithfully by learning an interpretable model locally around the predicted instance. As an extension of LIME, this paper proposes an high-interpretability and high-fidelity local explanation method, known as Local Explanation using feature Dependency Sampling and Nonlinear Approximation (LEDSNA). Given an instance being explained, LEDSNA enhances interpretability by feature sampling with intrinsic dependency. Besides, LEDSNA improves the local explanation fidelity by approximating nonlinear boundary of local decision. We evaluate our method with classification tasks in both image domain and text domain. Experiments show that LEDSNA's explanation of the back-box model achieves much better performance than original LIME in terms of interpretability and fidelity.