OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic Model Tree for Interpretable Data Stream Learning
Venue: arXiv cs.LG 2022
Year: 2022

Data streams are ubiquitous in modern business and society. In practice, data streams may evolve over time and cannot be stored indefinitely. Effective and transparent machine learning on data streams is thus often challenging. Hoeffding Trees have emerged as a state-of-the art for online predictive modelling. They are easy to train and provide meaningful convergence guarantees under a stationary process. Yet, at the same time, Hoeffding Trees often require heuristic and costly extensions to adjust to distributional change, which may considerably impair their interpretability. In this work, we revisit Model Trees for machine learning in evolving data streams. Model Trees are able to maintain more flexible and locally robust representations of the active data concept, making them a natural fit for data stream applications. Our novel framework, called Dynamic Model Tree, satisfies desirable consistency and minimality properties. In experiments with synthetic and real-world tabular streaming data sets, we show that the proposed framework can drastically reduce the number of splits required by existing incremental decision trees. At the same time, our framework often outperforms state-of-the-art models in terms of predictive quality -- especially when concept drift is involved. Dynamic Model Trees are thus a powerful online learning framework that contributes to more lightweight and interpretable machine learning in data streams.
Auto-MLM: Improved Contrastive Learning for Self-supervised Multi-lingual Knowledge Retrieval
Venue: arXiv cs.CL 2022
Year: 2022

Contrastive learning (CL) has become a ubiquitous approach for several natural language processing (NLP) downstream tasks, especially for question answering (QA). However, the major challenge, how to efficiently train the knowledge retrieval model in an unsupervised manner, is still unresolved. Recently the commonly used methods are composed of CL and masked language model (MLM). Unexpectedly, MLM ignores the sentence-level training, and CL also neglects extraction of the internal info from the query. To optimize the CL hardly obtain internal information from the original query, we introduce a joint training method by combining CL and Auto-MLM for self-supervised multi-lingual knowledge retrieval. First, we acquire the fixed dimensional sentence vector. Then, mask some words among the original sentences with random strategy. Finally, we generate a new token representation for predicting the masked tokens. Experimental results show that our proposed approach consistently outperforms all the previous SOTA methods on both AliExpress $\&$ LAZADA service corpus and openly available corpora in 8 languages.
End to End Lip Synchronization with a Temporal AutoEncoder
Venue: arXiv cs.CV 2022
Year: 2022

We study the problem of syncing the lip movement in a video with the audio stream. Our solution finds an optimal alignment using a dual-domain recurrent neural network that is trained on synthetic data we generate by dropping and duplicating video frames. Once the alignment is found, we modify the video in order to sync the two sources. Our method is shown to greatly outperform the literature methods on a variety of existing and new benchmarks. As an application, we demonstrate our ability to robustly align text-to-speech generated audio with an existing video stream. Our code and samples are available at https://github.com/itsyoavshalev/End-to-End-Lip-Synchronization-with-a-Temporal-AutoEncoder.
Research topic trend prediction of scientific papers based on spatial enhancement and dynamic graph convolution network
Venue: arXiv cs.IR 2022
Year: 2022

In recent years, with the increase of social investment in scientific research, the number of research results in various fields has increased significantly. Accurately and effectively predicting the trends of future research topics can help researchers discover future research hotspots. However, due to the increasingly close correlation between various research themes, there is a certain dependency relationship between a large number of research themes. Viewing a single research theme in isolation and using traditional sequence problem processing methods cannot effectively explore the spatial dependencies between these research themes. To simultaneously capture the spatial dependencies and temporal changes between research topics, we propose a deep neural network-based research topic hotness prediction algorithm, a spatiotemporal convolutional network model. Our model combines a graph convolutional neural network (GCN) and Temporal Convolutional Network (TCN), specifically, GCNs are used to learn the spatial dependencies of research topics a and use space dependence to strengthen spatial characteristics. TCN is used to learn the dynamics of research topics' trends. Optimization is based on the calculation of weighted losses based on time distance. Compared with the current mainstream sequence prediction models and similar spatiotemporal models on the paper datasets, experiments show that, in research topic prediction tasks, our model can effectively capture spatiotemporal relationships and the predictions outperform state-of-art baselines.
Characterizing YouTube and BitChute Content and Mobilizers During U.S. Election Fraud Discussions on Twitter
Venue: arXiv cs.SI 2022
Year: 2022

In this study, we characterize the cross-platform mobilization of YouTube and BitChute videos on Twitter during the 2020 U.S. Election fraud discussions. Specifically, we extend the VoterFraud2020 dataset to describe the prevalence of content supplied by both platforms, the mobilizers of that content, the suppliers of that content, and the content itself. We find that while BitChute videos promoting election fraud claims were linked to and engaged with in the Twitter discussion, they played a relatively small role compared to YouTube videos promoting fraud claims. This core finding points to the continued need for proactive, consistent, and collaborative content moderation solutions rather than the reactive and inconsistent solutions currently being used. Additionally, we find that cross-platform disinformation spread from video platforms was not prominently from bot accounts or political elites, but rather average Twitter users. This finding supports past work arguing that research on disinformation should move beyond a focus on bots and trolls to a focus on participatory disinformation spread.
Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control
Venue: arXiv cs.AI 2022
Year: 2022

A one-step two-critic deep reinforcement learning (OSTC-DRL) approach for inverter-based volt-var control (IB-VVC) in active distribution networks is proposed in this paper. Firstly, considering IB-VVC can be formulated as a single-period optimization problem, we formulate the IB-VVC as a one-step Markov decision process rather than the standard Markov decision process, which simplifies the DRL learning task. Then we design the one-step actor-critic DRL scheme which is a simplified version of recent DRL algorithms, and it avoids the issue of Q value overestimation successfully. Furthermore, considering two objectives of VVC: minimizing power loss and eliminating voltage violation, we utilize two critics to approximate the rewards of two objectives separately. It simplifies the approximation tasks of each critic, and avoids the interaction effect between two objectives in the learning process of critic. The OSTC-DRL approach integrates the one-step actor-critic DRL scheme and the two-critic technology. Based on the OSTC-DRL, we design two centralized DRL algorithms. Further, we extend the OSTC-DRL to multi-agent OSTC-DRL for decentralized IB-VVC and design two multi-agent DRL algorithms. Simulations demonstrate that the proposed OSTC-DRL has a faster convergence rate and a better control performance, and the multi-agent OSTC-DRL works well for decentralized IB-VVC problems.
An Algebraic Framework for Structured Epidemic Modeling
Venue: arXiv cs.PL 2022
Year: 2022

Pandemic management requires that scientists rapidly formulate and analyze epidemiological models in order to forecast the spread of disease and the effects of mitigation strategies. Scientists must modify existing models and create novel ones in light of new biological data and policy changes such as social distancing and vaccination. Traditional scientific modeling workflows detach the structure of a model -- its submodels and their interactions -- from its implementation in software. Consequently, incorporating local changes to model components may require global edits to the code-base through a manual, time-intensive, and error-prone process. We propose a compositional modeling framework that uses high-level algebraic structures to capture domain-specific scientific knowledge and bridge the gap between how scientists think about models and the code that implements them. These algebraic structures, grounded in applied category theory, simplify and expedite modeling tasks such as model specification, stratification, analysis, and calibration. With their structure made explicit, models also become easier to communicate, criticize, and refine in light of stakeholder feedback.
Vulnerability Detection in Open Source Software: An Introduction
Venue: arXiv cs.CR 2022
Year: 2022

This paper is an introductory discussion on the cause of open source software vulnerabilities, their importance in the cybersecurity ecosystem, and a selection of detection methods. A recent application security report showed 44% of applications contain critical vulnerabilities in an open source component, a concerning proportion. Most companies do not have a reliable way of being directly and promptly notified when zero-day vulnerabilities are found and then when patches are made available. This means attack vectors in open source exist longer than necessary. Conventional approaches to vulnerability detection are outlined alongside some newer research trends. A conclusion is made that it may not be possible to entirely replace expert human inspection of open source software, although it can be effectively augmented with techniques such as machine learning, IDE plug-ins and repository linking to make implementation and review less time intensive. Underpinning any technological advances should be better knowledge at the human level. Development teams need trained, coached and improved so they can implement open source more securely, know what vulnerabilities to look for and how to handle them. It is the use of this blended approach to detection which is key.
Explicitising The Implicit Intrepretability of Deep Neural Networks Via Duality
Venue: arXiv cs.LG 2022
Year: 2022

Recent work by Lakshminarayanan and Singh [2020] provided a dual view for fully connected deep neural networks (DNNs) with rectified linear units (ReLU). It was shown that (i) the information in the gates is analytically characterised by a kernel called the neural path kernel (NPK) and (ii) most critical information is learnt in the gates, in that, given the learnt gates, the weights can be retrained from scratch without significant loss in performance. Using the dual view, in this paper, we rethink the conventional interpretations of DNNs thereby explicitsing the implicit interpretability of DNNs. Towards this, we first show new theoretical properties namely rotational invariance and ensemble structure of the NPK in the presence of convolutional layers and skip connections respectively. Our theory leads to two surprising empirical results that challenge conventional wisdom: (i) the weights can be trained even with a constant 1 input, (ii) the gating masks can be shuffled, without any significant loss in performance. These results motivate a novel class of networks which we call deep linearly gated networks (DLGNs). DLGNs using the phenomenon of dual lifting pave way to more direct and simpler interpretation of DNNs as opposed to conventional interpretations. We show via extensive experiments on CIFAR-10 and CIFAR-100 that these DLGNs lead to much better interpretability-accuracy tradeoff.
Remember to correct the bias when using deep learning for regression!
Venue: arXiv cs.LG 2022
Year: 2022

When training deep learning models for least-squares regression, we cannot expect that the training error residuals of the final model, selected after a fixed training time or based on performance on a hold-out data set, sum to zero. This can introduce a systematic error that accumulates if we are interested in the total aggregated performance over many data points. We suggest to adjust the bias of the machine learning model after training as a default postprocessing step, which efficiently solves the problem. The severeness of the error accumulation and the effectiveness of the bias correction is demonstrated in exemplary experiments.
Identification of diffracted vortex beams at different propagation distances using deep learning
Venue: arXiv cs.LG 2022
Year: 2022

Orbital angular momentum of light is regarded as a valuable resource in quantum technology, especially in quantum communication and quantum sensing and ranging. However, the OAM state of light is susceptible to undesirable experimental conditions such as propagation distance and phase distortions, which hinders the potential for the realistic implementation of relevant technologies. In this article, we exploit an enhanced deep learning neural network to identify different OAM modes of light at multiple propagation distances with phase distortions. Specifically, our trained deep learning neural network can efficiently identify the vortex beam's topological charge and propagation distance with 97% accuracy. Our technique has important implications for OAM based communication and sensing protocols.
Graph Refinement for Coreference Resolution
Venue: arXiv cs.CL 2022
Year: 2022

The state-of-the-art models for coreference resolution are based on independent mention pair-wise decisions. We propose a modelling approach that learns coreference at the document-level and takes global decisions. For this purpose, we model coreference links in a graph structure where the nodes are tokens in the text, and the edges represent the relationship between them. Our model predicts the graph in a non-autoregressive manner, then iteratively refines it based on previous predictions, allowing global dependencies between decisions. The experimental results show improvements over various baselines, reinforcing the hypothesis that document-level information improves conference resolution.
Calibrating constitutive models with full-field data via physics informed neural networks
Venue: arXiv cs.LG 2022
Year: 2022

The calibration of solid constitutive models with full-field experimental data is a long-standing challenge, especially in materials which undergo large deformation. In this paper, we propose a physics-informed deep-learning framework for the discovery of constitutive model parameterizations given full-field displacement data and global force-displacement data. Contrary to the majority of recent literature in this field, we work with the weak form of the governing equations rather than the strong form to impose physical constraints upon the neural network predictions. The approach presented in this paper is computationally efficient, suitable for irregular geometric domains, and readily ingests displacement data without the need for interpolation onto a computational grid. A selection of canonical hyperelastic materials models suitable for different material classes is considered including the Neo-Hookean, Gent, and Blatz-Ko constitutive models as exemplars for general hyperelastic behavior, polymer behavior with lock-up, and compressible foam behavior respectively. We demonstrate that physics informed machine learning is an enabling technology and may shift the paradigm of how full-field experimental data is utilized to calibrate constitutive models under finite deformations.
NGSO Constellation Design for Global Connectivity
Venue: arXiv cs.NI 2022
Year: 2022

Non-geostationary orbit (NGSO) satellite constellations represent a cornerstone in the NewSpace paradigm and thus have become one of the hottest topics for the industry, academia, but also for national space agencies and regulators. For instance, numerous companies worldwide, including Starlink, OneWeb, Kepler, SPUTNIX, and Amazon have started or will soon start to deploy their own NGSO constellations, which aim to provide either broadband or IoT services. One of the major drivers for such a high interest on NGSO constellations is that, with an appropriate design, they are capable of providing global coverage and connectivity.
Autonomous Navigation of AGVs in Unknown Cluttered Environments: log-MPPI Control Strategy
Venue: arXiv cs.RO 2022
Year: 2022

Sampling-based model predictive control (MPC) optimization methods, such as Model Predictive Path Integral (MPPI), have recently shown promising results in various robotic tasks. However, it might produce an infeasible trajectory when the distributions of all sampled trajectories are concentrated within high-cost even infeasible regions. In this study, we propose a new method called log-MPPI equipped with a more effective trajectory sampling distribution policy which significantly improves the trajectory feasibility in terms of satisfying system constraints. The key point is to draw the trajectory samples from the normal log-normal (NLN) mixture distribution, rather than from Gaussian distribution. Furthermore, this work presents a method for collision-free navigation in unknown cluttered environments by incorporating the 2D occupancy grid map into the optimization problem of the sampling-based MPC algorithm. We first validate the efficiency and robustness of our proposed control strategy through extensive simulations of 2D autonomous navigation in different types of cluttered environments as well as the cartpole swing-up task. We further demonstrate, through real-world experiments, the applicability of log-MPPI for performing a 2D grid-based collision-free navigation in an unknown cluttered environment, showing its superiority to be utilized with the local costmap without adding additional complexity to the optimization problem. A video demonstrating the real-world and simulation results is available at https://youtu.be/_uGWQEFJSN0.
End-to-end Document Recognition and Understanding with Dessurt
Venue: arXiv cs.CV 2022
Year: 2022

We introduce Dessurt, a relatively simple document understanding transformer capable of being fine-tuned on a greater variety of document tasks than prior methods. It receives a document image and task string as input and generates arbitrary text autoregressively as output. Because Dessurt is an end-to-end architecture that performs text recognition in addition to the document understanding, it does not require an external recognition model as prior methods do. Dessurt is a more flexible model than prior methods and is able to handle a variety of document domains and tasks. We show that this model is effective at 9 different dataset-task combinations.
Quantifying the presence/absence of meso-scale structures in networks
Venue: arXiv cs.SI 2022
Year: 2022

Meso-scale structures are network features where nodes with similar properties are grouped together instead of being treated individually. In this work, we provide formal and mathematical definitions of three such structures: assortative communities, disassortative communities and core-periphery. We then leverage these definitions and a Bayesian framework to quantify the presence/absence of each structure in a network. This allows for probabilistic statements about the network structure as well as uncertainty estimates of the group labels and edge probabilities. The method is applied to real-world networks, yielding provocative results about well-known network data sets.
Controllable Augmentations for Video Representation Learning
Venue: arXiv cs.CV 2022
Year: 2022

This paper focuses on self-supervised video representation learning. Most existing approaches follow the contrastive learning pipeline to construct positive and negative pairs by sampling different clips. However, this formulation tends to bias to static background and have difficulty establishing global temporal structures. The major reason is that the positive pairs, i.e., different clips sampled from the same video, have limited temporal receptive field, and usually share similar background but differ in motions. To address these problems, we propose a framework to jointly utilize local clips and global videos to learn from detailed region-level correspondence as well as general long-term temporal relations. Based on a set of controllable augmentations, we achieve accurate appearance and motion pattern alignment through soft spatio-temporal region contrast. Our formulation is able to avoid the low-level redundancy shortcut by mutual information minimization to improve the generalization. We also introduce local-global temporal order dependency to further bridge the gap between clip-level and video-level representations for robust temporal modeling. Extensive experiments demonstrate that our framework is superior on three video benchmarks in action recognition and video retrieval, capturing more accurate temporal dynamics.
Robust Beamforming for Localization-Aided Millimeter Wave Communication Systems
Venue: arXiv cs.IT 2022
Year: 2022

In this letter, we investigate a robust beamforming problem for localization-aided millimeter wave (mmWave) communication systems. To handle this problem, we propose a novel restriction and relaxation (R&R) method. The proposed R&R method aims at minimizing the total transmit power while the positioning error follows a Gaussian distribution. Specifically, in the restriction phase of R&R, the probabilistic constraint is transformed into the deterministic form by using the Bernsteintype inequality. In the relaxation phase of R&R, the non-convex optimization problem is reformulated into a convex semidefinite program (SDP) by using semidefinite relaxation (SDR) and firstorder Taylor expansion methods. To the best of our knowledge, we first consider the impact of the distribution of the positioning error on the channel state information (CSI), which further influences the data rate. Numerical results present the trade-off of the beamforming between the communication and positioning.