OpenRaiser/Intern-Atlas huggingface dataset:
SimOn: A Simple Framework for Online Temporal Action Localization
Venue: arXiv cs.CV 2022
Year: 2022

Online Temporal Action Localization (On-TAL) aims to immediately provide action instances from untrimmed streaming videos. The model is not allowed to utilize future frames and any processing techniques to modify past predictions, making On-TAL much more challenging. In this paper, we propose a simple yet effective framework, termed SimOn, that learns to predict action instances using the popular Transformer architecture in an end-to-end manner. Specifically, the model takes the current frame feature as a query and a set of past context information as keys and values of the Transformer. Different from the prior work that uses a set of outputs of the model as past contexts, we leverage the past visual context and the learnable context embedding for the current query. Experimental results on the THUMOS14 and ActivityNet1.3 datasets show that our model remarkably outperforms the previous methods, achieving a new state-of-the-art On-TAL performance. In addition, the evaluation for Online Detection of Action Start (ODAS) demonstrates the effectiveness and robustness of our method in the online setting. The code is available at https://github.com/TuanTNG/SimOn
Safety-Critical Optimal Control for Robotic Manipulators in A Cluttered Environment
Venue: arXiv cs.RO 2022
Year: 2022

Designing safety-critical control for robotic manipulators is challenging, especially in a cluttered environment. First, the actual trajectory of a manipulator might deviate from the planned one due to the complex collision environments and non-trivial dynamics, leading to collision; Second, the feasible space for the manipulator is hard to obtain since the explicit distance functions between collision meshes are unknown. By analyzing the relationship between the safe set and the controlled invariant set, this paper proposes a data-driven control barrier function (CBF) construction method, which extracts CBF from distance samples. Specifically, the CBF guarantees the controlled invariant property for considering the system dynamics. The data-driven method samples the distance function and determines the safe set. Then, the CBF is synthesized based on the safe set by a scenario-based sum of square (SOS) program. Unlike most existing linearization based approaches, our method reserves the volume of the feasible space for planning without approximation, which helps find a solution in a cluttered environment. The control law is obtained by solving a CBF-based quadratic program in real time, which works as a safe filter for the desired planning-based controller. Moreover, our method guarantees safety with the proven probabilistic result. Our method is validated on a 7-DOF manipulator in both real and virtual cluttered environments. The experiments show that the manipulator is able to execute tasks where the clearance between obstacles is in millimeters.
Hyper-GST: Predict Metro Passenger Flow Incorporating GraphSAGE, Hypergraph, Social-meaningful Edge Weights and Temporal Exploitation
Venue: arXiv cs.LG 2022
Year: 2022

Predicting metro passenger flow precisely is of great importance for dynamic traffic planning. Deep learning algorithms have been widely applied due to their robust performance in modelling non-linear systems. However, traditional deep learning algorithms completely discard the inherent graph structure within the metro system. Graph-based deep learning algorithms could utilise the graph structure but raise a few challenges, such as how to determine the weights of the edges and the shallow receptive field caused by the over-smoothing issue. To further improve these challenges, this study proposes a model based on GraphSAGE with an edge weights learner applied. The edge weights learner utilises socially meaningful features to generate edge weights. Hypergraph and temporal exploitation modules are also constructed as add-ons for better performance. A comparison study is conducted on the proposed algorithm and other state-of-art graph neural networks, where the proposed algorithm could improve the performance.
ParGAN: Learning Real Parametrizable Transformations
Venue: arXiv cs.CV 2022
Year: 2022

Current methods for image-to-image translation produce compelling results, however, the applied transformation is difficult to control, since existing mechanisms are often limited and non-intuitive. We propose ParGAN, a generalization of the cycle-consistent GAN framework to learn image transformations with simple and intuitive controls. The proposed generator takes as input both an image and a parametrization of the transformation. We train this network to preserve the content of the input image while ensuring that the result is consistent with the given parametrization. Our approach does not require paired data and can learn transformations across several tasks and datasets. We show how, with disjoint image domains with no annotated parametrization, our framework can create smooth interpolations as well as learn multiple transformations simultaneously.
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes
Venue: arXiv cs.CL 2022
Year: 2022

Providing better language tools for low-resource and endangered languages is imperative for equitable growth. Recent progress with massively multilingual pretrained models has proven surprisingly effective at performing zero-shot transfer to a wide variety of languages. However, this transfer is not universal, with many languages not currently understood by multilingual approaches. It is estimated that only 72 languages possess a "small set of labeled datasets" on which we could test a model's performance, the vast majority of languages not having the resources available to simply evaluate performances on. In this work, we attempt to clarify which languages do and do not currently benefit from such transfer. To that end, we develop a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model. Our approach is derived from the hypothesis that if a model's understanding is insensitive to perturbations to text in a language, it is likely to have a limited understanding of that language. We construct a cross-lingual sentence similarity task to evaluate our approach empirically on 350, primarily low-resource, languages.
What is Wrong with Language Models that Can Not Tell a Story?
Venue: arXiv cs.CL 2022
Year: 2022

This paper argues that a deeper understanding of narrative and the successful generation of longer subjectively interesting texts is a vital bottleneck that hinders the progress in modern Natural Language Processing (NLP) and may even be in the whole field of Artificial Intelligence. We demonstrate that there are no adequate datasets, evaluation methods, and even operational concepts that could be used to start working on narrative processing.
A Note on the Compatibility of Different Robust Program Equilibria of the Prisoner's Dilemma
Venue: arXiv cs.GT 2022
Year: 2022

We study a program game version of the Prisoner's Dilemma, i.e., a two-player game in which each player submits a computer program, the programs are given read access to each other's source code and then choose whether to cooperate or defect. Prior work has introduced various programs that form cooperative equilibria against themselves in this game. For example, the $ε$-grounded Fair Bot cooperates with probability $ε$ and with the remaining probability runs its opponent's program and copies its action. If both players submit this program, then this is a Nash equilibrium in which both players cooperate. Others have proposed cooperative equilibria based on proof-based Fair Bots, which cooperate if they can prove that the opponent cooperates (and defect otherwise). We here show that these different programs are compatible with each other. For example, if one player submits $ε$-grounded Fair Bot and the other submits a proof-based Fair Bot, then this is also a cooperative equilibrium of the program game version of the Prisoner's Dilemma.
Galois Hull Dimensions of Gabidulin Codes
Venue: arXiv cs.IT 2022
Year: 2022

For a prime power $q$, an integer $m$ and $0\leq e\leq m-1$ we study the $e$-Galois hull dimension of Gabidulin codes $G_k(\boldsymbolα)$ of length $m$ and dimension $k$ over $\mathbb{F}_{q^m}$. Using a self-dual basis $\boldsymbolα$ of $\mathbb{F}_{q^m}$ over $\mathbb{F}_q$, we first explicitly compute the hull dimension of $G_k(\boldsymbolα)$. Then a necessary and sufficient condition of $G_k(\boldsymbolα)$ to be linear complementary dual (LCD), self-orthogonal and self-dual will be provided. We prove the existence of $e$-Galois (where $e=\frac{m}{2}$) self-dual Gabidulin codes of length $m$ for even $q$, which is in contrast to the known fact that Euclidean self-dual Gabidulin codes do not exist for even $q$. As an application, we construct two classes of entangled-assisted quantum error-correcting codes (EAQECCs) whose parameters have more flexibility compared to known codes in this context.
Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twitter
Venue: arXiv cs.CL 2022
Year: 2022

Misinformation spread over social media has become an undeniable infodemic. However, not all spreading claims are made equal. If propagated, some claims can be destructive, not only on the individual level, but to organizations and even countries. Detecting claims that should be prioritized for fact-checking is considered the first step to fight against spread of fake news. With training data limited to a handful of languages, developing supervised models to tackle the problem over lower-resource languages is currently infeasible. Therefore, our work aims to investigate whether we can use existing datasets to train models for predicting worthiness of verification of claims in tweets in other languages. We present a systematic comparative study of six approaches for cross-lingual check-worthiness estimation across pairs of five diverse languages with the help of Multilingual BERT (mBERT) model. We run our experiments using a state-of-the-art multilingual Twitter dataset. Our results show that for some language pairs, zero-shot cross-lingual transfer is possible and can perform as good as monolingual models that are trained on the target language. We also show that in some languages, this approach outperforms (or at least is comparable to) state-of-the-art models.
Large Language Models with Controllable Working Memory
Venue: arXiv cs.CL 2022
Year: 2022

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities. Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored. As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge. This enables model predictions to be grounded in the context, which can then be used to update or correct specific model predictions without frequent retraining. By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge. In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs. We demonstrate that state-of-the-art T5 and PaLM (both pretrained and finetuned) could exhibit poor controllability and robustness, which do not scale with increasing model size. As a solution, we propose a novel method - Knowledge Aware FineTuning (KAFT) - to strengthen both controllability and robustness by incorporating counterfactual and irrelevant contexts to standard supervised datasets. Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.
ChromoSkein: Untangling Three-Dimensional Chromatin Fiber With a Web-Based Visualization Framework
Venue: arXiv cs.HC 2022
Year: 2022

We present ChromoSkein, a web-based tool for visualizing three-dimensional chromatin models. The spatial organization of chromatin is essential to its function. Experimental methods, namely Hi-C, reveal the spatial conformation but output a 2D matrix representation. Biologists leverage simulation to bring this information back to 3D, assembling a 3D chromatin shape prediction using the 2D matrices as constraints. Our overview of existing chromatin visualization software shows that the available tools limit the utility of 3D through ineffective shading and a lack of advanced interactions. We designed ChromoSkein to encourage analytical work directly with the 3D representation. Our tool features a 3D view that supports understanding the shape of the highly tangled chromatin fiber and the spatial relationships of its parts. Users can explore and filter the 3D model using two interactions. First, they can manage occlusion both by toggling the visibility of semantic parts and by adding cutting planes. Second, they can segment the model through the creation of custom selections. To complement the 3D view, we link the spatial representation with non-spatial genomic data, such as 2D Hi-C maps and 1D genomic signals. We demonstrate the utility of ChromoSkein in two exemplary use cases that examine functional genomic loci in the spatial context of chromosomes and the whole genome.
Towards Reasoning-Aware Explainable VQA
Venue: arXiv cs.CL 2022
Year: 2022

The domain of joint vision-language understanding, especially in the context of reasoning in Visual Question Answering (VQA) models, has garnered significant attention in the recent past. While most of the existing VQA models focus on improving the accuracy of VQA, the way models arrive at an answer is oftentimes a black box. As a step towards making the VQA task more explainable and interpretable, our method is built upon the SOTA VQA framework by augmenting it with an end-to-end explanation generation module. In this paper, we investigate two network architectures, including Long Short-Term Memory (LSTM) and Transformer decoder, as the explanation generator. Our method generates human-readable textual explanations while maintaining SOTA VQA accuracy on the GQA-REX (77.49%) and VQA-E (71.48%) datasets. Approximately 65.16% of the generated explanations are approved by humans as valid. Roughly 60.5% of the generated explanations are valid and lead to the correct answers.
An Open, Multi-Platform Software Architecture for Online Education in the Metaverse
Venue: arXiv cs.HC 2022
Year: 2022

Use of online platforms for education is a vibrant and growing arena, incorporating a variety of software platforms and technologies, including various modalities of extended reality. We present our Enhanced Reality Teaching Concierge, an open networking hub architected to enable efficient and easy connectivity between a wide variety of services or applications to a wide variety of clients, designed to showcase 3D for academic purposes across web technologies, virtual reality, and even virtual worlds. The agnostic nature of the system, paired with efficient architecture, and simple and open protocols furnishes an ecosystem that can easily be tailored to maximize the innate characteristics of each 3D display environment while sharing common data and control systems with the ultimate goal of a seamless, expandable, nimble education metaverse.
HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions
Venue: arXiv cs.CL 2022
Year: 2022

With the fast development of Machine Translation (MT) systems, especially the new boost from Neural MT (NMT) models, the MT output quality has reached a new level of accuracy. However, many researchers criticised that the current popular evaluation metrics such as BLEU can not correctly distinguish the state-of-the-art NMT systems regarding quality differences. In this short paper, we describe the design and implementation of a linguistically motivated human-in-the-loop evaluation metric looking into idiomatic and terminological Multi-word Expressions (MWEs). MWEs have played a bottleneck in many Natural Language Processing (NLP) tasks including MT. MWEs can be used as one of the main factors to distinguish different MT systems by looking into their capabilities in recognising and translating MWEs in an accurate and meaning equivalent manner.
Smaller Low-Depth Circuits for Kronecker Powers
Venue: arXiv cs.DS 2022
Year: 2022

We give new, smaller constructions of constant-depth linear circuits for computing any matrix which is the Kronecker power of a fixed matrix. A standard argument (e.g., the mixed product property of Kronecker products, or a generalization of the Fast Walsh-Hadamard transform) shows that any such $N \times N$ matrix has a depth-2 circuit of size $O(N^{1.5})$. We improve on this for all such matrices, and especially for some such matrices of particular interest: - For any integer $q > 1$ and any matrix which is the Kronecker power of a fixed $q \times q$ matrix, we construct a depth-2 circuit of size $O(N^{1.5 - a_q})$, where $a_q > 0$ is a positive constant depending only on $q$. No bound beating size $O(N^{1.5})$ was previously known for any $q>2$. - For the case $q=2$, i.e., for any matrix which is the Kronecker power of a fixed $2 \times 2$ matrix, we construct a depth-2 circuit of size $O(N^{1.446})$, improving the prior best size $O(N^{1.493})$ [Alman, 2021]. - For the Walsh-Hadamard transform, we construct a depth-2 circuit of size $O(N^{1.443})$, improving the prior best size $O(N^{1.476})$ [Alman, 2021]. - For the disjointness matrix (the communication matrix of set disjointness, or equivalently, the matrix for the linear transform that evaluates a multilinear polynomial on all $0/1$ inputs), we construct a depth-2 circuit of size $O(N^{1.258})$, improving the prior best size $O(N^{1.272})$ [Jukna and Sergeev, 2013]. Our constructions also generalize to improving the standard construction for any depth $\leq O(\log N)$. Our main technical tool is an improved way to convert a nontrivial circuit for any matrix into a circuit for its Kronecker powers. Our new bounds provably could not be achieved using the approaches of prior work.
Automatic Number Plate Recognition (ANPR) with YOLOv3-CNN
Venue: arXiv cs.CV 2022
Year: 2022

We present a YOLOv3-CNN pipeline for detecting vehicles, segregation of number plates, and local storage of final recognized characters. Vehicle identification is performed under various image correction schemes to determine the effect of environmental factors (angle of perception, luminosity, motion-blurring, and multi-line custom font etc.). A YOLOv3 object detection model was trained to identify vehicles from a dataset of traffic images. A second YOLOv3 layer was trained to identify number plates from vehicle images. Based upon correction schemes, individual characters were segregated and verified against real-time data to calculate accuracy of this approach. While characters under direct view were recognized accurately, some numberplates affected by environmental factors had reduced levels of accuracy. We summarize the results under various environmental factors against real-time data and produce an overall accuracy of the pipeline model.
SimuShips -- A High Resolution Simulation Dataset for Ship Detection with Precise Annotations
Venue: arXiv cs.CV 2022
Year: 2022

Obstacle detection is a fundamental capability of an autonomous maritime surface vessel (AMSV). State-of-the-art obstacle detection algorithms are based on convolutional neural networks (CNNs). While CNNs provide higher detection accuracy and fast detection speed, they require enormous amounts of data for their training. In particular, the availability of domain-specific datasets is a challenge for obstacle detection. The difficulty in conducting onsite experiments limits the collection of maritime datasets. Owing to the logistic cost of conducting on-site operations, simulation tools provide a safe and cost-efficient alternative for data collection. In this work, we introduce SimuShips, a publicly available simulation-based dataset for maritime environments. Our dataset consists of 9471 high-resolution (1920x1080) images which include a wide range of obstacle types, atmospheric and illumination conditions along with occlusion, scale and visible proportion variations. We provide annotations in the form of bounding boxes. In addition, we conduct experiments with YOLOv5 to test the viability of simulation data. Our experiments indicate that the combination of real and simulated images improves the recall for all classes by 2.9%.
Network Security Roadmap
Venue: arXiv cs.CR 2022
Year: 2022

Users may already have some perception of provided security based on experience with earlier generations. To maintain the stability and coherent integration of 5G services, it is imperative that security and privacy features prevalent in earlier generations are also present in 5G. However, it is not sufficient just to provide the same security features as in the legacy systems due to the new threat model introduced by the integration of new technologies like SDN, virtualization and SBA. 5G systems are expected to be more service-oriented. This suggests there will be an additional emphasis on security and privacy requirements that spawn from the new dimension of service-oriented security architecture.
The Friendship Paradox and Social Network Participation
Venue: arXiv cs.SI 2022
Year: 2022

The friendship paradox implies that a person will, on average, have fewer friends than their friends do. Prior work has shown how the friendship paradox can lead to perception biases regarding behaviors that correlate with the number of friends: for example, people tend to perceive their friends as being more socially engaged than they are. Here, we investigate the consequences of this type of social comparison in the conceptual setting of content creation ("sharing") in an online social network. Suppose people compare the amount of feedback that their content receives to the amount of feedback that their friends' content receives, and suppose they modify their sharing behavior as a result of that comparison. How does that impact overall sharing on the social network over time? We run simulations over model-generated synthetic networks, assuming initially uniform sharing and feedback rates. Thus, people's initial modifications of their sharing behavior in response to social comparisons are entirely driven by the friendship paradox. These modifications induce inhomogeneities in sharing rates that can further alter perception biases. If people's responses to social comparisons are monotonic (i.e., the larger the disparity, the larger the modification in sharing behavior), our simulations suggest that overall sharing in the network gradually declines. Meanwhile, convex responses can sustain or grow overall sharing in the network. We focus entirely on synthetic graphs in the present work and have not yet extended our simulations to real-world network topologies. Nevertheless, we do discuss practical implications, such as how interventions can be tailored to sustain long-term sharing, even in the presence of adverse social-comparison effects.