OpenRaiser/Intern-Atlas huggingface dataset:
Weightless Neural Networks for Efficient Edge Inference
Venue: arXiv cs.AR 2022
Year: 2022

Weightless Neural Networks (WNNs) are a class of machine learning model which use table lookups to perform inference. This is in contrast with Deep Neural Networks (DNNs), which use multiply-accumulate operations. State-of-the-art WNN architectures have a fraction of the implementation cost of DNNs, but still lag behind them on accuracy for common image recognition tasks. Additionally, many existing WNN architectures suffer from high memory requirements. In this paper, we propose a novel WNN architecture, BTHOWeN, with key algorithmic and architectural improvements over prior work, namely counting Bloom filters, hardware-friendly hashing, and Gaussian-based nonlinear thermometer encodings to improve model accuracy and reduce area and energy consumption. BTHOWeN targets the large and growing edge computing sector by providing superior latency and energy efficiency to comparable quantized DNNs. Compared to state-of-the-art WNNs across nine classification datasets, BTHOWeN on average reduces error by more than than 40% and model size by more than 50%. We then demonstrate the viability of the BTHOWeN architecture by presenting an FPGA-based accelerator, and compare its latency and resource usage against similarly accurate quantized DNN accelerators, including Multi-Layer Perceptron (MLP) and convolutional models. The proposed BTHOWeN models consume almost 80% less energy than the MLP models, with nearly 85% reduction in latency. In our quest for efficient ML on the edge, WNNs are clearly deserving of additional attention.
The Best of Both Worlds: Reinforcement Learning with Logarithmic Regret and Policy Switches
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we study the problem of regret minimization for episodic Reinforcement Learning (RL) both in the model-free and the model-based setting. We focus on learning with general function classes and general model classes, and we derive results that scale with the eluder dimension of these classes. In contrast to the existing body of work that mainly establishes instance-independent regret guarantees, we focus on the instance-dependent setting and show that the regret scales logarithmically with the horizon $T$, provided that there is a gap between the best and the second best action in every state. In addition, we show that such a logarithmic regret bound is realizable by algorithms with $O(\log T)$ switching cost (also known as adaptivity complexity). In other words, these algorithms rarely switch their policy during the course of their execution. Finally, we complement our results with lower bounds which show that even in the tabular setting, we cannot hope for regret guarantees lower than $o(\log T)$.
Disperse rotation operator DRT and use in some stream ciphers
Venue: arXiv cs.CR 2022
Year: 2022

The rotation operator is frequently used in several stream ciphers, including HC-128, Rabbit, and Salsa20, the final candidates for eSTREAM. This is because the rotation operator (ROT) is simple but has very good dispersibility. In this paper, we propose a disperse rotation operator (DRT), which has the same structure as ROT but has better dispersibility. In addition, the use of DRT instead of ROT has shown that the quality of the output stream of all three stream ciphers was significantly improved. However, the use of DRT instead of ROT in the HC-128 stream cipher prevents the expansion of differential attacks based on LSB.
Neural Graph Matching for Pre-training Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Recently, graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. However, when adapted to downstream tasks, it usually requires abundant task-specific labeled data, which can be extremely scarce in practice. A promising solution to data scarcity is to pre-train a transferable and expressive GNN model on large amounts of unlabeled graphs or coarse-grained labeled graphs. Then the pre-trained GNN is fine-tuned on downstream datasets with task-specific fine-grained labels. In this paper, we present a novel Graph Matching based GNN Pre-Training framework, called GMPT. Focusing on a pair of graphs, we propose to learn structural correspondences between them via neural graph matching, consisting of both intra-graph message passing and inter-graph message passing. In this way, we can learn adaptive representations for a given graph when paired with different graphs, and both node- and graph-level characteristics are naturally considered in a single pre-training task. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training. We further propose an approximate contrastive training strategy to significantly reduce time/memory consumption. Extensive experiments on multi-domain, out-of-distribution benchmarks have demonstrated the effectiveness of our approach. The code is available at: https://github.com/RUCAIBox/GMPT.
Physics-Data Driven Machine Learning Based Model: A Hybrid Way for Nonlinear, Dynamic, and Open-loop Identification of IPMC Soft Artificial Muscles
Venue: arXiv cs.LG 2022
Year: 2022

Ionic Polymer Metal Composites (IPMCs) are one of the most preferred choices among biocompatible materials for industrial and biomedical applications. Despite their advantages, some of their drawbacks include non-linear and hysteretic behavior, which complicates the modeling process. In previous works, usually autoregressive models were used to predict the behavior of an IPMC actuator. The main drawback of using an autoregressive model is that it cannot be used in mobile and real-time applications. In this study, we proposed a hybrid analytical intelligent model for an IPMC actuator. The most outstanding feature of this model is its non-autoregressive structure. The hybrid concept proposed in this study can be generalized to various problems other than IPMCs. The structure used in this work comprises an analytical model and a deep neural network, providing a non-linear, dynamic, and non-autoregressive model for the IPMC actuator. Lastly, the average NMSE achieved using the proposed hybrid model is 9.5781e-04 showing a significant drop in the error rate compared to other non-autoregressive structures.
Cross-Modality Earth Mover's Distance for Visible Thermal Person Re-Identification
Venue: arXiv cs.CV 2022
Year: 2022

Visible thermal person re-identification (VT-ReID) suffers from the inter-modality discrepancy and intra-identity variations. Distribution alignment is a popular solution for VT-ReID, which, however, is usually restricted to the influence of the intra-identity variations. In this paper, we propose the Cross-Modality Earth Mover's Distance (CM-EMD) that can alleviate the impact of the intra-identity variations during modality alignment. CM-EMD selects an optimal transport strategy and assigns high weights to pairs that have a smaller intra-identity variation. In this manner, the model will focus on reducing the inter-modality discrepancy while paying less attention to intra-identity variations, leading to a more effective modality alignment. Moreover, we introduce two techniques to improve the advantage of CM-EMD. First, the Cross-Modality Discrimination Learning (CM-DL) is designed to overcome the discrimination degradation problem caused by modality alignment. By reducing the ratio between intra-identity and inter-identity variances, CM-DL leads the model to learn more discriminative representations. Second, we construct the Multi-Granularity Structure (MGS), enabling us to align modalities from both coarse- and fine-grained levels with the proposed CM-EMD. Extensive experiments show the benefits of the proposed CM-EMD and its auxiliary techniques (CM-DL and MGS). Our method achieves state-of-the-art performance on two VT-ReID benchmarks.
Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation
Venue: arXiv cs.CL 2022
Year: 2022

Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years. As a countermeasure, adversarial defense has been explored, but relatively few efforts have been made to detect adversarial examples. However, detecting adversarial examples may be crucial for automated tasks (e.g. review sentiment analysis) that wish to amass information about a certain population and additionally be a step towards a robust defense system. To this end, we release a dataset for four popular attack methods on four datasets and four models to encourage further research in this field. Along with it, we propose a competitive baseline based on density estimation that has the highest AUC on 29 out of 30 dataset-attack-model combinations. Source code is available in https://github.com/anoymous92874838/text-adv-detection.
Open Source Routers: A Survey
Venue: arXiv cs.NI 2022
Year: 2022

Variety, size and complexity of data types, services and applications in Internet is continuously growing up. This increasing of complexity needs more powerful and sophisticated equipment's. One group of these devices that has essential role are routers. Some of vendors produce some elaborate and complex products but the commercial solutions are too closed and inflexible. The term "Open Source Routers" covers a lot of implementations of free software routers. Open Source Routers are solutions to overcome commercial solutions with closed platforms. In this article, we survey the existing implementations and a wide array of past and state-of-the-art projects on open software routers followed by a discussion of major challenges in this area.
Detecting High-Quality GAN-Generated Face Images using Neural Networks
Venue: arXiv cs.CR 2022
Year: 2022

In the past decades, the excessive use of the last-generation GAN (Generative Adversarial Networks) models in computer vision has enabled the creation of artificial face images that are visually indistinguishable from genuine ones. These images are particularly used in adversarial settings to create fake social media accounts and other fake online profiles. Such malicious activities can negatively impact the trustworthiness of users identities. On the other hand, the recent development of GAN models may create high-quality face images without evidence of spatial artifacts. Therefore, reassembling uniform color channel correlations is a challenging research problem. To face these challenges, we need to develop efficient tools able to differentiate between fake and authentic face images. In this chapter, we propose a new strategy to differentiate GAN-generated images from authentic images by leveraging spectral band discrepancies, focusing on artificial face image synthesis. In particular, we enable the digital preservation of face images using the Cross-band co-occurrence matrix and spatial co-occurrence matrix. Then, we implement these techniques and feed them to a Convolutional Neural Networks (CNN) architecture to identify the real from artificial faces. Additionally, we show that the performance boost is particularly significant and achieves more than 92% in different post-processing environments. Finally, we provide several research observations demonstrating that this strategy improves a comparable detection method based only on intra-band spatial co-occurrences.
Do Perceived Gender Biases in Retrieval Results Affect Relevance Judgements?
Venue: arXiv cs.IR 2022
Year: 2022

This work investigates the effect of gender-stereotypical biases in the content of retrieved results on the relevance judgement of users/annotators. In particular, since relevance in information retrieval (IR) is a multi-dimensional concept, we study whether the value and quality of the retrieved documents for some bias-sensitive queries can be judged differently when the content of the documents represents different genders. To this aim, we conduct a set of experiments where the genders of the participants are known as well as experiments where the participants genders are not specified. The set of experiments comprise of retrieval tasks, where participants perform a rated relevance judgement for different search query and search result document compilations. The shown documents contain different gender indications and are either relevant or non-relevant to the query. The results show the differences between the average judged relevance scores among documents with various gender contents. Our work initiates further research on the connection of the perception of gender stereotypes in users with their judgements and effects on IR systems, and aim to raise awareness about the possible biases in this domain.
Characterizing Grounded Theory Approaches in Visualization
Venue: arXiv cs.HC 2022
Year: 2022

Grounded theory (GT) is a research methodology that entails a systematic workflow for theory generation grounded on emergent data. In this paper, we juxtapose GT workflows with typical workflows in visualization and visual analytics, shortly VIS, and underline the characteristics shared by these workflows. We explore the research landscape of VIS to observe where GT has been applied to generate VIS theories, explicitly as well as implicitly. We propose a "why" typology for characterizing aspects in VIS where GT can potentially play a significant role. We outline a "how" methodology for conducting GT research in VIS, which addresses the need for theoretical advancement in VIS while benefitting from other methods and techniques in VIS. We exemplify this "how" methodology by adopting GT approaches in studying the messages posted on VisGuides - an Open Discourse Forum for discussing visualization guidelines.
Automatic Facial Paralysis Estimation with Facial Action Units
Venue: arXiv cs.CV 2022
Year: 2022

Facial palsy is unilateral facial nerve weakness or paralysis of rapid onset with unknown causes. Automatically estimating facial palsy severeness can be helpful for the diagnosis and treatment of people suffering from it across the world. In this work, we develop and experiment with a novel model for estimating facial palsy severity. For this, an effective Facial Action Units (AU) detection technique is incorporated into our model, where AUs refer to a unique set of facial muscle movements used to describe almost every anatomically possible facial expression. In this paper, we propose a novel Adaptive Local-Global Relational Network (ALGRNet) for facial AU detection and use it to classify facial paralysis severity. ALGRNet mainly consists of three main novel structures: (i) an adaptive region learning module that learns the adaptive muscle regions based on the detected landmarks; (ii) a skip-BiLSTM that models the latent relationships among local AUs; and (iii) a feature fusion&refining module that investigates the complementary between the local and global face. Quantitative results on two AU benchmarks, i.e., BP4D and DISFA, demonstrate our ALGRNet can achieve promising AU detection accuracy. We further demonstrate the effectiveness of its application to facial paralysis estimation by migrating ALGRNet to a facial paralysis dataset collected and annotated by medical professionals.
Nonlinear Reduced Order Modelling of Soil Structure Interaction Effects via LSTM and Autoencoder Neural Networks
Venue: arXiv cs.CE 2022
Year: 2022

In the field of structural health monitoring (SHM), inverse problems which require repeated analyses are common. With the increase in the use of nonlinear models, the development of nonlinear reduced order modelling techniques is of paramount interest. Of considerable research interest, is the use of flexible and scalable machine learning methods which can learn to approximate the behaviour of nonlinear dynamic systems using input and output data. One such nonlinear system of interest, in the context of wind turbine structures, is the soil structure interaction (SSI) problem. Soil demonstrates strongly nonlinear behaviour with regards to its restoring force and has been shown to considerably influence the dynamic response of wind turbine structures. In this work, we demonstrate the application of a recently developed nonlinear reduced order modelling method, which leverages Autoencoder and LSTM neural networks, to a nonlinear soil structure interaction problem of a wind turbine monopile subject to realistic loading at the seabed level. The accuracy and efficiency of the methodology is compared to full order simulations carried out using Abaqus. The ROM was shown to have good fidelity and a considerable reduction in computational time for the system considered.
Context Enhanced Short Text Matching using Clickthrough Data
Venue: arXiv cs.CL 2022
Year: 2022

The short text matching task employs a model to determine whether two short texts have the same semantic meaning or intent. Existing short text matching models usually rely on the content of short texts which are lack information or missing some key clues. Therefore, the short texts need external knowledge to complete their semantic meaning. To address this issue, we propose a new short text matching framework for introducing external knowledge to enhance the short text contextual representation. In detail, we apply a self-attention mechanism to enrich short text representation with external contexts. Experiments on two Chinese datasets and one English dataset demonstrate that our framework outperforms the state-of-the-art short text matching models.
Robustness and Adaptation to Hidden Factors of Variation
Venue: arXiv cs.LG 2022
Year: 2022

We tackle here a specific, still not widely addressed aspect, of AI robustness, which consists of seeking invariance / insensitivity of model performance to hidden factors of variations in the data. Towards this end, we employ a two step strategy that a) does unsupervised discovery, via generative models, of sensitive factors that cause models to under-perform, and b) intervenes models to make their performance invariant to these sensitive factors' influence. We consider 3 separate interventions for robustness, including: data augmentation, semantic consistency, and adversarial alignment. We evaluate our method using metrics that measure trade offs between invariance (insensitivity) and overall performance (utility) and show the benefits of our method for 3 settings (unsupervised, semi-supervised and generalization).
LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction
Venue: arXiv cs.CV 2022
Year: 2022

Multi-agent trajectory prediction is a fundamental problem in autonomous driving. The key challenges in prediction are accurately anticipating the behavior of surrounding agents and understanding the scene context. To address these problems, we propose LatentFormer, a transformer-based model for predicting future vehicle trajectories. The proposed method leverages a novel technique for modeling interactions among dynamic objects in the scene. Contrary to many existing approaches which model cross-agent interactions during the observation time, our method additionally exploits the future states of the agents. This is accomplished using a hierarchical attention mechanism where the evolving states of the agents autoregressively control the contributions of past trajectories and scene encodings in the final prediction. Furthermore, we propose a multi-resolution map encoding scheme that relies on a vision transformer module to effectively capture both local and global scene context to guide the generation of more admissible future trajectories. We evaluate the proposed method on the nuScenes benchmark dataset and show that our approach achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%. We further investigate the contributions of various components of the proposed technique via extensive ablation studies.
OptiTrap: Optimal Trap Trajectories for Acoustic Levitation Displays
Venue: arXiv cs.HC 2022
Year: 2022

Acoustic levitation has recently demonstrated the ability to create volumetric content by trapping and quickly moving particles along reference paths to reveal shapes in mid-air. However, the problem of specifying physically feasible trap trajectories to display desired shapes remains unsolved. Even if only the final shape is of interest to the content creator, the trap trajectories need to determine where and when the traps need to be, for the particle to reveal the intended shape. We propose OptiTrap, the first structured numerical approach to compute trap trajectories for acoustic levitation displays. Our approach generates trap trajectories that are physically feasible and nearly time-optimal, and reveal generic mid-air shapes, given only a reference path (i.e., a shape with no time information). We provide a multi-dimensional model of the acoustic forces around a trap to model the trap-particle system dynamics and compute optimal trap trajectories by formulating and solving a non-linear path following problem. We formulate our approach and evaluate it, demonstrating how OptiTrap consistently produces feasible and nearly optimal paths, with increases in size, frequency, and accuracy of the shapes rendered, allowing us to demonstrate larger and more complex shapes than ever shown to date.
LegalVis: Exploring and Inferring Precedent Citations in Legal Documents
Venue: arXiv cs.HC 2022
Year: 2022

To reduce the number of pending cases and conflicting rulings in the Brazilian Judiciary, the National Congress amended the Constitution, allowing the Brazilian Supreme Court (STF) to create binding precedents (BPs), i.e., a set of understandings that both Executive and lower Judiciary branches must follow. The STF's justices frequently cite the 58 existing BPs in their decisions, and it is of primary relevance that judicial experts could identify and analyze such citations. To assist in this problem, we propose LegalVis, a web-based visual analytics system designed to support the analysis of legal documents that cite or could potentially cite a BP. We model the problem of identifying potential citations (i.e., non-explicit) as a classification problem. However, a simple score is not enough to explain the results; that is why we use an interpretability machine learning method to explain the reason behind each identified citation. For a compelling visual exploration of documents and BPs, LegalVis comprises three interactive visual components: the first presents an overview of the data showing temporal patterns, the second allows filtering and grouping relevant documents by topic, and the last one shows a document's text aiming to interpret the model's output by pointing out which paragraphs are likely to mention the BP, even if not explicitly specified. We evaluated our identification model and obtained an accuracy of 96%; we also made a quantitative and qualitative analysis of the results. The usefulness and effectiveness of LegalVis were evaluated through two usage scenarios and feedback from six domain experts.
High Welfare Matching Markets via Descending Price
Venue: arXiv cs.GT 2022
Year: 2022

We consider design of monetary mechanisms for two-sided matching. Mechanisms in the tradition of the deferred acceptance algorithm, even in variants incorporating money, tend to focus on the criterion of stability. Instead, in this work we seek a simple auction-inspired mechanism with social welfare guarantees. We consider a descending-price mechanism called the Marshallian Match, proposed (but not analyzed) by Waggoner and Weyl (2019). When all values for potential matches are positive, we show the Marshallian Match with a "rebate" payment rule achieves constant price of anarchy. This result extends to models with costs for acquiring information about one's values, and also to matching on hypergraphs. With possibly-negative valuations, which capture e.g. job markets, the problem becomes harder. We introduce notions of approximate stability and show that they have beneficial welfare implications. However, the main problem of proving constant factor welfare guarantees in "ex ante stable equilibrium" remains open.