OpenRaiser/Intern-Atlas huggingface dataset:
M-SENA: An Integrated Platform for Multimodal Sentiment Analysis
Venue: arXiv cs.AI 2022
Year: 2022

M-SENA is an open-sourced platform for Multimodal Sentiment Analysis. It aims to facilitate advanced research by providing flexible toolkits, reliable benchmarks, and intuitive demonstrations. The platform features a fully modular video sentiment analysis framework consisting of data management, feature extraction, model training, and result analysis modules. In this paper, we first illustrate the overall architecture of the M-SENA platform and then introduce features of the core modules. Reliable baseline results of different modality features and MSA benchmarks are also reported. Moreover, we use model evaluation and analysis tools provided by M-SENA to present intermediate representation visualization, on-the-fly instance test, and generalization ability test results. The source code of the platform is publicly available at https://github.com/thuiar/M-SENA.
3D Adapted Random Forest Vision (3DARFV) for Untangling Heterogeneous-Fabric Exceeding Deep Learning Semantic Segmentation Efficiency at the Utmost Accuracy
Venue: arXiv cs.CV 2022
Year: 2022

Planetary exploration depends heavily on 3D image data to characterize the static and dynamic properties of the rock and environment. Analyzing 3D images requires many computations, causing efficiency to suffer lengthy processing time alongside large energy consumption. High-Performance Computing (HPC) provides apparent efficiency at the expense of energy consumption. However, for remote explorations, the conveyed surveillance and the robotized sensing need faster data analysis with ultimate accuracy to make real-time decisions. In such environments, access to HPC and energy is limited. Therefore, we realize that reducing the number of computations to optimal and maintaining the desired accuracy leads to higher efficiency. This paper demonstrates the semantic segmentation capability of a probabilistic decision tree algorithm, 3D Adapted Random Forest Vision (3DARFV), exceeding deep learning algorithm efficiency at the utmost accuracy.
Refine-Net: Normal Refinement Neural Network for Noisy Point Clouds
Venue: arXiv cs.CV 2022
Year: 2022

Point normal, as an intrinsic geometric property of 3D objects, not only serves conventional geometric tasks such as surface consolidation and reconstruction, but also facilitates cutting-edge learning-based techniques for shape analysis and generation. In this paper, we propose a normal refinement network, called Refine-Net, to predict accurate normals for noisy point clouds. Traditional normal estimation wisdom heavily depends on priors such as surface shapes or noise distributions, while learning-based solutions settle for single types of hand-crafted features. Differently, our network is designed to refine the initial normal of each point by extracting additional information from multiple feature representations. To this end, several feature modules are developed and incorporated into Refine-Net by a novel connection module. Besides the overall network architecture of Refine-Net, we propose a new multi-scale fitting patch selection scheme for the initial normal estimation, by absorbing geometry domain knowledge. Also, Refine-Net is a generic normal estimation framework: 1) point normals obtained from other methods can be further refined, and 2) any feature module related to the surface geometric structures can be potentially integrated into the framework. Qualitative and quantitative evaluations demonstrate the clear superiority of Refine-Net over the state-of-the-arts on both synthetic and real-scanned datasets. Our code is available at https://github.com/hrzhou2/refinenet.
A Survey on Cross-Lingual Summarization
Venue: arXiv cs.CL 2022
Year: 2022

Cross-lingual summarization is the task of generating a summary in one language (e.g., English) for the given document(s) in a different language (e.g., Chinese). Under the globalization background, this task has attracted increasing attention of the computational linguistics community. Nevertheless, there still remains a lack of comprehensive review for this task. Therefore, we present the first systematic critical review on the datasets, approaches, and challenges in this field. Specifically, we carefully organize existing datasets and approaches according to different construction methods and solution paradigms, respectively. For each type of datasets or approaches, we thoroughly introduce and summarize previous efforts and further compare them with each other to provide deeper analyses. In the end, we also discuss promising directions and offer our thoughts to facilitate future research. This survey is for both beginners and experts in cross-lingual summarization, and we hope it will serve as a starting point as well as a source of new ideas for researchers and engineers interested in this area.
Computational historical linguistics and language diversity in South Asia
Venue: arXiv cs.CL 2022
Year: 2022

South Asia is home to a plethora of languages, many of which severely lack access to new language technologies. This linguistic diversity also results in a research environment conducive to the study of comparative, contact, and historical linguistics -- fields which necessitate the gathering of extensive data from many languages. We claim that data scatteredness (rather than scarcity) is the primary obstacle in the development of South Asian language technology, and suggest that the study of language history is uniquely aligned with surmounting this obstacle. We review recent developments in and at the intersection of South Asian NLP and historical-comparative linguistics, describing our and others' current efforts in this area. We also offer new strategies towards breaking the data barrier.
Winternitz stack protocols
Venue: arXiv cs.CR 2022
Year: 2022

This paper proposes and evaluates a new bipartite post-quantum digital signature protocol based on Winternitz chains and the HORS oracle. Mutually mistrustful Alice and Bob are able to agree and sign a series of documents in a way that makes it impossible (within the assumed security model) to repudiate their signatures. The number of signatures supported by a single public key is limited by a large number but the security of the signature scheme is not diminished by repeated application. A single public key supports both parties. Some ramifications are discussed, security parameters evaluated and an application area delineated for the proposed concept.
SerialTrack: ScalE and Rotation Invariant Augmented Lagrangian Particle Tracking
Venue: arXiv cs.RO 2022
Year: 2022

We present a new particle tracking algorithm to accurately resolve large deformation and rotational motion fields, which takes advantage of both local and global particle tracking algorithms. We call this method the ScalE and Rotation Invariant Augmented Lagrangian Particle Tracking (SerialTrack). This method builds an iterative scale and rotation invariant topology-based feature for each particle within a multi-scale tracking algorithm. The global kinematic compatibility condition is applied as a global augmented Lagrangian constraint to enhance the tracking accuracy. An open source software package implementing this numerical approach to track both 2D and 3D, incremental and cumulative deformation fields is provided.
ZOOMER: Boosting Retrieval on Web-scale Graphs by Regions of Interest
Venue: arXiv cs.IR 2022
Year: 2022

We introduce ZOOMER, a system deployed at Taobao, the largest e-commerce platform in China, for training and serving GNN-based recommendations over web-scale graphs. ZOOMER is designed for tackling two challenges presented by the massive user data at Taobao: low training/serving efficiency due to the huge scale of the graphs, and low recommendation quality due to the information overload which distracts the recommendation model from specific user intentions. ZOOMER achieves this by introducing a key concept, Region of Interests (ROI) in GNNs for recommendations, i.e., a neighborhood region in the graph with significant relevance to a strong user intention. ZOOMER narrows the focus from the whole graph and "zooms in" on the more relevant ROIs, thereby reducing the training/serving cost and mitigating the information overload at the same time. With carefully designed mechanisms, ZOOMER identifies the interest expressed by each recommendation request, constructs an ROI subgraph by sampling with respect to the interest, and guides the GNN to reweigh different parts of the ROI towards the interest by a multi-level attention module. Deployed as a large-scale distributed system, ZOOMER supports graphs with billions of nodes for training and thousands of requests per second for serving. ZOOMER achieves up to 14x speedup when downsizing sampling scales with comparable (even better) AUC performance than baseline methods. Besides, both the offline evaluation and online A/B test demonstrate the effectiveness of ZOOMER.
CorpusVis: Visual Analysis of Digital Sheet Music Collections
Venue: arXiv cs.HC 2022
Year: 2022

Manually investigating sheet music collections is challenging for music analysts due to the magnitude and complexity of underlying features, structures, and contextual information. However, applying sophisticated algorithmic methods would require advanced technical expertise that analysts do not necessarily have. Bridging this gap, we contribute CorpusVis, an interactive visual workspace, enabling scalable and multi-faceted analysis. Our proposed visual analytics dashboard provides access to computational methods, generating varying perspectives on the same data. The proposed application uses metadata including composers, type, epoch, and low-level features, such as pitch, melody, and rhythm. To evaluate our approach, we conducted a pair analytics study with nine participants. The qualitative results show that CorpusVis supports users in performing exploratory and confirmatory analysis, leading them to new insights and findings. In addition, based on three exemplary workflows, we demonstrate how to apply our approach to different tasks, such as exploring musical features or comparing composers.
SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics
Venue: arXiv cs.SE 2022
Year: 2022

Learning-based program repair has achieved good results in a recent series of papers. Yet, we observe that the related work fails to repair some bugs because of a lack of knowledge about 1) the application domain of the program being repaired, and 2) the fault type being repaired. In this paper, we solve both problems by changing the learning paradigm from supervised training to self-supervised training in an approach called SelfAPR. First, SelfAPR generates training samples on disk by perturbing a previous version of the program being repaired, enforcing the neural model to capture projectspecific knowledge. This is different from the previous work based on mined past commits. Second, SelfAPR executes all training samples and extracts and encodes test execution diagnostics into the input representation, steering the neural model to fix the kind of fault. This is different from the existing studies that only consider static source code as input. We implement SelfAPR and evaluate it in a systematic manner. We generate 1 039 873 training samples obtained by perturbing 17 open-source projects. We evaluate SelfAPR on 818 bugs from Defects4J, SelfAPR correctly repairs 110 of them, outperforming all the supervised learning repair approaches.
Energy-Efficient VM Placement in PON-based Data Center Architectures with Cascaded AWGRs
Venue: arXiv cs.NI 2022
Year: 2022

Data centers based on Passive Optical Networks (PONs) can offer scalability, low cost and high energy-efficiency. Application in data centers can use Virtual Machines (VMs) to provide efficient utilization of the physical resources. This paper investigates the impact of VM placement on the energyefficiency in a PON-based data center architecture that utilizes cascaded Arrayed Waveguide Grating Routers (AWGRs). In this paper, we develop a Mixed Integer Linear Programming (MILP) optimization model to optimize the VM placement in the proposed PON-based data center architecture. This optimization aims to minimize the power consumption of the networking and computing by placing the VMs and their demands in the optimum number of resources (i.e., servers and networking devices) in the data center. We first minimize the processing power consumption only and then we minimize the processing and networking power consumption. The results show that a reduction in the networking power consumption by up to 75% is achieved when performing joint minimization of processing and networking power consumption compared to considering the minimization of the processing power consumption only.
A platform for causal knowledge representation and inference in industrial fault diagnosis based on cubic DUCG
Venue: arXiv cs.AI 2022
Year: 2022

The working conditions of large-scale industrial systems are very complex. Once a failure occurs, it will affect industrial production, cause property damage, and even endanger the workers' lives. Therefore, it is important to control the operation of the system to accurately grasp the operation status of the system and find out the failure in time. The occurrence of system failure is a gradual process, and the occurrence of the current system failure may depend on the previous state of the system, which is sequential. The fault diagnosis technology based on time series can monitor the operating status of the system in real-time, detect the abnormal operation of the system within the allowable time interval, diagnose the root cause of the fault and predict the status trend. In order to guide the technical personnel to troubleshoot and solve related faults, in this paper, an industrial fault diagnosis system is implemented based on the cubic DUCG theory. The diagnostic model of the system is constructed based on expert knowledge and experience. At the same time, it can perform real-time fault diagnosis based on time sequence, which solves the problem of fault diagnosis of industrial systems without sample data.
Random Forest Regression for continuous affect using Facial Action Units
Venue: arXiv cs.CV 2022
Year: 2022

In this paper we describe our approach to the arousal and valence track of the 3rd Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). We extracted facial features using OpenFace and used them to train a multiple output random forest regressor. Our approach performed comparable to the baseline approach.
Efficient Answering of Historical What-if Queries
Venue: arXiv cs.DB 2022
Year: 2022

We introduce historical what-if queries, a novel type of what-if analysis that determines the effect of a hypothetical change to the transactional history of a database. For example, "how would revenue be affected if we would have charged an additional $6 for shipping?" Such queries may lead to more actionable insights than traditional what-if queries as their results can be used to inform future actions, e.g., increasing shipping fees. We develop efficient techniques for answering historical what-if queries, i.e., determining how a modified history affects the current database state. Our techniques are based on reenactment, a replay technique for transactional histories. We optimize this process using program and data slicing techniques that determine which updates and what data can be excluded from reenactment without affecting the result. Using an implementation of our techniques in Mahif (a Middleware for Answering Historical what-IF queries) we demonstrate their effectiveness experimentally.
Facial Expression Classification using Fusion of Deep Neural Network in Video for the 3rd ABAW3 Competition
Venue: arXiv cs.CV 2022
Year: 2022

For computers to recognize human emotions, expression classification is an equally important problem in the human-computer interaction area. In the 3rd Affective Behavior Analysis In-The-Wild competition, the task of expression classification includes eight classes with six basic expressions of human faces from videos. In this paper, we employ a transformer mechanism to encode the robust representation from the backbone. Fusion of the robust representations plays an important role in the expression classification task. Our approach achieves 30.35\% and 28.60\% for the $F_1$ score on the validation set and the test set, respectively. This result shows the effectiveness of the proposed architecture based on the Aff-Wild2 dataset.
Optimal MIMO Combining for Blind Federated Edge Learning with Gradient Sparsification
Venue: arXiv cs.IT 2022
Year: 2022

We provide the optimal receive combining strategy for federated learning in multiple-input multiple-output (MIMO) systems. Our proposed algorithm allows the clients to perform individual gradient sparsification which greatly improves performance in scenarios with heterogeneous (non i.i.d.) training data. The proposed method beats the benchmark by a wide margin.
Evaluation of IoT Self-healing Mechanisms using Fault-Injection in Message Brokers
Venue: arXiv cs.SE 2022
Year: 2022

The widespread use of Internet-of-Things (IoT) across different application domains leads to an increased concern regarding their dependability, especially as the number of potentially mission-critical systems becomes considerable. Fault-tolerance has been used to reduce the impact of faults in systems, and their adoption in IoT is becoming a necessity. This work focuses on how to exercise fault-tolerance mechanisms by deliberately provoking its malfunction. We start by describing a proof-of-concept fault-injection add-on to a commonly used publish/subscribe broker. We then present several experiments mimicking real-world IoT scenarios, focusing on injecting faults in systems with (and without) active self-healing mechanisms and comparing their behavior to the baseline without faults. We observe evidence that fault-injection can be used to (a) exercise in-place fault-tolerance apparatus, and (b) detect when these mechanisms are not performing nominally, providing insights into enhancing in-place fault-tolerance techniques.
Analyzing Process-Aware Information System Updates Using Digital Twins of Organizations
Venue: arXiv cs.AI 2022
Year: 2022

Digital transformation often entails small-scale changes to information systems supporting the execution of business processes. These changes may increase the operational frictions in process execution, which decreases the process performance. The contributions in the literature providing support to the tracking and impact analysis of small-scale changes are limited in scope and functionality. In this paper, we use the recently developed Digital Twins of Organizations (DTOs) to assess the impact of (process-aware) information systems updates. More in detail, we model the updates using the configuration of DTOs and quantitatively assess different types of impacts of information system updates (structural, operational, and performance-related). We implemented a prototype of the proposed approach. Moreover, we discuss a case study involving a standard ERP procure-to-pay business process.
Using Orientation to Distinguish Overlapping Chromosomes
Venue: arXiv cs.LG 2022
Year: 2022

A difficult step in the process of karyotyping is segmenting chromosomes that touch or overlap. In an attempt to automate the process, previous studies turned to Deep Learning methods, with some formulating the task as a semantic segmentation problem. These models treat separate chromosome instances as semantic classes, which we show to be problematic, since it is uncertain which chromosome should be classed as #1 and #2. Assigning class labels based on comparison rules, such as the shorter/longer chromosome alleviates, but does not fully resolve the issue. Instead, we separate the chromosome instances in a second stage, predicting the orientation of the chromosomes by the model and use it as one of the key distinguishing factors of the chromosomes. We demonstrate this method to be effective. Furthermore, we introduce a novel Double-Angle representation that a neural network can use to predict the orientation. The representation maps any direction and its reverse to the same point. Lastly, we present a new expanded synthetic dataset, which is based on Pommier's dataset, but addresses its issues with insufficient separation between its training and testing sets.
Benchmarking Visual Localization for Autonomous Navigation
Venue: arXiv cs.CV 2022
Year: 2022

This work introduces a simulator-based benchmark for visual localization in the autonomous navigation context. The dynamic benchmark enables investigation of how variables such as the time of day, weather, and camera perspective affect the navigation performance of autonomous agents that utilize visual localization for closed-loop control. The experimental part of the paper studies the effects of four such variables by evaluating state-of-the-art visual localization methods as part of the motion planning module of an autonomous navigation stack. The results show major variation in the suitability of the different methods for vision-based navigation. To the authors' best knowledge, the proposed benchmark is the first to study modern visual localization methods as part of a complete navigation stack. We make the benchmark available at https://github.com/lasuomela/carla_vloc_benchmark.
Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Facial expression recognition plays an important role in human-computer interaction. In this paper, we propose the Coarse-to-Fine Cascaded network with Smooth Predicting (CFC-SP) to improve the performance of facial expression recognition. CFC-SP contains two core components, namely Coarse-to-Fine Cascaded networks (CFC) and Smooth Predicting (SP). For CFC, it first groups several similar emotions to form a rough category, and then employs a network to conduct a coarse but accurate classification. Later, an additional network for these grouped emotions is further used to obtain fine-grained predictions. For SP, it improves the recognition capability of the model by capturing both universal and unique expression features. To be specific, the universal features denote the general characteristic of facial emotions within a period and the unique features denote the specific characteristic at this moment. Experiments on Aff-Wild2 show the effectiveness of the proposed CFSP. We achieved 3rd place in the Expression Classification Challenge of the 3rd Competition on Affective Behavior Analysis in-the-wild. The code will be released at https://github.com/BR-IDL/PaddleViT.