OpenRaiser/Intern-Atlas huggingface dataset:
A Spatio-Temporal Multilayer Perceptron for Gesture Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Gesture recognition is essential for the interaction of autonomous vehicles with humans. While the current approaches focus on combining several modalities like image features, keypoints and bone vectors, we present neural network architecture that delivers state-of-the-art results only with body skeleton input data. We propose the spatio-temporal multilayer perceptron for gesture recognition in the context of autonomous vehicles. Given 3D body poses over time, we define temporal and spatial mixing operations to extract features in both domains. Additionally, the importance of each time step is re-weighted with Squeeze-and-Excitation layers. An extensive evaluation of the TCG and Drive&Act datasets is provided to showcase the promising performance of our approach. Furthermore, we deploy our model to our autonomous vehicle to show its real-time capability and stable execution.
Go Wide or Go Deep: Levering Watermarking Performance with Computational Cost for Specific Images
Venue: arXiv cs.CR 2022
Year: 2022

Digital watermarking has been widely studied for the protection of intellectual property. Traditional watermarking schemes often design in a "wider" rule, which applies one general embedding mechanism to all images. But this will limit the scheme into a robustness-invisibility trade-off, where the improvements of robustness can only be achieved by the increase of embedding intensity thus causing the visual quality decay. However, a new scenario comes out at this stage that many businesses wish to give high level protection to specific valuable images, which requires high robustness and high visual quality at the same time. Such scenario makes the watermarking schemes should be designed in a "deeper" way which makes the embedding mechanism customized to specific images. To achieve so, we break the robustness-invisibility trade-off by introducing computation cost in, and propose a novel auto-decoder-like image-specified watermarking framework (ISMark). Based on ISMark, the strong robustness and high visual quality for specific images can be both achieved. In detail, we apply an optimization procedure (OPT) to replace the traditional embedding mechanism. Unlike existing schemes that embed watermarks using a learned encoder, OPT regards the cover image as the optimizable parameters to minimize the extraction error of the decoder, thus the features of each specified image can be effectively exploited to achieve superior performance. Extensive experiments indicate that ISMark outperforms the state-of-the-art methods by a large margin, which improves the average bit error rate by 4.64% (from 4.86% to 0.22%) and PSNR by 2.20dB (from 32.50dB to 34.70dB).
SELECTOR: Selecting a Representative Benchmark Suite for Reproducible Statistical Comparison
Venue: arXiv cs.NE 2022
Year: 2022

Fair algorithm evaluation is conditioned on the existence of high-quality benchmark datasets that are non-redundant and are representative of typical optimization scenarios. In this paper, we evaluate three heuristics for selecting diverse problem instances which should be involved in the comparison of optimization algorithms in order to ensure robust statistical algorithm performance analysis. The first approach employs clustering to identify similar groups of problem instances and subsequent sampling from each cluster to construct new benchmarks, while the other two approaches use graph algorithms for identifying dominating and maximal independent sets of nodes. We demonstrate the applicability of the proposed heuristics by performing a statistical performance analysis of five portfolios consisting of three optimization algorithms on five of the most commonly used optimization benchmarks. The results indicate that the statistical analyses of the algorithms' performance, conducted on each benchmark separately, produce conflicting outcomes, which can be used to give a false indication of the superiority of one algorithm over another. On the other hand, when the analysis is conducted on the problem instances selected with the proposed heuristics, which uniformly cover the problem landscape, the statistical outcomes are robust and consistent.
Research Status of Deep Learning Methods for Rumor Detection
Venue: arXiv cs.CY 2022
Year: 2022

To manage the rumors in social media to reduce the harm of rumors in society. Many studies used methods of deep learning to detect rumors in open networks. To comprehensively sort out the research status of rumor detection from multiple perspectives, this paper analyzes the highly focused work from three perspectives: Feature Selection, Model Structure, and Research Methods. From the perspective of feature selection, we divide methods into content feature, social feature, and propagation structure feature of the rumors. Then, this work divides deep learning models of rumor detection into CNN, RNN, GNN, Transformer based on the model structure, which is convenient for comparison. Besides, this work summarizes 30 works into 7 rumor detection methods such as propagation trees, adversarial learning, cross-domain methods, multi-task learning, unsupervised and semi-supervised methods, based knowledge graph, and other methods for the first time. And compare the advantages of different methods to detect rumors. In addition, this review enumerate datasets available and discusses the potential issues and future work to help researchers advance the development of field.
A global analysis of metrics used for measuring performance in natural language processing
Venue: arXiv cs.CL 2022
Year: 2022

Measuring the performance of natural language processing models is challenging. Traditionally used metrics, such as BLEU and ROUGE, originally devised for machine translation and summarization, have been shown to suffer from low correlation with human judgment and a lack of transferability to other tasks and languages. In the past 15 years, a wide range of alternative metrics have been proposed. However, it is unclear to what extent this has had an impact on NLP benchmarking efforts. Here we provide the first large-scale cross-sectional analysis of metrics used for measuring performance in natural language processing. We curated, mapped and systematized more than 3500 machine learning model performance results from the open repository 'Papers with Code' to enable a global and comprehensive analysis. Our results suggest that the large majority of natural language processing metrics currently used have properties that may result in an inadequate reflection of a models' performance. Furthermore, we found that ambiguities and inconsistencies in the reporting of metrics may lead to difficulties in interpreting and comparing model performances, impairing transparency and reproducibility in NLP research.
Adversarial Filtering Modeling on Long-term User Behavior Sequences for Click-Through Rate Prediction
Venue: arXiv cs.IR 2022
Year: 2022

Rich user behavior information is of great importance for capturing and understanding user interest in click-through rate (CTR) prediction. To improve the richness, collecting long-term behaviors becomes a typical approach in academy and industry but at the cost of increasing online storage and latency. Recently, researchers have proposed several approaches to shorten long-term behavior sequence and then model user interests. These approaches reduce online cost efficiently but do not well handle the noisy information in long-term user behavior, which may deteriorate the performance of CTR prediction significantly. To obtain better cost/performance trade-off, we propose a novel Adversarial Filtering Model (ADFM) to model long-term user behavior. ADFM uses a hierarchical aggregation representation to compress raw behavior sequence and then learns to remove useless behavior information with an adversarial filtering mechanism. The selected user behaviors are fed into interest extraction module for CTR prediction. Experimental results on public datasets and industrial dataset demonstrate that our method achieves significant improvements over state-of-the-art models.
Algorithms and Theory for Supervised Gradual Domain Adaptation
Venue: arXiv cs.LG 2022
Year: 2022

The phenomenon of data distribution evolving over time has been observed in a range of applications, calling the needs of adaptive learning algorithms. We thus study the problem of supervised gradual domain adaptation, where labeled data from shifting distributions are available to the learner along the trajectory, and we aim to learn a classifier on a target data distribution of interest. Under this setting, we provide the first generalization upper bound on the learning error under mild assumptions. Our results are algorithm agnostic, general for a range of loss functions, and only depend linearly on the averaged learning error across the trajectory. This shows significant improvement compared to the previous upper bound for unsupervised gradual domain adaptation, where the learning error on the target domain depends exponentially on the initial error on the source domain. Compared with the offline setting of learning from multiple domains, our results also suggest the potential benefits of the temporal structure among different domains in adapting to the target one. Empirically, our theoretical results imply that learning proper representations across the domains will effectively mitigate the learning errors. Motivated by these theoretical insights, we propose a min-max learning objective to learn the representation and classifier simultaneously. Experimental results on both semi-synthetic and large-scale real datasets corroborate our findings and demonstrate the effectiveness of our objectives.
Interactivity in Constructive Cryptography : Modeling and Applications to Updatable Encryption and Private Information Retrieval
Venue: arXiv cs.CR 2022
Year: 2022

In this work, we extend the Constructive Cryptography (CC) framework introduced by Maurer in 2011 so as to handle interactive protocols. We design and construct a so-called {\em Interactive Server Memory Resource} (ISMR), that is an augmented version of the basic instantiation of a client-server protocol in CC, namely the Server Memory Resource. We then apply our ISMR construction to two types of interactive cryptographic protocols for remote storage : Updatable Encryption (UE) and Private Information Retrieval (PIR). Concerning UE, our results are a composable version of those protocols, clarifying the security guarantees achieved by {\em any} UE scheme. Namely, we give the relevant security notion to consider according to a given leakage context. Letting USMR denote our ISMR adapted to the UE application, we prove that $\mathsf{IND}\text{-}\mathsf{UE}\text{-}\mathsf{CPA}$ security is sufficient for a secure construction of a confidential USMR that hides the age of ciphertexts; and $\mathsf{IND}\text{-}(\mathsf{ENC}+\mathsf{UPD})\text{-}\mathsf{CPA}$ security is sufficient for a secure construction of a confidential USMR in case of unrestricted leakage. As a consequence, contrary to what was claimed before, the $\mathsf{IND}\text{-}\mathsf{UE}$ security notion is not always stronger than the $\mathsf{IND}\text{-}(\mathsf{ENC+UPD})$ one. Concerning PIR, we also give a composable version of PIR protocols, yielding a unique model that unifies different notions of PIR : IT-PIR, C-PIR, one- or multi- server PIR. Using the flexibility of CC, we are also able to model PIR variants, such as SPIR.
Enhancing Marine Data Transmission with Socially-Aware Resilient Vessel Networks
Venue: arXiv cs.NI 2022
Year: 2022

With the multi-dimensional exploration towards oceans, enormous sensing data has been generated with significant volume, velocity, variety and heterogeneity. The resulted Big Marine Data (BMD) thus issue unprecedented architectural challenges on existing marine communication systems. Current dominant marine communication technologies, e.g., shore-based cellular stations, high frequency radio, and expensive satellites, extremely suffer from short coverage, low bandwidth, insecurity, and unavailable cross-domain transmission. In this paper, Resilient Vessel Network (RVN) is proposed to fundamentally enhance BMD transmission. RVNs with widespread self-organized vessels and opportunistic connections reveal advantages of ubiquity, resilience, low cost and cross-domain transmission. To efficiently manage opportunistic vessel-to-vessel (V2V) connections for optimal routing, Social Network Analysis (SNA) on historical vessel interactions is applied for vessel familiarity measurement and community detection. The performance of the proposed community-based routing (CBR) is comprehensively evaluated with real datasets of fishing vessel trajectories. It is demonstrated that CBR achieves much lower transmission cost with comparable delivery ratio compared to typical routing algorithms.
Sustainability in Software Architecture: A Systematic Mapping Study
Venue: arXiv cs.SE 2022
Year: 2022

Sustainability is an increasingly-studied topic in software engineering in general, and in software architecture in particular. There are already a number of secondary studies addressing sustainability in software engineering, but no such study focusing explicitly on software architecture. This work aims to fill this gap by conducting a systematic mapping study on the intersection between sustainability and software architecture research with the intention of (i) reflecting on the current state of the art, and (ii) identifying the needs for further research. Our results show that, overall, existing works have focused disproportionately on specific aspects of sustainability, and in particular on the most technical and "inward facing" ones. This comes at the expense of the holistic perspective required to address a multi-faceted concern such as sustainability. Furthermore, more reflection-oriented research works, and better coverage of the activities in the architecting life cycle are required to further the maturity of the area. Based on our findings we then propose a research agenda for sustainability-aware software architecture.
Estimation of Reliable Proposal Quality for Temporal Action Detection
Venue: arXiv cs.CV 2022
Year: 2022

Temporal action detection (TAD) aims to locate and recognize the actions in an untrimmed video. Anchor-free methods have made remarkable progress which mainly formulate TAD into two tasks: classification and localization using two separate branches. This paper reveals the temporal misalignment between the two tasks hindering further progress. To address this, we propose a new method that gives insights into moment and region perspectives simultaneously to align the two tasks by acquiring reliable proposal quality. For the moment perspective, Boundary Evaluate Module (BEM) is designed which focuses on local appearance and motion evolvement to estimate boundary quality and adopts a multi-scale manner to deal with varied action durations. For the region perspective, we introduce Region Evaluate Module (REM) which uses a new and efficient sampling method for proposal feature representation containing more contextual information compared with point feature to refine category score and proposal boundary. The proposed Boundary Evaluate Module and Region Evaluate Module (BREM) are generic, and they can be easily integrated with other anchor-free TAD methods to achieve superior performance. In our experiments, BREM is combined with two different frameworks and improves the performance on THUMOS14 by 3.6% and 1.0% respectively, reaching a new state-of-the-art (63.6% average mAP). Meanwhile, a competitive result of 36.2% average mAP is achieved on ActivityNet-1.3 with the consistent improvement of BREM. The codes are released at https://github.com/Junshan233/BREM.
Predicting Real-time Scientific Experiments Using Transformer models and Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Life and physical sciences have always been quick to adopt the latest advances in machine learning to accelerate scientific discovery. Examples of this are cell segmentation or cancer detection. Nevertheless, these exceptional results are based on mining previously created datasets to discover patterns or trends. Recent advances in AI have been demonstrated in real-time scenarios like self-driving cars or playing video games. However, these new techniques have not seen widespread adoption in life or physical sciences because experimentation can be slow. To tackle this limitation, this work aims to adapt generative learning algorithms to model scientific experiments and accelerate their discovery using in-silico simulations. We particularly focused on real-time experiments, aiming to model how they react to user inputs. To achieve this, here we present an encoder-decoder architecture based on the Transformer model to simulate real-time scientific experimentation, predict its future behaviour and manipulate it on a step-by-step basis. As a proof of concept, this architecture was trained to map a set of mechanical inputs to the oscillations generated by a chemical reaction. The model was paired with a Reinforcement Learning controller to show how the simulated chemistry can be manipulated in real-time towards user-defined behaviours. Our results demonstrate how generative learning can model real-time scientific experimentation to track how it changes through time as the user manipulates it, and how the trained models can be paired with optimisation algorithms to discover new phenomena beyond the physical limitations of lab experimentation. This work paves the way towards building surrogate systems where physical experimentation interacts with machine learning on a step-by-step basis.
4DAC: Learning Attribute Compression for Dynamic Point Clouds
Venue: arXiv cs.CV 2022
Year: 2022

With the development of the 3D data acquisition facilities, the increasing scale of acquired 3D point clouds poses a challenge to the existing data compression techniques. Although promising performance has been achieved in static point cloud compression, it remains under-explored and challenging to leverage temporal correlations within a point cloud sequence for effective dynamic point cloud compression. In this paper, we study the attribute (e.g., color) compression of dynamic point clouds and present a learning-based framework, termed 4DAC. To reduce temporal redundancy within data, we first build the 3D motion estimation and motion compensation modules with deep neural networks. Then, the attribute residuals produced by the motion compensation component are encoded by the region adaptive hierarchical transform into residual coefficients. In addition, we also propose a deep conditional entropy model to estimate the probability distribution of the transformed coefficients, by incorporating temporal context from consecutive point clouds and the motion estimation/compensation modules. Finally, the data stream is losslessly entropy coded with the predicted distribution. Extensive experiments on several public datasets demonstrate the superior compression performance of the proposed approach.
Discovering changes in birthing narratives during COVID-19
Venue: arXiv cs.CL 2022
Year: 2022

We investigate whether, and if so how, birthing narratives written by new parents on Reddit changed during COVID-19. Our results indicate that the presence of family members significantly decreased and themes related to induced labor significantly increased in the narratives during COVID-19. Our work builds upon recent research that analyze how new parents use Reddit to describe their birthing experiences.
Parallel coarsening of graph data with spectral guarantees
Venue: arXiv cs.DM 2022
Year: 2022

Finding coarse representations of large graphs is an important computational problem in the fields of scientific computing, large scale graph partitioning, and the reduction of geometric meshes. Of particular interest in all of these fields is the preservation of spectral properties with regards to the original graph. While many methods exist to perform this task, they typically require expensive linear algebraic operations and yield high work complexities. We adapt a spectral coarsening bound from the literature in order to develop a coarsening algorithm with a work complexity that is drastically smaller than previous work. We further show that this algorithm is easily parallelizable and presents impressive scaling results on meshes.
Online Deep Learning from Doubly-Streaming Data
Venue: arXiv cs.LG 2022
Year: 2022

This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. The challenges of this problem are two folds: 1) Data samples ceaselessly flowing in may carry shifted patterns over time, requiring learners to update hence adapt on-the-fly. 2) Newly emerging features are described by very few samples, resulting in weak learners that tend to make error predictions. A plausible idea to overcome the challenges is to establish relationship between the pre-and-post evolving feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional media streams with complex feature interplay, which suffers an tradeoff between onlineness (biasing shallow learners) and expressiveness(requiring deep learners). Motivated by this, we propose a novel OLD^3S paradigm, where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building intermediate feature mapping relationship. A key trait of OLD^3S is to treat the model capacity as a learnable semantics, yields optimal model depth and parameters jointly, in accordance with the complexity and non-linearity of the input data streams in an online fashion. Both theoretical analyses and empirical studies substantiate the viability and effectiveness of our proposal.
How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language
Venue: arXiv cs.CL 2022
Year: 2022

More than 43% of the languages spoken in the world are endangered, and language loss currently occurs at an accelerated rate because of globalization and neocolonialism. Saving and revitalizing endangered languages has become very important for maintaining the cultural diversity on our planet. In this work, we focus on discussing how NLP can help revitalize endangered languages. We first suggest three principles that may help NLP practitioners to foster mutual understanding and collaboration with language communities, and we discuss three ways in which NLP can potentially assist in language education. We then take Cherokee, a severely-endangered Native American language, as a case study. After reviewing the language's history, linguistic features, and existing resources, we (in collaboration with Cherokee community members) arrive at a few meaningful ways NLP practitioners can collaborate with community partners. We suggest two approaches to enrich the Cherokee language's resources with machine-in-the-loop processing, and discuss several NLP tools that people from the Cherokee community have shown interest in. We hope that our work serves not only to inform the NLP community about Cherokee, but also to provide inspiration for future work on endangered languages in general. Our code and data will be open-sourced at https://github.com/ZhangShiyue/RevitalizeCherokee