OpenRaiser/Intern-Atlas huggingface dataset:
Regulating trusted autonomous systems in Australia
Venue: arXiv cs.CY 2023
Year: 2023

Australia is a leader in autonomous systems technology, particularly in the mining industry, borne from necessity in a geographically dispersed and complex natural environment. Increasingly advanced autonomous systems are becoming more prevalent in Australia, particularly as the safety, environmental and efficiency benefits become better understood, and the increasing sophistication of technology improves capability and availability. Increasing use of these systems, including in the maritime domain and air domain, is placing pressure on the national safety regulators, who must either continue to apply their traditional regulatory approach requiring exemptions to enable operation of emerging technology, or seize the opportunity to put in place an agile and adaptive approach better suited to the rapid developments of the twenty first century. In Australia the key national safety regulators have demonstrated an appetite for working with industry to facilitate innovation, but their limited resources mean progress is slow. There is a critical role to be played by third parties from industry, government, and academia who can work together to develop, test and publish new assurance and accreditation frameworks for trusted autonomous systems, and assist in the transition to an adaptive and agile regulatory philosophy. This is necessary to ensure the benefits of autonomous systems can be realised, without compromising safety. This paper will identify the growing use cases for autonomous systems in Australia, in the maritime, air and land domains, assess the current regulatory framework, argue that Australia's regulatory approach needs to become more agile and anticipatory, and investigate how third party projects could positively impact the assurance and accreditation process for autonomous systems in the future.
Information-Theoretic Diffusion
Venue: arXiv cs.LG 2023
Year: 2023

Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation. We introduce a new mathematical foundation for diffusion models inspired by classic results in information theory that connect Information with Minimum Mean Square Error regression, the so-called I-MMSE relations. We generalize the I-MMSE relations to exactly relate the data distribution to an optimal denoising regression problem, leading to an elegant refinement of existing diffusion bounds. This new insight leads to several improvements for probability distribution estimation, including theoretical justification for diffusion model ensembling. Remarkably, our framework shows how continuous and discrete probabilities can be learned with the same regression objective, avoiding domain-specific generative models used in variational methods. Code to reproduce experiments is provided at http://github.com/kxh001/ITdiffusion and simplified demonstration code is at http://github.com/gregversteeg/InfoDiffusionSimple.
MACOptions: Multi-Agent Learning with Centralized Controller and Options Framework
Venue: arXiv cs.AI 2023
Year: 2023

These days automation is being applied everywhere. In every environment, planning for the actions to be taken by the agents is an important aspect. In this paper, we plan to implement planning for multi-agents with a centralized controller. We compare three approaches: random policy, Q-learning, and Q-learning with Options Framework. We also show the effectiveness of planners by showing performance comparison between Q-Learning with Planner and without Planner.
A Prototype-Oriented Clustering for Domain Shift with Source Privacy
Venue: arXiv cs.LG 2023
Year: 2023

Unsupervised clustering under domain shift (UCDS) studies how to transfer the knowledge from abundant unlabeled data from multiple source domains to learn the representation of the unlabeled data in a target domain. In this paper, we introduce Prototype-oriented Clustering with Distillation (PCD) to not only improve the performance and applicability of existing methods for UCDS, but also address the concerns on protecting the privacy of both the data and model of the source domains. PCD first constructs a source clustering model by aligning the distributions of prototypes and data. It then distills the knowledge to the target model through cluster labels provided by the source model while simultaneously clustering the target data. Finally, it refines the target model on the target domain data without guidance from the source model. Experiments across multiple benchmarks show the effectiveness and generalizability of our source-private clustering method.
DeepVATS: Deep Visual Analytics for Time Series
Venue: arXiv cs.LG 2023
Year: 2023

The field of Deep Visual Analytics (DVA) has recently arisen from the idea of developing Visual Interactive Systems supported by deep learning, in order to provide them with large-scale data processing capabilities and to unify their implementation across different data and domains. In this paper we present DeepVATS, an open-source tool that brings the field of DVA into time series data. DeepVATS trains, in a self-supervised way, a masked time series autoencoder that reconstructs patches of a time series, and projects the knowledge contained in the embeddings of that model in an interactive plot, from which time series patterns and anomalies emerge and can be easily spotted. The tool includes a back-end for data processing pipeline and model training, as well as a front-end with a interactive user interface. We report on results that validate the utility of DeepVATS, running experiments on both synthetic and real datasets. The code is publicly available on https://github.com/vrodriguezf/deepvats
CRAFT: Criticality-Aware Fault-Tolerance Enhancement Techniques for Emerging Memories-Based Deep Neural Networks
Venue: arXiv cs.AR 2023
Year: 2023

Deep Neural Networks (DNNs) have emerged as the most effective programming paradigm for computer vision and natural language processing applications. With the rapid development of DNNs, efficient hardware architectures for deploying DNN-based applications on edge devices have been extensively studied. Emerging Non-Volatile Memories (NVMs), with their better scalability, non-volatility and good read performance, are found to be promising candidates for deploying DNNs. However, despite the promise, emerging NVMs often suffer from reliability issues such as stuck-at faults, which decrease the chip yield/memory lifetime and severely impact the accuracy of DNNs. A stuck-at cell can be read but not reprogrammed, thus, stuck-at faults in NVMs may or may not result in errors depending on the data to be stored. By reducing the number of errors caused by stuck-at faults, the reliability of a DNN-based system can be enhanced. This paper proposes CRAFT, i.e., Criticality-Aware Fault-Tolerance Enhancement Techniques to enhance the reliability of NVM-based DNNs in the presence of stuck-at faults. A data block remapping technique is used to reduce the impact of stuck-at faults on DNNs accuracy. Additionally, by performing bit-level criticality analysis on various DNNs, the critical-bit positions in network parameters that can significantly impact the accuracy are identified. Based on this analysis, we propose an encoding method which effectively swaps the critical bit positions with that of non-critical bits when more errors (due to stuck-at faults) are present in the critical bits.
Geometric Perception based Efficient Text Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Every Scene Text Recognition (STR) task consists of text localization \& text recognition as the prominent sub-tasks. However, in real-world applications with fixed camera positions such as equipment monitor reading, image-based data entry, and printed document data extraction, the underlying data tends to be regular scene text. Hence, in these tasks, the use of generic, bulky models comes up with significant disadvantages compared to customized, efficient models in terms of model deployability, data privacy \& model reliability. Therefore, this paper introduces the underlying concepts, theory, implementation, and experiment results to develop models, which are highly specialized for the task itself, to achieve not only the SOTA performance but also to have minimal model weights, shorter inference time, and high model reliability. We introduce a novel deep learning architecture (GeoTRNet), trained to identify digits in a regular scene image, only using the geometrical features present, mimicking human perception over text recognition. The code is publicly available at https://github.com/ACRA-FL/GeoTRNet
Blockchain-based certificate authentication system with enabling correction
Venue: arXiv cs.CR 2023
Year: 2023

Blockchain has proven to be an emerging technology in the digital world, changing the way everyone thinks about data security and bringing efficiency to several industries. It has already been applied to a wide range of applications, from financial services and supply chain management to voting systems and identity verification. An organization must verify its candidates before selecting them. Choosing an unqualified candidate can ruin an organization's reputation. In this digital era, many key fraudulent schemes are rampant in many companies and one of them is certificate fraud. It is possible to validate a candidate's qualifications using traditional methods, but there are drawbacks such as security issues and time consumption. In this paper, a blockchain-based academic certificate authentication system will be used to ensure authenticity and make the assertion of the decentralized system secure. However, the system will generate, authenticate and make corrections on academic certificates. Ultimately, some blockchain-based authentication systems already exist, they can't correct any errors that occur during generation. The proposed system will help in many ways, such as providing a user-friendly university admission, and smooth job hiring process, etc. In conclusion, our proposed system can permanently eradicate certificate forgeries and create and promote trust in society.
Channel Estimation for Reconfigurable Intelligent Surface with a few Active Elements
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, a channel estimation technique for reconfigurable intelligent surface (RIS)-aided multi-user multiple-input single-output communication systems is proposed. By deploying a small number of active elements at the RIS, the RIS can receive and process the training signals. Through the partial channel state information (CSI) obtained from the active elements, the overall training overhead to estimate the entire channel can be dramatically reduced. To minimize the estimation complexity, the proposed technique is based on the linear combination of partial CSI, which only requires linear matrix operations. By exploiting the spatial correlation among the RIS elements, proper weights for the linear combination and normalization factors are developed. Numerical results show that the proposed technique outperforms other schemes using the active elements at the RIS in terms of the normalized mean squared error when the number of active elements is small, which is necessary to maintain the low cost and power consumption of RIS.
CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back
Venue: arXiv cs.SE 2023
Year: 2023

Representing code changes as numeric feature vectors, i.e., code change representations, is usually an essential step to automate many software engineering tasks related to code changes, e.g., commit message generation and just-in-time defect prediction. Intuitively, the quality of code change representations is crucial for the effectiveness of automated approaches. Prior work on code changes usually designs and evaluates code change representation approaches for a specific task, and little work has investigated code change encoders that can be used and jointly trained on various tasks. To fill this gap, this work proposes a novel Code Change Representation learning approach named CCRep, which can learn to encode code changes as feature vectors for diverse downstream tasks. Specifically, CCRep regards a code change as the combination of its before-change and after-change code, leverages a pre-trained code model to obtain high-quality contextual embeddings of code, and uses a novel mechanism named query back to extract and encode the changed code fragments and make them explicitly interact with the whole code change. To evaluate CCRep and demonstrate its applicability to diverse code-change-related tasks, we apply it to three tasks: commit message generation, patch correctness assessment, and just-in-time defect prediction. Experimental results show that CCRep outperforms the state-of-the-art techniques on each task.
Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution
Venue: arXiv cs.LG 2023
Year: 2023

Inductive one-bit matrix completion is motivated by modern applications such as recommender systems, where new users would appear at test stage with the ratings consisting of only ones and no zeros. We propose a unified graph signal sampling framework which enjoys the benefits of graph signal analysis and processing. The key idea is to transform each user's ratings on the items to a function (signal) on the vertices of an item-item graph, then learn structural graph properties to recover the function from its values on certain vertices -- the problem of graph signal sampling. We propose a class of regularization functionals that takes into account discrete random label noise in the graph vertex domain, then develop the GS-IMC approach which biases the reconstruction towards functions that vary little between adjacent vertices for noise reduction. Theoretical result shows that accurate reconstructions can be achieved under mild conditions. For the online setting, we develop a Bayesian extension, i.e., BGS-IMC which considers continuous random Gaussian noise in the graph Fourier domain and builds upon a prediction-correction update algorithm to obtain the unbiased and minimum-variance reconstruction. Both GS-IMC and BGS-IMC have closed-form solutions and thus are highly scalable in large data. Experiments show that our methods achieve state-of-the-art performance on public benchmarks.
Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving
Venue: arXiv cs.RO 2023
Year: 2023

Predicting the behaviors of other road users is crucial to safe and intelligent decision-making for autonomous vehicles (AVs). However, most motion prediction models ignore the influence of the AV's actions and the planning module has to treat other agents as unalterable moving obstacles. To address this problem, this paper proposes an interaction-aware motion prediction model that is able to predict other agents' future trajectories according to the ego agent's future plan, i.e., their reactions to the ego's actions. Specifically, we employ Transformers to effectively encode the driving scene and incorporate the AV's plan in decoding the predicted trajectories. To train the model to accurately predict the reactions of other agents, we develop an online learning framework, where the ego agent explores the environment and collects other agents' reactions to itself. We validate the decision-making and learning framework in three highly interactive simulated driving scenarios. The results reveal that our decision-making method significantly outperforms the reinforcement learning methods in terms of data efficiency and performance. We also find that using the interaction-aware model can bring better performance than the non-interaction-aware model and the exploration process helps improve the success rate in testing.
A Perceptual Study of Sound Ecology in Peripheral Sonification
Venue: arXiv cs.HC 2023
Year: 2023

Based on a case study on 3D printing, we have been experimenting on the sonification of multidimensional data for peripheral process monitoring. In a previous paper, we tested the effectiveness of a soundscape which combined intentionally incongruous natural and musical sounds. This was based on the hypothesis that auditory stimuli could better stand out from one another if they were less ecologically coherent, thus allowing for better reaction rates to various notifications. In this paper, we follow up on that hypothesis by testing two new acoustic ecologies, each exclusively consisting of either musical or natural sounds. We then run those ecologies through the same dual-task evaluation process as the previous one in order to compare them. The results seem to favor our hypothesis, as the new ecologies were not detected as accurately as the original. Though, the set of natural sounds seemed to be considered less intrusive by testers, and to allow for a better performance at an external primary task. We hope to see this work become part of a much larger corpus of studies, which may eventually provide a more definite answer on the effect of ecological coherence in peripheral soundscape design.
Dual-interest Factorization-heads Attention for Sequential Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

Accurate user interest modeling is vital for recommendation scenarios. One of the effective solutions is the sequential recommendation that relies on click behaviors, but this is not elegant in the video feed recommendation where users are passive in receiving the streaming contents and return skip or no-skip behaviors. Here skip and no-skip behaviors can be treated as negative and positive feedback, respectively. With the mixture of positive and negative feedback, it is challenging to capture the transition pattern of behavioral sequence. To do so, FeedRec has exploited a shared vanilla Transformer, which may be inelegant because head interaction of multi-heads attention does not consider different types of feedback. In this paper, we propose Dual-interest Factorization-heads Attention for Sequential Recommendation (short for DFAR) consisting of feedback-aware encoding layer, dual-interest disentangling layer and prediction layer. In the feedback-aware encoding layer, we first suppose each head of multi-heads attention can capture specific feedback relations. Then we further propose factorization-heads attention which can mask specific head interaction and inject feedback information so as to factorize the relation between different types of feedback. Additionally, we propose a dual-interest disentangling layer to decouple positive and negative interests before performing disentanglement on their representations. Finally, we evolve the positive and negative interests by corresponding towers whose outputs are contrastive by BPR loss. Experiments on two real-world datasets show the superiority of our proposed method against state-of-the-art baselines. Further ablation study and visualization also sustain its effectiveness. We release the source code here: https://github.com/tsinghua-fib-lab/WWW2023-DFAR.
Convolutional Neural Networks Trained to Identify Words Provide a Surprisingly Good Account of Visual Form Priming Effects
Venue: arXiv cs.CV 2023
Year: 2023

A wide variety of orthographic coding schemes and models of visual word identification have been developed to account for masked priming data that provide a measure of orthographic similarity between letter strings. These models tend to include hand-coded orthographic representations with single unit coding for specific forms of knowledge (e.g., units coding for a letter in a given position). Here we assess how well a range of these coding schemes and models account for the pattern of form priming effects taken from the Form Priming Project and compare these findings to results observed with 11 standard deep neural network models (DNNs) developed in computer science. We find that deep convolutional networks (CNNs) perform as well or better than the coding schemes and word recognition models, whereas transformer networks did less well. The success of CNNs is remarkable as their architectures were not developed to support word recognition (they were designed to perform well on object recognition), they classify pixel images of words (rather than artificial encodings of letter strings), and their training was highly simplified (not respecting many key aspects of human experience). In addition to these form priming effects, we find that the DNNs can account for visual similarity effects on priming that are beyond all current psychological models of priming. The findings add to the recent work of (Hannagan et al., 2021) and suggest that CNNs should be given more attention in psychology as models of human visual word recognition.
InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition
Venue: arXiv cs.LG 2023
Year: 2023

Recognizing facial activity is a well-understood (but non-trivial) computer vision problem. However, reliable solutions require a camera with a good view of the face, which is often unavailable in wearable settings. Furthermore, in wearable applications, where systems accompany users throughout their daily activities, a permanently running camera can be problematic for privacy (and legal) reasons. This work presents an alternative solution based on the fusion of wearable inertial sensors, planar pressure sensors, and acoustic mechanomyography (muscle sounds). The sensors were placed unobtrusively in a sports cap to monitor facial muscle activities related to facial expressions. We present our integrated wearable sensor system, describe data fusion and analysis methods, and evaluate the system in an experiment with thirteen subjects from different cultural backgrounds (eight countries) and both sexes (six women and seven men). In a one-model-per-user scheme and using a late fusion approach, the system yielded an average F1 score of 85.00% for the case where all sensing modalities are combined. With a cross-user validation and a one-model-for-all-user scheme, an F1 score of 79.00% was obtained for thirteen participants (six females and seven males). Moreover, in a hybrid fusion (cross-user) approach and six classes, an average F1 score of 82.00% was obtained for eight users. The results are competitive with state-of-the-art non-camera-based solutions for a cross-user study. In addition, our unique set of participants demonstrates the inclusiveness and generalizability of the approach.
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller networks. In this paper, we explore offline compression methods, meaning computationally-cheap approaches that do not require further fine-tuning of the compressed model. We challenge the classical matrix factorization methods by proposing a novel, better-performing autoencoder-based framework. We perform a comprehensive ablation study of our approach, examining its different aspects over a diverse set of evaluation settings. Moreover, we show that enabling collaboration between modules across layers by compressing certain modules together positively impacts the final model performance. Experiments on various NLP tasks demonstrate that our approach significantly outperforms commonly used factorization-based offline compression methods.