OpenRaiser/Intern-Atlas huggingface dataset:
Emotional Interaction Qualities: Vocabulary, Modalities, Actions, And Mapping
Venue: arXiv cs.HC 2023
Year: 2023

Have you ever typed particularly powerful on your keyboard, maybe even harsh, to write and send a message with some emphasis of your emotional state or message? Did it work? Probably not. It didn't affect how you typed or interacted with your mouse. But what if you had other, connected devices, with other modalities for inputs and outputs? Which would you have chosen, and how would you characterize your interactions with them? We researched with our multisensory and multimodal tool, the Loaded Dice, in co-design workshops the design space of IoT usage scenarios: what interaction qualities users want, characterized using an interaction vocabulary, and how they might map them to a selection of sensors and actuators. We discuss based on our experience some thoughts of such a mapping.
Learning Modulo Theories
Venue: arXiv cs.LG 2023
Year: 2023

Recent techniques that integrate \emph{solver layers} into Deep Neural Networks (DNNs) have shown promise in bridging a long-standing gap between inductive learning and symbolic reasoning techniques. In this paper we present a set of techniques for integrating \emph{Satisfiability Modulo Theories} (SMT) solvers into the forward and backward passes of a deep network layer, called SMTLayer. Using this approach, one can encode rich domain knowledge into the network in the form of mathematical formulas. In the forward pass, the solver uses symbols produced by prior layers, along with these formulas, to construct inferences; in the backward pass, the solver informs updates to the network, driving it towards representations that are compatible with the solver's theory. Notably, the solver need not be differentiable. We implement \layername as a Pytorch module, and our empirical results show that it leads to models that \emph{1)} require fewer training samples than conventional models, \emph{2)} that are robust to certain types of covariate shift, and \emph{3)} that ultimately learn representations that are consistent with symbolic knowledge, and thus naturally interpretable.
Secure synchronization of artificial neural networks used to correct errors in quantum cryptography
Venue: arXiv cs.CR 2023
Year: 2023

Quantum cryptography can provide a very high level of data security. However, a big challenge of this technique is errors in quantum channels. Therefore, error correction methods must be applied in real implementations. An example is error correction based on artificial neural networks. This paper considers the practical aspects of this recently proposed method and analyzes elements which influence security and efficiency. The synchronization process based on mutual learning processes is analyzed in detail. The results allowed us to determine the impact of various parameters. Additionally, the paper describes the recommended number of iterations for different structures of artificial neural networks and various error rates. All this aims to support users in choosing a suitable configuration of neural networks used to correct errors in a secure and efficient way.
Metaverse for Wireless Systems: Architecture, Advances, Standardization, and Open Challenges
Venue: arXiv cs.NI 2023
Year: 2023

The growing landscape of emerging wireless applications is a key driver toward the development of novel wireless system designs. Such a design can be based on the metaverse that uses a virtual model of the physical world systems along with other schemes/technologies (e.g., optimization theory, machine learning, and blockchain). A metaverse using a virtual model performs proactive intelligent analytics prior to a user request for efficient management of the wireless system resources. Additionally, a metaverse will enable self-sustainability to operate wireless systems with the least possible intervention from network operators. Although the metaverse can offer many benefits, it faces some challenges as well. Therefore, in this tutorial, we discuss the role of a metaverse in enabling wireless applications. We present an overview, key enablers, design aspects (i.e., metaverse for wireless and wireless for metaverse), and a novel high-level architecture of metaverse-based wireless systems. We discuss metaverse management, reliability, and security of the metaverse-based system. Furthermore, we discuss recent advances and standardization of metaverse-enabled wireless system. Finally, we outline open challenges and present possible solutions.
Personalised Federated Learning On Heterogeneous Feature Spaces
Venue: arXiv cs.LG 2023
Year: 2023

Most personalised federated learning (FL) approaches assume that raw data of all clients are defined in a common subspace i.e. all clients store their data according to the same schema. For real-world applications, this assumption is restrictive as clients, having their own systems to collect and then store data, may use heterogeneous data representations. We aim at filling this gap. To this end, we propose a general framework coined FLIC that maps client's data onto a common feature space via local embedding functions. The common feature space is learnt in a federated manner using Wasserstein barycenters while the local embedding functions are trained on each client via distribution alignment. We integrate this distribution alignement mechanism into a federated learning approach and provide the algorithmics of FLIC. We compare its performances against FL benchmarks involving heterogeneous input features spaces. In addition, we provide theoretical insights supporting the relevance of our methodology.
Dual-View Selective Instance Segmentation Network for Unstained Live Adherent Cells in Differential Interference Contrast Images
Venue: arXiv cs.CV 2023
Year: 2023

Despite recent advances in data-independent and deep-learning algorithms, unstained live adherent cell instance segmentation remains a long-standing challenge in cell image processing. Adherent cells' inherent visual characteristics, such as low contrast structures, fading edges, and irregular morphology, have made it difficult to distinguish from one another, even by human experts, let alone computational methods. In this study, we developed a novel deep-learning algorithm called dual-view selective instance segmentation network (DVSISN) for segmenting unstained adherent cells in differential interference contrast (DIC) images. First, we used a dual-view segmentation (DVS) method with pairs of original and rotated images to predict the bounding box and its corresponding mask for each cell instance. Second, we used a mask selection (MS) method to filter the cell instances predicted by the DVS to keep masks closest to the ground truth only. The developed algorithm was trained and validated on our dataset containing 520 images and 12198 cells. Experimental results demonstrate that our algorithm achieves an AP_segm of 0.555, which remarkably overtakes a benchmark by a margin of 23.6%. This study's success opens up a new possibility of using rotated images as input for better prediction in cell images.
Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation
Venue: arXiv cs.CL 2023
Year: 2023

Non-autoregressive translation (NAT) model achieves a much faster inference speed than the autoregressive translation (AT) model because it can simultaneously predict all tokens during inference. However, its translation quality suffers from degradation compared to AT. And existing NAT methods only focus on improving the NAT model's performance but do not fully utilize it. In this paper, we propose a simple but effective method called "Candidate Soups," which can obtain high-quality translations while maintaining the inference speed of NAT models. Unlike previous approaches that pick the individual result and discard the remainders, Candidate Soups (CDS) can fully use the valuable information in the different candidate translations through model uncertainty. Extensive experiments on two benchmarks (WMT'14 EN-DE and WMT'16 EN-RO) demonstrate the effectiveness and generality of our proposed method, which can significantly improve the translation quality of various base models. More notably, our best variant outperforms the AT model on three translation tasks with 7.6 times speedup.
Semi-Parametric Video-Grounded Text Generation
Venue: arXiv cs.CV 2023
Year: 2023

Efficient video-language modeling should consider the computational cost because of a large, sometimes intractable, number of video frames. Parametric approaches such as the attention mechanism may not be ideal since its computational cost quadratically increases as the video length increases. Rather, previous studies have relied on offline feature extraction or frame sampling to represent the video efficiently, focusing on cross-modal modeling in short video clips. In this paper, we propose a semi-parametric video-grounded text generation model, SeViT, a novel perspective on scalable video-language modeling toward long untrimmed videos. Treating a video as an external data store, SeViT includes a non-parametric frame retriever to select a few query-relevant frames from the data store for a given query and a parametric generator to effectively aggregate the frames with the query via late fusion methods. Experimental results demonstrate our method has a significant advantage in longer videos and causal video understanding. Moreover, our model achieves the new state of the art on four video-language datasets, iVQA (+4.8), Next-QA (+6.9), and Activitynet-QA (+4.8) in accuracy, and MSRVTT-Caption (+3.6) in CIDEr.
Learning Informative Representation for Fairness-aware Multivariate Time-series Forecasting: A Group-based Perspective
Venue: arXiv cs.LG 2023
Year: 2023

Performance unfairness among variables widely exists in multivariate time series (MTS) forecasting models since such models may attend/bias to certain (advantaged) variables. Addressing this unfairness problem is important for equally attending to all variables and avoiding vulnerable model biases/risks. However, fair MTS forecasting is challenging and has been less studied in the literature. To bridge such significant gap, we formulate the fairness modeling problem as learning informative representations attending to both advantaged and disadvantaged variables. Accordingly, we propose a novel framework, named FairFor, for fairness-aware MTS forecasting. FairFor is based on adversarial learning to generate both group-independent and group-relevant representations for the downstream forecasting. The framework first leverages a spectral relaxation of the K-means objective to infer variable correlations and thus to group variables. Then, it utilizes a filtering&fusion component to filter the group-relevant information and generate group-independent representations via orthogonality regularization. The group-independent and group-relevant representations form highly informative representations, facilitating to sharing knowledge from advantaged variables to disadvantaged variables to guarantee fairness. Extensive experiments on four public datasets demonstrate the effectiveness of our proposed FairFor for fair forecasting and significant performance improvement.
Targeted Attacks on Timeseries Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Real-world deep learning models developed for Time Series Forecasting are used in several critical applications ranging from medical devices to the security domain. Many previous works have shown how deep learning models are prone to adversarial attacks and studied their vulnerabilities. However, the vulnerabilities of time series models for forecasting due to adversarial inputs are not extensively explored. While the attack on a forecasting model might aim to deteriorate the performance of the model, it is more effective, if the attack is focused on a specific impact on the model's output. In this paper, we propose a novel formulation of Directional, Amplitudinal, and Temporal targeted adversarial attacks on time series forecasting models. These targeted attacks create a specific impact on the amplitude and direction of the output prediction. We use the existing adversarial attack techniques from the computer vision domain and adapt them for time series. Additionally, we propose a modified version of the Auto Projected Gradient Descent attack for targeted attacks. We examine the impact of the proposed targeted attacks versus untargeted attacks. We use KS-Tests to statistically demonstrate the impact of the attack. Our experimental results show how targeted attacks on time series models are viable and are more powerful in terms of statistical similarity. It is, hence difficult to detect through statistical methods. We believe that this work opens a new paradigm in the time series forecasting domain and represents an important consideration for developing better defenses.
Noncontact Haptic Rendering of Static Contact with Convex Surface Using Circular Movement of Ultrasound Focus on a Finger Pad
Venue: arXiv cs.HC 2023
Year: 2023

A noncontact tactile stimulus can be presented by focusing airborne ultrasound on the human skin. Focused ultrasound has recently been reported to produce not only vibration but also static pressure sensation on the palm by modulating the sound pressure distribution at a low frequency. This finding expands the potential for tactile rendering in ultrasound haptics as static pressure sensation is perceived with a high spatial resolution. In this study, we verified that focused ultrasound can render a static pressure sensation associated with contact with a small convex surface on a finger pad. This static contact rendering enables noncontact tactile reproduction of a fine uneven surface using ultrasound. In the experiments, four ultrasound foci were simultaneously and circularly rotated on a finger pad at 5 Hz. When the orbit radius was 3 mm, vibration and focal movements were barely perceptible, and the stimulus was perceived as static pressure. Moreover, under the condition, the pressure sensation rendered a contact with a small convex surface with a radius of 2 mm. The perceived intensity of the static contact sensation was equivalent to a physical contact force of 0.24 N on average, which was 12 times the radiation force physically applied to the skin.
Down the Rabbit Hole: Detecting Online Extremism, Radicalisation, and Politicised Hate Speech
Venue: arXiv cs.SI 2023
Year: 2023

Social media is a modern person's digital voice to project and engage with new ideas and mobilise communities $\unicode{x2013}$ a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms for Extremism, Radicalisation, and Hate speech (ERH) detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safety and free expression. Hence, we propose and examine the unique research field of ERH context mining to unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015-2021 51-study Systematic Literature Review (SLR) provides the first cross-examination of textual, network, and visual approaches to detecting extremist affiliation, hateful content, and radicalisation towards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.
A Green(er) World for A.I
Venue: arXiv cs.AI 2023
Year: 2023

As research and practice in artificial intelligence (A.I.) grow in leaps and bounds, the resources necessary to sustain and support their operations also grow at an increasing pace. While innovations and applications from A.I. have brought significant advances, from applications to vision and natural language to improvements to fields like medical imaging and materials engineering, their costs should not be neglected. As we embrace a world with ever-increasing amounts of data as well as research and development of A.I. applications, we are sure to face an ever-mounting energy footprint to sustain these computational budgets, data storage needs, and more. But, is this sustainable and, more importantly, what kind of setting is best positioned to nurture such sustainable A.I. in both research and practice? In this paper, we outline our outlook for Green A.I. -- a more sustainable, energy-efficient and energy-aware ecosystem for developing A.I. across the research, computing, and practitioner communities alike -- and the steps required to arrive there. We present a bird's eye view of various areas for potential changes and improvements from the ground floor of AI's operational and hardware optimizations for datacenters/HPCs to the current incentive structures in the world of A.I. research and practice, and more. We hope these points will spur further discussion, and action, on some of these issues and their potential solutions.
Adversarial Learning for Implicit Semantic-Aware Communications
Venue: arXiv cs.LG 2023
Year: 2023

Semantic communication is a novel communication paradigm that focuses on recognizing and delivering the desired meaning of messages to the destination users. Most existing works in this area focus on delivering explicit semantics, labels or signal features that can be directly identified from the source signals. In this paper, we consider the implicit semantic communication problem in which hidden relations and closely related semantic terms that cannot be recognized from the source signals need to also be delivered to the destination user. We develop a novel adversarial learning-based implicit semantic-aware communication (iSAC) architecture in which the source user, instead of maximizing the total amount of information transmitted to the channel, aims to help the recipient learn an inference rule that can automatically generate implicit semantics based on limited clue information. We prove that by applying iSAC, the destination user can always learn an inference rule that matches the true inference rule of the source messages. Experimental results show that the proposed iSAC can offer up to a 19.69 dB improvement over existing non-inferential communication solutions, in terms of symbol error rate at the destination user.
MissBeamNet: Learning Missing Doppler Velocity Log Beam Measurements
Venue: arXiv cs.RO 2023
Year: 2023

One of the primary means of sea exploration is autonomous underwater vehicles (AUVs). To perform these tasks, AUVs must navigate the rough challenging sea environment. AUVs usually employ an inertial navigation system (INS), aided by a Doppler velocity log (DVL), to provide the required navigation accuracy. The DVL transmits four acoustic beams to the seafloor, and by measuring changes in the frequency of the returning beams, the DVL can estimate the AUV velocity vector. However, in practical scenarios, not all the beams are successfully reflected. When only three beams are available, the accuracy of the velocity vector is degraded. When fewer than three beams are reflected, the DVL cannot estimate the AUV velocity vector. This paper presents a data-driven approach, MissBeamNet, to regress the missing beams in partial DVL beam measurement cases. To that end, a deep neural network (DNN) model is designed to process the available beams along with past DVL measurements to regress the missing beams. The AUV velocity vector is estimated using the available measured and regressed beams. To validate the proposed approach, sea experiments were made with the "Snapir" AUV, resulting in an 11 hours dataset of DVL measurements. Our results show that the proposed system can accurately estimate velocity vectors in situations of missing beam measurements. Our dataset and codebase implementing the described framework is available at our GitHub repository https://github.com/ansfl/MissBeamNet .
Turing Machines Equipped with CTC in Physical Universes
Venue: arXiv cs.CC 2023
Year: 2023

We study the paradoxical aspects of closed time-like curves and their impact on the theory of computation. After introducing the $\text{TM}_\text{CTC}$, a classical Turing machine benefiting CTCs for backward time travel, Aaronson et al. proved that $\text{P} = \text{PSPACE}$ and the $Δ_2$ sets, such as the halting problem, are computable within this computational model. Our critical view is the physical consistency of this model, which leads to proposing the strong axiom, explaining that every particle rounding on a CTC will be destroyed before returning to its starting time, and the weak axiom, describing the same notion, particularly for Turing machines. We claim that in a universe containing CTCs, the two axioms must be true; otherwise, there will be an infinite number of any particle rounding on a CTC in the universe. An immediate result of the weak axiom is the incapability of Turing machines to convey information for a full round on a CTC, leading to the proposed $\text{TM}_\text{CTC}$ programs for the aforementioned corollaries failing to function. We suggest our solution for this problem as the data transferring hypothesis, which applies another $\text{TM}_\text{CTC}$ as a means for storing data. A prerequisite for it is the existence of the concept of Turing machines throughout time, which makes it appear infeasible in our universe. Then, we discuss possible physical conditions that can be held for a universe containing CTCs and conclude that if returning to an approximately equivalent universe by a CTC was conceivable, the above corollaries would be valid.
Matching Linear Algebra and Tensor Code to Specialized Hardware Accelerators
Venue: arXiv cs.PL 2023
Year: 2023

Dedicated tensor accelerators demonstrate the importance of linear algebra in modern applications. Such accelerators have the potential for impressive performance gains, but require programmers to rewrite code using vendor APIs - a barrier to wider scale adoption. Recent work overcomes this by matching and replacing patterns within code, but such approaches are fragile and fail to cope with the diversity of real-world codes. We develop ATC, a compiler that uses program synthesis to map regions of code to specific APIs. The mapping space that ATC explores is combinatorially large, requiring the development of program classification, dynamic analysis, variable constraint generation and lexical distance matching techniques to make it tractable. We apply ATC to real-world tensor and linear algebra codes and evaluate them against four state-of-the-art approaches. We accelerate between 2.6x and 7x more programs, leading to over an order of magnitude performance improvement.
Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?
Venue: arXiv cs.CL 2023
Year: 2023

In this article, we use probing to investigate phenomena that occur during fine-tuning and knowledge distillation of a BERT-based natural language understanding (NLU) model. Our ultimate purpose was to use probing to better understand practical production problems and consequently to build better NLU models. We designed experiments to see how fine-tuning changes the linguistic capabilities of BERT, what the optimal size of the fine-tuning dataset is, and what amount of information is contained in a distilled NLU based on a tiny Transformer. The results of the experiments show that the probing paradigm in its current form is not well suited to answer such questions. Structural, Edge and Conditional probes do not take into account how easy it is to decode probed information. Consequently, we conclude that quantification of information decodability is critical for many practical applications of the probing paradigm.