OpenRaiser/Intern-Atlas huggingface dataset:
On the Performance Optimization of Two-way Hybrid VLC/RF based IoT System over Cellular Spectrum
Venue: arXiv cs.NI 2022
Year: 2022

This paper investigates the system outage performance of a useful architecture of two-way hybrid visible light communication/radio frequency (VLC/RF) communication using overlay mode of cooperative cognitive radio network (CCRN). The demand of high data rate application can be fulfilled using VLC link and communication over a wide area of coverage with high reliability can be achieved through RF link. In the proposed architecture, cooperative communication between two licensed user (LU) nodes is accomplished via an aggregation agent (AA). AA can perform like a relay node and in return, it can access the LU spectrum for two-way communications with Internet-of-Things (IoT) device. First, closed form expressions of outage probability of both LU and IoT communication are established. On the basis of these expressions, optimization problems are formulated to achieve minimum outage probability of both LU and IoT network. The impacts of both VLC and RF system parameters on these systems outage probability and throughput are finally shown in simulation results.
Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2022
Year: 2022

The emerging technology of Reconfigurable Intelligent Surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this paper, we consider multi-user and multi-RIS-empowered wireless systems, and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on Deep Reinforcement Learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment, while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth Generation (6G) era are presented along with some key open research challenges. Differently from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multi-armed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional Deep Q-Network (DQN) algorithm, but with lower implementation complexity.
Past and Future Motion Guided Network for Audio Visual Event Localization
Venue: arXiv cs.CV 2022
Year: 2022

In recent years, audio-visual event localization has attracted much attention. It's purpose is to detect the segment containing audio-visual events and recognize the event category from untrimmed videos. Existing methods use audio-guided visual attention to lead the model pay attention to the spatial area of the ongoing event, devoting to the correlation between audio and visual information but ignoring the correlation between audio and spatial motion. We propose a past and future motion extraction (pf-ME) module to mine the visual motion from videos ,embedded into the past and future motion guided network (PFAGN), and motion guided audio attention (MGAA) module to achieve focusing on the information related to interesting events in audio modality through the past and future visual motion. We choose AVE as the experimental verification dataset and the experiments show that our method outperforms the state-of-the-arts in both supervised and weakly-supervised settings.
PGADA: Perturbation-Guided Adversarial Alignment for Few-shot Learning Under the Support-Query Shift
Venue: arXiv cs.CV 2022
Year: 2022

Few-shot learning methods aim to embed the data to a low-dimensional embedding space and then classify the unseen query data to the seen support set. While these works assume that the support set and the query set lie in the same embedding space, a distribution shift usually occurs between the support set and the query set, i.e., the Support-Query Shift, in the real world. Though optimal transportation has shown convincing results in aligning different distributions, we find that the small perturbations in the images would significantly misguide the optimal transportation and thus degrade the model performance. To relieve the misalignment, we first propose a novel adversarial data augmentation method, namely Perturbation-Guided Adversarial Alignment (PGADA), which generates the hard examples in a self-supervised manner. In addition, we introduce Regularized Optimal Transportation to derive a smooth optimal transportation plan. Extensive experiments on three benchmark datasets manifest that our framework significantly outperforms the eleven state-of-the-art methods on three datasets.
Cross-lingual Adaptation for Recipe Retrieval with Mixup
Venue: arXiv cs.CV 2022
Year: 2022

Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different languages. Moreover, only recipes are available for training in the target domain. A novel recipe mixup method is proposed to learn transferable embedding features between the two domains. Specifically, recipe mixup produces mixed recipes to form an intermediate domain by discretely exchanging the section(s) between source and target recipes. To bridge the domain gap, recipe mixup loss is proposed to enforce the intermediate domain to locate in the shortest geodesic path between source and target domains in the recipe embedding space. By using Recipe 1M dataset as source domain (English) and Vireo-FoodTransfer dataset as target domain (Chinese), empirical experiments verify the effectiveness of recipe mixup for cross-lingual adaptation in the context of image-to-recipe retrieval.
Network Traffic Anomaly Detection Method Based on Multi scale Residual Feature
Venue: arXiv cs.NI 2022
Year: 2022

To address the problem that traditional network traffic anomaly detection algorithms do not suffi-ciently mine potential features in long time domain, an anomaly detection method based on mul-ti-scale residual features of network traffic is proposed. The original traffic is divided into subse-quences of different time spans using sliding windows, and each subsequence is decomposed and reconstructed into data sequences of different levels using wavelet transform technique; the stacked autoencoder (SAE) constructs similar feature space using normal network traffic, and gen-erates reconstructed error vector using the difference between reconstructed samples and input samples in the similar feature space; the multi-path residual group is used to learn reconstructed error The traffic classification is completed by a lightweight classifier. The experimental results show that the detection performance of the proposed method for anomalous network traffic is sig-nificantly improved compared with traditional methods; it confirms that the longer time span and more S transformation scales have positive effects on discovering potential diversity information in the original network traffic.
FOLPETTI: A Novel Multi-Armed Bandit Smart Attack for Wireless Networks
Venue: arXiv cs.CR 2022
Year: 2022

Channel hopping provides a defense mechanism against jamming attacks in large scale \ac{iot} networks.} However, a sufficiently powerful attacker may be able to learn the channel hopping pattern and efficiently predict the channel to jam. In this paper, we present FOLPETTI, a MAB-based attack to dynamically follow the victim's channel selection in real-time. Compared to previous attacks implemented via DRL, FOLPETTI does not require recurrent training phases to capture the victim's behavior, allowing hence a continuous attack. We assess the validity of FOLPETTI by implementing it to launch a jamming attack. We evaluate its performance against a victim performing random channel selection and a victim implementing a MAB defence strategy. We assume that the victim detects an attack when more than $20\%$ of the transmitted packets are not received, therefore this represents the limit for the attack to be stealthy. In this scenario, FOLPETTI achieves a $15\%$ success rate for the victim's random channel selection strategy, close to the $17.5\%$ obtained with a genie-aided approach. Conversely, the DRL-based approach reaches a success rate of $12.5\%$, which is $5.5\%$ less than FOLPETTI. We also confirm the results by confronting FOLPETTI with a MAB based channel hopping method. Finally, we show that FOLPETTI creates an additional energy demand independently from its success rate, therefore decreasing the lifetime of IoT devices.
Network-Coding-based Forwarding for LoRaWAN Gateways
Venue: arXiv cs.NI 2022
Year: 2022

LoRaWAN is a promising IoT access technology that is growing in popularity. This study addresses the issue of duplicate packets forwarding by LoRaWAN gateways and proposes a novel forwarding scheme to eliminate forwarding duplicate packets by utilizing inter-flow network coding. The proposed scheme is distributed and requires no coordination between gateways. The proposed scheme is evaluated under different network and traffic conditions. The results show that substantial savings can be achieved in bandwidth leading to enhanced scalability of the network without increasing outgoing traffic.
Building Machine Translation Systems for the Next Thousand Languages
Venue: arXiv cs.CL 2022
Year: 2022

In this paper we share findings from our effort to build practical machine translation (MT) systems capable of translating across over one thousand languages. We describe results in three research domains: (i) Building clean, web-mined datasets for 1500+ languages by leveraging semi-supervised pre-training for language identification and developing data-driven filtering techniques; (ii) Developing practical MT models for under-served languages by leveraging massively multilingual models trained with supervised parallel data for over 100 high-resource languages and monolingual datasets for an additional 1000+ languages; and (iii) Studying the limitations of evaluation metrics for these languages and conducting qualitative analysis of the outputs from our MT models, highlighting several frequent error modes of these types of models. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings.
Row-wise Accelerator for Vision Transformer
Venue: arXiv cs.AR 2022
Year: 2022

Following the success of the natural language processing, the transformer for vision applications has attracted significant attention in recent years due to its excellent performance. However, existing deep learning hardware accelerators for vision cannot execute this structure efficiently due to significant model architecture differences. As a result, this paper proposes the hardware accelerator for vision transformers with row-wise scheduling, which decomposes major operations in vision transformers as a single dot product primitive for a unified and efficient execution. Furthermore, by sharing weights in columns, we can reuse the data and reduce the usage of memory. The implementation with TSMC 40nm CMOS technology only requires 262K gate count and 149KB SRAM buffer for 403.2 GOPS throughput at 600MHz clock frequency.
Towards a High-performance and Secure Memory System and Architecture for Emerging Applications
Venue: arXiv cs.CR 2022
Year: 2022

In this dissertation, we propose a memory and computing coordinated methodology to thoroughly exploit the characteristics and capabilities of the GPU-based heterogeneous system to effectively optimize applications' performance and privacy. Specifically, 1) we propose a task-aware and dynamic memory management mechanism to co-optimize applications' latency and memory footprint, especially in multitasking scenarios. 2) We propose a novel latency-aware memory management framework that analyzes the application characteristics and hardware features to reduce applications' initialization latency and response time. 3) We develop a new model extraction attack that explores the vulnerability of the GPU unified memory system to accurately steal private DNN models. 4) We propose a CPU/GPU Co-Encryption mechanism that can defend against a timing-correlation attack in an integrated CPU/GPU platform to provide a secure execution environment for the edge applications. This dissertation aims at developing a high-performance and secure memory system and architecture in GPU heterogeneous platforms to deploy emerging AI-enabled applications efficiently and safely.
A Hybrid Approach: Utilising Kmeans Clustering and Naive Bayes for IoT Anomaly Detection
Venue: arXiv cs.CR 2022
Year: 2022

The proliferation and variety of Internet of Things devices means that they have increasingly become a viable target for malicious users. This has created a need for anomaly detection algorithms that can work across multiple devices. This thesis suggests a potential alternative to the current anomaly detection algorithms to be implemented within IoT systems that can be applied across different types of devices. This algorithm is comprised of both unsupverised and supervised machine areas of machine learning combining the strongest facet of each. The algorithm involves the initial k-means clustering of attacks and assigns them to clusters. Next, the clusters are then used by the AdaBoosted Naive Bayes supervised learning algorithm in order to teach itself which piece of data should be clustered to which specific attack. This increases the accuracy of the proposed algorithm by adding clustered data before the final classification step, ensuring a more accurate algorithm. The correct indentification percentage scores for this proposed algorithm range anywhere from 90% to 100%, as well as rating the proposed algorithms accuracy, precision and recall. These high scores achieve an accurate, flexible, scalable, optimised algorithm that could potentially be in different IoT devices, ensuring strong data integrity and privacy.
Photo-to-Shape Material Transfer for Diverse Structures
Venue: arXiv cs.GR 2022
Year: 2022

We introduce a method for assigning photorealistic relightable materials to 3D shapes in an automatic manner. Our method takes as input a photo exemplar of a real object and a 3D object with segmentation, and uses the exemplar to guide the assignment of materials to the parts of the shape, so that the appearance of the resulting shape is as similar as possible to the exemplar. To accomplish this goal, our method combines an image translation neural network with a material assignment neural network. The image translation network translates the color from the exemplar to a projection of the 3D shape and the part segmentation from the projection to the exemplar. Then, the material prediction network assigns materials from a collection of realistic materials to the projected parts, based on the translated images and perceptual similarity of the materials. One key idea of our method is to use the translation network to establish a correspondence between the exemplar and shape projection, which allows us to transfer materials between objects with diverse structures. Another key idea of our method is to use the two pairs of (color, segmentation) images provided by the image translation to guide the material assignment, which enables us to ensure the consistency in the assignment. We demonstrate that our method allows us to assign materials to shapes so that their appearances better resemble the input exemplars, improving the quality of the results over the state-of-the-art method, and allowing us to automatically create thousands of shapes with high-quality photorealistic materials. Code and data for this paper are available at https://github.com/XiangyuSu611/TMT.
Few-shot Mining of Naturally Occurring Inputs and Outputs
Venue: arXiv cs.CL 2022
Year: 2022

Creating labeled natural language training data is expensive and requires significant human effort. We mine input output examples from large corpora using a supervised mining function trained using a small seed set of only 100 examples. The mining consists of two stages -- (1) a biencoder-based recall-oriented dense search which pairs inputs with potential outputs, and (2) a crossencoder-based filter which re-ranks the output of the biencoder stage for better precision. Unlike model-generated data augmentation, our method mines naturally occurring high-quality input output pairs to mimic the style of the seed set for multiple tasks. On SQuAD-style reading comprehension, augmenting the seed set with the mined data results in an improvement of 13 F1 over a BART-large baseline fine-tuned only on the seed set. Likewise, we see improvements of 1.46 ROUGE-L on Xsum abstractive summarization.
Exploiting Digital Surface Models for Inferring Super-Resolution for Remotely Sensed Images
Venue: arXiv cs.CV 2022
Year: 2022

Despite the plethora of successful Super-Resolution Reconstruction (SRR) models applied to natural images, their application to remote sensing imagery tends to produce poor results. Remote sensing imagery is often more complicated than natural images and has its peculiarities such as being of lower resolution, it contains noise, and often depicting large textured surfaces. As a result, applying non-specialized SRR models on remote sensing imagery results in artifacts and poor reconstructions. To address these problems, this paper proposes an architecture inspired by previous research work, introducing a novel approach for forcing an SRR model to output realistic remote sensing images: instead of relying on feature-space similarities as a perceptual loss, the model considers pixel-level information inferred from the normalized Digital Surface Model (nDSM) of the image. This strategy allows the application of better-informed updates during the training of the model which sources from a task (elevation map inference) that is closely related to remote sensing. Nonetheless, the nDSM auxiliary information is not required during production and thus the model infers a super-resolution image without any additional data besides its low-resolution pairs. We assess our model on two remotely sensed datasets of different spatial resolutions that also contain the DSM pairs of the images: the DFC2018 dataset and the dataset containing the national Lidar fly-by of Luxembourg. Based on visual inspection, the inferred super-resolution images exhibit particularly superior quality. In particular, the results for the high-resolution DFC2018 dataset are realistic and almost indistinguishable from the ground truth images.
Age-driven Joint Sampling and Non-slot Based Scheduling for Industrial Internet of Things
Venue: arXiv cs.IT 2022
Year: 2022

Effective control of time-sensitive industrial applications depends on the real-time transmission of data from underlying sensors. Quantifying the data freshness through age of information (AoI), in this paper, we jointly design sampling and non-slot based scheduling policies to minimize the maximum time-average age of information (MAoI) among sensors with the constraints of average energy cost and finite queue stability. To overcome the intractability involving high couplings of such a complex stochastic process, we first focus on the single-sensor time-average AoI optimization problem and convert the constrained Markov decision process (CMDP) into an unconstrained Markov decision process (MDP) by the Lagrangian method. With the infinite-time average energy and AoI expression expended as the Bellman equation, the single-sensor time-average AoI optimization problem can be approached through the steady-state distribution probability. Further, we propose a low-complexity sub-optimal sampling and semi-distributed scheduling scheme for the multi-sensor scenario. The simulation results show that the proposed scheme reduces the MAoI significantly while achieving a balance between the sampling rate and service rate for multiple sensors.
Re-thinking Knowledge Graph Completion Evaluation from an Information Retrieval Perspective
Venue: arXiv cs.CL 2022
Year: 2022

Knowledge graph completion (KGC) aims to infer missing knowledge triples based on known facts in a knowledge graph. Current KGC research mostly follows an entity ranking protocol, wherein the effectiveness is measured by the predicted rank of a masked entity in a test triple. The overall performance is then given by a micro(-average) metric over all individual answer entities. Due to the incomplete nature of the large-scale knowledge bases, such an entity ranking setting is likely affected by unlabelled top-ranked positive examples, raising questions on whether the current evaluation protocol is sufficient to guarantee a fair comparison of KGC systems. To this end, this paper presents a systematic study on whether and how the label sparsity affects the current KGC evaluation with the popular micro metrics. Specifically, inspired by the TREC paradigm for large-scale information retrieval (IR) experimentation, we create a relatively "complete" judgment set based on a sample from the popular FB15k-237 dataset following the TREC pooling method. According to our analysis, it comes as a surprise that switching from the original labels to our "complete" labels results in a drastic change of system ranking of a variety of 13 popular KGC models in terms of micro metrics. Further investigation indicates that the IR-like macro(-average) metrics are more stable and discriminative under different settings, meanwhile, less affected by label sparsity. Thus, for KGC evaluation, we recommend conducting TREC-style pooling to balance between human efforts and label completeness, and reporting also the IR-like macro metrics to reflect the ranking nature of the KGC task.