OpenRaiser/Intern-Atlas huggingface dataset:
Uncovering the Inner Workings of STEGO for Safe Unsupervised Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Self-supervised pre-training strategies have recently shown impressive results for training general-purpose feature extraction backbones in computer vision. In combination with the Vision Transformer architecture, the DINO self-distillation technique has interesting emerging properties, such as unsupervised clustering in the latent space and semantic correspondences of the produced features without using explicit human-annotated labels. The STEGO method for unsupervised semantic segmentation contrastively distills feature correspondences of a DINO-pre-trained Vision Transformer and recently set a new state of the art. However, the detailed workings of STEGO have yet to be disentangled, preventing its usage in safety-critical applications. This paper provides a deeper understanding of the STEGO architecture and training strategy by conducting studies that uncover the working mechanisms behind STEGO, reproduce and extend its experimental validation, and investigate the ability of STEGO to transfer to different datasets. Results demonstrate that the STEGO architecture can be interpreted as a semantics-preserving dimensionality reduction technique.
GPULZ: Optimizing LZSS Lossless Compression for Multi-byte Data on Modern GPUs
Venue: arXiv cs.DC 2023
Year: 2023

Today's graphics processing unit (GPU) applications produce vast volumes of data, which are challenging to store and transfer efficiently. Thus, data compression is becoming a critical technique to mitigate the storage burden and communication cost. LZSS is the core algorithm in many widely used compressors, such as Deflate. However, existing GPU-based LZSS compressors suffer from low throughput due to the sequential nature of the LZSS algorithm. Moreover, many GPU applications produce multi-byte data (e.g., int16/int32 index, floating-point numbers), while the current LZSS compression only takes single-byte data as input. To this end, in this work, we propose GPULZ, a highly efficient LZSS compression on modern GPUs for multi-byte data. The contribution of our work is fourfold: First, we perform an in-depth analysis of existing LZ compressors for GPUs and investigate their main issues. Then, we propose two main algorithm-level optimizations. Specifically, we (1) change prefix sum from one pass to two passes and fuse multiple kernels to reduce data movement between shared memory and global memory, and (2) optimize existing pattern-matching approach for multi-byte symbols to reduce computation complexity and explore longer repeated patterns. Third, we perform architectural performance optimizations, such as maximizing shared memory utilization by adapting data partitions to different GPU architectures. Finally, we evaluate GPULZ on six datasets of various types with NVIDIA A100 and A4000 GPUs. Results show that GPULZ achieves up to 272.1X speedup on A4000 and up to 1.4X higher compression ratio compared to state-of-the-art solutions.
Holographic MIMO: How Many Antennas Do We Need for Energy Efficient Transmission?
Venue: arXiv cs.IT 2023
Year: 2023

Holographic multiple-input multiple-output (HMIMO) communication systems utilize spatially-constrained massive MIMO arrays containing large numbers of antennas with sub-wavelength spacing, and have emerged as a promising candidate technology for Sixth Generation (6G) networks. In this paper, we consider the downlink of a multi-user HMIMO communication system under a Fourier plane-wave series representation of the stochastic electromagnetic MIMO channel model, and make two important contributions. First, we present a closed-form expression of the ergodic achievable downlink rate under maximum ratio transmission (MRT) precoding at the base station (BS). The derived expression explicitly shows the effect of the side-lengths of the HMIMO surfaces at the BS and each user, and the number of antennas deployed in these surfaces on the user rates. Second, we formulate an energy efficiency (EE) maximization problem with respect to the number of antennas arranged within spatially-constrained HMIMO surfaces at the BS and each user. The resulting implicit solution for this problem is shown to be globally optimal. Numerical results yield useful insights into the EE performance of multi-user HMIMO systems in different operating regimes.
Peer-to-Peer Federated Continual Learning for Naturalistic Driving Action Recognition
Venue: arXiv cs.LG 2023
Year: 2023

Naturalistic driving action recognition (NDAR) has proven to be an effective method for detecting driver distraction and reducing the risk of traffic accidents. However, the intrusive design of in-cabin cameras raises concerns about driver privacy. To address this issue, we propose a novel peer-to-peer (P2P) federated learning (FL) framework with continual learning, namely FedPC, which ensures privacy and enhances learning efficiency while reducing communication, computational, and storage overheads. Our framework focuses on addressing the clients' objectives within a serverless FL framework, with the goal of delivering personalized and accurate NDAR models. We demonstrate and evaluate the performance of FedPC on two real-world NDAR datasets, including the State Farm Distracted Driver Detection and Track 3 NDAR dataset in the 2023 AICity Challenge. The results of our experiments highlight the strong competitiveness of FedPC compared to the conventional client-to-server (C2S) FLs in terms of performance, knowledge dissemination rate, and compatibility with new clients.
Identity Encoder for Personalized Diffusion
Venue: arXiv cs.CV 2023
Year: 2023

Many applications can benefit from personalized image generation models, including image enhancement, video conferences, just to name a few. Existing works achieved personalization by fine-tuning one model for each person. While being successful, this approach incurs additional computation and storage overhead for each new identity. Furthermore, it usually expects tens or hundreds of examples per identity to achieve the best performance. To overcome these challenges, we propose an encoder-based approach for personalization. We learn an identity encoder which can extract an identity representation from a set of reference images of a subject, together with a diffusion generator that can generate new images of the subject conditioned on the identity representation. Once being trained, the model can be used to generate images of arbitrary identities given a few examples even if the model hasn't been trained on the identity. Our approach greatly reduces the overhead for personalized image generation and is more applicable in many potential applications. Empirical results show that our approach consistently outperforms existing fine-tuning based approach in both image generation and reconstruction, and the outputs is preferred by users more than 95% of the time compared with the best performing baseline.
Few-shot Weakly-supervised Cybersecurity Anomaly Detection
Venue: arXiv cs.CR 2023
Year: 2023

With increased reliance on Internet based technologies, cyberattacks compromising users' sensitive data are becoming more prevalent. The scale and frequency of these attacks are escalating rapidly, affecting systems and devices connected to the Internet. The traditional defense mechanisms may not be sufficiently equipped to handle the complex and ever-changing new threats. The significant breakthroughs in the machine learning methods including deep learning, had attracted interests from the cybersecurity research community for further enhancements in the existing anomaly detection methods. Unfortunately, collecting labelled anomaly data for all new evolving and sophisticated attacks is not practical. Training and tuning the machine learning model for anomaly detection using only a handful of labelled data samples is a pragmatic approach. Therefore, few-shot weakly supervised anomaly detection is an encouraging research direction. In this paper, we propose an enhancement to an existing few-shot weakly-supervised deep learning anomaly detection framework. This framework incorporates data augmentation, representation learning and ordinal regression. We then evaluated and showed the performance of our implemented framework on three benchmark datasets: NSL-KDD, CIC-IDS2018, and TON_IoT.
Implementation of Digital Circuits on Three-Dimensional FPGAs Using Simulated Annealing
Venue: arXiv cs.AR 2023
Year: 2023

3D FPGAs have recently been produced as the next generation of the FPGA family to continue the integration of more transistors on a single chip seamlessly. In this paper, we propose a complete CAD flow to implement an arbitrary logic circuit on the 3D FPGA. The partitioning and placement stages of the flow are based on the simulated annealing algorithm. Furthermore, the routing stage is a modified version of the Pathfinder algorithm. The simulation results indicate that the comparison between 2D FPGA and 3D FPGA (including 2-tier) shows that the circuit speed increases by 28.66% and minimum channel width decrease by 29.92%, while the total area raises by 8.86%. Finally, the results of the comparison between 2-tier and 4-tier in 3D FPGA show that circuit speed and minimum channel width increase by 15.95% and 15.92% in 4-tier, respectively. Meanwhile, the total area increases only by 1.96%.
Resource Allocation for RIS-Assisted Device-to-Device Communications in Heterogeneous Cellular Networks
Venue: arXiv cs.IT 2023
Year: 2023

In recent years, with the explosive growth of data traffic, communication base stations (BSs) need to serve more and more users. Offloading traffic from BSs has become an efficient way to reduce the burden on BSs. Device-to-Device (D2D) communications have emerged to improve spectrum utilization by reusing the frequency spectrum of the cellular frequency band. In the general environment, Heterogeneous Cellular Networks (HCNs) including millimeter wave (mm-wave) have appeared. Since the D2D link allows to share of spectrum resources with the cellular user, it will bring potential interference to the cellular user. Fortunately, an emerging technology called Reconfigurable Intelligent Surface (RIS) can mitigate the severe interference caused by D2D links by shaping the incident beam and improving the multipath phase shift. In this paper, we study the resource allocation scheme to maximize the system sum rate, in the RISassisted single-cell heterogeneous D2D communication scenario. To solve the Block Coordinate Descent (BCD) problem, the problem of maximizing the sum rate is decomposed into three sub-problems. The resource allocation sub-problem is solved by a coalitional game method based on the game theory. The power allocation problem of the coalition converts the concave function into a convex optimization by mathematical transformation. The problem is solved by the gradient descent method. The local search method is adopted to find the optimum for the phase conversion problem. Then iterate until the difference of sum rate is less than the threshold. The simulation results show that the designed algorithm is superior to other benchmark schemes in the literature.
The 7th AI City Challenge
Venue: arXiv cs.CV 2023
Year: 2023

The AI City Challenge's seventh edition emphasizes two domains at the intersection of computer vision and artificial intelligence - retail business and Intelligent Traffic Systems (ITS) - that have considerable untapped potential. The 2023 challenge had five tracks, which drew a record-breaking number of participation requests from 508 teams across 46 countries. Track 1 was a brand new track that focused on multi-target multi-camera (MTMC) people tracking, where teams trained and evaluated using both real and highly realistic synthetic data. Track 2 centered around natural-language-based vehicle track retrieval. Track 3 required teams to classify driver actions in naturalistic driving analysis. Track 4 aimed to develop an automated checkout system for retail stores using a single view camera. Track 5, another new addition, tasked teams with detecting violations of the helmet rule for motorcyclists. Two leader boards were released for submissions based on different methods: a public leader board for the contest where external private data wasn't allowed and a general leader board for all results submitted. The participating teams' top performances established strong baselines and even outperformed the state-of-the-art in the proposed challenge tracks.
ID2image: Leakage of non-ID information into face descriptors and inversion from descriptors to images
Venue: arXiv cs.CV 2023
Year: 2023

Embedding a face image to a descriptor vector using a deep CNN is a widely used technique in face recognition. Via several possible training strategies, such embeddings are supposed to capture only identity information. Information about the environment (such as background and lighting) or changeable aspects of the face (such as pose, expression, presence of glasses, hat etc.) should be discarded since they are not useful for recognition. In this paper, we present a surprising result that this is not the case. We show that non-ID attributes, as well as landmark positions and the image histogram can be recovered from the ID embedding of state-of-the-art face embedding networks (VGGFace2 and ArcFace). In fact, these non-ID attributes can be predicted from ID embeddings with similar accuracy to a prediction from the original image. Going further, we present an optimisation strategy that uses a generative model (specifically StyleGAN2 for faces) to recover images from an ID embedding. We show photorealistic inversion from ID embedding to face image in which not only is the ID realistically reconstructed but the pose, lighting and background/apparel to some extent as well.
From Online Behaviours to Images: A Novel Approach to Social Bot Detection
Venue: arXiv cs.SI 2023
Year: 2023

Online Social Networks have revolutionized how we consume and share information, but they have also led to a proliferation of content not always reliable and accurate. One particular type of social accounts is known to promote unreputable content, hyperpartisan, and propagandistic information. They are automated accounts, commonly called bots. Focusing on Twitter accounts, we propose a novel approach to bot detection: we first propose a new algorithm that transforms the sequence of actions that an account performs into an image; then, we leverage the strength of Convolutional Neural Networks to proceed with image classification. We compare our performances with state-of-the-art results for bot detection on genuine accounts / bot accounts datasets well known in the literature. The results confirm the effectiveness of the proposal, because the detection capability is on par with the state of the art, if not better in some cases.
RoboREIT: an Interactive Robotic Tutor with Instructive Feedback Component for Requirements Elicitation Interview Training
Venue: arXiv cs.RO 2023
Year: 2023

[Context] Interviewing stakeholders is the most popular requirements elicitation technique among multiple methods. The success of an interview depends on the collaboration of the interviewee which can be fostered through the interviewer's preparedness and communication skills. Mastering these skills requires experience and practicing interviews. [Problem] Practical training is resource-heavy as it calls for the time and effort of a stakeholder for each student which may not be feasible for a large number of students. [Method] To address this scalability problem, this paper proposes RoboREIT, an interactive Robotic tutor for Requirements Elicitation Interview Training. The humanoid robotic component of RoboREIT responds to the questions of the interviewer, which the interviewer chooses from a set of predefined alternatives for a particular scenario. After the interview session, RoboREIT provides contextual feedback to the interviewer on their performance and allows the student to inspect their mistakes. RoboREIT is extensible with various scenarios. [Results] We performed an exploratory user study to evaluate RoboREIT and demonstrate its applicability in requirements elicitation interview training. The quantitative and qualitative analyses of the users' responses reveal the appreciation of RoboREIT and provide further suggestions about how to improve it. [Contribution] Our study is the first in the literature that utilizes a social robot in requirements elicitation interview education. RoboREIT's innovative design incorporates replaying faulty interview stages and allows the student to learn from mistakes by a second time practicing. All participants praised the feedback component, which is not present in the state-of-the-art, for being helpful in identifying the mistakes. A favorable response rate of 81% for the system's usefulness indicates the positive perception of the participants.
MA-ViT: Modality-Agnostic Vision Transformers for Face Anti-Spoofing
Venue: arXiv cs.CV 2023
Year: 2023

The existing multi-modal face anti-spoofing (FAS) frameworks are designed based on two strategies: halfway and late fusion. However, the former requires test modalities consistent with the training input, which seriously limits its deployment scenarios. And the latter is built on multiple branches to process different modalities independently, which limits their use in applications with low memory or fast execution requirements. In this work, we present a single branch based Transformer framework, namely Modality-Agnostic Vision Transformer (MA-ViT), which aims to improve the performance of arbitrary modal attacks with the help of multi-modal data. Specifically, MA-ViT adopts the early fusion to aggregate all the available training modalities data and enables flexible testing of any given modal samples. Further, we develop the Modality-Agnostic Transformer Block (MATB) in MA-ViT, which consists of two stacked attentions named Modal-Disentangle Attention (MDA) and Cross-Modal Attention (CMA), to eliminate modality-related information for each modal sequences and supplement modality-agnostic liveness features from another modal sequences, respectively. Experiments demonstrate that the single model trained based on MA-ViT can not only flexibly evaluate different modal samples, but also outperforms existing single-modal frameworks by a large margin, and approaches the multi-modal frameworks introduced with smaller FLOPs and model parameters.
SerPyTor: A distributed context-aware computational graph execution framework for durable execution
Venue: arXiv cs.DC 2023
Year: 2023

Distributed computation is always a tricky topic to deal with, especially in context of various requirements in various scenarios. A popular solution is to use Apache Spark with a setup of multiple systems forming a cluster. However, the prerequisite setup for a Spark cluster often induces an additional overhead, often limiting usage in constrained scenarios, especially in scenarios requiring context propagation. In this paper, we explore a relatively lightweight computational graph execution framework requiring little setup and fast speeds, coupled with context awareness.
GPSMirror: Expanding Accurate GPS Positioning to Shadowed and Indoor Regions with Backscatter
Venue: arXiv cs.NI 2023
Year: 2023

Despite the prevalence of GPS services, they still suffer from intermittent positioning with poor accuracy in partially shadowed regions like urban canyons, flyover shadows, and factories' indoor areas. Existing wisdom relies on hardware modifications of GPS receivers or power-hungry infrastructures requiring continuous plug-in power supply which is hard to provide in outdoor regions and some factories. This paper fills the gap with GPSMirror, the first GPS-strengthening system that works for unmodified smartphones with the assistance of newly-designed GPS backscatter tags. The key enabling techniques in GPSMirror include: (i) a careful hardware design with microwatt-level power consumption that pushes the limit of backscatter sensitivity to re-radiate extremely weak GPS signals with enough coverage approaching the regulation limit; and (ii) a novel GPS positioning algorithm achieving meter-level accuracy in shadowed regions as well as expanding locatable regions under inadequate satellites where conventional algorithms fail. We build a prototype of the GPSMirror tags and conduct comprehensive experiments to evaluate them. Our results show that a GPSMirror tag can provide coverage up to 27.7 m. GPSMirror achieves median positioning accuracy of 3.7 m indoors and 4.6 m in urban canyon environments, respectively.
Preventing Malicious Use of Keyloggers Using Anti-Keyloggers
Venue: arXiv cs.CR 2023
Year: 2023

Multinational corporations routinely track how its employees use their computers, the internet, or email. There are roughly a thousand devices on the market that enable businesses to monitor their workforce. Observe what their users do online, in their emails, and on their so-called personal laptops while at work. Our team will develop a key-logger for this Journal project that will allow us to capture every keystroke. Additionally, it will enable us to capture the mouse movements and clicks. This will enable us to keep an eye on how a person uses the internet and every other application on his or her "own" computer. Keyloggers, on the other hand, can also be used to steal data in the form of malware or something comparable. An anti-key logger will also be created to address this issue, allowing us to determine whether a key logger is already monitoring the system. We will be able to remain watchful and maintain the security of the data on our system thanks to the anti-keylogger.
An Instance Segmentation Dataset of Yeast Cells in Microstructures
Venue: arXiv cs.CV 2023
Year: 2023

Extracting single-cell information from microscopy data requires accurate instance-wise segmentations. Obtaining pixel-wise segmentations from microscopy imagery remains a challenging task, especially with the added complexity of microstructured environments. This paper presents a novel dataset for segmenting yeast cells in microstructures. We offer pixel-wise instance segmentation labels for both cells and trap microstructures. In total, we release 493 densely annotated microscopy images. To facilitate a unified comparison between novel segmentation algorithms, we propose a standardized evaluation strategy for our dataset. The aim of the dataset and evaluation strategy is to facilitate the development of new cell segmentation approaches. The dataset is publicly available at https://christophreich1996.github.io/yeast_in_microstructures_dataset/ .
Understanding Rug Pulls: An In-Depth Behavioral Analysis of Fraudulent NFT Creators
Venue: arXiv cs.CR 2023
Year: 2023

The explosive growth of non-fungible tokens (NFTs) on Web3 has created a new frontier for digital art and collectibles, but also an emerging space for fraudulent activities. This study provides an in-depth analysis of NFT rug pulls, which are fraudulent schemes aimed at stealing investors' funds. Using data from 758 rug pulls across 10 NFT marketplaces, we examine the structural and behavioral properties of these schemes, identify the characteristics and motivations of rug-pullers, and classify NFT projects into groups based on creators' association with their accounts. Our findings reveal that repeated rug pulls account for a significant proportion of the rise in NFT-related cryptocurrency crimes, with one NFT collection attempting 37 rug pulls within three months. Additionally, we identify the largest group of creators influencing the majority of rug pulls, and demonstrate the connection between rug-pullers of different NFT projects through the use of the same wallets to store and move money. Our study contributes to the understanding of NFT market risks and provides insights for designing preventative strategies to mitigate future losses.