Miracle-dz/UniFAR-FTUs huggingface dataset:
EdgePC: Efficient Deep Learning Analytics for Point Clouds on Edge Devices

Recently, point cloud (PC) has gained popularity in modeling various 3D objects (including both synthetic and real-life) and has been extensively utilized in a wide range of applications such as AR/VR, 3D reconstruction, and autonomous driving. For such applications, it is critical to analyze/understand the surrounding scenes properly. To achieve this, deep learning based methods (e.g., convolutional neural networks (CNNs)) have been widely employed for higher accuracy. Unlike the deep learning on conventional 2D images/videos, where the feature computation (matrix multiplication) is the major bottleneck, in point cloud-based CNNs, the sample and neighbor search stages are the primary bottlenecks, and collectively contribute to 54% (up to 80%) of the overall execution latency on a typical edge device. While prior efforts have attempted to solve this issue by designing custom ASICs or pipelining the neighbor search with other stages, to our knowledge, none of them has tried to "structurize" the unstructured PC data for improving computational efficiency. In this paper, we first explore the opportunities of structurizing PC data using Morton code (which is originally designed to map data from a high dimensional space to one dimension, while preserving spatial locality) and observe that there is a huge scope to "skip" the sample and neighbor search computation by operating on the "structurized" PC data. Based on this, we propose two approximation techniques for the sampling and neighbor search stages. We implemented our proposals on an NVIDIA Jetson AGX Xavier edge GPU board. The evaluation results collected on six different workloads show that our design can accelerate the sample and neighbor search stages by 3.68× (up to 5.21×) with minimal impact on inference accuracy. This acceleration in turn results in 1.55× speedup in the end-to-end execution latency and saves 33% of energy expenditure.
Harmony: Heterogeneous Multi-Modal Federated Learning through Disentangled Model Training

Multi-modal sensing systems are increasingly prevalent in real-world applications such as health monitoring and autonomous driving. Most multi-modal learning approaches need to access users' raw data, which poses significant concerns to users' privacy. Federated learning (FL) provides a privacy-aware distributed learning framework. However, current FL approaches have not addressed the unique challenges of heterogeneous multi-modal FL systems, such as modality heterogeneity and significantly longer training delay. In this paper, we propose Harmony, a new system for heterogeneous multi-modal federated learning. Harmony disentangles the multi-modal network training in a novel two-stage framework, namely modality-wise federated learning and federated fusion learning. By integrating a novel balance-aware resource allocation mechanism in modality-wise FL and exploiting modality biases in federated fusion learning, Harmony improves the model accuracy under non-i.i. d. data distributions and speeds up system convergence. We implemented Harmony on a real-world multi-modal sensor testbed deployed in the homes of 16 elderly subjects for Alzheimer's Disease monitoring. Our evaluation on the testbed and three large-scale public datasets of different applications show that, Harmony outperforms by up to 46.35% accuracy over state-of-the-art baselines and saves up to 30% training delay.
FDMAX: An Elastic Accelerator Architecture for Solving Partial Differential Equations

Partial Differential Equations (PDEs) are widely employed to describe natural phenomena in many science and engineering fields. Many PDEs do not have analytical solutions, hence, numerical methods have become prevalent for approximating PDE solutions. The most widely used numerical method is the Finite Difference Method (FDM), which requires fine grids and high-precision numerical iterations that are both compute- and memory-intensive. PDE-solving accelerators have been proposed in the literature, however, they usually focus on specific types of PDEs with rigid grid sizes which limits their broader applicability. Besides, they rarely provided insight into the optimizations of parallel computing and data accesses for solving PDEs, which hinders further improvements in performance and energy efficiency. This paper presents FDMAX, an elastic accelerator to efficiently support FDM for different types of PDEs with any grid size. FDMAX employs a customized Processing Element (PE) array architecture that maximizes data reuse with minimized interconnection overhead. The PE array can be reconfigured to break into a set of subarrays to adapt to different grid sizes for optimal efficiency. Moreover, the PE array exploits computation and data reuse for increased performance and energy efficiency, and is reconfigurable to support a wide range of PDEs such as elliptic, parabolic, and hyperbolic equations. Evaluated on four well-known PDEs, our simulation results show that FDMAX achieves on average 1189× speedup with 1123× energy reduction over Intel Xeon CPU, and 4.9× speedup with 6.3× energy reduction over NVIDIA RTX3090 GPU, and 2.9× speedup over Alrescha, the state-of-the-art PDE-solving accelerator.
United-and-Close: An interactive visual platform for assessing urban segregation within the 15-minutes paradigm

The ‘15-minute city’ paradigm is an urban model based on the concept of ‘hyper-proximity’: citizens should be able to access fundamental services and facilities (such as schools, shops, parks, doctors, and markets) within 15-20 minutes on foot, by bicycle or by public transport. Compliance with the ‘15-minute city’ paradigm is supposed to reduce pollution and social inequalities. It is supposed to bring the psychological fragility of the citizen back to the center of the urban redevelopment debate. Although the concept has gained great attention and interest from policymakers and urban designers, we still lack tools that can help to validate, on a data-driven basis, the assumption that hyper-proximity is eventually correlated with lower urban segregation, which is one of the driving forces that lead to social inequalities. We aim to define a data-driven methodology to analyze the urban areas where services should be accessible within 15 minutes; network analysis is exploited to estimate services proximity as well as the connectivity of different urban areas with each other, in order to gather signals of the general resilience or exposure to urban segregation. We also aim to compute a set of city-agnostic metrics that will include user-specified parameters and personalized weights for each Point of Interest’s category. United-and-Close is the resulting Web platform designed to be accessible to citizens, policy and decision-makers, and investors, but also for researchers involved in disciplines such as urban informatics that need support to better assess the 15-minute paradigm and its actual impact on our cities.
GMAP 2023: 2nd Workshop on Group Modeling, Adaptation and Personalization

Although most existing recommender systems support single users, there are many scenarios where these systems target the needs of groups. Traits such as group mood, emotional contagion, and interpersonal relationships are often ill-defined characteristics, tend to mutate over time, and are usually missing from the systems’ modeling, even though they play an indispensable part in group modeling. Furthermore, producing timely and accurate recommendations for groups that are explainable, fair, and privacy-protecting is a notoriously tricky challenge since group members may have divergent views and needs. The second GMAP workshop aims at bringing together a community of researchers focused on group modeling, adaptation, and personalization. The objective is to explore the challenges and opportunities of developing effective methods and tools to support group decision-making. The workshop, we brought together researchers from several disciplines, including Psychology, Computer Science, and Organizational Behavior, to discuss their latest research and ideas on this topic. It also provided opportunities for participants to share their research and experiences and to collaborate and network with other researchers in this field. The long-term goal is to foster a vibrant and inclusive community of researchers committed to advancing our understanding of group modeling, adaptation, and personalization by bringing together experts from different disciplines and perspectives. Throughout this workshop, we aim to identify critical challenges and opportunities in this area and develop a shared research agenda to guide future work.
Carbon-Aware Global Routing in Path-Aware Networks

The growing energy consumption of Information and Communication Technology (ICT) has raised concerns about its environmental impact. However, the carbon footprint of data transmission over the Internet has so far received relatively modest attention. This carbon footprint can be reduced by sending traffic over carbon-efficient inter-domain paths. However, challenges in estimating and disseminating carbon intensity of inter-domain paths have prevented carbon-aware path selection from becoming a reality. In this paper, we take advantage of path-aware network architectures to overcome these challenges. In particular, we design CIRo, a system for forecasting the carbon intensity of inter-domain paths and disseminating them across the Internet. We implement a proof of concept for CIRo on the codebase of the SCION path-aware Internet architecture and test it on the SCIONLab global research testbed. Further, through large-scale simulations, we demonstrate the potential of CIRo for reducing the carbon footprint of endpoints and end domains: With CIRo, half of domain pairs can reduce the carbon intensity of their inter-domain traffic by at least 47%, and 87% of end domains can reduce their carbon footprint of Internet use by at least 50%.
Demo: EMSAssist – An End-to-End Mobile Voice Assistant at the Edge for Emergency Medical Services

We present EMSAssist, the first end-to-end mobile voice assistant for emergency medical services (EMS). EMSAssist allows Emergency Medical Technicians (EMT) to verbally describe patients' signs and symptoms and uses EMTs' voice input to recommend top-5 EMS protocols. Through this demo, we allow the attendees to evaluate EMSAssist through a pair of Google Glass and a mobile phone. Both mobile devices can collect users' voices as input and output top-5 recommended protocols. A companion youtube video of using EMSAssist on the Google Glass is provided: https://www.youtube. com/watch? v=bj7aQJKf4aE
All Your PC Are Belong to Us: Exploiting Non-control-Transfer Instruction BTB Updates for Dynamic PC Extraction

Leaking a program's instruction address (PC) pattern, completely and precisely, has long been a sought-after capability for microarchitectural side-channel attackers. Case in point, such a primitive would be sufficient to construct powerful control-flow leakage attacks (inferring program secrets impacting control flow) that defeat existing control-flow leakage mitigations, or even reverse-engineer private binaries through PC-trace granular fingerprinting. However, current side-channel attack techniques only capture PCs at a coarse granularity or for only specific instruction types. In this paper, we propose the first micro-architectural side-channel attack that is capable of directly observing the exact PCs of arbitrary victim dynamic instructions---i.e., even the PCs of non-control-transfer instructions and even if the program code is private. Our attack exploits several previously overlooked characteristics in modern Intel Branch Target Buffers (BTBs). The core observation is perhaps counter-intuitive: despite being a structure related to control-flow prediction, the BTB incurs observable state changes after the execution of potentially any instruction, not just control-transfer instructions. Through reverse-engineering and analyzing said BTB vulnerabilities, we design and implement an attack framework named NightVision. We demonstrate how NightVision is capable of efficiently and accurately identifying a subset, or the entirety, of a victim program's dynamic PC trace (depending on the attacker's capabilities). We show how NightVision enables a new control-flow attack that bypasses prior defenses. Additionally, we show that when combined with code fingerprinting techniques, NightVision enables reverse-engineering of private programs.
LocRa: Enable Practical Long-Range Backscatter Localization for Low-Cost Tags

Long-range backscatter localization is a promising technology for the Internet of Things. Existing works cannot work well for distributed base stations and low-cost tags. We present LocRa, which provides accurate localization for long-range backscatter with distributed base stations. We present a novel method to extract accurate channel information and synchronize the phase of different base stations. To compensate for the frequency and phase error on low-cost tags, we combine multiple channel measurements and eliminate the error by aligning different channels. Finally, we exploit frequency domain characteristics of the backscatter signal to extend its bandwidth and improve the SNR, thereby enhancing the localization accuracy. We prototype LocRa tags using custom low-cost hardware and implement LocRa base stations on USRP. Through extensive experiments, we show that the localization error of LocRa is 6.8 cm and 88 cm when the tag is 5m and 50m away from the base station, which is 3.1× and 2.3× better than the state-of-the-arts methods.
LeCA: In-Sensor Learned Compressive Acquisition for Efficient Machine Vision on the Edge

With the rapid advances of deep learning-based computer vision (CV) technology, digital images are increasingly consumed, not by humans, but by downstream CV algorithms. However, capturing high-fidelity and high-resolution images is energy-intensive. It not only dominates the energy consumption of the sensor itself (i.e. in low-power edge devices), but also contributes to significant memory burdens and performance bottlenecks in the later storage, processing, and communication stages. In this paper, we systematically explore a new paradigm of in-sensor processing, termed "learned compressive acquisition" (LeCA). Targeting machine vision applications on the edge, the LeCA framework exploits the joint learning of a sensor autoencoder structure with the downstream CV algorithms to effectively compress the original image into low-dimensional features with adaptive bit depth. We employ column-parallel analog-domain processing directly inside the image sensor to perform the compressive encoding of the raw image, resulting in meaningful hardware savings, and energy efficiency improvements. Evaluated within a modern machine vision processing pipeline, LeCA achieves 4×, 6×, and 8× compression ratios prior to any digital compression, with minimal accuracy loss of 0.97%, 0.98%, and 2.01% on ImageNet, outperforming existing methods. Compared with the conventional full-resolution image sensor and the state-of-the-art compressive sensing sensor, our LeCA sensor is 6.3× and 2.2× more energy-efficient while reaching a 2× higher compression ratio.
R2D2: Removing ReDunDancy Utilizing Linearity of Address Generation in GPUs

A generally used GPU programming methodology is that adjacent threads access data in neighbor or specific-stride memory addresses and perform computations with the fetched data. This paper demonstrates that the memory addresses often exhibit a simple linear value pattern across GPU threads, as each thread uses built-in variables and constant values to compute the memory addresses. However, since the threads compute their context data individually, GPUs incur a heavy instruction overhead to calculate the memory addresses, even though they exhibit a simple pattern. We propose a GPU architecture called Removing ReDunDancy Utilizing Linearity of Address Generation (R2D2), reducing a large amount of the dynamic instruction count by detecting such linear patterns in the memory addresses and exploiting them for kernel computations. R2D2 detects linearities of the memory addresses with software support and pre-computes them before the threads execute the instructions. With the proposed scheme, each thread is able to compute its memory addresses with fewer dynamic instructions than conventional GPUs. In our evaluation, R2D2 achieves dynamic instruction reduction by 28%, 1.25x speedup, and energy consumption reduction by 17% over baseline GPU.
F4T: A Fast and Flexible FPGA-based Full-stack TCP Acceleration Framework

As complex workloads that run on many servers are pursuing higher networking throughput, more CPU cycles are consumed to support the TCP stack. To mitigate the high CPU burden from executing the compute-intensive TCP, prior works have proposed to offload TCP processing to the embedded processors, ASICs, or FPGAs in network devices. However, none of the approaches satisfy all of the critical requirements of TCP simultaneously, which are high performance, many connections, and high flexibility. Embedded processors do not provide enough performance to fully offload the TCP stack, while ASICs fail to provide enough flexibility. Meanwhile, existing FPGA-based TCP accelerators either fail to provide high performance or give up some of the critical features and requirements to achieve high performance due to their inefficient processing architecture. This paper proposes F4T, a fast and flexible FPGA-based full-stack TCP acceleration framework to effectively save CPU cycles by achieving high performance, supporting many connections, and providing high flexibility. By analyzing the existing FPGA-based TCP accelerators, we identify that the key performance bottleneck arises from two factors: (1) inefficient processing of stateful operations that incurs stalls and (2) lack of TCP state management necessary for utilizing multiple memory modules. To resolve the identified bottlenecks, we design F4T to process stateful operations back-to-back without stalls and efficiently manage the TCP states among multiple memory modules. F4T also provides a full software-hardware stack that allows applications to easily utilize F4T without any modifications. Our evaluation results show that F4T saturates the 100Gbps link bandwidth with only two CPU cores while supporting many connections and providing high flexibility. F4T saves 64% CPU cycles and provides 2.8× CPU cycles to applications compared to Linux's TCP stack when running the Nginx web server.
GenDP: A Framework of Dynamic Programming Acceleration for Genome Sequencing Analysis

Genomics is playing an important role in transforming healthcare. Genetic data, however, is being produced at a rate that far outpaces Moore's Law. Many efforts have been made to accelerate genomics kernels on modern commodity hardware such as CPUs and GPUs, as well as custom accelerators (ASICs) for specific genomics kernels. While ASICs provide higher performance and energy efficiency than general-purpose hardware, they incur a high hardware design cost. Moreover, in order to extract the best performance, ASICs tend to have significantly different architectures for different kernels. The divergence of ASIC designs makes it difficult to run commonly used modern sequencing analysis pipelines due to software integration and programming challenges. With the observation that many genomics kernels are dominated by dynamic programming (DP) algorithms, this paper presents GenDP, a framework of dynamic programming acceleration including DPAx, a DP accelerator, and DPMap, a graph partitioning algorithm that maps DP objective functions to the accelerator. DPAx supports DP kernels with various dependency patterns, such as 1D and 2D DP tables and long-range dependencies in the graph structure. DPAx also supports different DP objective functions and precisions required for genomics applications. GenDP is evaluated on genomics kernels in both short-read and long-read analysis pipelines, achieving 157.8× throughput/mm2 over GPU baselines and 132.0× throughput/mm2 over CPU baselines.
Fusing In-Storage and Near-Storage Acceleration of Convolutional Neural Networks

Video analytics have a wide range of applications and has attracted much interest over the years. While it can be both computationally and energy intensive, video analytics can greatly benefit from in/ near memory compute. The practice of moving compute closer to memory has continued to show improvements to performance and energy consumption and is seeing increasing adoption. Recent advancements in solid state drives (SSDs) have incorporated near memory Field Programmable Gate Arrays (FPGAs) with shared access to the drive’s storage cells. These near memory FPGAs are capable of running operations required by video analytic pipelines such as object detection and template matching. These operations are typically executed using Convolutional Neural Networks (CNNs). A CNN is composed of multiple individually processed layers which perform various image processing tasks. Due to lack of resources, a layer may be partitioned into more manageable sub-layers. These sub-layers are then processed sequentially, however some sub-layers can be processed simultaneously. Moreover, the storage cells within FPGA equipped SSD’s are capable of being augmented with in-storage compute to accelerate CNN workloads and exploit the intra parallelism within a CNN layer. To this end we present our work, which leverages heterogeneous architectures to create an in/near-storage acceleration solution for video analytics. We designed a NAND flash accelerator, and an FPGA accelerator, then mapped and evaluated several CNN benchmarks. We show how to utilize FPGAs, local DRAMs, and in-memory SSD compute to accelerate CNN workloads. Our work also demonstrates how to remove unnecessary memory transfers to save latency and energy.
Transformer for Hyperspectral Image Classification based on Multi-feature Learning

The CNN-based HSI classification methods are difficulty to capture the long-range spectral dependencies that resulting to lose the intrinsic and potentially spectral information. To address it, this letter proposes a model that combine transformer and portion of Inception V4 to learn multi-feature for the HSI classification, where multi-features are incorporated by multi-bands to address the problems from data features to redundant bands. The band clustering and selection strategy is applied to obtain multi-bank samples at first, subsequently, we utilize the first reduction block in Inception V4 network to extract the multi spatial features, then transformer is utilized to learn the long-range sequential spectra data, multilayer perceptron(MLP) is conducted the final classification task. The multi-head self-attention within the transformer is adopted to integrate information with different band cluster and emphasize the contributive spectral features. The proposed method is tested on two widely used hyperspectral datasets and shows the well results.
VVC for Immersive Video Streaming

Immersive video, ranging from 360° video to volumetric three-dimensional video, is expected to gain importance in the future. The recently finalized Versatile Video Coding (VVC) standard was designed to be suitable for a broad range of video content types and services, specifically including 360° video. In the same spirit, the format to encapsulate VVC in the ISO Base Media File Format (ISOBMFF) was tailor-made to support storage and streaming of immersive video. This paper reviews the features of VVC and its ISOBMFF storage for immersive video carriage. Furthermore, VVC-based profiles of the Omnidirectional Media Format standard are described as examples of using VVC and its ISOBMFF storage for immersive video streaming.