Reviewerly/exHarmony huggingface dataset:
Detection-segmentation convolutional neural network for autonomous vehicle perception

Object detection and segmentation are two core modules of an autonomous vehicle perception system.They should have high efficiency and low latency while reducing computational complexity.Currently, the most commonly used algorithms are based on deep neural networks, which guarantee high efficiency but require high-performance computing platforms.In the case of autonomous vehicles, i.e. cars, but also drones, it is necessary to use embedded platforms with limited computing power, which makes it difficult to meet the requirements described above.A reduction in the complexity of the network can be achieved by using an appropriate: architecture, representation (reduced numerical precision, quantisation, pruning), and computing platform.In this paper, we focus on the first factor -the use of so-called detection-segmentation networks as a component of a perception system.We considered the task of segmenting the drivable area and road markings in combination with the detection of selected objects (pedestrians, traffic lights, and obstacles).We compared the performance of three different architectures described in the literature: MultiTask V3, HybridNets, and YOLOP.We conducted the experiments on a custom dataset consisting of approximately 500 images of the drivable area and lane markings, and 250 images of detected objects.Of the three methods analysed, MultiTask V3 proved to be the best, achieving 99% mAP50 for detection, 97% MIoU for drivable area segmentation, and 91% MIoU for lane segmentation, as well as 124 fps on the RTX 3060 graphics card.This architecture is a good solution for embedded perception systems for autonomous vehicles.The code is available at: https://github.com/vision-agh/MMAR_2023.
Research and Implementation of DMA Based on SR-IOV

With the development of cloud computing, container cloud, and other technologies, CPU and memory virtualization becomes more mature. Since the performance of I/O virtualization affects the overall operational efficiency of the system, improving I/O performance and the utilization of transmission bandwidth of devices is the focus of virtualization technology research. Based on the theoretical research of Single Root I/O Virtualization (SR-IOV) technology, this paper designs and implements a Direct Memory Access(DMA) interaction architecture based on SR-IOV virtualization devices using Field Programmable Gate Array (FPGA), which solves the need for high-speed data transmission between cloud computing platforms and virtual devices. This architecture adopts the processing method of independent virtual device management descriptors and shared data movers, ensuring independent operation between virtual devices and reducing the resource utilization rate of the design. Firstly, the chain structure is employed flexibly and efficiently to achieve descriptor management. In the meantime, it also reduces the pressure of interrupts on the CPU. Secondly, the data mover implementation framework adopts asynchronous pipelining mode, with extremely high transmission performance. After testing and verification, the interactive architecture can achieve high-speed data communication, with read and write bandwidth utilization rates of up to 87% and 79%, respectively. At the same time, this architecture can balance the bandwidth allocation among virtual devices.
SiamIRPN: Siamese Visual Tracking with Improved Region Proposal Networks

Though introducing the Region Proposal Network (RPN) from object detection enabled Siamese trackers’ success, RPN-based trackers still struggle in challenging scenarios. We posit that the reason comes from two major limitations of the introduced RPN, where one is that the external structure of the RPN is simple and straight, the other is that the internal components used in RPN cannot cope with complex scenes. In this paper, we propose an Improved RPN (IRPN) suitable for visual tracking. Externally, we place a Convolutional Block Attention Mechanism (CBAM) to the IRPN, and internally, we adopt the advanced component groups for the IPRN. Using multiple IRPN blocks and deep architecture, we propose a Siamese tracker (SiamIRPN) having a layer-wise structure. Comprehensive experiments and ablation studies on five benchmarks (VOT-2019, OTB100, UAV123, GOT-10k, and NFS) show our proposed SiamIRPN achieves competitive performance.
End-to-End Learned Light Field Image Rescaling Using Joint Spatial-Angular and Epipolar Information

Light field (LF) rescaling is indispensable in accommodating different LF image resolutions for different applications. Unlikely most recent studies which only execute learned LF upscaling from a predefined downscaling method, we propose a novel LF rescaling framework by jointly optimizing learned LF downscaling and upscaling as a combined task. Specifically, our light field rescaling network (LFRN) simultaneously extracts features from different 2D subspaces of LF data (e.g., spatial-angular and epipolar subspaces) to fully handle 4D LF image information. Our newly designed attention fusion module (AFM) adaptively combines these two data features based on learnable embedding weights. Due to joint optimization of the learned LF downscaling and upscaling tasks, our LFRN method can achieve significant performance gain in both objective and subjective visual qualities compared to conventional predefined downscaling with learned LF upscaling task.
RFID-assisted visual multiple object tracking without using visual appearance and motion

Visual Multiple Object Tracking (MOT) typically utilizes appearance and motion clues for associations. However, these features may be limited under certain challenging scenarios, such as appearance ambiguity and frequent occlusions. In this paper, we introduce a novel deep RF-affinity neural network (DRFAN) that enhances visual tracking with the aid of a passive wireless positioning device, Radio Frequency Identification (RFID). DRFAN aims to solve object tracking by introducing a new concept of a "candidate trajectory" to indicate target movement. This approach fundamentally deviates from existing fusion methods that rely on known visual tracks. Instead, DRFAN exclusively uses detection bounding boxes and RFID signals. The proposed method overcomes the limitations of visual tracking by swiftly resuming correct tracking whenever a failure occurs. This is the first time using signals from low-cost passive RFID tags to achieve image-level localization, and a discriminative neural network is designed specifically for RFID-assisted visual association. Our experimental results validate the robustness and applicability of the proposed approach.
Realization of Software Artifact Knowledge Base with Text Classification

Software artifacts are deliverables created by developers throughout the software development lifecycle, including documents, codes, test cases, and other outputs. Effective management of software artifacts is essential for tracking software requirements and enhancing development efficiency. To manage the artifacts effectively and improve software development efficiency, we design and implement a software artifact knowledge base. To further enhance the organization of text software artifacts, we propose a text classification model utilized by software artifact knowledge base to classify software texts artifacts. Experiments are conducted to assess the performance of the text classification model.
Identity-Based Proxy Re-Encryption Based on LWE with Short Parameters

Proxy re-encryption (PRE) can guarantee the security of data sharing in the cloud making it important in ciphertext sharing in the cloud computing environment. Plenty of the current PRE algorithms based on the learning with errors (LWE) problem are constructed under the random oracle model or have problems such as long public parameter size, large ciphertext expansion rate, etc. We propose an identity-based PRE (IB-PRE) scheme using the LWE problem with short public parameters which can solve the mentioned challenges. Our scheme uses the blocking operation to decrease the public parameter size and a more useful trapdoor generation algorithm to reduce the trapdoor size. These two technologies improve the computational efficiency of the public key. We think that when we divide the identity id into 16 blocks, without much increase in computational overhead, the public key size can be reduced by 88.34%, and the scheme’s security level can remain unchanged. Our scheme achieves adaptive IND-ID-CPA security. It is unidirectional, collusion-resistant, and constructed under the standard model.
Shuffled Differentially Private Federated Learning for Time Series Data Analytics

Trustworthy federated learning aims to achieve optimal performance while ensuring clients’ privacy. Existing privacy-preserving federated learning approaches are mostly tailored for image data, lacking applications for time series data, which have many important applications, like machine health monitoring, human activity recognition, etc. Furthermore, protective noising on a time series data analytics model can significantly interfere with temporal-dependent learning, leading to a greater decline in accuracy. To address these issues, we develop a privacy-preserving federated learning algorithm for time series data. Specifically, we employ local differential privacy to extend the privacy protection trust boundary to the clients. We also incorporate shuffle techniques to achieve a privacy amplification, mitigating the accuracy decline caused by leveraging local differential privacy. Extensive experiments were conducted on five time series datasets. The evaluation results reveal that our algorithm experienced minimal accuracy loss compared to non-private federated learning in both small and large client scenarios. Under the same level of privacy protection, our algorithm demonstrated improved accuracy compared to the centralized differentially private federated learning in both scenarios.
Thinking of How to Use the Gaze Estimation in the Wild

The eye gaze of humans can directly reflect their attention on some objects, so it becomes one of the most critical factors in improving the user experience of HRI (Human-Robot Interaction) or other intelligent systems. Moreover, empowered by deep learning, gaze estimation methods’ accuracy has been improved and can be used in many applications. However, researchers and engineers still encounter some challenges when using state-of-the-art methods in the wild. One of the challenges is personalization, which means data distribution through people is different. Furthermore, the other challenge is the different devices’ distribution which we will focus on in this paper. The different distribution of images or video will need to be clarified for the model to extract features, while the different devices’ distribution will mislead the methods to project the PoG on the space. So in this paper, we propose a semantic representation for gaze(SRG) representation to alleviate this challenge caused by the different distribution of the devices. Based on the experiment, we got comparable results on different datasets. Moreover, this kind of gaze estimation method can be easily used on various devices.
Sample Augmentation and Orientation-aware Deep Neural Network for Logistics Objects Detection

Object detection is an important task in computer vision, with wide-ranging applications from autonomous driving to Logistics including image and video analysis. Among these, oriented object detection has emerged as a promising approach for detecting and localizing objects in images with arbitrary orientations. While traditional object detection methods rely on horizontal bounding boxes so that it is difficult for detecting oriented objects, oriented object detection leverages the rich representation power of deep neural networks to learn object-oriented features that capture both the appearance and orientation of objects in images. This paper introduces an orientation-aware neural network detector called Directional R-CNN with sample augmentation. The proposed framework addresses the issue of over-fitting by using various data augmentation techniques to improve the quality and quantity of the image samples. Furthermore, the effectiveness of the proposed approach is extensively evaluated through experiments conducted on real-world logistics applications, specifically for detecting oriented objects. The results indicate that the detection of enhanced image samples outperforms other methods with the mAP of 93.5%.
Privacy-preserving Online Ride-hailing Service System Based on Taking the Intersection of Private sets of Points of Interest

Online ride-hailing has gained recognition and become increasingly popular as a way to ease traffic congestion. However, it also poses a serious threat to people's data privacy. While existing schemes have made good progress in protecting the location privacy of drivers and passengers, there are also some problems. When ride-hailing platforms are faced with a large number of requests from passengers and drivers, the time complexity of conducting driver-passenger matching is relatively high; in some schemes based on hidden region division, drivers and passengers located at the boundary of adjacent regions with relatively similar locations cannot be matched successfully. In the paper, we propose a privacy-preserving online ride-hailing service system based on taking the intersection of private sets of points of interest (PIHS). The core idea of the scheme is to use a set of location points to represent the geographic location of a user in an online ride-hailing service. It converts the explicit set into a privacy set that is unrecognizable to an attacker and converts the distance between two location points into a problem of finding the intersection of the privacy sets, so that drivers and passengers with a significant number of intersections between the two privacy sets can be matched successfully. Under the traditional online ride-hailing architecture and the Cloud-Edge-End architecture for the Internet of Vehicles, we use existing cryptographic techniques to design a privacy set intersection solution that can be operated by a third party without frequent interactions, and let the online platform act as a third party to perform operations such as privacy set intersection to provide driver-passenger matching services. In contrast to the latest privacy protection schemes, this scheme performs better when the ride-hailing platform matches a large number of drivers and passengers. Under the premise of protecting the privacy and security of users, the larger the number of users, the more obvious the advantages of this scheme.
ASIFC: Auditable and Secure Information Flow Control for PaaS Cloud

In the cloud, data security issues such as data leakage and maliciously access frequently occurs. Deploying strict security controls on the information flows in the cloud can effectively prevent such problems from happening. How to accurately control the information flows in the cloud and provide an intuitive way of auditing has become an indispensable need to enhance data security. This paper proposes an auditable and secure information flow control (ASIFC) module for PaaS cloud. By using decentralized information flow control model, ASIFC strictly controls the inter-process communication such as pipes, shared memory, and message queues, accurately and intuitively controlling the information flows in the cloud. Moreover, based on NetworkX, a log-driven visualization interface which can display flow direction of information in PaaS cloud is offered, providing understandable content for auditing. The performance evaluation results show that the module outperforms FlowK and Flume in micro-benchmark.
Query by Activity Video in the Wild

This paper considers retrieval of videos containing human activity from just a video query. In the literature, a common assumption is that all activities have sufficient labelled examples when learning an embedding for retrieval. However, this assumption does not hold in practice, as only a portion of activities have many examples, while other activities are only described by few examples. In this paper, we propose a visual-semantic embedding network that explicitly deals with the imbalanced scenario for activity retrieval. Our network contains two novel modules. The visual alignment module performs a global alignment between the input video and visual feature bank representations for all activities. The semantic module performs an alignment between the input video and semantic activity representations. By matching videos with both visual and semantic activity representations over all activities, we no longer ignore infrequent activities during retrieval. Experiments on a new imbalanced activity retrieval benchmark show the effectiveness of our proposal.
Single Image Deraining via Feature-based Deep Convolutional Neural Network

It is challenging to remove rain-steaks from a single rainy image because the rain steaks are spatially varying in the rainy image. Although the CNN based methods have reported promising performance recently, there are still some defects, such as data dependency and insufficient interpretation. A single image deraining algorithm based on the combination of data-driven and model-based approaches is proposed. Firstly, an improved weighted guided image filter (iWGIF) is used to extract high-frequency information and learn the rain steaks to avoid interference from other information through the input image. Then, transfering the input image and rain steaks from the image domain to the feature domain adaptively to learn useful features for high-quality image deraining. Finally, networks with attention mechanisms is used to restore high-quality images from the latent features. Experiments show that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both qualitative and quantitative measures.
Truncated Weighted Nuclear Norm Regularization and Sparsity for Image Denoising

The attribute of signal sparsity is widely used to sparse representaion. The existing nuclear norm minimization and weighted nuclear norm minimization may achieve a suboptimal in real application with the inaccurate approximation of rank function. This paper presents a novel denoising method that preserves fine structures in the image by imposing L <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> norm constraints on the wavelet transform coefficients and low rank on high-frequency components of group similar patches. An efficient proximal operator of Truncated Weighted Nuclear Norm (TWNN) is proposed to accurately recover the underlying high-frequency components of low rank patches. By combining a wavelet domain sparse preservation prior with TWNN, the proposed method significantly improves the reconstruction accuracy, leading to a higher PSNR/SSIM and visual quality than state of the art approaches.
A Novel Subcharacter-enhanced Hybrid Network for Chinese Text Classification

Chinese text classification aimed to set pre-defined labels to given documents, and it is a very important research field. To deal with this problem, a novel subcharacter-enhanced hybrid network model called SHN is proposed. The proposed SHN can hierarchically extract the text semantic features. Firstly, our model decomposes Chinese words by Cangjie-code into subcharacters. Secondly, both word embeddings and subcharacters n-grams vectors are used as the input part of SHN, which is aimed to fuse both internal structure information and semantic information. Thirdly, SHN makes use of a bidirectional GRU module that contains word-level attention, to obtains semantic information from the sentences in document. Finally, a dynamic convolution neural network module is used by SHN, which aimed to extract high level features. Experiments carried out on two datasets show that SHN can obtain better results than conventional methods and is effective.
Simultaneous Watermarking and Draco 3D Object Compression Method

In our present society, 3D objects play an increasingly important role in many different domains, especially with the development of meta environments. Many applications demand large 3D objects, which makes their compression a requirement. Meanwhile, 3D objects are also exposed to various security problems during online storage and transmission. Consequently, security aspects such as copyright and authentication are essential for 3D objects. In this paper, we propose a simultaneous 3D object watermarking and compression method based on Draco. A watermarking step is integrated within Draco, Google’s 3D compression method, which is rapidly being universally adopted as standard. The proposed method enables a large bit embedding rate, which can be used for copyright information. The proposed method is also, to the best of our knowledge, the first watermarking method for 3D objects compressed with Draco.
Carefull Tread: A Lightweight Consensus in Blockchain to Trust Mobility Data using Q-Learning

Mobility data generated from Connected and Autonomous Vehicles (CAVs) and micro-mobility devices (e-scooters, e-bikes and smartphones) is critical to understand different safety and future Intelligent Transportation System (ITS) needs. Blockchain is used to endorse such mobility data in a secure manner, however, trusting large amount of data from massive number of devices is challenging as results in scalability issues along incurring longer delays. To cater this, this paper employs reinforcement learning, specifically, Q-learning towards a lightweight mobility data validation at scale. We derive the optimal policy for a node to endorse mobility data generated by nearby nodes in a blockchain network. We present a novel consensus protocol for the node to learn the block sampling rates and time delay thresholds based on the neighborhood topology and network connectivity. We simulate the proposed consensus protocol in NS3, and it has shown to achieve lower delays, overhead and high throughput for up to 50 nodes simultaneously endorsing each others mobility data in New York city.
An Effective DDPG-generated Task Scheduling Policy to Minimize Latency in Distributed Computing System

With the development of artificial intelligence technology in recent years, more and more computation-intensive applications are emerging on user devices, which are difficult to meet the needs of users due to the limitations of their own processing power and endurance. Offloading computing tasks to cloud servers with sufficient resources is an effective solution to this problem. Meanwhile, how to schedule computing tasks on cloud servers is critical. In this paper, we study the problem of how to assign tasks to distributed cloud computing servers such that the computation latency, i.e., their computing time, can be minimized, where tasks are dependent and can be modeled as directed acyclic graphs. We first formulate this problem as a mixed-integer programming problem with some linear and nonlinear constraints. A Markov decision process is built for solving this problem, where we define the state, action, and reward functions. The deep deterministic policy gradient algorithm is then successfully applied to build a task scheduling policy. Various tradeoffs are investigated in experiments, and the results confirm that our built-in scheduling policy is more effective than some classic heuristic scheduling approaches.