Reviewerly/exHarmony huggingface dataset:
Accelerating Continual Learning on Edge FPGA

Real-time edge AI systems operating in dynamic environments must learn quickly from streaming input samples without needing to undergo offline model training. We propose an FPGA accelerator for continual learning based on streaming linear discriminant analysis (SLDA), which is capable of class-incremental object classification. The proposed SLDA accelerator employs application-specific parallelism, efficient data reuse, resource sharing, and approximate computing to achieve high performance and power efficiency. Additionally, we introduce a new variant of SLDA and discuss the accuracy-efficiency trade-offs. The proposed SLDA accelerator is combined with a Convolutional Neural Network (CNN). which is implemented on Xilinx DPU to achieve full continual learning capability at nearly the same latency as inference. Experiments based on popular datasets for continual learning, CoRE50 and CUB200, demonstrate that the proposed SLDA accelerator outperforms the embedded CPU and GPU counterparts, in terms of speed and energy efficiency.
Inferring the Importance of Product Appearance with Semi-supervised Multi-modal Enhancement

Nowadays, almost all the online orders were placed through screened devices such as mobile phones, tablets, and computers. With the rapid development of the Internet of Things (IoT) and smart appliances, more and more screenless smart devices, e.g., smart speaker and smart refrigerator, appear in our daily lives. They open up new means of interaction and may provide an excellent opportunity to reach new customers and increase sales. However, not all the items are suitable for screenless shopping, since some items' appearance play an important role in consumer decision making. Typical examples include clothes, dolls, bags, and shoes. In this paper, we aim to infer the significance of every item's appearance in consumer decision making and identify the group of items that are suitable for screenless shopping. Specifically, we formulate the problem as a classification task that predicts if an item's appearance has a significant impact on people's purchase behavior. To solve this problem, we extract multi-modal features from three different views, and collect a set of necessary labels via crowdsourcing. We then propose an iterative semi-supervised learning framework with a carefully designed multi-modal enhancement module. Experimental results verify the effectiveness of the proposed method.
Robust Optimization-based Motion Planning for high-DOF Robots under Sensing Uncertainty

Motion planning for high degree-of-freedom (DOF) robots is challenging, especially when acting in complex environments under sensing uncertainty. While there is significant work on how to plan under state uncertainty for low-DOF robots, existing methods cannot be easily translated into the high-DOF case, due to the complex geometry of the robot’s body and its environment. In this paper, we present a method that enhances optimization-based motion planners to produce robust trajectories for high-DOF robots for convex obstacles. Our approach introduces robustness into planners that are based on sequential convex programming: We reformulate each convex subproblem as a robust optimization problem that "protects" the solution against deviations due to sensing uncertainty. The parameters of the robust problem are estimated by sampling from the distribution of noisy obstacles, and performing a first-order approximation of the signed distance function. The original merit function is updated to account for the new costs of the robust formulation at every step. The effectiveness of our approach is demonstrated on two simulated experiments that involve a full body square robot, that moves in randomly generated scenes, and a 7-DOF Fetch robot, performing tabletop operations. The results show nearly zero probability of collision for a reasonable range of the noise parameters for Gaussian and Uniform uncertainty.
Real-time Obstacle Avoidance with a Virtual Torque Approach for a Robotic Tool in the End Effector

This paper proposes a real-time obstacle avoidance control scheme for a 6-DOF manipulator with a tool in the end effector. The system consists of environment monitoring, robot-tool segmentation and collision-free motion planning of the manipulator. A Kinect V2 RGB-D camera is used to track obstacles including human and objects in the working environment. The K-D tree algorithm is then adopted to cluster point clouds of the tool and the obstacles. For robot-tool segmentation, we propose a method to model the tool in the end effector and predict its pose in order to solve the camera occlusion problem. For collision-free motion planning, a novel potential field algorithm is proposed to take into consideration of the pose of the tool. A virtual torque approach is proposed and added to the potential field in order to generate a smoother and shorter avoidance motion. The experimental results on a TM5-700 cobot show that the manipulator with a tool in the end effector effectively avoided an obstacle in real time and completed the assigned task. It is shown that the path length with the proposed virtual torque is shortened by 80.43% compared with the case without using the virtual torque.
Semantic Tag Augmented XlanV Model for Video Captioning

The key of video captioning is to leverage the cross-modal information from both vision and language perspectives. We propose to leverage the semantic tags to bridge the gap between these modalities rather than directly concatenating or attending to the visual and linguistic features as the previous works. The semantic tags are the object tags and the action tags detected in videos, which can be viewed as partial captions for the input video. To effectively exploit the semantic tags, we design a Semantic Tag augmented XlanV (ST-XlanV) model which encodes 4 kinds of visual and semantic features with X-Linear Attention based cross-attention modules. Moreover, tag related tasks are also designed in the pre-training stage to aid the model more fruitfully exploits the cross-modal information. The proposed model reaches the 5th place in the pre-training for video captioning challenge with the help of the semantic tags. Our codes will be available at: https://github.com/RubickH/ST-XlanV.
Development and Evaluation of a Novel Investment Decision System in Cryptocurrency Market

More and more people are entering the cryptocurrency market after Bitcoin (BTC) soared to nearly USD 20,000 in 2017. To promote the development of information technology and cryptocurrency marketing, various computerized systems integrating information technology with investment and financing are innovated continuously. In this study, the daily cryptocurrency prices were input to Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM); and the developing trend plots were drawn to predict and analyze the future cryptocurrency prices through deep learning. Finally, the business practices of cryptocurrency investment were modularized based on High-Level Fuzzy Petri Nets (HLFPNs) to make a better investment decision so that all investors can use this decision system to quickly understand the future cryptocurrency trend. The experimental results have shown that this decision system can provide effective investment information to achieve investors’ personal financial goals with the expectation of improving financial situations.
Collaborative Verifiable Delay Functions

We propose and define a new primitive, collaborative verifiable delay functions (coVDFs), an extension to VDFs allowing multiple parties to jointly compute a publicly verifiable delay, whilst encapsulating a personal input from each party. These personal inputs can contain information such as a hash of a bid in an auction, or some public identifier of the solving party. We highlight the differences between the single-party and the multi-party settings, and discuss some applications facilitated by the additional properties offered by coVDFs.We formalise this new primitive, as well as the relevant security properties, and we introduce the notions of robustness and traceability, which mitigate adversarial behaviour arising from the introduction of multiple parties. We propose two candidate constructions: the first is an extension of Wesolowski’s VDF construction from EUROCRYPT 2019, relying on repeated squaring in a finite abelian group. We prove that this extension satisfies the traceability property, however it is not robust. Our second construction is based on the hashgraph protocol proposed by Baird in 2016, and involves each of the n parties repeatedly implementing a gossip protocol to one another and computing a hash each time they do. The construction results in every party producing a copy of the same graph, and is robust, meaning it runs correctly, in the presence of up to n/3 malicious parties.
Infrastructural configurations of mobile telephony in urban Africa: vignettes from Buru Buru, Nairobi

Since its inception in the 1990s, mobile telephony in Africa has evolved, reflecting varied advances in technology. These advances have become particularly reminiscent of the role of mobile technologies in everyday facets of life. In this paper, we examine infrastructural configurations of mobile telephony in an urban African context. We demonstrate how urban Africa is being instrumented through the incoming of mobile telephony, but also how the convergence of the digital and the physical is materializing through the everyday use and appropriation of mobile phone-based technologies. We make this contribution through illustrations and vignettes – including Mkokoteni handcart operations, electricity “token” meter connections, and mobile phone kiosk processes – from Buru Buru, an estate in Nairobi. Thus, we place our analysis within recent area studies scholarship, drawing perspectives from science, technology and society studies, infrastructure studies and urban studies. We contend that the variegated infrastructural configurations of mobile telephony challenge ingrained accounts of technological determinism, not least within conventional area studies.
Autonomous Multi-View Navigation via Deep Reinforcement Learning

In this paper, we propose a novel deep reinforcement learning (DRL) system for the autonomous navigation of mobile robots that consists of three modules: map navigation, multi-view perception and multi-branch control. Our DRL system takes as the input a routed map provided by a global planner and three RGB images captured by a multi-camera setup to gather global and local information, respectively. In particular, we present a multi-view perception module based on an attention mechanism to filter out redundant information caused by multi-camera sensing. We also replace raw RGB images with low-dimensional representations via a specifically designed network, which benefits a more robust sim2real transfer learning. Extensive experiments in both simulated and real-world scenarios demonstrate that our system outperforms state-of-the-art approaches.
Finding Solutions to the Integer Case Constraint Satisfiability Problem Using Grover’s Algorithm

Constraint satisfiability problems, crucial to several applications, are solved on a quantum computer using Grover's search algorithm, leading to a quadratic improvement over the classical case. The solutions are obtained with high probability for several cases and are illustrated for the cases involving two variables for both 3- and 4-bit numbers. Methods are defined for inequality comparisons, and these are combined according to the form of the satisfiability formula, to form the oracle for the algorithm. The circuit is constructed using IBM Qiskit and is verified on an IBM simulator. It is further executed on one of the Noisy Intermediate-Scale Quantum (NISQ) processors from IBM on the cloud. Noise levels in the processor at present are found to be too high for successful execution. Running the algorithm on the simulator with a custom noise model lets us identify the noise threshold for successful execution.
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies

We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with only relevant states as input. These policies are pretrained using the internal model with domain randomization over the relevant states. The policy network weights are then transferred to the target domain (e.g., the real world) for fine tuning. We perform extensive policy transfer experiments in simulation for two representative manipulation tasks: block stacking and crate opening. Our policies are shown to be more robust to domain shifts, more sample efficient to learn, and scale to more complex settings with larger state spaces. We also show improved zero-shot sim- to-real transfer of our policies for the block stacking task.
Exploring the power of lightweight YOLOv4

Research on deep learning has always had two main streams: (1) design a powerful network architecture and train it with existing learning methods to achieve the best results, and (2) design better learning methods so that the existing network architecture can achieve the best capbility after training. In recent years, because mobile device has become popular, the requirement of low power consumption becomes a must. Under the requirement of low power consumption, we hope to design low-cost lightweight networks that can be effectively deployed at the edge, while it must have enough resources to be used and the inference speed must be fast enough. In this work, we set a very ambitious goal of exploring the power of lightweight neural networks. We utilize the analysis of data space, model’s representational capacity, and knowledge projection space to construct an automated machine learning pipeline. Through this mechanism, we systematically derive the most suitable knowledge projection space between the data and the model. Our method can indeed automatically find learning strategies suitable for the target model and target application through exploration. Experiment results show that the proposed method can significantly enhance the accuracy of lightweight neural networks for object detection. We directly apply the lightweight model trained by our proposed method to a Jetson Xavier NX embedded module and a Kneron KL720 edge AI SoC as system solutions.
Deep-Learning-Based Super-Resolution of Video Satellite Imagery by the Coupling of Multiframe and Single-Frame Models

Image super-resolution (SR) is an effective solution to the limitation of the spatial resolution of video satellite images, which is caused by the degradation and compression in the imaging phase. For the processing of satellite videos, the commonly employed deep-learning-based single-frame SR (SFSR) framework has limited performance without using complementary information between the video frames. On the other side, the multiframe SR (MFSR) can utilize temporal subpixel information to super-resolve the high-resolution (HR) imagery. However, although deeper and wider deep learning network provides powerful feature representations for SR methods, it has always been a challenge to accurately reconstruct the boundaries of ground objects in video satellite images. In this article, to address these issues, we propose an edge-guided video SR (EGVSR) framework for video satellite image SR, which couples the MFSR model and the edge-SFSR (E-SFSR) model in a unified network. The EGVSR framework is composed of an MFSR branch and an edge branch. The MFSR branch is used to extract the complementary features from the consecutive video frames. Concurrently, the edge branch acts as an SFSR model to translate the edge maps from the low-resolution modality to the HR one. At the final SR stage, the DBFM is built to focus on the promising inner representations of the features of the two branches and fuse them. Extensive experiments on video satellite imagery show that the proposed EGVSR method can achieve superior performance compared to the representative deep-learning-based SR methods.
EFM-Net: Feature Extraction and Filtration with Mask Improvement Network for Object Detection in Remote Sensing Images

Object detection is an essential task in computer vision. Many methods have made significant progress in ordinary object detection. Due to the particularity of remote sensing images, the detection target is tiny, the background is messy, dense, and has mutual occlusion, which makes the general detection method challenging to apply to remote sensing images. For these problems, we propose a new detection framework feature extraction and filtration method with a mask improvement network (EFM-Net) to enhance object detection ability. In EFM-Net, we designed a multi-branched feature extraction (MBFE) module to better capture the information in the feature graph. In order to suppress the background interference, we designed a background filtering module based on attention mechanisms to enhance the attention of objects. Finally, we proposed a mask generate the boundary improvement method to make the network more robust to occlusion detection. We tested the DOTA v1.0, NWPU VHR-10, and UCAS-AOD datasets, and the experimental results show that our method has excellent effects.
Communication and Personality

With the COVID-19 pandemic, governments have used applications to help citizens to be informed about the disease. Some applications use chatbots, which despite the advances, still represent challenges for researchers and designers in the domain of human-computer interaction (HCI). Therefore, this paper aims to evaluate three Brazilian government chatbots focused on COVID-19, considering two perspectives: communication (which involves communicability and language) and personality. For this, we used the Semiotic Inspection Method (SIM) to evaluate the chatbots' communicability and best practice guidelines for evaluating the chatbots' language and personality. As a result, we show that chatbots make little use of communicability strategies. In addition, they do not convey personality traits. We also discussed the relationship between personality and the designer's self-expression, making room for potential future work in the area. The research contributes to the extent that the study of COVID-19 chatbots in the current pandemic scenario is important and presents a methodology for assessing the language and personality of chatbots.
Image enhancement via texture protection Retinex

Images obtained in dim light do not clearly represent the target scene, limiting information transmission on the image carrier. This study proposes a texture-preserving image enhancement method, i.e. ETPR. The proposed method draws the illumination map of a low light image by Max-RGB, and then applies the weighted median filter algorithm and Retinex to improve the illumination map further. Then, the enhanced image is changed from RGB mode to YCbCr mode and the texture of the luminance component Y is described. It is achieved by denoising the texture of the image to be enhanced, and then the final enhanced image is obtained by the method of sub-region image fusion denoising. The effectiveness of ETPR is established by comparing it with existing enhancement technologies using public data and real images.
DBDNet for denoising in ESPI wrapped phase patterns with high density and high speckle noise

In this paper, we propose a dilated-blocks-based deep convolution neural network, named DBDNet, for denoising in electronic speckle pattern interferometry (ESPI) wrapped phase patterns with high density and high speckle noise. In our method, the proposed dilated blocks have a specific sequence of dilation rate and a multilayer cascading fusion structure, which can better improve the effect of speckle noise reduction, especially for phase patterns with high noise and high density. Furthermore, we have built an abundant training dataset with varieties of densities and noise levels to train our network; thus, the trained model has a good generalization and can denoise ESPI wrapped phase in various circumstances. The network can get denoised results directly and does not need any pre-process or post-process. We test our method on one group of computer-simulated ESPI phase patterns and one group of experimentally obtained ESPI phase patterns. The test images have a high degree of speckle noise and different densities. We compare our method with two representative methods in the spatial domain and frequency domain, named oriented-couple partial differential equation and windowed Fourier low pass filter (LPF), and a method based on deep learning, named fast and flexible denoising convolutional neural network (FFDNet). The denoising performance is evaluated quantitatively and qualitatively. The results demonstrate that our method can reduce high speckle noise and restore the dense areas of ESPI phase patterns, and get better results than the compared methods. We also apply our method to a series of phase patterns from a dynamic measurement and get successful results.
Marker‐less augmented reality based on monocular vision for falx meningioma localization

The existing augmented reality (AR) based neuronavigation systems typically require markers and additional tracking devices for model registration, which causes excessive preparatory steps.For fast and accurate intraoperative navigation, this work proposes a marker-less AR system that tracks the head features with the monocular camera. After the semi-automatic initialization process, the feature points between the captured image and the pre-loaded keyframes are matched for obtaining correspondences. The camera pose is estimated by solving the Perspective-n-Point problem.The localization error of AR visualization on scalp and falx meningioma is 0.417 ± 0.057 and 1.413 ± 0.282 mm, respectively. The maximum localization error is less than 2 mm. The AR system is robust to occlusions and changes in viewpoint and scale.We demonstrate that the developed system can successfully display the augmented falx meningioma with enough accuracy and provide guidance for neurosurgeons to locate the tumour in brain.
Performance Evaluation of JPEG Standards, WebP and PNG in Terms of Compression Ratio and Time for Lossless Encoding

Still images have a significant share in the data created today. These images take up a lot of space in their raw form. Image compression algorithms provide a much smaller representation of images in order to solve this problem. The JPEG group offered many alternatives to the JPEG algorithm, which became insufficient due to the increase in resolution and color detail. In this study, lossless compression ratios and encoding-decoding times of JPEG-LS, JPEG 2000, JPEG XR, JPEG XT, JPEG XL and also WebP and PNG algorithms, were examined on selected data sets. For the experiments, images in three categories as SDR, 16-bit and HDR were obtained. In obtained results, JPEG XL gives the most successful results, with the best in compression ratio and good compression and decompression speeds. For 16 bit images, JPEG XR gives the best compression speed and PNG’s only advantage is its decompression speed.