Reviewerly/exHarmony huggingface dataset:
Neurons Perception Dataset for RoboMaster AI Challenge

From virtual game to physical robot, games have witnessed the development of artificial intelligence (AI) technology, especially the data-driven technology represented by deep learning. Compared with virtual games, a physical robot game such as RoboMaster AI challenge needs to build a complete closed-loop architecture composed of perception, planning, control, and decision-making to support autonomous confrontation. Perception, as the eye of the robot, its performance in the complex environment depends on a massive dataset. Although there are many open perception datasets, these datasets are difficult to meet the needs of RoboMaster AI challenge due to the high dynamics of the task, the distinctiveness of the objects, and limited computing resources. In this paper, we release a dataset named Neurons <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Neurons is a team dedicated to promoting the development of robot with deep neural network. We will release the code and dataset at https://github.com/DRL-CASIA/NeuronsDataset. perception dataset for RoboMaster AI challenge, which covers 3 tasks including monocular depth estimation, lightweight object detection, and multi-view 3D object detection, and makes up the data blank in this field. In addition, we also evaluate State-Of-The-Art (SOTA) methods on each task, hoping to provide an impartial benchmark for the development of perception algorithm.
Adaptive Deep Learning Network With Multi-Scale and Multi-Dimensional Features for Underwater Image Enhancement

Owing to the turbidity, light absorption, and scattering of water, the quality of the acquired underwater images is worse and seriously affects the processing of subsequent vision tasks. To solve this problem, a high-performance underwater image enhancement model is proposed in this paper. We design a convolutional structure and a pooling structure. The convolution structure is named ‘flower convolution’, which can well expand the perceptual field and retain the ability to extract local features. The pooling method is called ‘dewater pooling’, which not only considers the overall features but also highlights the salient features. In addition, we propose a novel attention mechanism and a multi-dimensional feature fusion method, which can well fuse high and low-dimensional features, strengthen salient features, and suppress unimportant features. Moreover, we design an adaptive loss function, which can evaluate the quality of underwater images more reasonably. At last, we train and test the model on a real-world underwater image/video dataset. The experimental results demonstrate that the model outperforms the state-of-the-art methods in both qualitative and quantitative evaluation. The experimental results of the ablation study demonstrate the usefulness of the various components. Further applied research also shows the superior capability of the model.
Multi-Scale Attention and Structural Relation Graph for Local Feature Matching

Building a dense correspondence between two images is a fundamental vision problem. Most existing methods use local features, but global features cannot be ignored. Local features are often not enough to disambiguate similar regions without global features. Computing relevant features between images requires structural relationship and the importance of local features. For that, We propose novel multi-scale attention and structural relation graph (MASRG) for local feature matching. The MASRG adopts an overall architecture that first builds coarse-level matches on a coarse feature map and then refines fine matches on a fine-level feature map. We propose a structural relation graph module and a multi-scale attention module. We introduce global context information into the overall architecture. Using global information to separately assist in learning the structural information between local descriptors, the features of different receptive fields, and the importance of modeling single local information, a limited number of possible matches can be obtained with high confidence. Finally, the matching relationship is predicted. In this way, the network significantly improves the matching reliability and localization accuracy. Our proposed method has 5.6%, 6.7%, and 6.3% performance increases over the baseline method(See I) under different conditions in the HPatches. Extensive experiments on three large-scale datasets (i.e., HPatches, InLoc, and Aachen Day-Night v1.1) demonstrate that our proposed MASRG method is superior to state-of-the-art local feature matching approaches.
How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulation

Reinforcement learning (RL) has been shown to be effective at learning control from experience. However, RL typically requires a large amount of online interaction with the environment. This limits its applicability to real-world settings, such as in robotics, where such interaction is expensive. In this work we investigate ways to minimize online interactions in a target task, by reusing a suboptimal policy we might have access to, for example from training on related prior tasks, or in simulation. To this end, we develop two RL algorithms that can speed up training by using not only the action distributions of teacher policies, but also data collected by such policies on the task at hand. We conduct a thorough experimental study of how to use suboptimal teachers on a challenging robotic manipulation benchmark on vision-based stacking with diverse objects. We compare our methods to offline, online, offline-to-online, and kickstarting RL algorithms. By doing so, we find that training on data from both the teacher and student, enables the best performance for limited data budgets. We examine how to best allocate a limited data budget - on the target task - between the teacher and the student policy, and report experiments using varying budgets, two teachers with different degrees of suboptimality, and five stacking tasks that require a diverse set of behaviors. Our analysis, both in simulation and in the real world, shows that our approach is the best across data budgets, while standard offline RL from teacher rollouts is surprisingly effective when enough data is given.
Adversarial Fine-tune with Dynamically Regulated Adversary

Adversarial training is an effective method to boost model robustness to malicious, adversarial attacks. However, such improvement in model robustness often leads to a significant sacrifice of standard performance on clean images. In many real-world applications such as health diagnosis and autonomous surgical robotics, the standard performance is more valued over model robustness against such extremely malicious attacks. This leads us to the question: to what extent can we improve the robustness of the model without sacrificing standard performance? This work tackles this problem and proposes a simple yet effective transfer learning based adversarial training strategy that disentangles the negative effects of adversarial samples on model's standard performance. In addition, we introduce a training-friendly adversarial attack algorithm, which facilitates the boost of adversarial robustness without introducing significant training complexity. Extensive experiments show that the proposed approach outperforms previous adversarial training algorithms with the following objective: to improve the robustness of the model while preserving model's standard accuracy on clean data.
Vietnamese Document Analysis: Dataset, Method and Benchmark Suite

Document image understanding is increasingly useful since the number of digital documents is increasing day-by-day and the need for automation is increasing.Object detection plays a significant role in detecting vital objects and layouts in document images and contributes to providing a clearer understanding of the documents.Nonetheless, previous research mainly focuses on English document images, and studies on Vietnamese document images are limited.In this study, we extensively benchmark state-of-the-art object detectors and analyze the performance of each method on Vietnamese document images.Moreover, we also investigate the effectiveness of four different loss functions on the experimental object detection methods.Extensive experiments on the UIT-DODV dataset are conducted to provide insightful discussions.
Integrated Autoencoder-Level Set Method Outperforms Autoencoder for Novelty Detection

Novelty detection (ND), also known as one-class classification or anomaly/outlier detection, has attracted a lot of research interest across a range of applications, where abnormal data are limited or are extremely rare, e.g., credit card [26], mobile phone fraud detection [1], mobile robotics [2], sensor networks[3], rumor detection [4], [5], video surveillance [6]–[8] and healthcare [9]–[11] areas. ND aim to train the detectors with the target (normal, negative) class and then identify the deviated data as novelties (abnormal, positive) class using the trained detectors.
PG-YOLO: A Novel Lightweight Object Detection Method for Edge Devices in Industrial Internet of Things

With the rapid development of Industrial Internet of Things (IIoT) technology, video surveillance devices and video data in IIoT environments are massively growing and increasingly important.Deploying rapidly evolving deep learning-based object detection algorithms in IIoT can improve the efficiency of video data utilization and increase the automation and intelligence of the IIoT.Facing the transmission latency problem of massive video data, the algorithms are better deployed in edge devices.However, due to the large size and high computing power requirements of existing object detection algorithms, it is difficult to deploy them on edge devices with limited performance.Therefore, to solve this problem, we propose a smaller and less computationally intensive object detection algorithm, PG-YOLO.We replaced all the convolutional modules of YOLOV5 with low computational ghost modules and streamlined the backbone network to improve the network performance.Then, we propose a better pruning algorithm to compress the network, and finally improve the accuracy by distillation to obtain PG-YOLO.PG-YOLO is smaller and faster and can be deployed in edge devices.In addition, we conducted experimental validation on the SHWD dataset.The experimental results show that PG-YOLO can compress the volume of the model by 9 times compared with YOLOv5s, and the compressed detection accuracy reaches 0.934, with only 0.1% loss of accuracy.Also, the time required for inference is reduced by 32.7%, and the inference speed is improved by 10 FPS.Compared with other object detection algorithms, PG-YOLO also has advantages.
Prediction of Breast Cancer using Machine Learning Techniques

Breast cancer is a topic that is frequently discussed these days. It is one of the most widespread diseases and forms of cancer. The National Cancer Institute says that the second most frequent malignancy in women is breast cancer. Every year, around 2000 new instances of breast cancer are diagnosed in males, while approximately 2,30,000 new cases are diagnosed in women. The diagnosis of this disease is crucial so that women can get it treated in its early stages. It is best for an precise and timely diagnosis. This is a critical phase in the therapy and recovery process. Breast cancer detection is done with the help of mammograms, which are basically X-rays of the breasts. It's a tool that is used to detect and help diagnose breast cancer. But detection is not easy due to different kinds of uncertainties in using these mammograms. Breast cancer can be detected using machine learning algorithms. These approaches can be used to create tools for clinicians that can be used to identify and diagnose breast cancer at an early stage. This will significantly improve patient survival rates.
SaGCN: Semantic-Aware Graph Calibration Network for Temporal Sentence Grounding

Temporal sentence grounding is a challenging task that aims to localize the semantic corresponding segment from the untrimmed video according to the given query language description. Existing methods either utilize a cross-modal matching architecture following a scan-and-rank pipeline or directly predict the probabilities of being the target boundary for each frame based on the entire video content. However, such methods are weak when some of the critical semantic concepts in the query are actually relevant to multiple video segments or the desired video segment contains a query-irrelevant scene due to ignoring query semantic concepts and local and global cross-modal context. In this paper, we propose a novel semantic-aware graph calibration network (SaGCN) to address the issues mentioned above. Specifically, we first introduce a semantic-aware local relational graph module to capture the inherent relationships among the specific semantic concept relevant local contextual information for fine-grained cross-modal information interactions. Then, a semantic-aware global relational graph module is derived for global contextual information integration and achieving cross-modal alignment. Finally, an attention-based calibration module is designed for eliminating the irrelevant information maintained in the visual modality under the guidance of query description. Extensive experiments verify the effectiveness of our proposed SaGCN on two widely used datasets (Charades-STA and TACoS), in which we achieve significant and consistent improvement compared to the state-of-the-art approaches.
Segmentation-Based Formulation of Slot Filling Task for Better Generative Modeling

Slot filling is a fundamental task in spoken language understanding that is usually formulated as a sequence labeling problem and solved using discriminative models such as conditional random fields and recurrent neural networks. One of the weak points of this discriminative approach is robustness against incomplete annotations. For obtaining a more robust method, this paper leverages an overlooked property of slot filling tasks: Non-slot parts of utterance follow a specific pattern depending on the user’s intent. To this end, we propose a generative model that estimates the underlying pattern of utterances based on a segmentation-based formulation of slot-filling tasks. The proposed method adopts nonparametric Bayesian models that enjoy the flexibility of the phrase distribution modeling brought by the new formulation. The experimental result demonstrates that the proposed method performs better in a situation that the training data with incomplete annotations in comparison to the BiLSTM-CRF and HMM.
Robust Cross-Modal Retrieval by Adversarial Training

Cross-modal retrieval is usually implemented based on cross-modal representation learning, which is used to extract semantic information from cross-modal data. Recent work shows that cross-modal representation learning is vulnerable to adversarial attacks, even using large-scale pre-trained networks. By attacking the representation, it can be simple to attack the downstream tasks, especially for cross-modal retrieval tasks. Adversarial attacks on any modality will easily lead to obvious retrieval errors, which brings the challenge to improve the adversarial robustness of cross-modal retrieval. In this paper, we propose a robust cross-modal retrieval method (RoCMR), which generates adversarial examples for both the query modality and candidate modality and performs adversarial training for cross-modal retrieval. Specifically, we generate adversarial examples for both image and text modalities and train the model with benign and adversarial examples in the framework of contrastive learning. We evaluate the proposed RoCMR on two datasets and show its effectiveness in defending against gradient-based attacks.
Semi-Automatic Infrared Calibration for Augmented Reality Systems in Surgery

Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Several studies attempt this calibration through manual alignment or with additional fiducial markers in the surgical scene. We propose a calibration system that offers a direct method for the calibration of AR head-mounted displays (HMDs) with CAOS systems, by using infrared-reflective marker-arrays widely used in CAOS. In our fast, user-agnostic setup, a HoloLens 2 detected the pose of marker arrays using infrared response and time-of-flight depth obtained through sensors onboard the HMD. Registration with a commercially available CAOS system was achieved when an IR marker-array was visible to both devices. Study tests found relative-tracking mean errors of 2.03 mm and 1.12° when calculating the relative pose between two static marker-arrays at short ranges. When using the calibration result to provide in-situ holographic guidance for a simulated wire- insertion task, a pre-clinical test reported mean errors of 2.07 mm and 1.54° when compared to a pre-planned trajectory.
Secure and Efficient Agreement Signing Atop Blockchain and Decentralized Identity

In recent years, we have witnessed an exorbitant growth of online interactions, especially during the pandemic of COVID-19, which requires diversified digital agreements in different application scenarios. In essence, multi-party agreement signing (MPAS) can be reckoned as a special case of the multi-party fair exchange (MPFE) for signatures over a digital agreement. However, such general MPAS protocols have not seen wide adoptions in practice, possibly due to the lack of incentives to maintain the signing platform. Practically, monetary-incentivized enterprises exist to act as a trusted third party (TTP) and provide signing services. However, such an agreement signing flow still suffers from certain limitations in terms of insecure and inefficient operations. To this end, we propose a secure and efficient framework for multi-party agreement signing based on decentralized identity, blockchain and decentralized storage network (DSN). The framework consists of two subsystems where the identity subsystem contains an extensible three-tier user identity model atop decentralized identity, and the agreement signing subsystem, empowered by the identity subsystem, DSN and several novel designs, achieves security and efficiency design goals. For the convenience of explanation, we suppose that a centralized signing service provider (SSP) is properly involved, i.e., sensitive information is still protected against the SSP, and acts as a TTP to ensure the crucial properties such as fairness and coordinate the signing activities. Our design is also compatible with other methods, e.g., ensuring fairness via blockchain, which removes the single point of failure of SSP. We prototype the framework and the performance evaluation shows effectiveness and efficiency.
A Deep Learning-Based Interactive Medical Image Segmentation Framework

Image segmentation is an essential component in medical image analysis. The case of 3D images such as MRI is particularly challenging and time consuming. Interactive or semi-automatic methods are thus highly desirable. While deep learning outperforms classical methods in automatic segmentation, its use in interactive frameworks is still limited. The main reason is that most neural networks do not lend themselves well to the required user interaction loop. We propose a general deep learning-based interactive framework for image segmentation, which embeds a base network in a user interaction loop with a user feedback memory. We propose to model the memory explicitly as a sequence of consecutive framework states, from which the features can be learned. A major difficulty is related to training, as the network inputs include the user feedback and thus depend on the network’s previous output. We propose to introduce a virtual user in the training process, modelled by simulating the user feedback from the current segmentation. We demonstrate our framework on the task of female pelvis MRI segmentation, using a new dataset. We evaluate our framework against existing work with the standard metrics and conduct a user evaluation. Our framework outperforms existing systems.
Distance Based Security using Quantum Entanglement:a survey

Quantum computing is a swiftly blooming technology that straps up the process of quantum mechanics to solve problems too complex for conventional computers. Quantum Cryptography applies algorithms to encrypt messages that it is never read by anyone outside of the unauthorized recipient. Using Quantum mechanics, for secure communication, we have to follow either a superposition or entanglement algorithm. When compared to superposition, entanglement algorithms are providing more security. Why because it is difficult for intruders to identify how the qubits maintain the relationship. In the existing system, Quantum Key Distribution for short distances has already implemented its even commercially available using entanglement algorithm (Artur Ekert E91 Protocol). In the proposed system, quantum communication over very long distances. In this paper, using Quantum entanglement; the keys are exchanged securely and identify eavesdropping in the communication channel.
Gaze-Enhanced User Interface for Real-Time Video Surveillance

Image analysis is a demanding task, particularly, if real-time analysis of motion imagery is required. Hence, the human image analysts needs assistance by a customized software system. In this contribution, a user interface for real-time video surveillance is proposed. The objective is to provide a concept that enables a single user to perform sensor operation and basic observation and detection tasks at the same time. In order to provide efficient and comfortable support, our approach enhances the typical joystick input with gaze input and automated image exploitation algorithms; an alternative concept uses a gamepad instead of the joystick. First qualitative assessment by expert video analysts identified different advantages of the different concepts.
To Adapt or to Fine-Tune: A Case Study on Abstractive Summarization

Recent advances in the field of abstractive summarization leverage pre-trained language models rather than train a model from scratch. However, such models are sluggish to train and accompanied by a massive overhead. Researchers have proposed a few lightweight alternatives such as smaller adapters to mitigate the drawbacks. Nonetheless, it remains uncertain whether using adapters benefits the task of summarization, in terms of improved efficiency without an unpleasant sacrifice in performance. In this work, we carry out multifaceted investigations on fine-tuning and adapters for summarization tasks with varying complexity: language, domain, and task transfer. In our experiments, fine-tuning a pre-trained language model generally attains a better performance than using adapters; the performance gap positively correlates with the amount of training data used. Notably, adapters exceed fine-tuning under extremely low-resource conditions. We further provide insights on multilinguality, model convergence, and robustness, hoping to shed light on the pragmatic choice of fine-tuning or adapters in abstractive summarization.