Reviewerly/exHarmony huggingface dataset:
Prototype Knowledge Distillation for Medical Segmentation with Missing Modality

Multi-modality medical imaging is crucial in clinical treatment as it can provide complementary information for medical image segmentation. However, collecting multi-modal data in clinical is difficult due to the limitation of the scan time and other clinical situations. As such, it is clinically meaningful to develop an image segmentation paradigm to handle this missing modality problem. In this paper, we propose a prototype knowledge distillation (ProtoKD) method to tackle the challenging problem, especially for the toughest scenario when only single modal data can be accessed. Specifically, our ProtoKD can not only distillate the pixel-wise knowledge of multi-modality data to single-modality data but also transfer intra-class and inter-class feature variations, such that the student model could learn more robust feature representation from the teacher model and inference with only one single modality data. Our method achieves state-of-the-art performance on BraTS benchmark.
Adaptable End-to-End ASR Models Using Replaceable Internal LMs and Residual Softmax

End-to-end (E2E) automatic speech recognition (ASR) implicitly learns the token sequence distribution of paired audio-transcript training data. However, it still suffers from domain shifts from training to testing, and domain adaptation is still challenging. To alleviate this problem, this paper designs a replaceable internal language model (RILM) method, which makes it feasible to directly replace the internal language model (LM) of E2E ASR models with a target-domain LM in the decoding stage when a domain shift is encountered. Furthermore, this paper proposes a residual softmax (R-softmax) that is designed for CTC-based E2E ASR models to adapt to the target domain without re-training during inference. For E2E ASR models trained on the LibriSpeech corpus, experiments showed that the proposed methods gave a 2.6% absolute WER reduction on the Switchboard data and a 1.0% WER reduction on the AESRC2020 corpus while maintaining intra-domain ASR results.
JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition

We propose JEIT, a joint end-to-end (E2E) model and internal language model (ILM) training method to inject large-scale unpaired text into ILM during E2E training which improves rare-word speech recognition. With JEIT, the E2E model computes an E2E loss on audio-transcript pairs while its ILM estimates a cross-entropy loss on unpaired text. The E2E model is trained to minimize a weighted sum of E2E and ILM losses. During JEIT, ILM absorbs knowledge from unpaired text while the E2E training serves as regularization. Unlike ILM adaptation methods, JEIT does not require a separate adaptation step and avoids the need for Kullback-Leibler divergence regularization of ILM. We also show that modular hybrid autoregressive transducer (MHAT) performs better than HAT in the JEIT framework, and is much more robust than HAT during ILM adaptation. To push the limit of unpaired text injection, we further propose a combined JEIT and JOIST training (CJJT) that benefits from modality matching, encoder text injection and ILM training. Both JEIT and CJJT can foster a more effective LM fusion. With 100B unpaired sentences, JEIT/CJJT improves rare-word recognition accuracy by up to 16.4% over a model trained without unpaired text.
Prefallkd: Pre-Impact Fall Detection Via CNN-ViT Knowledge Distillation

Fall accidents are critical issues in an aging and aged society. Recently, many researchers developed "pre-impact fall detection systems" using deep learning to support wearable-based fall protection systems for preventing severe injuries. However, most works only employed simple neural network models instead of complex models considering the usability in resource-constrained mobile devices and strict latency requirements. In this work, we propose a novel pre-impact fall detection via CNN-ViT knowledge distillation, namely PreFallKD, to strike a balance between detection performance and computational complexity. The proposed PreFallKD transfers the detection knowledge from the pre-trained teacher model (vision transformer) to the student model (lightweight convolutional neural networks). Additionally, we apply data augmentation techniques to tackle issues of data imbalance. We conduct the experiment on the KFall public dataset and compare PreFallKD with other state-of-the-art models. The experiment results show that PreFallKD could boost the student model during the testing phase and achieves reliable F1-score (92.66%) and lead time (551.3 ms).
Parameter-Efficient Transfer Learning of Pre-Trained Transformer Models for Speaker Verification Using Adapters

Recently, the pre-trained Transformer models have received a rising interest in the field of speech processing thanks to their great success in various downstream tasks. However, most fine-tuning approaches update all the parameters of the pre-trained model, which becomes prohibitive as the model size grows and sometimes results in over-fitting on small datasets. In this paper, we conduct a comprehensive analysis of applying parameter-efficient transfer learning (PETL) methods to reduce the required learnable parameters for adapting to speaker verification tasks. Specifically, during the fine-tuning process, the pre-trained models are frozen, and only lightweight modules inserted in each Transformer block are trainable (a method known as adapters). Moreover, to boost the performance in a cross-language low-resource scenario, the Transformer model is further tuned on a large intermediate dataset before directly fine-tuning it on a small dataset. With updating fewer than 4% of parameters, (our proposed) PETL-based methods achieve comparable performances with full fine-tuning methods (Vox1-O: 0.55%, Vox1-E: 0.82%, Vox1-H:1.73%).
Code-Switching Text Generation and Injection in Mandarin-English ASR

Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance. Automatic Speech Recognition (ASR) with End-to-End (E2E) modeling for such speech can be a challenging task due to the lack of data. In this study, we investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T), in Mandarin-English code-switching speech recognition. We first propose a strategy to generate codeswitching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models, i.e., 16% relative Token-based Error Rate (TER) reduction averaged on three evaluation sets, and the approach of tying speech and text latent spaces is superior to that of TTS conversion on the evaluation set which contains more homogeneous data with the training set.
Light Field Compression Via Compact Neural Scene Representation

In this paper, we propose a novel light field compression method based on a low rank-constrained neural scene representation. While most existing methods directly compress the light field views, our method first learns a Multi-Layer Perceptron (MLP)-based Neural Radiance Field (NeRF) from the input views. To be able to efficiently compress the NeRF scene representation, the weights of the MLP are optimized under a low-rank constraint using the Alternating Direction Method of Multipliers (ADMM) optimization method. The weights of NeRF are then decomposed into Tensor Train (TT) components which allow us to distill original NeRF network into a slimmer one. The slim NeRF is then refined using a quantization-aware training procedure. Experimental results show that this low rank-constrained NeRF-based light field compression method can achieve better rate-distortion than reference methods, while keeping the free-viewpoint reconstruction capability.
A Dual-Path Transformer Network for Scene Text Detection

The prosperity of deep learning contributes to the rapid progress of scene text detection. Among all the methods, segmentation-based methods have drawn extensive attention due to their superiority in detecting text instances of arbitrary shapes and extreme aspect ratios. However, the bottom-up methods are limited to the performance of their segmentation models. In this paper, we propose DPTNet (Dual-Path Transformer Network), a simple yet effective network to utilize both global and local information for the scene text detection task. Moreover, we propose a parallel design that integrates the convolutional network with a powerful self-attention mechanism to provide complementary clues. In addition, a bi-directional interaction module across two paths is developed to provide complementary clues along the channel and spatial dimensions. Our DPTNet achieves state-of-the-art results on several standard benchmarks in terms of both detection accuracy and speed.
Rethink Long-Tailed Recognition with Vision Transforms

In the real world, data tends to follow long-tailed distributions w.r.t. class or attribution, motivating the challenging Long-Tailed Recognition (LTR) problem. In this paper, we revisit recent LTR methods with promising Vision Transformers (ViT). We figure out that 1) ViT is hard to train with longtailed data. 2) ViT learns generalized features in an unsupervised manner, like mask generative training, either on longtailed or balanced datasets. Hence, we propose to adopt unsupervised learning to utilize long-tailed data. Furthermore, we propose the Predictive Distribution Calibration (PDC) as a novel metric for LTR, where the model tends to simply classify inputs into common classes. Our PDC can measure the model calibration of predictive preferences quantitatively. On this basis, we find many LTR approaches alleviate it slightly, despite the accuracy improvement. Extensive experiments on benchmark datasets validate that PDC reflects the model’s predictive preference precisely, which is consistent with the visualization.
A Survey of Efficient Light Weight Cryptography Algorithm for Internet of Medical Things

The evolution of Internet of Things (IoT) promotes patient’s monitoring and diagnose is tremendously. A wireless sensors of minute size are implanted into the victim’s body to measure physiological parameters such as heart rate, glucose level etc continuously. The nodes present inside Wireless Sensor Networks (WSN) are monitored through the internet. So, it is much prone to the wearable devices being overwhelmed and accessed by an unauthorized user as a result of its physical nature. The physiological information of an individual is highly emotional. Therefore, security is considered as a vital requirement for healthcare applications notably if a patient struggles from a severe disease. Thus, Light Weight Cryptography (LWC) encryption schemes were introduced as a primitive measure to secure the required data from threats. The essential requirements of LWC algorithm are reduced area, power consumption, high security, and efficient Processing. In this work, AES (Advanced Encryption Standard) algorithm with Decoder -Switch - Encoder (DSE) - Substitution (S) box technique are used followed by modular inverse and affine transformation process for high security and smaller area. Furthermore, pipelining, branch parallel operations and loop unrolling are also utilized for efficient processing.
A Study On Ciphertext Policy Attribute Based Encryption

In many cyber-physical systems, including the Internet of Things, attribute-based encryption and ciphertext policy (CP-ABE) are frequently employed to provide data security and secure cloud computing. Both policy-based ciphertext encryption and policy-based key-based encryption are different types of encryptions. We generate the key in KP-ABE using a policy that takes the qualities into consideration. And to access the offered properties, CP-ABE employs a tree structure. The encryptor can change the policy governing who is permitted to decrypt the encrypted communication in a Ciphertext Policy Attribute-based Encryption system. A characteristic-based policy can be developed. Along with the access policy, the ciphertext in CP-ABE is transmitted. To protect the identity of the encryptor, we offer a solution that does not need sharing the access policy along with the ciphertext. Even if the storage server is unreliable, encrypted data may be kept private using our approaches since they are secure against collusion assaults.
Confidence-Based Event-Centric Online Video Question Answering on a Newly Constructed ATBS Dataset

Deep neural networks facilitate video question answering (VideoQA), but the real-world applications on video streams such as CCTV and live cast place higher demands on the solver. To address the challenges of VideoQA on long videos of unknown length, we define a new set of problems called Online Open-ended Video Question Answering (O <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> VQA). It requires an online state-updating mechanism for the solver to decide if the collected information is sufficient to conclude an answer. We then propose a Confidence-based Event-centric Online Video Question Answering (CEO-VQA) model to solve this problem. Furthermore, a dataset called Answer Target in Background Stream (ATBS) is constructed to evaluate this newly developed online VideoQA application. Compared to the baseline VideoQA method that watches the whole video, the experimental results show that the proposed method achieves a significant performance gain.
Speech Summarization of Long Spoken Document: Improving Memory Efficiency of Speech/Text Encoders

Speech summarization requires processing several minute-long speech sequences to allow exploiting the whole context of a spoken document. A conventional approach is a cascade of automatic speech recognition (ASR) and text summarization (TS). However, the cascade systems are sensitive to ASR errors. Moreover, the cascade system cannot be optimized for input speech and utilize para-linguistic information. Recently, there has been an increased interest in end-to-end (E2E) approaches optimized to output summaries directly from speech. Such systems can thus mitigate the ASR errors of cascade approaches. However, E2E speech summarization requires massive computational resources because it needs to encode long speech sequences. We propose a speech summarization system that enables E2E summarization from 100 seconds, which is the limit of the conventional method, to up to 10 minutes (i.e., the duration of typical instructional videos on YouTube). However, the modeling capability of this model for minute-long speech sequences is weaker than the conventional approach. We thus exploit auxiliary text information from ASR transcriptions to improve the modeling capabilities. The resultant system consists of a dual speech/text encoder decoder-based summarization system. We perform experiments on the How2 dataset showing the proposed system improved METEOR scores by up to 2.7 points by fully exploiting the long spoken documents.
Towards Zero-Shot Code-Switched Speech Recognition

In this work, we seek to build effective code-switched (CS) automatic speech recognition systems (ASR) under the zero-shot set-ting where no transcribed CS speech data is available for training. Previously proposed frameworks which conditionally factorize the bilingual task into its constituent monolingual parts are a promising starting point for leveraging monolingual data efficiently. However, these methods require the monolingual modules to perform language segmentation. That is, each monolingual module has to simultaneously detect CS points and transcribe speech segments of one language while ignoring those of other languages – not a trivial task. We propose to simplify each monolingual module by allowing them to transcribe all speech segments indiscriminately with a monolingual script (i.e. transliteration). This simple modification passes the responsibility of CS point detection to subsequent bilingual modules which determine the final output by considering multiple monolingual transliterations along with external language model information. We apply this transliteration-based approach in an end-to-end differentiable neural network and demonstrate its efficacy for zero-shot CS ASR on Mandarin-English SEAME test sets.
A Contrastive Knowledge Transfer Framework for Model Compression and Transfer Learning

Knowledge Transfer (KT) achieves competitive performance and is widely used for image classification tasks in model compression and transfer learning. Existing KT works transfer the information from a large model ("teacher") to train a small model ("student") by minimizing the difference of their conditionally independent output distributions. However, these works overlook the high-dimension structural knowledge from the intermediate representations of the teacher, which leads to limited effectiveness, and they are motivated by various heuristic intuitions, which makes it difficult to generalize. This paper proposes a novel Contrastive Knowledge Transfer Framework (CKTF), which enables the transfer of sufficient structural knowledge from the teacher to the student by optimizing multiple contrastive objectives across the intermediate representations between them. Also, CKTF provides a generalized agreement to existing KT techniques and increases their performance significantly by deriving them as specific cases of CKTF. The extensive evaluation shows that CKTF consistently outperforms the existing KT works by 0.04% to 11.59% in model compression and by 0.4% to 4.75% in transfer learning on various models and datasets.
A comprehensive survey on non‐cooperative collision avoidance for micro aerial vehicles: Sensing and obstacle detection

Abstract In recent years, unmanned aerial vehicles (UAVs) have been confirmed as a powerful tool for countless applications in nearly every industry, in which collision avoidance plays a vital role for micro aerial vehicles (MAVs) performing autonomous missions. The complexity of collision avoidance systems is directly related to the complexity of the navigation environment, the avoidance of highly uncertain obstacles being a more thoroughly researched critical aspect. The non‐cooperative collision avoidance strategy is necessary when communication between the agents is unavailable, under the case with many existing passive obstacles and non‐cooperative flying objects. In this article, we review the topic of non‐cooperative collision avoidance for MAVs in dynamic environments by covering two main issues: onboard sensing and obstacle description algorithms. A detailed discussion of these two topics is provided to face non‐cooperative collision avoidance technology for MAVs. Finally, the paper summarizes the open challenges and foreseen future directions in this field.
Evaluation of Human Behaviour Detection and Interaction with Information Projection for Drone-Based Night-Time Security

Night security is known for its long hours and heavy tasks. In Japan, a labor shortage of security guards has become an issue in recent years. To solve these problems, an increasing number of robotic security methods are being used. However, several problems exist with existing security robots. For example, wheeled robots traveling on the ground have difficulty in dealing with obstacles such as steps, while most drones are only for monitoring and do not have a function to help people. In this study, an aerial ubiquitous display (AUD) night security drone has been developed to solve the problems of existing security robots. The AUD is equipped with an infrared camera and a projector to detect human behavior at night and present information to people in need. In this paper, an experiment was conducted with the AUD to evaluate whether it can provide adequate nighttime security. In the experiment, real-time monitoring and information projection from the air were achieved. In addition, new security methods using the AUD were shown to be effective. Replacing security guards with the AUD to provide security at night will improve labor shortages in the future, and better security methods will be developed.
Siamese neural networks in recommendation

Abstract Recommender systems are widely adopted as an increasing research and development area, since they provide users with diverse and useful information tailored to their needs. Several strategies have been proposed, and in most of them some concept of similarity is used as a core part of the approach, either between items or between users. At the same time, Siamese Neural Networks are being used to capture the similarity of items in the image domain, as they are defined as a subtype of Artificial Neural Networks built with (at least two) identical networks that share their weights. In this review, we study the proposals done in the intersection of these two fields, that is, how Siamese Networks are being used for recommendation. We propose a classification that considers different recommendation problems and algorithmic approaches. Some research directions are pointed out to encourage future research. To the best of our knowledge, this paper is the first comprehensive survey that focuses on the usage of Siamese Neural Networks for Recommender Systems.
Antecedents and Outcomes of Blockchain Technology Adoption: Meta-Analysis

ABSTRACTABSTRACTThis study examines the organizational adoption of blockchain technology (BCT) by synthesizing empirical findings reported in 37 studies using a meta-analysis of bivariate relationships. An extended technology-organization-environment (TOE) framework is applied to examine the antecedents and outcomes of BCT adoption. Our results show that various technological, organizational, and environmental factors influence BCT adoption, including technology compatibility, perceived ease of use, top management support, and coercive pressure. Overall, technological factors had a stronger impact than organizational and environmental factors concerning BCT adoption. Further, our analysis identifies the outcomes of BCT adoption, including improvements in operational performance, innovation capability, and sustainability. BCT adoption has the strongest impact on operational performance. Our study contributes to the existing literature by offering a comprehensive and synthesized overview of the antecedents and outcomes of BCT adoption.KEYWORDS: Meta-analysisblockchain technologyTOE framework Disclosure statementNo potential conflict of interest was reported by the author(s).
A lightweight improvement of YOLOv5 for insulator fault detection

Abstract Along with the development of artificial intelligence, mobile terminal equipment patrol inspection has become the mainstream of power grid line patrol inspection. Insulator defect detection is an important part of power patrol inspection. To increase the detection speed under the condition of guaranteeing high precision of insulator detecting, an improved lightweight YOLOv5 algorithm is presented to achieve insulator defect detection. This algorithm uses the lightweight Ghost convolution to improve the general convolution and the Ghost Bottleneck module to improve the head module in YOLOv5. Based on the original Ghost lightweight, the algorithm improves the channel data suitable for insulator detection and decreases the number of convolutions. In the same research data and experimental environment setting, the effect is better than the unmodified Ghost optimization. Experiment results indicate that the mean precision of detecting insulator is 81%, the number of algorithm models and parameters is reduced, the speed of detection is increased under the premise of ensuring accuracy, and the improved algorithm model is more lightweight and easy to deploy and use in embedded mobile terminals.