Reviewerly/exHarmony huggingface dataset:
Learning Shadow Removal From Unpaired Samples via Reciprocal Learning

We focus on addressing the problem of shadow removal for an image, and attempt to make a weakly supervised learning model that does not depend on the pixelwise-paired training samples, but only uses the samples with image-level labels that indicate whether an image contains shadow or not. To this end, we propose a deep reciprocal learning model that interactively optimizes the shadow remover and the shadow detector to improve the overall capability of the model. On the one hand, shadow removal is modeled as an optimization problem with a latent variable of the detected shadow mask. On the other hand, a shadow detector can be trained using the prior from the shadow remover. A self-paced learning strategy is employed to avoid fitting to intermediate noisy annotation during the interactive optimization. Furthermore, a color-maintenance loss and a shadow-attention discriminator are both designed to facilitate model optimization. Extensive experiments on the pairwise ISTD dataset, SRD dataset, and unpaired USR dataset demonstrate the superiority of the proposed deep reciprocal model.
A Multisubobject Approach to Dynamic Formation Target Tracking Using Random Matrices

Bird flocks are typical group targets with various linear formations and high dynamics due to swarm intelligence. This leads to several problems in traditional multisubobject group target tracking such as shape model mismatch and false correlations. This article proposes a multisubobject approach to dynamic formation target tracking. The algebraic graph theory is introduced to analyze the structure of formation targets, then measurements are clustered and combined into subobjects. Based on the existing random matrix approach, two additional filtering branches, including collective filtering and internal structure filtering, are introduced to achieve the robust tracking performance of formation targets. The Kullback–Leibler divergence between the prediction and updated densities of the collective filtering is used to determine the change of formation shape. The correct association between the measurements and the subobjects is realized by the guidance of the internal structure filter. Finally, the effectiveness of the proposed method is verified by the simulation and experimental results.
Cryptography Using Modern Ciphers (Xor and Transposition)

In the present paper the author has suggested a symmetric key cryptographic system where two modern ciphers, namely XOR and Transposition Ciphers have been used. The cipher text thus obtained is hard to crack because a teraflop super computer may need upto 3400 Years to find all possible permutations even for a plain text consisting of 10 characters.
Harmony: Heterogeneous Multi-Modal Federated Learning through Disentangled Model Training

Multi-modal sensing systems are increasingly prevalent in real-world applications such as health monitoring and autonomous driving. Most multi-modal learning approaches need to access users' raw data, which poses significant concerns to users' privacy. Federated learning (FL) provides a privacy-aware distributed learning framework. However, current FL approaches have not addressed the unique challenges of heterogeneous multi-modal FL systems, such as modality heterogeneity and significantly longer training delay. In this paper, we propose Harmony, a new system for heterogeneous multi-modal federated learning. Harmony disentangles the multi-modal network training in a novel two-stage framework, namely modality-wise federated learning and federated fusion learning. By integrating a novel balance-aware resource allocation mechanism in modality-wise FL and exploiting modality biases in federated fusion learning, Harmony improves the model accuracy under non-i.i.d. data distributions and speeds up system convergence. We implemented Harmony on a real-world multi-modal sensor testbed deployed in the homes of 16 elderly subjects for Alzheimer's Disease monitoring. Our evaluation on the testbed and three large-scale public datasets of different applications show that, Harmony outperforms by up to 46.35% accuracy over state-of-the-art baselines and saves up to 30% training delay.
NN-Stretch: Automatic Neural Network Branching for Parallel Inference on Heterogeneous Multi-Processors

Mobile devices are increasingly equipped with heterogeneous multiprocessors, e.g., CPU + GPU + DSP. Yet existing Neural Network (NN) inference fails to fully utilize the computing power of the heterogeneous multi-processors due to the sequential structures of NN models. Towards this end, this paper proposes NN-Stretch, a new model adaption strategy, as well as the supporting system. It automatically branches a given model according to the processor architecture characteristics. Compared to other popular model adaption techniques such as model pruning that often sacrifices accuracy, NN-Stretch accelerates inference while preserving accuracy.
EMSAssist: An End-to-End Mobile Voice Assistant at the Edge for Emergency Medical Services

Accurate and prompt delivery of Emergency Medical Services (EMS) is critical in emergency incidents, e.g., man-made or natural disaster areas. However, quickly selecting the correct EMS protocol(s) (which dictate the medical procedures to be administered to patients) in complex medical scenarios, remains a key, demanding task for Emergency Medical Technicians (EMT). In this paper, we present EMSAssist, the first end-to-end mobile voice assistant at the edge for EMS. EMSAssist consists of three major components that address technical challenges present in state-of-the-art solutions: 1) For the first time, EMSAssist proposes and applies a few-sample fine-tuning technique in medical speech recognition task, that achieves a faster and more accurate speech transcription on our EMS audio dataset, when compared to Google Cloud Speech-to-Text; 2) A WordPiece tokenizer helps boosting the end-to-end EMS protocol selection accuracy by retrieving useful information from incorrect transcriptions; 3) A novel data customization framework that enables our data-driven EMSMobileBERT model to become the new state-of-the-art for EMS protocol selection. Extensive end-to-end evaluation results at the edge show EMSAssist can more accurately select EMS protocols (Top-5 accuracy above 96%) for EMTs, with end-to-end latencies of around 4.2 seconds.
Radon transform of image monotonic rearrangements as feature for noise sensor signature

Source camera identification represents a delicate, crucial but challenging task in digital forensics, especially when an image has to be used as a proof in a court of law. This paper investigates some properties of the Photo Response Non Uniformity (PRNU) pattern noise that represents the fingerprint of any acquisition sensor. The main goal is to define specific and distinctive features for this noise source that enable the identification of the acquisition sensor by simply analysing a few images. These features are required to be independent of image size, modifications, storage mode, etc. The discrimination power of the decreasing rearrangement of a function, combined with the Radon transform, has been investigated in this paper. Preliminary tests show that a proper rearrangement of PRNU image provides specific and device-dependent geometric structures that can be properly coded through the Radon transform. In particular, the empirical distribution of the Radon Transform of rearranged Flat Field images alone is capable to correctly characterize each device with high accuracy, showing robusteness to some standard image modifications, such as quantization and blurring; in addition, it guarantees independence of image size.
A secure emotion aware intelligent system for Internet of healthcare

Emotional aware intelligent healthcare mechanism is seen as one of the emergent techniques that can be used to overcome the traditional communication issues like storage overhead and delay in transmission process in healthcare systems. It is a smart technique primarily used to ensure faster connectivity and transmission as well as healthcare data processing through various heterogeneous networks by understanding human emotions. Researchers have focused on various processing and transmitting techniques of emotional aware intelligent systems. However, very few of them have focused on the need for security while transmitting or reading the patients’ record. The aim of this paper is to propose a secure and transparent communication mechanism in intelligent emotional aware systems using Analytical Hierarchical Process (AHP), blockchain system and a mathematical model. AHP and the mathematical model are used to ensure accuracy during the data transmission process by analysing the legitimacy of each device. In addition, the identity based trust and blockchain mechanism is used to ensure the continuous analysis and transparency among network entities while transferring the information. The proposed framework is further analysed against existing models over various security matrices such as data alteration, report generation, accuracy etc. The out performance of proposed mechanism is approximate 89% as compare to existing mechanism.
How virtual reality is used in industrial maritime design processes: Two case studies

Virtual reality (VR) is increasingly integrated into maritime design processes, and there is a need to better understand how it can support ship designers in their work. Through mapping and interviews, we investigated how VR was used in two design projects in the maritime industry. We explored two questions: What type of design activities and conversations were facilitated by VR as a collaborative medium? When VR was not selected, what other media were used instead, and for what reasons? We found that despite access to VR, most design conversations were facilitated by emails containing screenshots, visual interpretations, and text annotations. VR was used on a few occasions, together with executable files and flythrough videos made with real-time rendered models generated in a game engine, and a web-based 3D model-sharing tool. The use of VR as a collaboration medium was enabled by a variety of design activities ranging from field studies to 2D sketching and 3D modeling. The cases show how VR needed to be integrated with other forms of communication and was seen as a supplement and not the main communication channel for design. The cases also show that the pipeline required for VR scenes creation enabled other forms of collaboration across different supports, such as web-based online collaboration tools.
VMSE: Verifiable multi-keyword searchable encryption in multi-user setting supporting keywords updating

Since the powerful storage capacity of the cloud platform, more and more people are willing to store their files on the cloud service provider (CSP). In order to prevent the privacy of outsourced files from being leaked, data owners will encrypt files before uploading. However, encryption will significantly limit the accurate location of the files. Therefore, this paper mainly introduces a verifiable multi-keyword searchable encryption scheme supporting keywords updating (VMSE). First of all, our proposed VMSE scheme can achieve multiple keywords search while protecting the privacy of the outsourced files and users. Then, our VMSE scheme can also implement the integrity verification and decryption of search results, which are not considered in many existing multi-keyword searchable encryption schemes. Currently, the dynamics in search schemes mostly refer to the dynamic update of outsourced files or user’s search permissions. We consider the keyword dynamic update of outsourced files in this paper, which is a design challenge for searchable encryption schemes and achieved by few searchable encryption schemes. Finally, performance evaluation shows our scheme is efficient and feasible in practical application.
Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Spatial Attention Aberration Repressed Correlation Filters

In recent years, correlation filtering has been widely used in the field of UAV target tracking for its high efficiency and good robustness, even on a common CPU. However, the existing correlation filter-based tracking methods still have major problems when dealing with challenges such as fast moving targets, camera shake, and partial occlusion in UAV scenarios. Furthermore, the lack of reasonable attention mechanism for distortion information as well as background information prevents the limited computational resources from being used for the part of the object most severely affected by interference. In this paper, we propose the spatial attention aberration repressed correlation filter, which models the aberrations, makes full use of the spatial information of aberrations and assigns different attentions to them, and can better cope with these challenges. In addition, we propose a mechanism for the intermittent learning of the global context to balance the efficient use of limited computational resources and cope with various complex scenarios. We also tested the mechanism on challenging UAV benchmarks such as UAVDT and Visdrone2018, and the experiments show that SAARCF has better performance than state-of-the-art trackers.
An Unsupervised Image Denoising Method Using a Nonconvex Low-Rank Model with TV Regularization

In real-world scenarios, images may be affected by additional noise during compression and transmission, which interferes with postprocessing such as image segmentation and feature extraction. Image noise can also be induced by environmental variables and imperfections in the imaging equipment. Robust principal component analysis (RPCA), one of the traditional approaches for denoising images, suffers from a failure to efficiently use the background’s low-rank prior information, which lowers its effectiveness under complex noise backgrounds. In this paper, we propose a robust PCA method based on a nonconvex low-rank approximation and total variational regularization (TV) to model the image denoising problem in order to improve the denoising performance. Firstly, we use a nonconvex γ-norm to address the issue that the traditional nuclear norm penalizes large singular values excessively. The rank approximation is more accurate than the nuclear norm thanks to the elimination of matrix elements with substantial approximation errors to reduce the sparsity error. The method’s robustness is improved by utilizing the low sensitivity of the γ-norm to outliers. Secondly, we use the l1-norm to increase the sparsity of the foreground noise. The TV norm is used to improve the smoothness of the graph structure in accordance with the sparsity of the image in the gradient domain. The denoising effectiveness of the model is increased by employing the alternating direction multiplier strategy to locate the global optimal solution. It is important to note that our method does not require any labeled images, and its unsupervised denoising principle enables the generalization of the method to different scenarios for application. Our method can perform denoising experiments on images with different types of noise. Extensive experiments show that our method can fully preserve the edge structure information of the image, preserve important features of the image, and maintain excellent visual effects in terms of brightness smoothing.
JAMsFace: joint adaptive margins loss for deep face recognition

Abstract Deep feature learning has become crucial in large-scale face recognition, and margin-based loss functions have demonstrated impressive success in this field. These methods aim to enhance the discriminative power of the softmax loss by increasing the feature margin between different classes. These methods assume class balance, where a fixed margin is sufficient to squeeze intra-class variation equally. However, real-face datasets often exhibit imbalanced classes, where the fixed margin is suboptimal, limiting the discriminative power and generalizability of the face recognition model. Furthermore, margin-based approaches typically focus on enhancing discrimination either in the angle or cosine space, emphasizing one boundary while disregarding the other. To overcome these limitations, we propose a joint adaptive margins loss function (JAMsFace) that learns class-related margins for both angular and cosine spaces. This approach allows adaptive margin penalties to adjust adaptively for different classes. We explain and analyze the proposed JAMsFace geometrically and present comprehensive experiments on multiple face recognition benchmarks. The results show that JAMsFace outperforms existing face recognition losses in mainstream face recognition tasks. Specifically, JAMsFace advances the state-of-the-art face recognition performance on LFW, CPLFW, and CFP-FP and achieves comparable results on CALFW and AgeDB-30. Furthermore, for the challenging IJB-B and IJB-C benchmarks, JAMsFace achieves impressive true acceptance rates (TARs) of 89.09% and 91.81% at a false acceptance rate (FAR) of 1e-4, respectively.
Automation of knowledge extraction for degradation analysis

Degradation analysis relies heavily on capturing degradation data manually and its interpretation using knowledge to deduce an assessment of the health of a component. Health monitoring requires automation of knowledge extraction to improve the analysis, quality and effectiveness over manual degradation analysis. This paper proposes a novel approach to achieve automation by combining natural language processing methods, ontology and a knowledge graph to represent the extracted degradation causality and a rule based decision-making system to enable a continuous learning process. The effectiveness of this approach is demonstrated by using an aero-engine component as a use-case.
Self-training on graph neural networks for recommendation with implicit feedback

Graph Convolutional Networks (GCNs) gain success for recommendation, but still face great challenges of data sparseness and negative sampling in implicit feedback-based recommendation. In particular, they ignore the unique graph structure for information propagation and thus fail to fully explore the potentials of GCN. To tackle the problem, we propose a GCN-based self-training approach, called STL, which exploits the learning results in the training procedure, and the potential relations in the embedding space of GCN. First, to handle the data sparsity, we modify the interaction graph structure by adding edges linking a selected part of users and their potential positive items. Second, we adaptively expand the set of positive samples that are used in the pairwise loss function, which not only supplements the dataset but also avoids sampling noises. Further, similarity of structural neighbors on graph is used to mine hard negative sample for improving the sample quality. Experiments on three representative GCN-based recommenders and four widely used public datasets show that STL alleviates the problem of data sparsity, thereby improving recommendation performance compared to normal training.
TDRLM: Stylometric learning for authorship verification by Topic-Debiasing

Authorship verification is the task of determining if a candidate is the real author of the provided sequences of texts. Stylometric learning is the study of the writing style by analyzing the textual data through computational approaches for many linguistic tasks such as authorship verification, authorship identification, etc. In recent years, Deep learning has been used as an effective solution to stylometric learning for authorship verification. However, most of the existing studies do not consider the semantic biases captured by the model, and the learned stylometric features include a strong indicator of the topical biases. During the extraction of stylometric features, we found that specific words related to the latent topics impact the performance of machine learning algorithms and, therefore, are not generalizable toward out-of-sample authors. In this paper, we propose a latent topic score dictionary with a Topic-Debiasing Representation Learning Model (TDRLM) for stylometric representation learning on the problem of authorship verification. The model applies position-specific topic scores on a topic-debiasing attention mechanism in order to adjust the tokenized texts based on the topical bias. The experimental results show that our proposed approach achieves the best performance with the highest Area Under Curve (AUC) of 92.47% for the Twitter-Foursquare dataset and 93.11% for the (ICWSM) Twitter dataset, both better than the state-of-the-art stylometric learning models and the up-trained language models.
Training on LSA lifeboat operation using Mixed Reality

This work aims to provide an overview of the Mixed Reality (MR) technology's use in maritime industry for training purposes. Current training procedures cover a broad range of procedural operations for Life-Saving Appliances (LSA) lifeboats; however, several gaps and limitations have been identified related to the practical training that can be addressed through the use of MR. Augmented, Virtual and Mixed Reality applications are already used in various fields in maritime industry, but its full potential has not been yet exploited. SafePASS project aims to exploit MR advantages in the maritime training by introducing a relevant application focusing on use and maintenance of LSA lifeboats. An MR Training application is proposed supporting the training of crew members in equipment usage and operation, as well as in maintenance activities and procedures. The application consists of the training tool that trains crew members on handling lifeboats, the training evaluation tool that allows trainers to assess the performance of trainees and the maintenance tool that supports crew members to perform maintenance activities and procedures on lifeboats. For each tool, an indicative session and scenario workflow are implemented, along with the main supported interactions of the trainee with the equipment. The application has been tested and validated both in lab environment and using a real LSA lifeboat, resulting to improved experience for the users that provided feedback and recommendations for further development. The application has also been demonstrated onboard a cruise ship, showcasing the supported functionalities to relevant stakeholders that recognized the added value of the application and suggested potential future exploitation areas. The MR Training application has been evaluated as very promising in providing a user-friendly training environment that can support crew members LSA lifeboat operation and maintenance, while it is still subject to improvement and further expansion.
Remote Assistance in Cervical Cancer Screening using Microsoft Hololens 2: An Augmented-Reality Based Approach

India accounts for one of the highest populations of women with cervical cancer. The chances of mortality can be highly reduced if cancer is detected in the initial phases. Early screening via inspecting the biopsy samples is an effective way to detect cancer at its early stages. But this procedure requires a well-trained doctor to identify suspected regions for sample collection. The lack of specialized hospitals and adequately trained healthcare workers in rural regions of the country might lead to undetected cervical cancer. In this paper, we propose a system to remotely assist a healthcare worker in collecting a biopsy sample from the cervix by a trained practitioner using the Microsoft Hololens 2 headset. For this study, the proposed augmented reality system projects a 3D model of the cervix into the real environment, which can be remotely annotated. We plan to use this system for onsite data collection in later stages. Although the proposed solution has been tested for simulation environments with good results, we can foresee its huge potential for remote cervical cancer screening.