Reviewerly/exHarmony huggingface dataset:
Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation

Panoramic images with their 360° directional view encompass exhaustive information about the surrounding space, providing a rich foundation for scene understanding. To unfold this potential in the form of robust panoramic segmentation models, large quantities of expensive, pixel-wise annotations are crucial for success. Such annotations are available, but predominantly for narrow-angle, pinhole-camera images which, off the shelf, serve as sub-optimal resources for training panoramic models. Distortions and the distinct image-feature distribution in 360° panoramas impede the transfer from the annotation-rich pinhole domain and therefore come with a big dent in performance. To get around this domain difference and bring together semantic annotations from pinhole- and 360° surround-visuals, we propose to learn object deformations and panoramic image distortions in the Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) components which blend into our Transformer for PAnoramic Semantic Segmentation (Trans4PASS) model. Finally, we tie together shared semantics in pinhole- and panoramic feature embeddings by generating multi-scale prototype features and aligning them in our Mutual Prototypical Adaptation (MPA) for unsupervised domain adaptation. On the indoor Stanford2D3D dataset, our Trans4PASS with MPA maintains comparable performance to fully-supervised state-of-the-arts, cutting the need for over 1,400 labeled panoramas. On the outdoor DensePASS dataset, we break state-of-the-art by 14.39% mIoU and set the new bar at 56.38%.
Towards Practical Certifiable Patch Defense with Vision Transformer

Patch attacks, one of the most threatening forms of physical attack in adversarial examples, can lead networks to induce misclassification by modifying pixels arbitrarily in a continuous region. Certifiable patch defense can guarantee robustness that the classifier is not affected by patch attacks. Existing certifiable patch defenses sacrifice the clean accuracy of classifiers and only obtain a low certified accuracy on toy datasets. Furthermore, the clean and certified accuracy of these methods is still significantly lower than the accuracy of normal classification networks, which limits their application in practice. To move towards a practical certifiable patch defense, we introduce Vision Transformer (ViT) into the framework of Derandomized Smoothing (DS). Specifically, we propose a progressive smoothed image modeling task to train Vision Transformer, which can capture the more discriminable local context of an image while preserving the global semantic information. For efficient inference and deployment in the real world, we innovatively reconstruct the global self-attention structure of the original ViT into isolated band unit self-attention. On ImageNet, under 2% area patch attacks our method achieves 41.70% certified accuracy, a nearly 1-fold increase over the previous best method (26.00%). Simultaneously, our method achieves 78.58% clean accuracy, which is quite close to the normal ResNet-101 accuracy. Extensive experiments show that our method obtains state-of-the-art clean and certified accuracy with inferring efficiently on CIFAR-10 and ImageNet.
Preventive Measures to Secure Arc Fault using Active and Passive Protection

An arc fault in switchgear is a catastrophic failure with tremendous ramifications and a considerable danger of damage to anybody who comes into contact with the arc fault. As a result, switchgear makers are always looking for ways to mitigate this danger potential. The methods utilized in industrial practice may be classified into two broad categories: Protection may be both active and passive. Efforts are made via active protection to eliminate the probability of an arc fault occurring. Passive protection reduces the severity of the consequences of an arc fault that has already occurred. Both essential approaches, on the other hand, have their own set of drawbacks. A protection approach that recognizes the likelihood of an arc fault but reacts prior to the development of a high-current, destruction-intensive arc fault would be advantageous for both safety and economic reasons. Preventing the arc faults before they occur is the objective of our work.
Cohort design and natural language processing to reduce bias in electronic health records research

Electronic health record (EHR) datasets are statistically powerful but are subject to ascertainment bias and missingness. Using the Mass General Brigham multi-institutional EHR, we approximated a community-based cohort by sampling patients receiving longitudinal primary care between 2001-2018 (Community Care Cohort Project [C3PO], n = 520,868). We utilized natural language processing (NLP) to recover vital signs from unstructured notes. We assessed the validity of C3PO by deploying established risk models for myocardial infarction/stroke and atrial fibrillation. We then compared C3PO to Convenience Samples including all individuals from the same EHR with complete data, but without a longitudinal primary care requirement. NLP reduced the missingness of vital signs by 31%. NLP-recovered vital signs were highly correlated with values derived from structured fields (Pearson r range 0.95-0.99). Atrial fibrillation and myocardial infarction/stroke incidence were lower and risk models were better calibrated in C3PO as opposed to the Convenience Samples (calibration error range for myocardial infarction/stroke: 0.012-0.030 in C3PO vs. 0.028-0.046 in Convenience Samples; calibration error for atrial fibrillation 0.028 in C3PO vs. 0.036 in Convenience Samples). Sampling patients receiving regular primary care and using NLP to recover missing data may reduce bias and maximize generalizability of EHR research.
Modified Q-learning with distance metric and virtual target on path planning of mobile robot

Path planning is an essential element in mobile robot navigation. One of the popular path planners is Q-learning – a type of reinforcement learning that learns with little or no prior knowledge of the environment. Despite the successful implementation of Q-learning reported in numerous studies, its slow convergence associated with the curse of dimensionality may limit the performance in practice. To solve this problem, an Improved Q-learning (IQL) with three modifications is introduced in this study. First, a distance metric is added to Q-learning to guide the agent moves towards the target. Second, the Q function of Q-learning is modified to overcome dead-ends more effectively. Lastly, the virtual target concept is introduced in Q-learning to bypass dead-ends. Experimental results across twenty types of navigation maps show that the proposed strategies accelerate the learning speed of IQL in comparison with the Q-learning. Besides, performance comparison with seven well-known path planners indicates its efficiency in terms of the path smoothness, time taken, shortest distance and total distance used.
Deep Residual Network for Smartwatch-Based User Identification through Complex Hand Movements

Wearable technology has advanced significantly and is now used in various entertainment and business contexts. Authentication methods could be trustworthy, transparent, and non-intrusive to guarantee that users can engage in online communications without consequences. An authentication system on a security framework starts with a process for identifying the user to ensure that the user is permitted. Establishing and verifying an individual's appearance usually requires a lot of effort. Recent years have seen an increase in the usage of activity-based user identification systems to identify individuals. Despite this, there has not been much research into how complex hand movements can be used to determine the identity of an individual. This research used a one-dimensional residual network with squeeze-and-excitation (SE) configurations called the 1D-ResNet-SE model to investigate hand movements and user identification. According to the findings, the SE modules have enhanced the one-dimensional residual network's identification ability. As a deep learning model, the proposed methodology is capable of effectively identifying features from the input smartwatch sensor and could be utilized as an end-to-end model to clarify the modeling process. The 1D-ResNet-SE identification model is superior to the other models. Hand movement assessment based on deep learning is an effective technique to identify smartwatch users.
FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning

Federated distillation (FD) is a popular novel algorithmic paradigm for Federated learning (FL), which achieves training performance competitive to prior parameter averaging-based methods, while additionally allowing the clients to train different model architectures, by distilling the client predictions on an unlabeled auxiliary set of data into a student model. In this work, we propose FedAUX, an extension to FD, which, under the same set of assumptions, drastically improves the performance by deriving maximum utility from the unlabeled auxiliary data. FedAUX modifies the FD training procedure in two ways: First, unsupervised pre-training on the auxiliary data is performed to find a suitable model initialization for the distributed training. Second, (ε, δ) -differentially private certainty scoring is used to weight the ensemble predictions on the auxiliary data according to the certainty of each client model. Experiments on large-scale convolutional neural networks (CNNs) and transformer models demonstrate that our proposed method achieves remarkable performance improvements over state-of-the-art FL methods, without adding appreciable computation, communication, or privacy cost. For instance, when training ResNet8 on non-independent identically distributed (i.i.d.) subsets of CIFAR10, FedAUX raises the maximum achieved validation accuracy from 30.4% to 78.1%, further closing the gap to centralized training performance. Code is available at https://github.com/fedl-repo/fedaux.
Weakly Supervised Deep Ordinal Cox Model for Survival Prediction From Whole-Slide Pathological Images

Whole-Slide Histopathology Image (WSI) is generally considered the gold standard for cancer diagnosis and prognosis. Given the large inter-operator variation among pathologists, there is an imperative need to develop machine learning models based on WSIs for consistently predicting patient prognosis. The existing WSI-based prediction methods do not utilize the ordinal ranking loss to train the prognosis model, and thus cannot model the strong ordinal information among different patients in an efficient way. Another challenge is that a WSI is of large size (e.g., 100,000-by-100,000 pixels) with heterogeneous patterns but often only annotated with a single WSI-level label, which further complicates the training process. To address these challenges, we consider the ordinal characteristic of the survival process by adding a ranking-based regularization term on the Cox model and propose a weakly supervised deep ordinal Cox model (BDOCOX) for survival prediction from WSIs. Here, we generate amounts of bags from WSIs, and each bag is comprised of the image patches representing the heterogeneous patterns of WSIs, which is assumed to match the WSI-level labels for training the proposed model. The effectiveness of the proposed method is well validated by theoretical analysis as well as the prognosis and patient stratification results on three cancer datasets from The Cancer Genome Atlas (TCGA).
A Novel Direct Trajectory Planning Approach Based on Generative Adversarial Networks and Rapidly-Exploring Random Tree

Trajectory planning is essential for self-driving vehicles and has stringent requirements for accuracy and efficiency. The existing trajectory planning methods have limitations in the feasibility of planned trajectories and computational efficiency. This paper proposes a life-long learning framework to achieve effective and high-accuracy direct trajectory planning (DTP) tasks. Based on generative adversarial networks (GANs), this study develops a lightweight GDTP model to map the initial/final states and the control action sequence. Additionally, by embedding the GDTP into the rapidly-exploring random tree (RRT), a GDTP-RRT algorithm is further designed for long-distance and multi-stage planning tasks. Taking the tractor-trailer as an application case, we test the proposed method in multiple scenarios with varying characteristics. The experimental results show that the method can plan highly feasible trajectories in a short time, compared with the most applied algorithm – the cubic curve RRT* (CCRRT*). It is found that the tracking errors of our method are 29.1% and 44.1% lower than the CCRRT* in terms of position and heading angle. This paper provides an effective and stable vehicle trajectory planning method for complex self-driving tasks.
Stepwise Domain Adaptation (SDA) for Object Detection in Autonomous Vehicles Using an Adaptive CenterNet

In recent years, deep learning technologies for object detection have made great progress and have powered the emergence of state-of-the-art models to address object detection problems. Since the domain shift can make detectors unstable or even crash, the detection of cross-domain becomes very important for the design of object detectors. However, traditional deep learning technologies for object detection always rely on a large amount of reliable ground-truth labelling that is laborious, costly, and time-consuming. Although an advanced approach CycleGAN has been proposed for cross-domain object detection tasks, the ability of CycleGAN to reduce the divergence across domains at the feature level is limited. In this paper, a stepwise domain adaptation (SDA) detection method is proposed to further improve the performance of CycleGAN by minimizing the divergence in cross-domain object detection tasks. Specifically, the domain shift is addressed in two steps. In the first step, to bridge the domain gap, an unpaired image-to-image translator is trained to construct a fake target domain by translating the source images to the similar ones in the target domain. In the second step, to further minimize divergence across domains, an adaptive CenterNet is designed to align distributions at the feature level in an adversarial learning manner. Our proposed method is evaluated in domain shift scenarios based on the driving datasets including Cityscapes, Foggy Cityscapes, SIM10k, and BDD100K. The results show that our method is superior to the state-of-the-art methods and is effective for object detection in domain shift scenarios.
Review of Machine Learning Approach on Credit Card Fraud Detection

Abstract Massive usage of credit cards has caused an escalation of fraud. Usage of credit cards has resulted in the growth of online business advancement and ease of the e-payment system. The use of machine learning (methods) are adapted on a larger scale to detect and prevent fraud. ML algorithms play an essential role in analysing customer data. In this research article, we have conducted a comparative analysis of the literature review considering the ML techniques for credit card fraud detection (CCFD) and data confidentiality. In the end, we have proposed a hybrid solution, using the neural network (ANN) in a federated learning framework. It has been observed as an effective solution for achieving higher accuracy in CCFD while ensuring privacy.
Pretrained domain-specific language model for natural language processing tasks in the AEC domain

As an essential task for the architecture, engineering, and construction (AEC) industry, information retrieval (IR) from unstructured textual data based on natural language processing (NLP) is gaining increasing attention. Although various deep learning (DL) models for IR tasks have been investigated in the AEC domain, it is still unclear how domain corpora and domain-specific pretrained DL models can improve performance in various IR tasks. To this end, this work systematically explores the impacts of domain corpora and various transfer learning techniques on the performance of DL models for IR tasks and proposes a pretrained domain-specific language model for the AEC domain. First, both in-domain and close-domain corpora are developed. Then, two types of pretrained models, including traditional wording embedding models and BERT-based models, are pretrained based on various domain corpora and transfer learning strategies. Finally, several widely used DL models for IR tasks are further trained and tested based on various configurations and pretrained models. The result shows that domain corpora have opposite effects on traditional word embedding models for text classification and named entity recognition tasks but can further improve the performance of BERT-based models in all tasks. Meanwhile, BERT-based models dramatically outperform traditional methods in all IR tasks, with maximum improvements of 5.4% and 10.1% in the F1 score, respectively. This research contributes to the body of knowledge in two ways: 1) demonstrating the advantages of domain corpora and pretrained DL models and 2) opening the first domain-specific dataset and pretrained language model for the AEC domain, to the best of our knowledge. Thus, this work sheds light on the adoption and application of pretrained models in the AEC domain.
I-Am: Implicitly Authenticate Me—Person Authentication on Mobile Devices Through Ear Shape and Arm Gesture

Today, identity verification is required in many common activities, and it is arguably true that most people would like to be authenticated in the easiest and most transparent way, without having to remember a personal identification number. To this regard, this paper presents a multibiometric system based on the observation that the instinctive gesture of responding to a phone call can be used to capture two different biometrics, namely ear and arm gesture, which are complementary due to their, respectively, physical and behavioral nature. We conducted a comprehensive set of experiments aimed at assessing the contribution of each of the two biometrics as well as the advantage in their fusion to the system's overall performance. Experiments also provide objective measurement of both saliency and correlation of data captured by each sensor involved (accelerometer, gyroscope, and camera) according to various features extraction, features matching, and data-fusion techniques. The reports provide evidences about the potential of the proposed system and method for user authentication “in-the-wild,” whilst its eventual usage for person identification is also investigated. All of the experiments have been carried out on a specifically built, publicly available ear-arm database, including multibiometric captures of more than 100 subjects performed during different sessions, that represents an additional contribution of this paper.
Lightweight Ciphers and Their Side-Channel Resilience

Side-channel attacks represent a powerful category of attacks against cryptographic devices. Still, side-channel analysis for lightweight ciphers is much less investigated than for instance for AES. Although intuition may lead to the conclusion that lightweight ciphers are weaker in terms of side-channel resistance, that remains to be confirmed and quantified. In this paper, we consider various side-channel analysis metrics which should provide an insight on the resistance of lightweight ciphers against side-channel attacks. In particular, for the non-profiled scenario we use the theoretical confusion coefficient and empirical optimal distinguisher. Our study considers side-channel attacks on the first, the last, or both rounds simultaneously. Furthermore, we conduct a profiled side-channel analysis using various machine learning attacks to recover 4-bit and 8-bit intermediate states of the cipher. Our results show that the difference between AES and lightweight ciphers is smaller than one would expect, and even find scenarios in which lightweight ciphers may be more resistant. Interestingly, we observe that the studied 4-bit S-boxes have a different side-channel resilience, while the difference in the 8-bit ones is only theoretically present.
Cyber-Enabled Well-Being Oriented Daily Living Support Based on Personal Data Analysis

We are living in a cyber-physical-social environment with a variety of lifestyles and values.Living support has become important in such a diverse society.Owing to the ability to collect a large amount of personal data or life logs in the cyber-physical-social environment, it is now possible for us to provide living support based on personal data analysis.Moreover, analyzing such data can facilitate a deep understanding of an individual.In this study, we focus on the provision of cyber-enabled well-being oriented daily living support for an individual based on personal data analysis.Three categories of personal data are identified from an individual's daily life data.In this paper, we discuss the basic concept, model, and framework for well-being oriented personal data analysis in order to offer suggestions and advices to improve the living quality of an individual.Finally, we report a feasibility study with an application scenario by using personal and environmental data.
Co-Clustering to Reveal Salient Facial Features for Expression Recognition

Facial expressions are a strong visual intimation of gestural behaviors. The intelligent ability to learn these non-verbal cues of the humans is the key characteristic to develop efficient human computer interaction systems. Extracting an effective representation from facial expression images is a crucial step that impacts the recognition accuracy. In this paper, we propose a novel feature selection strategy using singular value decomposition (SVD) based co-clustering to search for the most salient regions in terms of facial features that possess a high discriminating ability among all expressions. To the best of our knowledge, this is the first known attempt to explicitly perform co-clustering in the facial expression recognition domain. In our method, Gabor filters are used to extract local features from an image and then discriminant features are selected based on the class membership in co-clusters. Experiments demonstrate that co-clustering localizes the salient regions of the face image. Not only does the procedure reduce the dimensionality but also improves the recognition accuracy. Experiments on CK plus, JAFFE and MMI databases validate the existence and effectiveness of these learned facial features.
The Approximate Duality Gap Technique: A Unified Theory of First-Order Methods

We present a general technique for the analysis of first-order methods. The technique relies on the construction of a duality gap for an appropriate approximation of the objective function, where the function approximation improves as the algorithm converges. We show that in continuous time enforcement of an invariant that this approximate duality gap decreases at a certain rate exactly recovers a wide range of first-order continuous-time methods. We characterize the discretization errors incurred by different discretization methods, and show how iteration-complexity-optimal methods for various classes of problems cancel out the discretization error. The techniques are illustrated on various classes of problems -- including convex minimization for Lipschitz-continuous objectives, smooth convex minimization, composite minimization, smooth and strongly convex minimization, solving variational inequalities with monotone operators, and convex-concave saddle-point optimization -- and naturally extend to other settings.