Reviewerly/exHarmony huggingface dataset:
Robust Multi-Modality Person Re-identification

To avoid the illumination limitation in visible person re-identification (Re-ID) and the heterogeneous issue in cross-modality Re-ID, we propose to utilize complementary advantages of multiple modalities including visible (RGB), near infrared (NI) and thermal infrared (TI) ones for robust person Re-ID. A novel progressive fusion network is designed to learn effective multi-modal features from single to multiple modalities and from local to global views. Our method works well in diversely challenging scenarios even in the presence of missing modalities. Moreover, we contribute a comprehensive benchmark dataset, RGBNT201, including 201 identities captured from various challenging conditions, to facilitate the research of RGB-NI-TI multi-modality person Re-ID. Comprehensive experiments on RGBNT201 dataset comparing to the state-of-the-art methods demonstrate the contribution of multi-modality person Re-ID and the effectiveness of the proposed approach, which launch a new benchmark and a new baseline for multi-modality person Re-ID.
Hierarchical Neural Dynamic Policies

We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input.The family of nonlinear dynamical system-based methods have successfully demonstrated dynamic robot behaviors but have difficulty in generalizing to unseen configurations as well as learning from image inputs.Recent works approach this issue by using deep network policies and reparameterize actions to embed the structure of dynamical systems but still struggle in domains with diverse configurations of image goals, and hence, find it difficult to generalize.In this paper, we address this dichotomy by leveraging embedding the structure of dynamical systems in a hierarchical deep policy learning framework, called Hierarchical Neural Dynamical Policies (H-NDPs).Instead of fitting deep dynamical systems to diverse data directly, H-NDPs form a curriculum by learning local dynamical system-based policies on small regions in state-space and then distill them into a global dynamical system-based policy that operates only from high-dimensional images.H-NDPs additionally provide smooth trajectories, a strong safety benefit in the real world.We perform extensive experiments on dynamic tasks both in the real world (digit writing, scooping, and pouring) and simulation (catching, throwing, picking).We show that H-NDPs are easily integrated with both imitation as well as reinforcement learning setups and achieve state-of-the-art results.Video results are at https://shikharbahl.github.io/hierarchical-ndps/.
An Improved Aggregated-Mosaic Method for the Sparse Object Detection of Remote Sensing Imagery

Object detection based on remote sensing imagery has become increasingly popular over the past few years. Unlike natural images taken by humans or surveillance cameras, the scale of remote sensing images is large, which requires the training and inference procedure to be on a cutting image. However, objects appearing in remote sensing imagery are often sparsely distributed and the labels for each class are imbalanced. This results in unstable training and inference. In this paper, we analyze the training characteristics of the remote sensing images and propose the fusion of the aggregated-mosaic training method, with the assigned-stitch augmentation and auto-target-duplication. In particular, based on the ground truth and mosaic image size, the assigned-stitch augmentation enhances each training sample with an appropriate account of objects, facilitating the smooth training procedure. Hard to detect objects, or those in classes with rare samples, are randomly selected and duplicated by the auto-target-duplication, which solves the sample imbalance or classes with insufficient results. Thus, the training process is able to focus on weak classes. We employ VEDAI and NWPU VHR-10, remote sensing datasets with sparse objects, to verify the proposed method. The YOLOv5 adopts the Mosaic as the augmentation method and is one of state-of-the-art detectors, so we choose Mosaic (YOLOv5) as the baseline. Results demonstrate that our method outperforms Mosaic (YOLOv5) by 2.72% and 5.44% on 512 × 512 and 1024 × 1024 resolution imagery, respectively. Moreover, the proposed method outperforms Mosaic (YOLOv5) by 5.48% under the NWPU VHR-10 dataset.
Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction

Relation Extraction (RE) is a vital step to complete Knowledge Graph (KG) by extracting entity relations from texts. However, it usually suffers from the long-tail issue. This paper proposes a novel approach to learn relation prototypes from unlabeled texts, to facilitate long-tail RE by transferring knowledge from relation types with sufficient training data. We learn relation prototypes as an implicit factor between entities, which reflects meanings of relations and their proximities. We construct a co-occurrence graph from texts, and capture both first-order and second-order entity proximities for embedding learning. By optimize the distance from entity pairs to corresponding prototypes, our method can be easily adapted to almost arbitrary RE frameworks. Thus, the learning of infrequent or even unseen relation types will benefit from semantically proximate relations through pairs of entities and large-scale textual information. Extensive experiments on two publicly available datasets present promising improvements (4.1% F1 on average). Ablation studies on long-tail relations, main components, and different RE models demonstrate the effectiveness of the learned relation prototypes. Finally, we analyze several example cases to give intuitive impressions as qualitative analysis. Our codes and data can be found in https://github.com/CrisJk/PA-TRP.
Compatibility-aware Heterogeneous Visual Search

We tackle the problem of visual search under resource constraints. Existing systems use the same embedding model to compute representations (embeddings) for the query and gallery images. Such systems inherently face a hard accuracy-efficiency trade-off: the embedding model needs to be large enough to ensure high accuracy, yet small enough to enable query-embedding computation on resource-constrained platforms. This trade-off could be mitigated if gallery embeddings are generated from a large model and query embeddings are extracted using a compact model. The key to building such a system is to ensure representation compatibility between the query and gallery models. In this paper, we address two forms of compatibility: One enforced by modifying the parameters of each model that computes the embeddings. The other by modifying the architectures that compute the embeddings, leading to compatibility-aware neural architecture search (Cmp-NAS). We test Cmp-NAS on challenging retrieval tasks for fashion images (DeepFashion2), and face images (IJB-C). Compared to ordinary (homogeneous) visual search using the largest embedding model (paragon), Cmp-NAS achieves 80-fold and 23-fold cost reduction while maintaining accuracy within 0.3% and 1.6% of the paragon on DeepFashion2 and IJB-C respectively.
A strategy for validation of variables derived from large-scale electronic health record data

Standardized approaches for rigorous validation of phenotyping from large-scale electronic health record (EHR) data have not been widely reported. We proposed a methodologically rigorous and efficient approach to guide such validation, including strategies for sampling cases and controls, determining sample sizes, estimating algorithm performance, and terminating the validation process, hereafter referred to as the San Diego Approach to Variable Validation (SDAVV). We propose sample size formulae which should be used prior to chart review, based on pre-specified critical lower bounds for positive predictive value (PPV) and negative predictive value (NPV). We also propose a stepwise strategy for iterative algorithm development/validation cycles, updating sample sizes for data abstraction until both PPV and NPV achieve target performance. We applied the SDAVV to a Department of Veterans Affairs study in which we created two phenotyping algorithms, one for distinguishing normal colonoscopy cases from abnormal colonoscopy controls and one for identifying aspirin exposure. Estimated PPV and NPV both reached 0.970 with a 95% confidence lower bound of 0.915, estimated sensitivity was 0.963 and specificity was 0.975 for identifying normal colonoscopy cases. The phenotyping algorithm for identifying aspirin exposure reached a PPV of 0.990 (a 95% lower bound of 0.950), an NPV of 0.980 (a 95% lower bound of 0.930), and sensitivity and specificity were 0.960 and 1.000. A structured approach for prospectively developing and validating phenotyping algorithms from large-scale EHR data can be successfully implemented, and should be considered to improve the quality of “big data” research.
Improving Sequential Recommendation Consistency with Self-Supervised Imitation

Most sequential recommendation models capture the features of consecutive items in a user-item interaction history. Though effective, their representation expressiveness is still hindered by the sparse learning signals. As a result, the sequential recommender is prone to make inconsistent predictions. In this paper, we propose a model, SSI, to improve sequential recommendation consistency with Self-Supervised Imitation. Precisely, we extract the consistency knowledge by utilizing three self-supervised pre-training tasks, where temporal consistency and persona consistency capture user-interaction dynamics in terms of the chronological order and persona sensitivities, respectively. Furthermore, to provide the model with a global perspective, global session consistency is introduced by maximizing the mutual information among global and local interaction sequences. Finally, to comprehensively take advantage of all three independent aspects of consistency-enhanced knowledge, we establish an integrated imitation learning framework. The consistency knowledge is effectively internalized and transferred to the student model by imitating the conventional prediction logit as well as the consistency-enhanced item representations. In addition, the flexible self-supervised imitation framework can also benefit other student recommenders. Experiments on four real-world datasets show that SSI effectively outperforms the state-of-the-art sequential recommendation methods.
Multi-Objective Cooperated Path Planning of Multiple Unmanned Aerial Vehicles Based on Revisit Time

This paper investigates multi-objective optimization of coordinated patrolling flight of multiple unmanned aerial vehicles in the vicinity of terrain, while respecting their performance parameters.A new efficient modified A-star (A*) algorithm with a novel defined criterion known as individual revisit time cell value is introduced and extended to the whole area of the 3D mountainous environment.As a contribution to solving trade-offs in the optimization problem, revisit time is conjugated with other contrary costs effective in flight planning through Pareto analysis.By introducing the revisit time and applying a specific setup to mitigate computational complexity, the proposed algorithm efficiently revisits the desired zones, which are more important to be revisited during the patrolling mission.The results of the introduced modified A* algorithm are compared in various scenarios with two different algorithms including a complete and optimal algorithm known as Dijkstra, and an evolutionary algorithm known as the genetic algorithm.Simulation results demonstrate that the proposed algorithm generates faster and more efficient trajectories in complex multiagent scenarios due to the introduced cell selection method and dynamic-based simplifications applied in this research.
Conditional Self-Supervised Learning for Few-Shot Classification

How to learn a transferable feature representation from limited examples is a key challenge for few-shot classification. Self-supervision as an auxiliary task to the main supervised few-shot task is considered to be a conceivable way to solve the problem since self-supervision can provide additional structural information easily ignored by the main task. However, learning a good representation by traditional self-supervised methods is usually dependent on large training samples. In few-shot scenarios, due to the lack of sufficient samples, these self-supervised methods might learn a biased representation, which more likely leads to the wrong guidance for the main tasks and finally causes the performance degradation. In this paper, we propose conditional self-supervised learning (CSS) to use auxiliary information to guide the representation learning of self-supervised tasks. Specifically, CSS leverages supervised information as prior knowledge to shape and improve the learning feature manifold of self-supervision without auxiliary unlabeled data, so as to reduce representation bias and mine more effective semantic information. Moreover, CSS exploits more meaningful information through supervised and the improved self-supervised learning respectively and integrates the information into a unified distribution, which can further enrich and broaden the original representation. Extensive experiments demonstrate that our proposed method without any fine-tuning can achieve a significant accuracy improvement on the few-shot classification scenarios compared to the state-of-the-art few-shot learning methods.
Axial Attitude Estimation of Spacecraft in Orbit Based on ISAR Image Sequence

Attitude estimation is a significant task for monitoring noncooperative spacecraft in orbit. Inverse synthetic aperture radar (ISAR) is an effective method to obtain the structure, size, and attitude of spacecraft. Most existing methods of attitude estimation based on ISAR images generally rely on characteristic point matching or rectangular feature matching among ISAR images. They often need high quality ISAR images, so the robustness is usually poor. A novel method for estimating spacecraft's axial attitude in ISAR image is proposed in this article. This method does not need to utilize a simulation database, or to match sequence of specific features, so it has good robustness. Our method bridges Range-Doppler images and attitude parameters with the accommodation of spacecraft's orbit parameters and the ISAR geometric projection model. Based on the ISAR imaging turntable model, the accurate position of the imaging plane of the spacecraft in orbit is derived. Then, attitude parameters are estimated by an optimization of the projection of spacecraft's three-dimensional model to the imaging plane. Experimental results demonstrate the effectiveness and robustness of the proposed method to estimate attitude of spacecraft in orbit by the simulated and measured ISAR images.
DeepHistoClass: A Novel Strategy for Confident Classification of Immunohistochemistry Images Using Deep Learning

A multitude of efforts worldwide aim to create a single-cell reference map of the human body, for fundamental understanding of human health, molecular medicine, and targeted treatment. Antibody-based proteomics using immunohistochemistry (IHC) has proven to be an excellent technology for integration with large-scale single-cell transcriptomics datasets. The golden standard for evaluation of IHC staining patterns is manual annotation, which is expensive and may lead to subjective errors. Artificial intelligence holds much promise for efficient and accurate pattern recognition, but confidence in prediction needs to be addressed. Here, the aim was to present a reliable and comprehensive framework for automated annotation of IHC images. We developed a multilabel classification of 7848 complex IHC images of human testis corresponding to 2794 unique proteins, generated as part of the Human Protein Atlas (HPA) project. Manual annotation data for eight different cell types was generated as a basis for training and testing a proposed Hybrid Bayesian Neural Network. By combining the deep learning model with a novel uncertainty metric, DeepHistoClass (DHC) Confidence Score, the average diagnostic performance improved from 86.9% to 96.3%. This metric not only reveals which images are reliably classified by the model, but can also be utilized for identification of manual annotation errors. The proposed streamlined workflow can be developed further for other tissue types in health and disease and has important implications for digital pathology initiatives or large-scale protein mapping efforts such as the HPA project.
Forecasting of Photovoltaic Power by Means of Non-Linear Auto-Regressive Exogenous Artificial Neural Network and Time Series Analysis

In this research paper, a nonlinear autoregressive with exogenous input (NARX) model of the nonlinear system based on neural network and time series analysis is proposed to deal with the one-month forecast of the produced power from photovoltaic modules (PVM). The PVM is a monocrystalline cell with a rated production of 175 watts that is placed at Heliopolis University, Bilbéis city, Egypt. The NARX model is considered powerful enough to emulate the nonlinear dynamic state-space model. It is extensively performed to resolve a variety of problems and is mainly important in complex process control. Moreover, the NARX method is selected because of its quick learning and completion times, as well as high appropriateness, and is distinguished by advantageous dynamics and interference resistance. The neural network (NN) is trained and optimized with three algorithms, the Levenberg–Marquardt Algorithm (NARX-LMA), the Bayesian Regularization Algorithm (NARX-BRA) and the Scaled Conjugate Gradient Algorithm (NARX-SCGA), to attain the best performance. The forecasted results using the NARX method based on the three algorithms are compared with experimentally measured data. The NARX-LMA, NARX-BRA and NARX-SCGA models are validated using statistical criteria. In general, weather conditions have a significant impact on the execution and quality of the results.
Cryptocurrency network factors and gold

Both cryptocurrencies and gold are scarce, expensive for extraction, and less affected by money supply. We focus on these similarities and investigate whether cryptocurrency network affects impact on expected return on gold. Our results show that the number of cryptocurrency wallet users is positively related to the expected return on gold. Moreover, we employed a machine-learning approach and considered the interactions among predictors. We reveal that network factors have a greater impact on gold than returns on Bitcoin and other macroeconomic and financial variables.
Self-Supervised Dialogue Learning for Spoken Conversational Question Answering

In spoken conversational question answering (SCQA), the answer to the corresponding question is generated by retrieving and then analyzing a fixed spoken document, including multi-part conversations. Most SCQA systems have considered only retrieving information from ordered utterances. However, the sequential order of dialogue is important to build a robust spoken conversational question answering system, and the changes of utterances order may severely result in low-quality and incoherent corpora. To this end, we introduce a self-supervised learning approach, including incoherence discrimination, insertion detection, and question prediction, to explicitly capture the coreference resolution and dialogue coherence among spoken documents. Specifically, we design a joint learning framework where the auxiliary self-supervised tasks can enable the pre-trained SCQA systems towards more coherent and meaningful spoken dialogue learning. We also utilize the proposed self-supervised learning tasks to capture intra-sentence coherence. Experimental results demonstrate that our proposed method provides more coherent, meaningful, and appropriate responses, yielding superior performance gains compared to the original pre-trained language models. Our method achieves state-of-the-art results on the Spoken-CoQA dataset.
RETRACTED ARTICLE: Human–Computer Interaction on IoT-Based College Physical Education

College physical education system is an essential component of the national health plan. Promoting the technical and modernized construction of the physical education curriculum in colleges and universities is crucial to enhance higher education's science and performance. In this technological era, the Internet of Things (IoT) is used in physical education to train and record physical activities. In this research, the AI-based IoT system (AI-IoTS) Wearable technology is promoted for IoT-based Human–Computer Interaction for College Physical education. This AI-IoTS consist of a Cloud Platform and three layers of AI. The AI-IoTS recognizes the data required for the students. Collect the data from the cloud using an IoT platform and processes it with the help of AI. The student can train themselves using wearable technology without the help of the Physical instructor. The simulation method of the proposed framework "AI-IoTS" proved that it could collect and teach students independently. The proposed AI-based IoT System (AI-IoTS) Wearable technology for IoT-based Human–Computer Interaction for College Physical education has been validated based on the optimization parameter, which outperforms conventional methods.
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression

Recent studies on compression of pretrained language models (e.g., BERT) usually use preserved accuracy as the metric for evaluation.In this paper, we propose two new metrics, label loyalty and probability loyalty that measure how closely a compressed model (i.e., student) mimics the original model (i.e., teacher).We also explore the effect of compression with regard to robustness under adversarial attacks.We benchmark quantization, pruning, knowledge distillation and progressive module replacing with loyalty and robustness.By combining multiple compression techniques, we provide a practical strategy to achieve better accuracy, loyalty and robustness.
Enriching Source Style Transfer in Recognition-Synthesis Based Non-Parallel Voice Conversion

Current voice conversion (VC) methods can successfully convert timbre of the audio. As modeling source audio's prosody effectively is a challenging task, there are still limitations of transferring source style to the converted speech. This study proposes a source style transfer method based on recognition-synthesis framework. Previously in speech generation task, prosody can be modeled explicitly with prosodic features or implicitly with a latent prosody extractor. In this paper, taking advantages of both, we model the prosody in a hybrid manner, which effectively combines explicit and implicit methods in a proposed prosody module. Specifically, prosodic features are used to explicit model prosody, while VAE and reference encoder are used to implicitly model prosody, which take Mel spectrum and bottleneck feature as input respectively. Furthermore, adversarial training is introduced to remove speaker-related information from the VAE outputs, avoiding leaking source speaker information while transferring style. Finally, we use a modified self-attention based encoder to extract sentential context from bottleneck features, which also implicitly aggregates the prosodic aspects of source speech from the layered representations. Experiments show that our approach is superior to the baseline and a competitive system in terms of style transfer; meanwhile, the speech quality and speaker similarity are well maintained.