Reviewerly/exHarmony huggingface dataset:
Dynamic Instance Domain Adaptation

Most existing studies on unsupervised domain adaptation (UDA) assume that each domain's training samples come with domain labels (e.g., painting, photo). Samples from each domain are assumed to follow the same distribution and the domain labels are exploited to learn domain-invariant features via feature alignment. However, such an assumption often does not hold true-there often exist numerous finer-grained domains (e.g., dozens of modern painting styles have been developed, each differing dramatically from those of the classic styles). Therefore, forcing feature distribution alignment across each artificially-defined and coarse-grained domain can be ineffective. In this paper, we address both single-source and multi-source UDA from a completely different perspective, which is to view each instance as a fine domain. Feature alignment across domains is thus redundant. Instead, we propose to perform dynamic instance domain adaptation (DIDA). Concretely, a dynamic neural network with adaptive convolutional kernels is developed to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance. This enables a shared classifier to be applied to both source and target domain data without relying on any domain annotation. Further, instead of imposing intricate feature alignment losses, we adopt a simple semi-supervised learning paradigm using only a cross-entropy loss for both labeled source and pseudo labeled target data. Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets including Digits, Office-Home, DomainNet, Digit-Five, and PACS.
Autoencoders

In this chapter, we look at autoencoders. This chapter is a theoretical one, covering the mathematics and the fundamental concepts of autoencoders. We discuss what they are, what their limitations are, the typical use cases, and then look at some examples. We start with a general introduction to autoencoders, and we discuss the role of the activation function in the output layer and the loss function. We then discuss what the reconstruction error is. Finally, we look at typical applications, such as dimensionality reduction, classification, denoising, and anomaly detection.
Data Science and Data Analytics: Artificial Intelligence and Machine Learning Integrated Based Approach

Data analytics is used to make the decision by analyzing the raw data. Data analytics is categorized into four categories: descriptive, diagnostic, prescriptive, and predictive analytics. Predictive analysis is the most powerful tool for data analytics and involves advanced statistical, modeling, data mining, artificial intelligence, deep learning to dig into data, and allows analytics to make predictions. Predictive analysis can be classified into two types, namely regression analysis and classification. When the dependent variables in the form of binary, logistic regression is the best method to perform. Regression analysis is an effective mathematical tool in which the association between two or more elements of interest may be analyzed by the researcher. An efficient data classification mechanism is also valuable because it can help the system decide the required degree of control in order to protect the security and credibility of the data. To achieve the statistics for predictive analysis, machine learning acts as the core principle for it. Machine learning is used to help a computer to interpret information, comprehend connections, and use observations to solve problems and/or enrich information. Predictive analytics extensively used machine learning for data modeling due to its ability to accurately process a vast amount of data and recognize patterns.
ReDFeat: Recoupling Detection and Description for Multimodal Feature Learning

Deep-learning-based local feature extraction algorithms that combine detection and description have made significant progress in visible image matching. However, the end-to-end training of such frameworks is notoriously unstable due to the lack of strong supervision of detection and the inappropriate coupling between detection and description. The problem is magnified in cross-modal scenarios, in which most methods heavily rely on the pre-training. In this paper, we recouple independent constraints of detection and description of multimodal feature learning with a mutual weighting strategy, in which the detected probabilities of robust features are forced to peak and repeat, while features with high detection scores are emphasized during optimization. Different from previous works, those weights are detached from back propagation so that the detected probability of indistinct features would not be directly suppressed and the training would be more stable. Moreover, we propose the Super Detector, a detector that possesses a large receptive field and is equipped with learnable non-maximum suppression layers, to fulfill the harsh terms of detection. Finally, we build a benchmark that contains cross visible, infrared, near-infrared and synthetic aperture radar image pairs for evaluating the performance of features in feature matching and image registration tasks. Extensive experiments demonstrate that features trained with the recoulped detection and description, named ReDFeat, surpass previous state-of-the-arts in the benchmark, while the model can be readily trained from scratch. The code is released at https://github.com/ACuOoOoO/ReDFeat.
Positive, Negative and Neutral

News recommendation for anonymous readers is a useful but challenging task for many news portals, where interactions between readers and articles are limited within a temporary login session. Previous works tend to formulate session-based recommendation as a next item prediction task, while they neglect the implicit feedback from user behaviors, which indicates what users really like or dislike. Hence, we propose a comprehensive framework to model user behaviors through positive feedback (i.e., the articles they spend more time on) and negative feedback (i.e., the articles they choose to skip without clicking in). Moreover, the framework implicitly models the user using their session start time, and the article using its initial publishing time, in what we call neutral feedback. Empirical evaluation on three real-world news datasets shows the framework's promising performance of more accurate, diverse and even unexpectedness recommendations than other state-of-the-art session-based recommendation approaches.
Positive-Unlabeled Learning with Adversarial Data Augmentation for Knowledge Graph Completion

Most real-world knowledge graphs (KG) are far from complete and comprehensive. This problem has motivated efforts in predicting the most plausible missing facts to complete a given KG, i.e., knowledge graph completion (KGC). However, existing KGC methods suffer from two main issues, 1) the false negative issue, i.e., the sampled negative training instances may include potential true facts; and 2) the data sparsity issue, i.e., true facts account for only a tiny part of all possible facts. To this end, we propose positive-unlabeled learning with adversarial data augmentation (PUDA) for KGC. In particular, PUDA tailors positive-unlabeled risk estimator for the KGC task to deal with the false negative issue. Furthermore, to address the data sparsity issue, PUDA achieves a data augmentation strategy by unifying adversarial training and positive-unlabeled learning under the positive-unlabeled minimax game. Extensive experimental results on real-world benchmark datasets demonstrate the effectiveness and compatibility of our proposed method.
Neural-network quantum state tomography

We revisit the application of neural networks techniques to quantum state tomography. We confirm that the positivity constraint can be successfully implemented with trained networks that convert outputs from standard feed-forward neural networks to valid descriptions of quantum states. Any standard neural-network architecture can be adapted with our method. Our results open possibilities to use state-of-the-art deep-learning methods for quantum state reconstruction under various types of noise.
SytaB: A Class of Smooth-Transition Hybrid Terrestrial/Aerial Bicopters

This work details the design, modeling and control of SytaB, a vehicle capable of hybrid terrestrial/aerial mobility with smooth transition, where the structure embedding a bicopter is adopted for the first time. In contrast to previous hybrid terrestrial/aerial vehicles with a quadrotor embedded, SytaB not only requires less energy for the same takeoff weight, but also regulates its attitude less frequently to restrain the vibration of sensors. Three modes, the terrestrial/aerial/transitional modes, are considered <styled-content style="color:black" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">for the vehicle</styled-content> , and the dynamics modeling and controller design of each mode are <styled-content style="color:black" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">carried out</styled-content> . The transition between terrestrial and aerial locomotions is smooth compared with the case of non-inclusion <styled-content style="color:black" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">of</styled-content> transitional mode such that the bounce and shake of the vehicle are alleviated. The energy efficiency is compared between the terrestrial and aerial modes, and between the energy-saving and high-maneuverability paradigms (a choice enabled in the terrestrial mode of SytaB). Experimental results are presented to demonstrate the potential of SytaB, the effectiveness of the designed controllers, and the necessity of considering the transition process.
A Nelder Mead-infused INFO algorithm for optimization of mechanical design problems

Abstract Nature-inspired metaheuristic algorithms have wide applications that have greater emphasis over the classical optimization techniques. The INFO algorithm is developed on the basis of the weighted mean of the vectors, which enhances the superior vector position that enables to get the global optimal solution. Moreover, it evaluates the fitness function within the updating stage, vectors combining, and local search stage. Accordingly, in the present article, a population-based algorithm named weighted mean of vectors (INFO) is hybridized with the Nelder–Mead algorithm (HINFO-NM) and adapted to optimize the standard benchmark function structural optimization of the vehicle suspension arm. This provides a superior convergence rate, prevention of trapping in the local search domain, and class balance between the exploration and exploitation phase. The pursued results suggest that the HINFO-NM algorithm is the robust optimizer that provides the best results compared to the rest of the algorithms. Moreover, the scalability of this algorithm can be realized by having the least standard deviation in the results. The HINFO-NM algorithm can be adopted in a wide range of optimization challenges by assuring superior results obtained in the present article.
An Algorithm–Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers

The Transformer has been an indispensable staple in deep learning. However, for real-life applications, it is very challenging to deploy efficient Transformers due to immense parameters and operations of models. To relieve this burden, exploiting sparsity is an effective approach to accelerate Transformers. Newly emerging Ampere GPUs leverage a 2:4 sparsity pattern to achieve model acceleration, while it can hardly meet the diverse algorithm and hardware constraints when deploying models. By contrast, we propose an algorithm-hardware co-optimized framework to flexibly and efficiently accelerate Transformers by utilizing general N:M sparsity patterns. (1) From algorithm perspective, we propose a sparsity inheritance mechanism along with an inherited dynamic pruning (IDP) method to obtain a series of N:M sparse candidate Transformers rapidly. A model compression scheme is further proposed to significantly reduce the storage requirement for deployment. (2) From hardware perspective, we present a flexible and efficient hardware architecture, namely STA, to achieve significant speedup when deploying N:M sparse Transformers. STA features not only a computing engine unifying both sparse-dense and dense-dense matrix multiplications with high computational efficiency but also a scalable softmax module eliminating the latency from intermediate off-chip data communication. Experimental results show that compared to other methods, N:M sparse Transformers, generated using IDP, achieves an average of 6.7% improvement on accuracy with high training efficiency. Moreover, STA can achieve 14.47x and 11.33x speedup compared to Intel i9-9900X and NVIDIA RTX 2080 Ti, respectively, and perform 2.00-19.47x faster inference than the state-of-the-art FPGA-based accelerators for Transformers.
Hadoop MapReduce for Mobile Clouds

The new generations of mobile devices have high processing power and storage, but they lag behind in terms of software systems for big data storage and processing. Hadoop is a scalable platform that provides distributed storage and computational capabilities on clusters of commodity hardware. Building Hadoop on a mobile network enables the devices to run data intensive computing applications without direct knowledge of underlying distributed systems complexities. However, these applications have severe energy and reliability constraints (e.g., caused by unexpected device failures or topology changes in a dynamic network). As mobile devices are more susceptible to unauthorized access, when compared to traditional servers, security is also a concern for sensitive data. Hence, it is paramount to consider reliability, energy efficiency and security for such applications. The MDFS (Mobile Distributed File System) [1] addresses these issues for big data processing in mobile clouds. We have developed the Hadoop MapReduce framework over MDFS and have studied its performance by varying input workloads in a real heterogeneous mobile cluster. Our evaluation shows that the implementation addresses all constraints in processing large amounts of data in mobile clouds. Thus, our system is a viable solution to meet the growing demands of data processing in a mobile environment.
Splines and wavelets on circulant graphs

We present novel families of wavelets and associated filterbanks for the analysis and representation of functions defined on circulant graphs. In this work, we leverage the inherent vanishing moment property of the circulant graph Laplacian operator, and by extension, the e-graph Laplacian, which is established as a parameterization of the former with respect to the degree per node, for the design of vertex-localized and critically-sampled higher-order graph (e-)spline wavelet filterbanks, which can reproduce and annihilate classes of (exponential) polynomial signals on circulant graphs. In addition, we discuss similarities and analogies of the detected properties and resulting constructions with splines and spline wavelets in the Euclidean domain. Ultimately, we consider generalizations to arbitrary graphs in the form of graph approximations, with focus on graph product decompositions. In particular, we proceed to show how the use of graph products facilitates a multi-dimensional extension of the proposed constructions and properties.
PersonaIA: A Lightweight Implicit Authentication System Based on Customized User Behavior Selection

Motivated by the great potential of implicit and seamless user authentication, we attempt to build an implicit authentication (IA) system with adaptive sampling that automatically selects dynamic sets of activities for user behavior extraction. Various activities, such as user location, application usage, user motion, and battery usage have been popular choices to generate behaviors, the soft biometrics, for implicit authentication. Unlike password-based or hard biometric-based authentication, implicit authentication does not require explicit user action or expensive hardware. However, user behaviors can change unpredictably which renders it more challenging to develop systems that depend on them. In addition to dynamic behavior extraction, the proposed implicit authentication system differs from the existing systems in terms of energy efficiency for battery-powered mobile devices. Since implicit authentication systems including the proposed one rely on machine learning, the expensive training process needs be outsourced to the remote server. However, mobile devices may not always have reliable network connections to send real-time data to the server for training. We overcome this limitation by proposing a W-layer, an overlay that provides a practical and energy-efficient solution for implicit authentication on mobile devices. We implemented partially labeled Dirichlet allocation (PLDA) on the server side for more accurate feature extraction, and achieved 93.3 percent precision and 98.6 percent accuracy in the synthetic dataset. Furthermore, we tested the power consumption of the smartphones used for our experiments and found that our method consumed 14.5 percent of the devices' total battery usage.
Refinement-Based Specification and Security Analysis of Separation Kernels

Assurance of information-flow security by formal methods is mandated in security certification of separation kernels. As an industrial standard for improving safety, ARINC 653 has been complied with by mainstream separation kernels. Due to the new trend of integrating safe and secure functionalities into one separation kernel, security analysis of ARINC 653 as well as a formal specification with security proofs are thus significant for the development and certification of ARINC 653 compliant Separation Kernels (ARINC SKs). This paper presents a specification development and security analysis method for ARINC SKs based on refinement. We propose a generic security model and a stepwise refinement framework. Two levels of functional specification are developed by the refinement. A major part of separation kernel requirements in ARINC 653 are modeled, such as kernel initialization, two-level scheduling, partition and process management, and inter-partition communication. The formal specification and its security proofs are carried out in the Isabelle/HOL theorem prover. We have reviewed the source code of one industrial and two open-source ARINC SK implementations, i.e., VxWorks 653, XtratuM, and POK, in accordance with the formal specification. During the verification and code review, six security flaws, which can cause information leakage, are found in the ARINC 653 standard and the implementations.
Numerically stable online estimation of variance in particle filters

This paper discusses variance estimation in sequential Monte Carlo methods, alternatively termed particle filters.The variance estimator that we propose is a natural modification of that suggested by H.P. Chan and T.L. Lai [Ann.Statist.41 (2013) 2877-2904], which allows the variance to be estimated in a single run of the particle filter by tracing the genealogical history of the particles.However, due particle lineage degeneracy, the estimator of the mentioned work becomes numerically unstable as the number of sequential particle updates increases.Thus, by tracing only a part of the particles' genealogy rather than the full one, our estimator gains long-term numerical stability at the cost of a bias.The scope of the genealogical tracing is regulated by a lag, and under mild, easily checked model assumptions, we prove that the bias tends to zero geometrically fast as the lag increases.As confirmed by our numerical results, this allows the bias to be tightly controlled also for moderate particle sample sizes.
A mathematical programming approach for integrated multiple linear regression subset selection and validation

Subset selection for multiple linear regression aims to construct a regression model that minimizes errors by selecting a small number of explanatory variables. Once a model is built, various statistical tests and diagnostics are conducted to validate the model and to determine whether the regression assumptions are met. Most traditional approaches require human decisions at this step. For example, the user adding or removing a variable until a satisfactory model is obtained. However, this trial-and-error strategy cannot guarantee that a subset that minimizes the errors while satisfying all regression assumptions will be found. In this paper, we propose a fully automated model building procedure for multiple linear regression subset selection that integrates model building and validation based on mathematical programming. The proposed model minimizes mean squared errors while ensuring that the majority of the important regression assumptions are met. We also propose an efficient constraint to approximate the constraint for the coefficient t-test. When no subset satisfies all of the considered regression assumptions, our model provides an alternative subset that satisfies most of these assumptions. Computational results show that our model yields better solutions (i.e., satisfying more regression assumptions) compared to the state-of-the-art benchmark models while maintaining similar explanatory power.
Learning to Refine Expansion Terms for Biomedical Information Retrieval Using Semantic Resources

With the rapid development of biomedicine, the number of biomedical articles has increased accordingly, which presents a great challenge for biologists trying to keep up with the latest research. Information retrieval technologies seek to meet this challenge by searching among a large number of articles based on a given query and providing the most relevant ones to fulfill information needs. As an effective information retrieval technique, query expansion has some room for improvement to achieve the desired performance when directly applied for biomedical information retrieval because there exist many domain-related terms both in users' queries and in related articles. To solve this problem, we propose a biomedical query expansion framework based on learning-to-rank methods, in which we refine the candidate expansion terms by training term-ranking models to select the most relevant terms for enriching the original query. To train the term-ranking models, we first propose a pseudo-relevance feedback method based on MeSH to select candidate expansion terms and then represent the candidate terms as feature vectors by defining both the corpus-based term features and the resource-based term features. Experimental results obtained for TREC genomics datasets show that our method can capture more relevant terms to expand the original query and effectively improve biomedical information retrieval performance.
Monitoring decoherence via measurement of quantum coherence

A multi-slit interference experiment, with which-way detectors, in the presence of environment induced decoherence, is theoretically analyzed. The effect of environment is modeled via a coupling to a bath of harmonic oscillators. Through an exact analysis, an expression for C, a recently introduced measure of coherence, of the particle at the detecting screen is obtained as a function of the parameters of the environment. It is argued that the effect of decoherence can be quantified using the measured coherence value which lies between zero and one. For the specific case of two slits, it is shown that the decoherence time can be obtained from the measured value of the coherence, C, thus providing a novel way to quantify the effect of decoherence via direct measurement of quantum coherence. This would be of significant value in many current studies that seek to exploit quantum superpositions for quantum information applications and scalable quantum computation.