Reviewerly/exHarmony huggingface dataset:
A Variational Autoencoder for Heterogeneous Temporal and Longitudinal Data

The variational autoencoder (VAE) is a popular deep latent variable model used to analyse high-dimensional datasets by learning a low-dimensional latent representation of the data. It simultaneously learns a generative model and an inference network to perform approximate posterior inference. Recently proposed extensions to VAEs that can handle temporal and longitudinal data have applications in healthcare, behavioural modelling, and predictive maintenance. However, these extensions do not account for heterogeneous data, i.e., data comprising of continuous and discrete attributes, which is common in many real-life applications. In this work, we propose the heterogeneous longitudinal VAE (HL-VAE) that extends the existing temporal and longitudinal VAEs to heterogeneous data for imputing missing values and unseen time point prediction. HL-VAE provides efficient inference for high-dimensional datasets and includes likelihood models for continuous, count, categorical, and ordinal data while accounting for missing observations. We demonstrate our model’s efficacy through simulated as well as clinical datasets, and show that our proposed model achieves competitive performance in missing value imputation and predictive accuracy.
Deep Baseline Network for Time Series Modeling and Anomaly Detection

Deep learning has seen increasing applications in time series in recent years. For time series anomaly detection scenarios, such as in finance, Internet of Things, data center operations, etc., time series usually show very flexible baselines depending on various external factors. Anomalies unveil themselves by lying far away from the baseline. However, the detection is not always easy due to some challenges including baseline shifting, lacking of labels, noise interference, real time detection in streaming data, result interpretability, etc. In this paper, we develop a novel deep architecture to properly extract the baseline from time series, namely Deep Baseline Network (DBLN). By using this deep network, we can easily locate the baseline position and then provide reliable and interpretable anomaly detection result. Empirical evaluation on both synthetic and public real-world datasets shows that our purely unsupervised algorithm achieves superior performance compared with state-of-art methods and has good practical applications.
Addressing Sample Efficiency and Model-bias in Model-based Reinforcement Learning

Model-based reinforcement learning promises to be an effective way to bring reinforcement learning to real-world robotic systems by offering a sample efficient learning approach compared to model-free reinforcement learning. However, model-based reinforcement learning approaches at present struggle to match the performance of model-free ones. This work attempts to fill this gap by improving the performance of model-based reinforcement learning while further improving its sample efficiency. To improve the sample efficiency, an exploration strategy is formulated which maximizes the information gain. The asymptotic performance is improved by compensating for the model-bias using a model-free critic. We have evaluated our proposed approach on four reinforcement learning benchmarking tasks in the openAI gym framework.
Empirical analysis of the convergence of Double DQN in relation to reward sparsity

Q-Networks are used in Reinforcement Learning to model the expected return from every action at a given state. When training Q-Networks, external reward signals are propagated to the previously performed actions leading up to each reward. If many actions are required before experiencing a reward, the reward signal is distributed across all those actions, where some actions may have greater impact on the reward than others. As the number of significant actions between rewards increases, the relative importance of each action decreases. If actions have too small importance, their impact might be overshadowed by noise in a deep neural network model, potentially causing convergence issues. In this work, we empirically test the limits of increasing the number of actions leading up to a reward in a simple grid-world environment. We show in our experiments that even though the training error surpasses the reward signal attributed to each action, the model is still able to learn a smooth enough value representation.
Logit Calibration for Non-IID and Long-Tailed Data in Federated Learning

Federated learning (FL) strives to enable collaborative training of deep models on the distributed clients of different data without centrally aggregating raw data and hence improving data privacy. Nevertheless, a central challenge in training classification models in the federated system is learning with non-IID data. Most of the existing work is dedicated to eliminating the heterogeneous influence of non-IID data in a federated system. However, in many real-world FL applications, the co-occurrence of data heterogeneity and long-tailed distribution is unavoidable. The universal class distribution is long-tailed, causing them to become easily biased towards head classes, which severely harms the global model performance. In this work, we also discovered an intriguing fact that the classifier logit vector (i.e., pre-softmax output) introduces a heterogeneity drift during the learning process of local training and global optimization, which harms the convergence as well as model performance. Therefore, motivated by the above finding, we propose a novel logit calibration FL method to solve the joint problem of non-IID and long-tailed data in federated learning, called Federated Learning with Logit Calibration (FedLC). First, we presented a method to mitigate the local update drift by calculating the Wasserstein distance among adjacent client logits and then aggregating similar clients to regulate local training. Second, based on the model ensemble, a new distillation method with logit calibration and class weighting was proposed by exploiting the diversity of local models trained on heterogeneous data, which effectively alleviates the global drift problem under long-tailed distribution. Finally, we evaluated FedLC using a highly non-IID and long-tailed experimental setting, comprehensive experiments on several benchmark datasets demonstrated that FedLC achieved superior performance compared with state-of-the-art FL methods, which fully illustrated the effectiveness of the logit calibration strategy.
Blockchain-based Distributed Storage System for Task Offloading in Vehicular Edge Computing

With the deepening requirements of development for the Internet of Vehicles technology, intelligent vehicles need to complete the complex task in a fast response time to meet the needs, which brings great challenges to reliable vehicle service provision. In order to solve this problem, we propose a task offloading scheme into the Internet of Vehicles, which can reduce the execution time of tasks. In this paper, we introduce a location-aware distributed storage system called LDSS to support task offloading. Data can be written, stored, computed and read out through different mobile edge servers. Firstly, the identity of the vehicle is stored on the blockchain, and then the vehicle needs to offload the data to the nearest mobile edge server. Then, the storage node and computing node are selected based on the time optimization strategy. Finally, by predicting the location of the vehicle, the mobile edge server nearest to the predicted vehicle location is selected as the access node for result downloading. After validating the identity of vehicle based on blockchain technology, the output result is returned to the vehicle. The experiments show that our proposed task offloading scheme based on blockchain can effectively achieve the effect of low latency and faster task response, as well as ensure data security.
Score-based Image-to-Image Regression with Synchronized Diffusion

Image-to-image regression is an important computer vision task. In this paper, we propose a novel image-to-image regression model following the recent trend in generative modeling that employs Stochastic Differential Equations (SDEs) and score matching. We first apply diffusion processes to regression data using designed SDEs, and then perform inference by gradually reversing the processes. In particular, our method uses synchronized diffusion, which simultaneously applies diffusion to both input and response images to stabilize diffusion and subsequent parameter learning. Furthermore, based on the Expectation-Maximization (EM) algorithm, we develop an effective algorithm for prediction. We implement a conditional U-Net architecture with pre-trained DenseNet encoder for our proposed model and refer to it as DenseSocre. Our new model is able to generate diverse outcomes for image colorization, and the proposed prediction algorithm is able to achieve close to state-of-art performance on high-resolution monocular depth estimation.
Sliding Focal Loss for Class Imbalance Classification in Federated XGBoost

As a very popular framework, federated learning can help heterogeneous participants cooperate training global models without the local data being exposed. It not only takes advantage of massive raw data, but also fundamentally protects the privacy of participants. An unavoidable challenge is that class imbalance brought by many participants will seriously affect the model performance and even damage the convergence. Introducing Focal loss to dynamically adjust the weight of samples in the training process is a good choice for relieving this issue. In our experiments, we find a trade-off between the convergence and the final accuracy of using focal loss and cross-entropy in traditional federated XGBoost. For this property, we propose hyperparametric linear and exponential sliding to combine the advantages of both methods. A complete experiment proved that sliding rather than direct cohesion was necessary. Meanwhile, linear sliding performs well than just using focal loss in three of the four class imbalances scenarios, and exponential sliding performs the best in all four scenarios. There are even two that exceed the cross-entropy with a finite number of communication rounds.
An Automatic Modeling Method for Web Service Business Processes towards CPN Model Checking

With the development of the Internet, the ever-increasing business demands have led to the emergence of web service composition technology. The Business Process Execution Language (BPEL) is a widely used web service composition standard, which reorganizes existing services to define new ones. However, the complexity and concurrency of the combined business process make it difficult to ensure its correctness, so it is particularly important to verify the business process. In this paper, we propose a method to automatically generate a Colored Petri Net (CPN) model of a BPEL process and use model checking to find errors in the process. We first preprocess the BPEL program, and then use the preprocessing information to generate different model modules according to the conversion rules, and finally form a complete CPN model according to the relationship between the processes. The automatically generated model can be opened with CPN Tools and combined with ASK-CTL for model checking. On the one hand, automated modeling can solve the shortcomings of manual modeling that are prone to errors and inefficient. On the other hand, model checking can not only find properties such as deadlocks and liveness in BPEL processes, but also detect missing input and redundant output errors in data flow. The experimental results verify the feasibility and correctness of the method.
Learning Dynamic Behavior Patterns for Fraud Detection

Telecom fraud detection is considered a needle in a haystack task due to the fact that fraudsters are usually hidden among a large number of normal users. In fact, a fraudster will usually act in a continuous manner over a period of time to achieve fraudulent purposes. At the same time, fraudsters are constantly looking for new fraudulent targets. Therefore, we consider a behavioral pattern-based fraudster identification system called HADyn-LSTM for telecommunication network fraud detection. Motivated by the breakthroughs in deep learning (DL), we make an effort to judge whether a user is a fraudster is based on the user's behavior sequence in the social network, which means that we study the difference between normal users and fraudsters through dynamic behavior patterns rather than static statistical attributes. Particularly, Based on sequential neural networks and attention mechanisms, we design an end-to-end system that learns and extracts the dynamic patterns of user behavior sequences. Furthermore, We tested our system on three datasets of different sizes. The test results show that the HADyn-LSTM system performs better than the static attribute-based method.
Semi-Supervised Machine Learning for Analyzing COVID-19 Related Twitter Data for Asian Hate Speech

The COVID-19 pandemic left a lot of people sick, tired, and frustrated. Many people expressed their feelings on social media through comments and posts. Detecting hate speech on social media is important to help reduce the spread of racist comments. Machine learning algorithms can be used to classify hate speech. In our experiments, we implement semi-supervised machine learning algorithms to classify Twitter data. We used a count vectorizer as the feature and a support vector machine (SVM) classifier to classify COVID-19 related Twitter data while changing the amount of labeled data available. We found that self-training semi-supervised machine learning has similar effectiveness to supervised learning when there is significantly less training data available.
Using Artificial Intelligence to Predict Patient Electronic Health Record Access Points

Electronic Health Records (EHRs) offer benefits to patients and healthcare providers, however, patient records are often split between multiple providers. To understand the distribution of EHRs per patient in multiple counties throughout southeastern North Carolina, data from the Carolina Coastal Health Alliance and the Coastal Connect Health Information Exchange (CCHIE) was provided to researchers. Utilizing Python programming methods, data analysis was performed to find the total unique EHRs available to patients in the study, as well as averages for EHR usage within different age ranges. Machine learning models were developed to predict the multiple aspects of the data set, including the most prevalent EHR provider, total EHRs per patient, and all potential EHRs per patient based on demographic and geographic information. Analysis of results determined that certain models were more successful in the prediction based on statistical measures of accuracy, precision, and recall. Results were cataloged in tables and an interactive mapping tool to highlight discrepancies in the overall data set that may influence results and EHR access for populations in North Carolina.
Machine Learning for Classifying Images with Motion Blur

Motion blur in an image results from movements of objects within a scene or from the imaging system itself. In applications such as high-speed license plate recognition, motion blur introduces image artifacts leading to challenges in image classification. In this work, we investigate machine learning techniques for classifying images with motion blur using convolutional neural networks. In particular, we explore how different motion directions and lengths affect the predictive performance of our classification model. We used the MNIST dataset, which contains 70,000 images of handwritten digits, to generate training and testing datasets of images with motion blur using MATLAB. Specifically, we considered motion blurs at various angles and lengths to analyze the effects of different motion blurs on the classification accuracy of images of digits. We found that our model shows very high accuracy when training and testing on datasets with the same type of motion blur. In contrast, training and testing on datasets with different motion blurs result in lower accuracy. We describe how we can improve overall classification performance and offer insights on what additional information can be inferred from the MNIST dataset with motion blur.
LA-vIOMMU: An Efficient Hardware-Software Co-design of IOMMU Virtualization

I/O Memory Management Unit (IOMMU) is im-portant hardware support for I/O virtualization, and it is widely used in device passthrough. However, IOMMU sacrifices memory utilization due to the static mapping requirement. Moreover, it lacks DMA security guarantees inside the guest. IOMMU virtual-ization is a decent solution among existing studies to address these problems. Nevertheless, pure software IOMMU virtualization suffers from high overhead, while hardware-assisted IOMMU virtualization needs complex hardware redesign. Therefore, en-abling efficient design with low hardware overhead remains chal-lenging. In this paper, we propose an efficient hardware-software co-design of IOMMU virtualization, named LA-vIOMMU, to achieve high I/O performance with low hardware overhead. LA-vIOMMU is designed based on the existing vIOMMU mode and can be adapted to other platforms. We describe the design and implementation of LA-vIOMMU. The LA-vIOMMU combines hardware expansion and software optimization to minimize the performance cost. Furthermore, we evaluate the effectiveness and security of LA-vIOMMU and analyze the advantages compared with the existing design in x86. The experiment results show that LA-vIOMMU can get superior throughput than existing software approaches and achieve throughput pretty close to hardware-assisted virtualization with fewer hardware design changes and less verification overhead.
Anomaly Detection from Multilinear Observations via Time-Series Analysis and 3DTPCA

In the era of big data, there is massive demand for new techniques to forecast and analyze multi-dimensional data. One task that has seen great interest in the community is anomaly detection of streaming data. Toward this end, the current research develops a novel approach to anomaly detection of streaming 2-dimensional observations via multilinear time-series analysis and 3-dimensional tensor principal component analysis (3DTPCA). We approach this problem utilizing dimensionality reduction and probabilistic inference in a low-dimensional space. We first propose a natural extension to 2-dimensional tensor principal component analysis (2DTPCA) to perform data dimensionality reduction on 4-dimensional tensor objects, aptly named 3DTPCA. We then represent the sub-sequences of our time-series observations as a 4-dimensional tensor utilizing a sliding window. Finally, we use 3DTPCA to compute reconstruction errors for inferring anomalous instances within the multilinear data stream. Experimental validation is presented via MovingMNIST data. Results illustrate that the proposed approach has a significant speedup in training time compared with deep learning, while performing competitively in terms of accuracy.
Using Natural Language Processing to Predict Costume Core Vocabulary of Historical Artifacts

Historic dress artifacts are a valuable source for human studies. In particular, they can provide important insights into the social aspects of their corresponding era. These insights are commonly drawn from garment pictures as well as the accompanying descriptions and are usually stored in a standardized and controlled vocabulary that accurately describes garments and costume items, called the Costume Core Vocabulary. Building an accurate Costume Core from garment descriptions can be challenging because the historic garment items are often donated, and the accompanying descriptions can be based on untrained individuals and use a language common to the period of the items. In this paper, we present an approach to use Natural Language Processing (NLP) to map the free-form text descriptions of the historic items to that of the controlled vocabulary provided by the Costume Core. Despite the limited dataset, we were able to train an NLP model based on the Universal Sentence Encoder to perform this mapping with more than 90% test accuracy for a subset of the Costume Core vocabulary. We describe our methodology, design choices, and development of our approach, and show the feasibility of predicting the Costume Core for unseen descriptions. With more garment descriptions still being curated to be used for training, we expect to have higher accuracy for better generalizability.
Distributional Knowledge Transfer for Heterogeneous Federated Learning

Federated learning (FL) produces an effective global model by aggregating multiple client weights trained on their private data. However, it is common that the data are not independently and identically distributed (non-IID) across different clients, which greatly degrades the performance of the global model. We observe that existing FL approaches mostly ignore the distribution information of client-side private data. Actually, the distribution information is a kind of structured knowledge about the data itself, and it also represents the mutual clustering relations of data examples. In this work, we propose a novel approach, namely Federated Distribution Knowledge Transfer (FedDKT), that alleviates heterogeneous FL by extracting and transferring the distribution knowledge from diverse data. Specifically, the server learns a lightweight generator to generate data and broadcasts it to the sampled clients, FedDKT decouples the feature representations of the generated data and transfers the distribution knowledge to assist model training. In other words, we exploit the similarity and shared parts of the generated data and local private data to improve the generalization ability of the FL global model and promote representation learning. Further, we also propose the similarity measure and attention measure strategies, which implement FedDKT by capturing the correlations and key dependencies among data examples, respectively. The comprehensive experiments demonstrate that FedDKT significantly improves the performance and convergence rate of the FL global model, especially when the data are extremely non-IID. In addition, FedDKT is also effective when the data are identically distributed, which fully illustrates the generalization and effectiveness of the distribution knowledge.
Class-wise and reduced calibration methods

For many applications of probabilistic classifiers it is important that the predicted confidence vectors reflect true probabilities (one says that the classifier is calibrated). It has been shown that common models fail to satisfy this property, making reliable methods for measuring and improving calibration important tools. Unfortunately, obtaining these is far from trivial for problems with many classes. We propose two techniques that can be used in tandem. First, a reduced calibration method transforms the original problem into a simpler one. We prove for several notions of calibration that solving the reduced problem minimizes the corresponding notion of miscalibration in the full problem, allowing the use of non-parametric recalibration methods that fail in higher dimensions. Second, we propose class-wise calibration methods, based on intuition building on a phenomenon called neural collapse and the observation that most of the accurate classifiers found in practice can be thought of as a union of K different functions which can be recalibrated separately, one for each class. These typically out-perform their non class-wise counterparts, especially for classifiers trained on imbalanced data sets. Applying the two methods together results in class-wise reduced calibration algorithms, which are powerful tools for reducing the prediction and per-class calibration errors. We demonstrate our methods on real and synthetic datasets and release all code as open source in [1], [2].