OpenRaiser/Intern-Atlas huggingface dataset:
Learning Robust Recommender from Noisy Implicit Feedback
Venue: arXiv cs.IR 2021
Year: 2021

The ubiquity of implicit feedback makes it indispensable for building recommender systems. However, it does not actually reflect the actual satisfaction of users. For example, in E-commerce, a large portion of clicks do not translate to purchases, and many purchases end up with negative reviews. As such, it is of importance to account for the inevitable noises in implicit feedback. However, little work on recommendation has taken the noisy nature of implicit feedback into consideration. In this work, we explore the central theme of denoising implicit feedback for recommender learning, including training and inference. By observing the process of normal recommender training, we find that noisy feedback typically has large loss values in the early stages. Inspired by this observation, we propose a new training strategy named Adaptive Denoising Training (ADT), which adaptively prunes the noisy interactions by two paradigms (i.e., Truncated Loss and Reweighted Loss). Furthermore, we consider extra feedback (e.g., rating) as auxiliary signal and propose three strategies to incorporate extra feedback into ADT: finetuning, warm-up training, and colliding inference. We instantiate the two paradigms on the widely used binary cross-entropy loss and test them on three representative recommender models. Extensive experiments on three benchmarks demonstrate that ADT significantly improves the quality of recommendation over normal training without using extra feedback. Besides, the proposed three strategies for using extra feedback largely enhance the denoising ability of ADT.
Multi-scale simulation of COVID-19 epidemics
Venue: arXiv cs.MA 2021
Year: 2021

Over a year after the start of the COVID-19 epidemics, we are still facing the virus and it is hard to correctly predict its future spread over weeks to come, as well as the impacts of potential political interventions. Current epidemic models mainly fall in two approaches: compartmental models, divide the population in epidemiological classes and rely on the mathematical resolution of differential equations to give a macroscopic view of the epidemical dynamics, allowing to evaluate its spread a posteriori; agent-based models are computer models that give a microscopic view of the situation, since each human is modelled as one autonomous agent, allowing to study the epidemical dynamics in relation to (heterogeneous) individual behaviours. In this work, we compared both methodologies and combined them to try and take advantage of the benefits of each, and to overcome their limits. In particular, agent-based simulation can be used to refine the values of the parameters of a compartmental model, or to predict how these values evolve depending on sanitary policies applied. In this report we discuss the conditions of such a combination of approaches, and future improvements.
Age of Information in Prioritized Random Access
Venue: arXiv cs.IT 2021
Year: 2021

Age of information (AoI) is a performance metric that captures the freshness of status updates. While AoI has been studied thoroughly for point-to-point links, the impact of modern random-access protocols on this metric is still unclear. In this paper, we extend the recent results by Munari to prioritized random access where devices are divided into different classes according to different AoI requirements. We consider the irregular repetition slotted ALOHA protocol and analyze the AoI evolution by means of a Markovian analysis following similar lines as in Munari (2021). We aim to design the protocol to satisfy the AoI requirements for each class while minimizing the power consumption. To this end, we optimize the update probability and the degree distributions of each class, such that the probability that their AoI exceeds a given threshold lies below a given target and the average number of transmitted packets is minimized.
Adaptive Group Collaborative Artificial Bee Colony Algorithm
Venue: arXiv cs.NE 2021
Year: 2021

As an effective algorithm for solving complex optimization problems, artificial bee colony (ABC) algorithm has shown to be competitive, but the same as other population-based algorithms, it is poor at balancing the abilities of global searching in the whole solution space (named as exploration) and quick searching in local solution space which is defined as exploitation. For improving the performance of ABC, an adaptive group collaborative ABC (AgABC) algorithm is introduced where the population in different phases is divided to specific groups and different search strategies with different abilities are assigned to the members in groups, and the member or strategy which obtains the best solution will be employed for further searching. Experimental results on benchmark functions show that the proposed algorithm with dynamic mechanism is superior to other algorithms in searching accuracy and stability. Furthermore, numerical experiments show that the proposed method can generate the optimal solution for the complex scheduling problem.
A Crowdsourced Contact Tracing Model to Detect COVID-19 Patients using Smartphones
Venue: arXiv cs.CY 2021
Year: 2021

Millions of people have died all across the world because of the COVID-19 outbreak. Researchers worldwide are working together and facing many challenges to bring out the proper vaccines to prevent this infectious virus. Therefore, in this study, a system has been designed which will be adequate to stop the outbreak of COVID-19 by spreading awareness of the COVID-19 infected patient situated area. The model has been formulated for Location base COVID-19 patient identification using mobile crowdsourcing. In this system, the government will update the information about inflected COVID-19 patients. It will notify other users in the vulnerable area to stay at 6 feet or 1.8-meter distance to remain safe. We utilized the Haversine formula and circle formula to generate the unsafe area. Ten thousand valid information has been collected to support the results of this research. The algorithm is tested for 10 test cases every time, and the datasets are increased by 1000. The run time of that algorithm is growing linearly. Thus, we can say that the proposed algorithm can run in polynomial time. The algorithm's correctness is also being tested where it is found that the proposed algorithm is correct and efficient. We also implement the system, and the application is evaluated by taking feedback from users. Thus, people can use our system to keep themselves in a safe area and decrease COVID patients' rate.
The Benefits of Edge Computing in Healthcare, Smart Cities, and IoT
Venue: arXiv cs.CY 2021
Year: 2021

Recent advancements in technology now allow for the generation of massive quantities of data. There is a growing need to transmit this data faster and more securely such that it cannot be accessed by malicious individuals. Edge computing has emerged in previous research as a method capable of improving data transmission times and security before the data ends up in the cloud. Edge computing has an impressive transmission speed based on fifth generation (5G) communication which transmits data with low latency and high bandwidth. While edge computing is sufficient to extract important features from the raw data to prevent large amounts of data requiring excessive bandwidth to be transmitted, cloud computing is used for the computational processes required for developing algorithms and modeling the data. Edge computing also improves the quality of the user experience by saving time and integrating quality of life (QoL) features. QoL features are important for the healthcare sector by helping to provide real-time feedback of data produced by healthcare devices back to patients for a faster recovery. Edge computing has better energy efficiency, can reduce the electricity cost, and in turn help people reduce their living expenses. This paper will take a detailed look into edge computing applications around Internet of Things (IoT) devices, smart city infrastructure, and benefits to healthcare.
The empirical study of e-learning post-acceptance after the spread of COVID-19: A multi-analytical approach based hybrid SEM-ANN
Venue: arXiv cs.CY 2021
Year: 2021

There are several reasons why the fear of vaccination has caused population rejection. Questions have been raised by students regarding the effectiveness of vaccines, which in turn has led to vaccination hesitancy. Students perceptions are influenced by vaccination hesitancy, which affects the acceptance of e-learning platforms. Hence, this research aimed to examine the post-acceptance of e-learning platforms on the basis of a conceptual model that employs different variables. Distinct contribution is made by every variable to the post-acceptance of e-learning platforms. A hybrid model was used in the current study in which technology acceptance model (TAM) determinants were employed along with other external factors such as fear of vaccination, perceived routine use, perceived enjoyment, perceived critical mass, and self-efficiency which are directly linked to post-acceptance of e-learning platforms. The focus of earlier studies on this topic has been on the significance of e-learning acceptance in various environments and countries. However, in this study, the newly-spread use of e-learning platforms in the gulf area was examined using a hybrid conceptual model. The empirical studies carried out in the past mainly used structural equation modelling (SEM) analysis; however, this study used an evolving hybrid analysis approach, in which SEM and the artificial neural network (ANN) that are based on deep learning were employed. The importance-performance map analysis (IPMA) was also used in this study to determine the significance and performance of each factor. The proposed model is backed by the findings of data analysis.
3rd Place Solution for NeurIPS 2021 Shifts Challenge: Vehicle Motion Prediction
Venue: arXiv cs.CV 2021
Year: 2021

Shifts Challenge: Robustness and Uncertainty under Real-World Distributional Shift is a competition held by NeurIPS 2021. The objective of this competition is to search for methods to solve the motion prediction problem in cross-domain. In the real world dataset, It exists variance between input data distribution and ground-true data distribution, which is called the domain shift problem. In this report, we propose a new architecture inspired by state of the art papers. The main contribution is the backbone architecture with self-attention mechanism and predominant loss function. Subsequently, we won 3rd place as shown on the leaderboard.
Tesselating a Pascal-like tetrahedron for the subdivision of high order tetrahedral finite elements
Venue: arXiv cs.CG 2021
Year: 2021

Three-dimensional $N^{th}$ order nodal Lagrangian tetrahedral finite elements ($P_N$ elements) can be generated using Pascal's tetrahedron $\mathcal{H}$ where each node in 3D element space corresponds to an entry in $\mathcal{H}$. For the purposes of visualization and post-processing, it is desirable to "subdivide" these high-order tetrahedral elements into sub-tetrahedra which cover the whole space without intersections and without introducing new exterior edges or vertices. That is, the exterior triangulation of the element should be congruent with the "natural" triangulation of the $2D$ Pascal's triangle. This work attempts to describe that process of subdivision for arbitrary $N$.
ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors
Venue: arXiv cs.CL 2021
Year: 2021

Fusion technique is a key research topic in multimodal sentiment analysis. The recent attention-based fusion demonstrates advances over simple operation-based fusion. However, these fusion works adopt single-scale, i.e., token-level or utterance-level, unimodal representation. Such single-scale fusion is suboptimal because that different modality should be aligned with different granularities. This paper proposes a fusion model named ScaleVLAD to gather multi-Scale representation from text, video, and audio with shared Vectors of Locally Aggregated Descriptors to improve unaligned multimodal sentiment analysis. These shared vectors can be regarded as shared topics to align different modalities. In addition, we propose a self-supervised shifted clustering loss to keep the fused feature differentiation among samples. The backbones are three Transformer encoders corresponding to three modalities, and the aggregated features generated from the fusion module are feed to a Transformer plus a full connection to finish task predictions. Experiments on three popular sentiment analysis benchmarks, IEMOCAP, MOSI, and MOSEI, demonstrate significant gains over baselines.
Flood Analytics Information System (FAIS) Version 4.00 Manual
Venue: arXiv cs.LG 2021
Year: 2021

This project was the first attempt to use big data analytics approaches and machine learning along with Natural Language Processing (NLP) of tweets for flood risk assessment and decision making. Multiple Python packages were developed and integrated within the Flood Analytics Information System (FAIS). FAIS workflow includes the use of IoTs-APIs and various machine learning approaches for transmitting, processing, and loading big data through which the application gathers information from various data servers and replicates it to a data warehouse (IBM database service). Users are allowed to directly stream and download flood related images/videos from the US Geological Survey (USGS) and Department of Transportation (DOT) and save the data on a local storage. The outcome of the river measurement, imagery, and tabular data is displayed on a web based remote dashboard and the information can be plotted in real-time. FAIS proved to be a robust and user-friendly tool for flood data analysis at regional scale that could help stakeholders for rapid assessment of flood situation and damages. FAIS also provides flood frequency analysis (FFA) to estimate flood quantiles including the associated uncertainties that combine the elements of observational analysis, stochastic probability distribution and design return periods. FAIS is publicly available and deployed on the Clemson-IBM cloud service.
Training Efficiency and Robustness in Deep Learning
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning has revolutionized machine learning and artificial intelligence, achieving superhuman performance in several standard benchmarks. It is well-known that deep learning models are inefficient to train; they learn by processing millions of training data multiple times and require powerful computational resources to process large batches of data in parallel at the same time rather than sequentially. Deep learning models also have unexpected failure modes; they can be fooled into misbehaviour, producing unexpectedly incorrect predictions. In this thesis, we study approaches to improve the training efficiency and robustness of deep learning models. In the context of learning visual-semantic embeddings, we find that prioritizing learning on more informative training data increases convergence speed and improves generalization performance on test data. We formalize a simple trick called hard negative mining as a modification to the learning objective function with no computational overhead. Next, we seek improvements to optimization speed in general-purpose optimization methods in deep learning. We show that a redundancy-aware modification to the sampling of training data improves the training speed and develops an efficient method for detecting the diversity of training signal, namely, gradient clustering. Finally, we study adversarial robustness in deep learning and approaches to achieve maximal adversarial robustness without training with additional data. For linear models, we prove guaranteed maximal robustness achieved only by appropriate choice of the optimizer, regularization, or architecture.
Safe Reinforcement Learning for Grid Voltage Control
Venue: arXiv cs.LG 2021
Year: 2021

Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions, yet this scheme usually trips a massive amount of load inefficiently. Reinforcement learning (RL) has been adopted as a promising approach to circumvent the issues; however, RL approach usually cannot guarantee the safety of the systems under control. In this paper, we discuss a couple of novel safe RL approaches, namely constrained optimization approach and Barrier function-based approach, that can safely recover voltage under emergency events. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark are performed to demonstrate the effectiveness of the proposed safe RL emergency control.
Machine Learning-Based Classification Algorithms for the Prediction of Coronary Heart Diseases
Venue: arXiv cs.CV 2021
Year: 2021

Coronary heart disease, which is a form of cardiovascular disease (CVD), is the leading cause of death worldwide. The odds of survival are good if it is found or diagnosed early. The current report discusses a comparative approach to the classification of coronary heart disease datasets using machine learning (ML) algorithms. The current study created and tested several machine-learning-based classification models. The dataset was subjected to Smote to handle unbalanced classes and feature selection technique in order to assess the impact on two distinct performance metrics. The results show that logistic regression produced the highest performance score on the original dataset compared to the other algorithms employed. In conclusion, this study suggests that LR on a well-processed and standardized dataset can predict coronary heart disease with greater accuracy than the other algorithms.
A Fast Knowledge Distillation Framework for Visual Recognition
Venue: arXiv cs.CV 2021
Year: 2021

While Knowledge Distillation (KD) has been recognized as a useful tool in many visual tasks, such as supervised classification and self-supervised representation learning, the main drawback of a vanilla KD framework is its mechanism, which consumes the majority of the computational overhead on forwarding through the giant teacher networks, making the entire learning procedure inefficient and costly. ReLabel, a recently proposed solution, suggests creating a label map for the entire image. During training, it receives the cropped region-level label by RoI aligning on a pre-generated entire label map, allowing for efficient supervision generation without having to pass through the teachers many times. However, as the KD teachers are from conventional multi-crop training, there are various mismatches between the global label-map and region-level label in this technique, resulting in performance deterioration. In this study, we present a Fast Knowledge Distillation (FKD) framework that replicates the distillation training phase and generates soft labels using the multi-crop KD approach, while training faster than ReLabel since no post-processes such as RoI align and softmax operations are used. When conducting multi-crop in the same image for data loading, our FKD is even more efficient than the traditional image classification framework. On ImageNet-1K, we obtain 79.8% with ResNet-50, outperforming ReLabel by ~1.0% while being faster. On the self-supervised learning task, we also show that FKD has an efficiency advantage. Our project page: http://zhiqiangshen.com/projects/FKD/index.html, source code and models are available at: https://github.com/szq0214/FKD.
Theoretical Analysis of an XGBoost Framework for Product Cannibalization
Venue: arXiv cs.LG 2021
Year: 2021

This paper is an extension of our work where we presented a three-stage XGBoost algorithm for forecasting sales under product cannibalization scenario. Previously we developed the model based on our intuition and provided empirical evidence on its performance. In this study we would briefly go over the algorithm and then provide mathematical reasoning behind its working.
HMC with Normalizing Flows
Venue: arXiv cs.LG 2021
Year: 2021

We propose using Normalizing Flows as a trainable kernel within the molecular dynamics update of Hamiltonian Monte Carlo (HMC). By learning (invertible) transformations that simplify our dynamics, we can outperform traditional methods at generating independent configurations. We show that, using a carefully constructed network architecture, our approach can be easily scaled to large lattice volumes with minimal retraining effort. The source code for our implementation is publicly available online at https://github.com/nftqcd/fthmc.
TransZero: Attribute-guided Transformer for Zero-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

Zero-shot learning (ZSL) aims to recognize novel classes by transferring semantic knowledge from seen classes to unseen ones. Semantic knowledge is learned from attribute descriptions shared between different classes, which act as strong priors for localizing object attributes that represent discriminative region features, enabling significant visual-semantic interaction. Although some attention-based models have attempted to learn such region features in a single image, the transferability and discriminative attribute localization of visual features are typically neglected. In this paper, we propose an attribute-guided Transformer network, termed TransZero, to refine visual features and learn attribute localization for discriminative visual embedding representations in ZSL. Specifically, TransZero takes a feature augmentation encoder to alleviate the cross-dataset bias between ImageNet and ZSL benchmarks, and improves the transferability of visual features by reducing the entangled relative geometry relationships among region features. To learn locality-augmented visual features, TransZero employs a visual-semantic decoder to localize the image regions most relevant to each attribute in a given image, under the guidance of semantic attribute information. Then, the locality-augmented visual features and semantic vectors are used to conduct effective visual-semantic interaction in a visual-semantic embedding network. Extensive experiments show that TransZero achieves the new state of the art on three ZSL benchmarks. The codes are available at: \url{https://github.com/shiming-chen/TransZero}.
Make A Long Image Short: Adaptive Token Length for Vision Transformers
Venue: arXiv cs.CV 2021
Year: 2021

The vision transformer splits each image into a sequence of tokens with fixed length and processes the tokens in the same way as words in natural language processing. More tokens normally lead to better performance but considerably increased computational cost. Motivated by the proverb "A picture is worth a thousand words" we aim to accelerate the ViT model by making a long image short. To this end, we propose a novel approach to assign token length adaptively during inference. Specifically, we first train a ViT model, called Resizable-ViT (ReViT), that can process any given input with diverse token lengths. Then, we retrieve the "token-length label" from ReViT and use it to train a lightweight Token-Length Assigner (TLA). The token-length labels are the smallest number of tokens to split an image that the ReViT can make the correct prediction, and TLA is learned to allocate the optimal token length based on these labels. The TLA enables the ReViT to process the image with the minimum sufficient number of tokens during inference. Thus, the inference speed is boosted by reducing the token numbers in the ViT model. Our approach is general and compatible with modern vision transformer architectures and can significantly reduce computational expanse. We verified the effectiveness of our methods on multiple representative ViT models (DeiT, LV-ViT, and TimesFormer) across two tasks (image classification and action recognition).