Reviewerly/exHarmony huggingface dataset:
Fast Partitioning for VVC Intra-Picture Encoding with a CNN Minimizing the Rate-Distortion-Time Cost

This paper presents a CNN to reduce the encoding time of a VVC-based intra-picture encoder. For encoding a 32 × 32 block, the CNN estimates two partitioning parameters that restrict the allowed coding block width and height. To estimate them such that the encoder skips testing inefficient partitioning modes, we train the CNN as follows: First, we generate training data by encoding sequences without the CNN. While encoding, we test all combinations of the two parameters for each 32 × 32 block and store the resulting Lagrangian rate-distortion-time (RDT) cost. We use the recorded cost to derive the loss function when training the CNN. Consequently, the CNN is trained such that it minimizes the Lagrangian RDT cost. Our CNN reduces the encoding time by 50% with a bit rate increase of 0.9%, which outperforms existing CNN-based approaches. Our generic training approach could also be applied for other encoder parameters.
Integrity Auditing for Multi-Copy in Cloud Storage Based on Red-Black Tree

With the rapid development of cloud storage, cloud users are willing to store data in the cloud storage system, and at the same time, the requirements for the security, integrity, and availability of data storage are getting higher and higher. Although many cloud audit schemes have been proposed, the data storage overhead is too large and the data cannot be dynamically updated efficiently when most of the schemes are in use. In order to solve these problems, a cloud audit scheme for multi-copy dynamic data integrity based on red-black tree full nodes is proposed. This scheme uses ID-based key authentication, and improves the classic Merkel hash tree MHT to achieve multi-copy storage and dynamic data manipulation, which improves the efficiency of real-time dynamic data update (insertion, deletion, modification). The third-party audit organization replaces users to verify the integrity of data stored on remote cloud servers, which reduces the computing overhead and system communication overhead. The security analysis proves that the security model based on the CDH problem and the DL problem is safe. Judging from the results of the simulation experiment, the scheme is safe and efficient.
Temporal Extension Module for Skeleton-Based Action Recognition

We present a module that extends the temporal graph of a graph convolutional network (GCN) for action recognition with a sequence of skeletons. Existing methods attempt to represent a more appropriate spatial graph on an intra-frame, but disregard optimization of the temporal graph on the interframe. Concretely, these methods connect between vertices corresponding only to the same joint on the inter-frame. In this work, we focus on adding connections to neighboring multiple vertices on the inter-frame and extracting additional features based on the extended temporal graph. Our module is a simple yet effective method to extract correlated features of multiple joints in human movement. Moreover, our module aids in further performance improvements, along with other GCN methods that optimize only the spatial graph. We conduct extensive experiments on two large datasets, NTU RGB+D and Kinetics-Skeleton, and demonstrate that our module is effective for several existing models and our final model achieves state-of-the-art performance.
AttDMM: An Attentive Deep Markov Model for Risk Scoring in Intensive Care Units

Clinical practice in intensive care units (ICUs) requires early warnings when a patient's condition is about to deteriorate so that preventive measures can be undertaken. To this end, prediction algorithms have been developed that estimate the risk of mortality in ICUs. In this work, we propose a novel generative deep probabilistic model for real-time risk scoring in ICUs. Specifically, we develop an attentive deep Markov model called AttDMM. To the best of our knowledge, AttDMM is the first ICU prediction model that jointly learns both long-term disease dynamics (via attention) and different disease states in health trajectory (via a latent variable model). Our evaluations were based on an established baseline dataset (MIMIC-III) with 53,423 ICU stays. The results confirm that compared to state-of-the-art baselines, our AttDMM was superior: AttDMM achieved an area under the receiver operating characteristic curve (AUROC) of 0.876, which yielded an improvement over the state-of-the-art method by 2.2%. In addition, the risk score from the AttDMM provided warnings several hours earlier. Thereby, our model shows a path towards identifying patients at risk so that health practitioners can intervene early and save patient lives.
UAVs Path Planning under a Bi-Objective Optimization Framework for Smart Cities

Unmanned aerial vehicles (UAVs) have been used extensively for search and rescue operations, surveillance, disaster monitoring, attacking terrorists, etc. due to their growing advantages of low-cost, high maneuverability, and easy deployability. This study proposes a mixed-integer programming model under a multi-objective optimization framework to design trajectories that enable a set of UAVs to execute surveillance tasks. The first objective maximizes the cumulative probability of target detection to aim for mission planning success. The second objective ensures minimization of cumulative path length to provide a higher resource utilization goal. A two-step variable neighborhood search (VNS) algorithm is offered, which addresses the combinatorial optimization issue for determining the near-optimal sequence for cell visiting to reach the target. Numerical experiments and simulation results are evaluated in numerous benchmark instances. Results demonstrate that the proposed approach can favorably support practical deployability purposes.
Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization

Neural networks trained with SGD were recently shown to rely preferentially on linearly-predictive features and can ignore complex, equally-predictive ones. This simplicity bias can explain their lack of robustness out of distribution (OOD). The more complex the task to learn, the more likely it is that statistical artifacts (i.e. selection biases, spurious correlations) are simpler than the mechanisms to learn. We demonstrate that the simplicity bias can be mitigated and OOD generalization improved. We train a set of similar models to fit the data in different ways using a penalty on the alignment of their input gradients. We show theoretically and empirically that this induces the learning of more com-plex predictive patterns. OOD generalization fundamentally requires information beyond i. i.d. examples, such as multiple training environ-ments, counterfactual examples, or other side information. Our approach shows that we can defer this requirement to an independent model selection stage. We obtain SOTA re-sults in visual recognition on biased data and generalization across visual domains. The method - the first to evade the simplicity bias - highlights the need for a better under-standing and control of inductive biases in deep learning.
A Multi-Metric Ranking Approach for Library Migration Recommendations

The wide adoption of third-party libraries in software projects is beneficial but also risky. An already-adopted third-party library may be abandoned by its maintainers, may have license incompatibilities, or may no longer align with current project requirements. Under such circumstances, developers need to migrate the library to another library with similar functionalities, but the migration decisions are often opinion-based and sub-optimal with limited information at hand. Therefore, several filtering-based approaches have been proposed to mine library migrations from existing software data to leverage "the wisdom of crowd," but they suffer from either low precision or low recall with different thresholds, which limits their usefulness in supporting migration decisions. In this paper, we present a novel approach that utilizes multiple metrics to rank and therefore recommend library migrations. Given a library to migrate, our approach first generates candidate target libraries from a large corpus of software repositories, and then ranks them by combining the following four metrics to capture different dimensions of evidence from development histories: Rule Support, Message Support, Distance Support, and API Support. We evaluate the performance of our approach with 773 migration rules (190 source libraries) that we borrow from previous work and recover from 21,358 Java GitHub projects. The experiments show that our metrics are effective to help identify real migration targets, and our approach significantly outperforms existing works, with MRR of 0.8566, top-1 precision of 0.7947, top-10 NDCG of 0.7702, and top-20 recall of 0.8939. To demonstrate the generality of our approach, we manually verify the recommendation results of 480 popular libraries not included in prior work, and we confirm 661 new migration rules from 231 of the 480 libraries with comparable performance. The source code, data, and supplementary materials are provided at: https://github.com/hehao98/MigrationHelper.
Formative Education Online: Teaching the Whole Person During the Global COVID-19 Pandemic

The COVID-19 pandemic, an unprecedented public health emergency, challenged higher education and threatened students’ well-being in several ways. With the abrupt shift to online learning, were instructors able to maintain a focus on educating whole students, in addition to teaching subject matter? We answer this question by investigating “formative education,” an approach to teaching and learning that emphasizes holistic development, exploring formative education online during the pandemic. This qualitative study investigates the strategies of 37 college faculty who provided successful formative education online. A cross-subject analysis of data from faculty interviews and supplemental materials (course artifacts, course evaluations, student interviews) uncovered three teaching approaches that faculty used to achieve formative education online: empathic (centering students’ emotions), reflective (facilitating deep inquiry), and adaptive (having flexibility in meeting students’ needs). These approaches could help instructors design online education that engages the whole person.
IDA-GAN: A Novel Imbalanced Data Augmentation GAN

Class imbalance is a widely existed and challenging problem in real-world applications such as disease diagnosis, fraud detection, network intrusion detection and so on. Due to the scarce of data, it could significantly deteriorate the accuracy of classification. To address this challenge, we propose a novel Imbalanced Data Augmentation Generative Adversarial Networks (GAN) named IDA-GAN as an augmentation tool to deal with the imbalanced dataset. This is a great challenge because it is hard to train a GAN model under this situation. We address this issue by coupling variational autoencoder along with GAN training. In this paper, specifically, we introduce the variational autoencoder to learn the majority and minority class distributions in the latent space, and use the generative model to utilize each class distribution for the subsequent GAN training. The generative model learns useful features to generate target minority-class samples. Compared with the state-of-the-art GAN model, the experimental results demonstrate that our proposed IDA-GAN could generate more diverse minority samples with better qualities, and it could benefits the imbalanced classification task in terms of several widely-used evaluation metrics on five benchmark datasets: MNIST, Fashion-MNIST, SVHN, CIFAR-10 and GTSRB.
Sentiment Analysis Algorithm Based on BERT and Convolutional Neural Network

The explosion of Internet information makes timely analysis and monitoring of Internet public opinion more and more important, and sentiment analysis of public opinion events is even more important. The traditional word2vec model cannot fully express the information contained in words. It is proposed to use the BERT model as the article feature extraction model; and use the deep convolutional neural network to extract the local information of the article, and then connect the fully connected network to classify the article, so as to achieve sentiment analysis purpose. Experimental results on public data sets show that sentiment analysis algorithms based on BERT and convolutional neural networks are better than traditional sentiment analysis algorithms.
CESCR: CP-ABE for efficient and secure sharing of data in collaborative ehealth with revocation and no dummy attribute

With the rapid advancement of information and communication technologies, there is a growing transformation of healthcare systems. A patient's health data can now be centrally stored in the cloud and be shared with multiple healthcare stakeholders, enabling the patient to be collaboratively treated by more than one healthcare institution. However, several issues, including data security and privacy concerns still remain unresolved. Ciphertext-policy attribute-based encryption (CP-ABE) has shown promising potential in providing data security and privacy in cloud-based systems. Nevertheless, the conventional CP-ABE scheme is inadequate for direct adoption in a collaborative ehealth system. For one, its expressiveness is limited as it is based on a monotonic access structure. Second, it lacks an attribute/user revocation mechanism. Third, the computational burden on both the data owner and data users is linear with the number of attributes in the ciphertext. To address these inadequacies, we propose CESCR, a CP-ABE for efficient and secure sharing of health data in collaborative ehealth systems with immediate and efficient attribute/user revocation. The CESCR scheme is unbounded, i.e., it does not bind the size of the attribute universe to the security parameter, it is based on the expressive and non-restrictive ordered binary decision diagram (OBDD) access structure, and it securely outsources the computationally demanding attribute operations of both encryption and decryption processes without requiring a dummy attribute. Security analysis shows that the CESCR scheme is secure in the selective model. Simulation and performance comparisons with related schemes also demonstrate that the CESCR scheme is expressive and efficient.
Playing a Part: Speaker Verification at the movies

The goal of this work is to investigate the performance of popular speaker recognition models on speech segments from movies, where often actors intentionally disguise their voice to play a character. We make the following three contributions: (i) We collect a novel, challenging speaker recognition dataset called VoxMovies, with speech for 856 identities from almost 4000 movie clips. VoxMovies contains utterances with varying emotion, accents and background noise, and therefore comprises an entirely different domain to the interview-style, emotionally calm utterances in current speaker recognition datasets such as VoxCeleb; (ii) We provide a number of domain adaptation evaluation sets, and benchmark the performance of state-of-the-art speaker recognition models on these evaluation pairs. We demonstrate that both speaker verification and identification performance drops steeply on this new data, showing the challenge in transferring models across domains; and finally (iii) We show that simple domain adaptation paradigms improve performance, but there is still large room for improvement.
Multi-Task Learning for Sentiment Analysis with Hard-Sharing and Task Recognition Mechanisms

In the era of big data, multi-task learning has become one of the crucial technologies for sentiment analysis and classification. Most of the existing multi-task learning models for sentiment analysis are developed based on the soft-sharing mechanism that has less interference between different tasks than the hard-sharing mechanism. However, there are also fewer essential features that the model can extract with the soft-sharing method, resulting in unsatisfactory classification performance. In this paper, we propose a multi-task learning framework based on a hard-sharing mechanism for sentiment analysis in various fields. The hard-sharing mechanism is achieved by a shared layer to build the interrelationship among multiple tasks. Then, we design a task recognition mechanism to reduce the interference of the hard-shared feature space and also to enhance the correlation between multiple tasks. Experiments on two real-world sentiment classification datasets show that our approach achieves the best results and improves the classification accuracy over the existing methods significantly. The task recognition training process enables a unique representation of the features of different tasks in the shared feature space, providing a new solution reducing interference in the shared feature space for sentiment analysis.
Multi-modal visual adversarial Bayesian personalized ranking model for recommendation

• We propose two multi-modal datasets for recommendation. • We propose a novel recommendation system called the MVABPR . • We design a set of EMK features to boost recommendation performance. • We use cross-modal semantics and adversarial learning to promote performance. • The implicit feeling tone of a recommended item can be captured accurately. Recommendation system is facing the “data sparseness” issue. Additional information, including images, texts, and videos, contributes to alleviating this issue. We propose a new multi-modal visual adversarial Bayesian personalized ranking (MVABPR) model to address the issue. The proposed model takes new features, cross-modal semantics, adversarial learning, and visual interface into account. Two multi-modal datasets are created based on the MovieLens datasets and the correlated images. Besides the shape, texture, color, and deep learning-based features, a set of efficient match kernel features are proposed. More discriminative but low-dimensional cross-modal semantics among these features is mined to characterize each item effectively, which is absorbed into the MVABPR model through a visual interface. A new adversarial learning strategy is employed to optimize the whole training procedure. This makes the MVABPR model more robust and stable. Experimental results demonstrate that the MVABPR model is effective and robust for recommendation. It outperforms other competitive baselines. As another advantage, it can learn visual information and users’ rating jointly, effectively, combined with adversarial learning. And the implicit feeling tone of a recommended item can be accurately captured by the proposed model. More importantly, the model achieves better performance on a large-scale sparser dataset, demonstrating its higher practicality.
Neural Inverse Text Normalization

While there have been several contributions exploring state of the art techniques for text normalization, the problem of inverse text normalization (ITN) remains relatively unexplored. The best known approaches leverage finite state transducer (FST) based models which rely on manually curated rules and are hence not scalable. We propose an efficient and robust neural solution for ITN leveraging transformer based seq2seq models and FST-based text normalization techniques for data preparation. We show that this can be easily extended to other languages without the need for a linguistic expert to manually curate them. We then present a hybrid framework for integrating Neural ITN with an FST to overcome common recoverable errors in production environments. Our empirical evaluations show that the proposed solution minimizes incorrect perturbations (insertions, deletions and substitutions) to ASR output and maintains high quality even on out of domain data. A transformer based model infused with pretraining consistently achieves a lower WER across several datasets and is able to outperform baselines on English, Spanish, German and Italian datasets.
Improving Viewing Experiences of First-Person Shooter Gameplays with Automatically-Generated Motion Effects

In recent times, millions of people enjoy watching video gameplays at an eSports stadium or home. We seek a method that improves gameplay spectator or viewer experiences by presenting multisensory stimuli. Using a motion chair, we provide the motion effects automatically generated from the audiovisual stream to the viewers watching a first-person shooter (FPS) gameplay. The motion effects express the game character's movement and gunfire action. We describe algorithms for the computation of such motion effects developed using computer vision techniques and deep learning. By a user study, we demonstrate that our method of providing motion effects significantly improves the viewing experiences of FPS gameplay. The contributions of this paper are with the motion synthesis algorithms integrated for FPS games and the empirical evidence for the benefits of experiencing multisensory gameplays.
MFPP: Morphological Fragmental Perturbation Pyramid for Black-Box Model Explanations

Deep neural networks (DNNs) have recently been applied and used in many advanced and diverse tasks, such as medical diagnosis, automatic driving, etc. Due to the lack of transparency of the deep models, DNNs are often criticized for their prediction that cannot be explainable by human. In this paper, we propose a novel Morphological Fragmental Perturbation Pyramid (MFPP) method to solve the Explainable AI problem. In particular, we focus on the black-box scheme, which can identify the input area that is responsible for the output of the DNN without having to understand the internal architecture of the DNN. In the MFPP method, we divide the input image into multi-scale fragments and randomly mask out fragments as perturbation to generate a saliency map, which indicates the significance of each pixel for the prediction result of the black box model. Compared with the existing input sampling perturbation method, the pyramid structure fragment has proved to be more effective. It can better explore the morphological information of the input image to match its semantic information, and does not need any value inside the DNN. We qualitatively and quantitatively prove that MFPP meets and exceeds the performance of state-of-the-art (SOTA) black-box interpretation method on multiple DNN models and datasets.
Braving Citational Justice in Human-Computer Interaction

Citations are central to the production and sharing of knowledge, and how, why, and where citations are used has been an intense subject of study across disciplines. We discuss citational practices and the politics of knowledge production within the field of Human-Computer Interaction (HCI), drawing on parallels from related fields, and reflecting on our own experiences of being cited and not cited, citing and not citing. We also present recommendations for making concrete changes across the individual and the structural in HCI, related to how citations are viewed, and how the field might advance in solidarity towards greater citational justice.