Reviewerly/exHarmony huggingface dataset:
ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table

Because of the superior feature representation ability of deep learning, various deep Click-Through Rate (CTR) models are deployed in the commercial systems by industrial companies. To achieve better performance, it is necessary to train the deep CTR models on huge volume of training data efficiently, which makes speeding up the training process an essential problem. Different from the models with dense training data, the training data for CTR models is usually high-dimensional and sparse. To transform the high-dimensional sparse input into low-dimensional dense real-value vectors, almost all deep CTR models adopt the embedding layer, which easily reaches hundreds of GB or even TB. Since a single GPU cannot afford to accommodate all the embedding parameters, when performing distributed training, it is not reasonable to conduct the data-parallelism only. Therefore, existing distributed training platforms for recommendation adopt model-parallelism. Specifically, they use CPU (Host) memory of servers to maintain and update the embedding parameters and utilize GPU worker to conduct forward and backward computations. Unfortunately, these platforms suffer from two bottlenecks: (1) the latency of pull & push operations between Host and GPU; (2) parameters update and synchronization in the CPU servers. To address such bottlenecks, in this paper, we propose the ScaleFreeCTR: a MixCache-based distributed training system for CTR models. Specifically, in SFCTR, we also store huge embedding table in CPU but utilize GPU instead of CPU to conduct embedding synchronization efficiently. To reduce the latency of data transfer between both GPU-Host and GPU-GPU, the MixCache mechanism and Virtual Sparse Id operation are proposed. Comprehensive experiments are conducted to demonstrate the effectiveness and efficiency of SFCTR. In addition, our system will be open-source based on MindSpore in the near future.
Good Bot, Bad Bot: Characterizing Automated Browsing Activity

As the web keeps increasing in size, the number of vulnerable and poorly-managed websites increases commensurately. Attackers rely on armies of malicious bots to discover these vulnerable websites, compromising their servers, and exfiltrating sensitive user data. It is, therefore, crucial for the security of the web to understand the population and behavior of malicious bots.In this paper, we report on the design, implementation, and results of Aristaeus, a system for deploying large numbers of "honeysites", i.e., websites that exist for the sole purpose of attracting and recording bot traffic. Through a seven-month-long experiment with 100 dedicated honeysites, Aristaeus recorded 26.4 million requests sent by more than 287K unique IP addresses, with 76,396 of them belonging to clearly malicious bots. By analyzing the type of requests and payloads that these bots send, we discover that the average honeysite received more than 37K requests each month, with more than 50% of these requests attempting to brute-force credentials, fingerprint the deployed web applications, and exploit large numbers of different vulnerabilities. By comparing the declared identity of these bots with their TLS handshakes and HTTP headers, we uncover that more than 86.2% of bots are claiming to be Mozilla Firefox and Google Chrome, yet are built on simple HTTP libraries and command-line tools.
Indicative conditionals: probabilities and relevance

Abstract We propose a new account of indicative conditionals, giving acceptability and logical closure conditions for them. We start from Adams’ Thesis: the claim that the acceptability of a simple indicative equals the corresponding conditional probability. The Thesis is widely endorsed, but arguably false and refuted by empirical research. To fix it, we submit, we need a relevance constraint: we accept a simple conditional $$\varphi \rightarrow \psi$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>φ</mml:mi> <mml:mo>→</mml:mo> <mml:mi>ψ</mml:mi> </mml:mrow> </mml:math> to the extent that (i) the conditional probability $$\mathrm{p}(\psi |\varphi )$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>p</mml:mi> <mml:mo>(</mml:mo> <mml:mi>ψ</mml:mi> <mml:mo>|</mml:mo> <mml:mi>φ</mml:mi> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> is high, provided that (ii) $$\varphi$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>φ</mml:mi> </mml:math> is relevant for $$\psi$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ψ</mml:mi> </mml:math> . How (i) should work is well-understood. It is (ii) that holds the key to improve our understanding of conditionals. Our account has (i) a probabilistic component, using Popper functions; (ii) a relevance component, given via an algebraic structure of topics or subject matters. We present a probabilistic logic for simple indicatives, and argue that its (in)validities are both theoretically desirable and in line with empirical results on how people reason with conditionals.
Adversarial attacks through architectures and spectra in face recognition

The ability of Deep Neural Networks (DNNs) to make fast predictions with high accuracy made them very popular in real-time applications. DNNs are nowadays in use for secure access to services or mobile devices. However, as DNNs use increased, at the same time attack techniques are born to “break” them. This paper presents a particular way to fool DNNs by moving from one spectrum to another one. The application field we explore is face recognition. The attack is first built on a trained Face DNN on Visible, Near Infrared or Thermal images, then transposed to another spectrum to fool another DNN. The attacks performed are based on the Fast Gradient Sign Method with the aim to misclassify the subject knowing the DNN to attack (White-Box Attack) but without knowing the DNN on which the attack will be transposed (Black-Box Attack). Results show that this cross-spectral attack is able to fool the most popular DNN architectures. In worst cases the DNN becomes useless to perform face recognition after the attack.
A cross-modal fusion based approach with scale-aware deep representation for RGB-D crowd counting and density estimation

To simultaneously estimate the crowd count and the density map from the crowd images, this paper presents a novel cross-modal fusion based approach for RGB-D crowd counting. For the RGB-D crowd counting task, the depth data is often utilized into the procedure of detecting heads of the crowd to enhance the counting performance so as to reduce the underestimation from the small heads of the crowd. Different from the traditional methods utilizing depth image for the target task, the proposed approach is essentially designed as a density estimation-based regression framework to learn the more abundant deep representation from the original images through cross-modal interactions in multiple locations of the framework, which is more beneficial to crowd counting in various scenes, especially the congested scenes. Meanwhile, modeling the global and local contexts is designed to facilitate the proposed approach to learn the more adequate scale-aware representation for the counting task. Extensive experiments on MICC and large-scale ShanghaiRGBD benchmarks demonstrate that the performance of the proposed approach is superior to the state-of-the-art methods for RGB-D crowd counting and density estimation. Further, the proposed approach could be extended to RGB crowd counting task and the experimental results show that it achieves the comparable performance with the existing crowd counting methods.
Unsupervised Monocular Depth Estimation via Recursive Stereo Distillation

Existing unsupervised monocular depth estimation methods resort to stereo image pairs instead of ground-truth depth maps as supervision to predict scene depth. Constrained by the type of monocular input in testing phase, they fail to fully exploit the stereo information through the network during training, leading to the unsatisfactory performance of depth estimation. Therefore, we propose a novel architecture which consists of a monocular network (Mono-Net) that infers depth maps from monocular inputs, and a stereo network (Stereo-Net) that further excavates the stereo information by taking stereo pairs as input. During training, the sophisticated Stereo-Net guides the learning of Mono-Net and devotes to enhance the performance of Mono-Net without changing its network structure and increasing its computational burden. Thus, monocular depth estimation with superior performance and fast runtime can be achieved in testing phase by only using the lightweight Mono-Net. For the proposed framework, our core idea lies in: 1) how to design the Stereo-Net so that it can accurately estimate depth maps by fully exploiting the stereo information; 2) how to use the sophisticated Stereo-Net to improve the performance of Mono-Net. To this end, we propose a recursive estimation and refinement strategy for Stereo-Net to boost its performance of depth estimation. Meanwhile, a multi-space knowledge distillation scheme is designed to help Mono-Net amalgamate the knowledge and master the expertise from Stereo-Net in a multi-scale fashion. Experiments demonstrate that our method achieves the superior performance of monocular depth estimation in comparison with other state-of-the-art methods.
Smile or Scowl? Looking at Infographic Design Through the Affective Lens

Infographics are frequently promoted for their ability to communicate data to audiences affectively. To facilitate the creation of affect-stirring infographics, it is important to characterize and understand people's affective responses to infographics and derive practical design guidelines for designers. To address these research questions, we first conducted two crowdsourcing studies to identify 12 infographic-associated affective responses and collect user feedback explaining what triggered affective responses in infographics. Then, by coding the user feedback, we present a taxonomy of design heuristics that exemplifies the affect-related design factors in infographics. We evaluated the design heuristics with 15 designers. The results showed that our work supports assessing the affective design in infographics and facilitates the ideation and creation of affective infographics.
NN-EMD: Efficiently Training <i>Neural Networks</i> Using <i>Encrypted Multi-Sourced Datasets</i>

Training complex neural network models using third-party cloud-based infrastructure among multiple data sources is a promising approach among existing machine learning solutions. However, privacy concerns of large-scale data collections and recent regulations have restricted the availability and use of privacy sensitive data in the third-party infrastructure. To address such privacy issues, a promising emerging approach is to train a neural network model over an encrypted dataset. Specifically, the model training process can be outsourced to a third party such as a cloud service that is backed by significant computing power, while the encrypted training data keeps the data confidential from the third party. Compared to training a traditional machine learning model over encrypted data, however, it is extremely challenging to train a deep neural network (DNN) model over encrypted data for two reasons: first, it requires large-scale computation over huge datasets; second, the existing solutions for computation over encrypted data, such as using homomorphic encryption, is inefficient. Further, for enhanced performance of a DNN model, we also need to use huge training datasets composed of data from multiple data sources that may not have pre-established trust relationships among each other. We propose a novel framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NN-EMD</i> , to train DNN over <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">encrypted multiple datasets</i> collected from multiple sources. Toward this, we propose a set of secure computation protocols using hybrid functional encryption schemes. We evaluate our framework for performance with regards to the training time and model accuracy on the MNIST datasets. We show that compared to other existing frameworks, our proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NN-EMD</i> framework can significantly reduce the training time, while providing comparable model accuracy and privacy guarantees as well as supporting multiple data sources. Furthermore, the depth and complexity of neural networks do not affect the training time despite introducing a privacy-preserving <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NN-EMD</i> setting.
Developing and Using BioSIM<sup>AR</sup>, an Augmented Reality Program to Visualize and Learn about Chemical Structures in a Virtual Environment on Any Internet-Connected Device

Visualization can be a motivating way to teach students about molecules. Nowadays, the available experimental data and accurate computational results allow students to build realistic and accurate molecular models. These models include the representation of complex systems such as proteins, membranes, or nanotubes. However, the visualization of these three-dimensional (3D) structures can be challenging, complex, or even abstract for most people. To make the visualization easier and more motivating, BioSIM Augmented Reality (BioSIMAR) was developed as a free online software program to provide a new way to visualize and interact with 3D molecular models in a virtual environment. Using augmented reality (AR) and quick response code technologies, BioSIMAR provides tangible models of molecules to help illustrate chemical concepts. BioSIMAR runs on any electronic device, including mobile phones, tablets, laptops, or desktop computers with a camera and an internet connection, without requiring the installation of additional software. This software program helps people visualize and understand concepts about atoms and molecules, and their characteristics. BioSIMAR is freely available at https://ar.biosim.pt/.
The Future of Computational Linguistics: On Beyond Alchemy

Over the decades, fashions in Computational Linguistics have changed again and again, with major shifts in motivations, methods and applications. When digital computers first appeared, linguistic analysis adopted the new methods of information theory, which accorded well with the ideas that dominated psychology and philosophy. Then came formal language theory and the idea of AI as applied logic, in sync with the development of cognitive science. That was followed by a revival of 1950s-style empiricism—AI as applied statistics—which in turn was followed by the age of deep nets. There are signs that the climate is changing again, and we offer some thoughts about paths forward, especially for younger researchers who will soon be the leaders.
Minimizing Training Time of Distributed Machine Learning by Reducing Data Communication

Due to the additive property of most machine learning objective functions, the training can be distributed to multiple machines. Distributed machine learning is an efficient way to deal with the rapid growth of data volume at the cost of extra inter-machine communication. One common implementation is the parameter server system which contains two types of nodes: worker nodes, which are used for calculating updates, and server nodes, which are used for maintaining parameters. We observe that inefficient communication between workers and servers may slow down the system. Therefore, we propose a graph partition problem to partition data among workers and parameters among servers such that the total training time is minimized. Our problem is NP-Complete. We investigate a two-step heuristic approach that first partitions data, and then partitions parameters. We consider the trade-off between partition time and the saving in training time. Besides, we adopt a multilevel graph partition approach to fit the bipartite graph partitioning. We implement both approaches based on an open-source parameter server platform-PS-lite. Experiment results on synthetic and real-world datasets show that both approaches could significantly improve the communication efficiency up to 14 times compared with the random partition.
Distilling Knowledge from Publicly Available Online EMR Data to Emerging Epidemic for Prognosis

Due to the characteristics of COVID-19, the epidemic develops rapidly and overwhelms health service systems worldwide. Many patients suffer from life-threatening systemic problems and need to be carefully monitored in ICUs. An intelligent prognosis can help physicians take an early intervention, prevent adverse outcomes, and optimize the medical resource allocation, which is urgently needed, especially in this ongoing global pandemic crisis. However, in the early stage of the epidemic outbreak, the data available for analysis is limited due to the lack of effective diagnostic mechanisms, the rarity of the cases, and privacy concerns. In this paper, we propose a distilled transfer learning framework, which leverages the existing publicly available online Electronic Medical Records to enhance the prognosis for inpatients with emerging infectious diseases. It learns to embed the COVID-19-related medical features based on massive existing EMR data. The transferred parameters are further trained to imitate the teacher model’s representation based on distillation, which embeds the health status more comprehensively on the source dataset. We conduct Length-of-Stay prediction experiments for patients in ICUs on real-world COVID-19 datasets. The experiment results indicate that our proposed model consistently outperforms competitive baseline methods. In order to further verify the scalability of o deal with different clinical tasks on different EMR datasets, we conduct an additional mortality prediction experiment on End-Stage Renal Disease datasets. The extensive experiments demonstrate that an benefit the prognosis for emerging pandemics and other diseases with limited EMR.
AMIS: Edge Computing Based Adaptive Mobile Video Streaming

This work proposes AMIS, an edge computing-based adaptive video streaming system. AMIS explores the power of edge computing in three aspects. First, with video contents pre-cached in the local buffer, AMIS is content-aware which adapts the video playout strategy based on the scene features of video contents and quality of experience (QoE) of users. Second, AMIS is channel-aware which measures the channel conditions in real-time and estimates the wireless bandwidth. Third, by integrating the content features and channel estimation, AMIS applies the deep reinforcement learning model to optimize the playout strategy towards the best QoE. Therefore, AMIS is an intelligent content- and channel-aware scheme which fully explores the intelligence of edge computing and adapts to general environments and QoE requirements. Using trace-driven simulations, we show that AMIS can succeed in improving the average QoE by 14%-46% as compared to the state-of-the-art adaptive bitrate algorithms.
Adversarial-Enhanced Hybrid Graph Network for User Identity Linkage

In this work, we investigate the user identity linkage task across different social media platforms based on heterogeneous multi-modal posts and social connections. This task is non-trivial due to the following two challenges. 1) As each user involves both intra multi-modal posts and inter social connections, how to accurately fulfil the user representation learning from both intra and inter perspectives constitutes the main challenge. And 2) even representations distributed on different platforms of the same identity tend to be distinct (i.e., the semantic gap problem) owing to discrepant data distribution of different platforms. Hence, how to alleviate the semantic gap problem poses another tough challenge. To this end, we propose a novel adversarial-enhanced hybrid graph network (AHG-Net), consisting of three key components: user representation extraction, hybrid user representation learning, and adversarial learning. Specifically, AHG-Net first employs advanced deep learning techniques to extract the user's intermediate representations from his/her heterogeneous multi-modal posts and social connections. Then AHG-Net unifies the intra-user representation learning and inter-user representation learning with a hybrid graph network. Finally, AHG-Net adopts adversarial learning to encourage the learned user presentations of the same identity to be similar using a semantic discriminator. Towards evaluation, we create a multi-modal user identity linkage dataset by augmenting an existing dataset with 62,021 images collected from Twitter and Foursquare. Extensive experiments validate the superiority of the proposed network. Meanwhile, we release the dataset, codes, and parameters to facilitate the research community.
Conditional Uncorrelation and Efficient Subset Selection in Sparse Regression

Given m d -dimensional responsors and n d -dimensional predictors, sparse regression finds at most k predictors for each responsor for linear approximation, 1 ≤ k ≤ d-1 . The key problem in sparse regression is subset selection, which usually suffers from high computational cost. In recent years, many improved approximate methods of subset selection have been published. However, less attention has been paid to the nonapproximate method of subset selection, which is very necessary for many questions in data analysis. Here, we consider sparse regression from the view of correlation and propose the formula of conditional uncorrelation. Then, an efficient nonapproximate method of subset selection is proposed in which we do not need to calculate any coefficients in the regression equation for candidate predictors. By the proposed method, the computational complexity is reduced from O([1/6]k3+(m+1)k2+mkd) to O([1/6]k3+[1/2](m+1)k2) for each candidate subset in sparse regression. Because the dimension d is generally the number of observations or experiments and large enough, the proposed method can greatly improve the efficiency of nonapproximate subset selection. We also apply the proposed method in real scenarios of dental age assessment and sparse coding to validate the efficiency of the proposed method.
Stochastic mutual information gradient estimation for dimensionality reduction networks

Feature ranking and selection is a widely used approach in various applications of supervised dimensionality reduction in discriminative machine learning. Nevertheless there exists significant evidence on feature ranking and selection algorithms based on any criterion leading to potentially sub-optimal solutions for class separability. In that regard, we introduce emerging information theoretic feature transformation protocols as an end-to-end neural network training approach. We present a dimensionality reduction network (MMINet) training procedure based on the stochastic estimate of the mutual information gradient. The network projects high-dimensional features onto an output feature space where lower dimensional representations of features carry maximum mutual information with their associated class labels. Furthermore, we formulate the training objective to be estimated non-parametrically with no distributional assumptions. We experimentally evaluate our method with applications to high-dimensional biological data sets, and relate it to conventional feature selection algorithms to form a special case of our approach.