Reviewerly/exHarmony huggingface dataset:
One Model to Serve All

Traditional industry recommendation systems usually use data in a single domain to train models and then serve the domain. However, a large-scale commercial platform often contains multiple domains, and its recommendation system often needs to make click-through rate (CTR) predictions for multiple domains. Generally, different domains may share some common user groups and items, and each domain may have its own unique user groups and items. Moreover, even the same user may have different behaviors in different domains. In order to leverage all the data from different domains, a single model can be trained to serve all domains. However, it is difficult for a single model to capture the characteristics of various domains and serve all domains well. On the other hand, training an individual model for each domain separately does not fully use the data from all domains. In this paper, we propose the Star Topology Adaptive Recommender (STAR) model to train a single model to serve all domains by leveraging data from all domains simultaneously, capturing the characteristics of each domain, and modeling the commonalities between different domains. Essentially, the net- work of each domain consists of two factorized networks: one centered network shared by all domains and the domain-specific network tailored for each domain. For each domain, we combine these two factorized networks and generate a unified network by element-wise multiplying the weights of the shared network and those of the domain-specific network, although these two factorized networks can be combined using other functions, which is open for further research. Most importantly, STAR can learn the shared network from all the data and adapt domain-specific parameters according to the characteristics of each domain. The experimental results from production data validate the superiority of the proposed STAR model. Since late 2020, STAR has been deployed in the display advertising system of Alibaba, obtaining 8.0% improvement on CTR and 6.0% increase on RPM (Revenue Per Mille).
ZeroNAS: Differentiable Generative Adversarial Networks Search for Zero-Shot Learning

In recent years, remarkable progress in zero-shot learning (ZSL) has been achieved by generative adversarial networks (GAN). To compensate for the lack of training samples in ZSL, a surge of GAN architectures have been developed by human experts through trial-and-error testing. Despite their efficacy, however, there is still no guarantee that these hand-crafted models can consistently achieve good performance across diversified datasets or scenarios. Accordingly, in this paper, we turn to neural architecture search (NAS) and make the first attempt to bring NAS techniques into the ZSL realm. Specifically, we propose a differentiable GAN architecture search method over a specifically designed search space for zero-shot learning, referred to as ZeroNAS. Considering the relevance and balance of the generator and discriminator, ZeroNAS jointly searches their architectures in a min-max player game via adversarial training. Extensive experiments conducted on four widely used benchmark datasets demonstrate that ZeroNAS is capable of discovering desirable architectures that perform favorably against state-of-the-art ZSL and generalized zero-shot learning (GZSL) approaches. Source code is at https://github.com/caixiay/ZeroNAS.
S<sup>2</sup>-MLP: Spatial-Shift MLP Architecture for Vision

Recently, visual Transformer (ViT) and its following works abandon the convolution and exploit the self-attention operation, attaining a comparable or even higher accuracy than CNN. More recently, MLP-mixer abandons both the convolution and the self-attention operation, proposing an architecture containing only MLP layers. To achieve cross-patch communications, it devises an additional token-mixing MLP besides the channel-mixing MLP. It achieves promising results when training on an extremely large-scale dataset such as JFT-300M. But it cannot achieve as outstanding performance as its CNN and ViT counterparts when training on medium-scale datasets such as ImageNet-1K. The performance drop of MLP-mixer motivates us to rethink the token-mixing MLP. We discover that token-mixing operation in MLP-mixer is a variant of depthwise convolution with a global reception field and spatial-specific configuration. In this paper, we propose a novel pure MLP architecture, spatial-shift MLP (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -MLP). Different from MLP-mixer, our S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -MLP only contains channel-mixing MLP. We devise a spatial-shift operation for achieving the communication between patches. It has a local reception field and is spatial-agnostic. Meanwhile, it is parameter-free and efficient for computation. The proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -MLP attains higher recognition accuracy than MLP-mixer when training on ImageNet1K dataset. Meanwhile, S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -MLP accomplishes as excellent performance as ViT on ImageNet-1K dataset with considerably simpler architecture and fewer FLOPs and parameters.
Evolutionary deep learning: A survey

As an advanced artificial intelligence technique for solving learning problems, deep learning (DL) has achieved great success in many real-world applications and attracted increasing attention in recent years. However, as the performance of DL depends on many factors such as the architecture and hyperparameters, how to optimize DL has become a hot research topic in the field of DL and artificial intelligence. Evolutionary computation (EC), including evolutionary algorithm and swarm intelligence, is a kind of efficient and intelligent optimization methodology inspired by the mechanisms of biological evolution and behaviors of swarm organisms. Therefore, a large number of researches have proposed EC algorithms to optimize DL, so called evolutionary deep learning (EDL), which have obtained promising results. Given the great progress and rapid development of EDL in recent years, it is quite necessary to review these developments in order to summarize previous research experiences and knowledge, as well as provide references to benefit the development of more researches and applications. For this aim, this paper categorizes existing works in a two-level taxonomy. The higher level includes four categories based on when the EC can be adopted in optimizing the DL, which are the four procedures of the whole DL lifetime, including data processing, model search, model training, and model evaluation and utilization. In the lower level, related works in each category are further classified according to the functionality and the aim of using EC in the corresponding DL procedure, i.e., why using EC in this DL procedure. As a result, the taxonomy can clearly show how an EC algorithm can be used to optimize and improve DL. Moreover, this survey also discusses the potential research directions to provide the prospect of EDL in the future.
MoNuSAC2020: A Multi-Organ Nuclei Segmentation and Classification Challenge

Detecting various types of cells in and around the tumor matrix holds a special significance in characterizing the tumor micro-environment for cancer prognostication and research. Automating the tasks of detecting, segmenting, and classifying nuclei can free up the pathologists' time for higher value tasks and reduce errors due to fatigue and subjectivity. To encourage the computer vision research community to develop and test algorithms for these tasks, we prepared a large and diverse dataset of nucleus boundary annotations and class labels. The dataset has over 46,000 nuclei from 37 hospitals, 71 patients, four organs, and four nucleus types. We also organized a challenge around this dataset as a satellite event at the International Symposium on Biomedical Imaging (ISBI) in April 2020. The challenge saw a wide participation from across the world, and the top methods were able to match inter-human concordance for the challenge metric. In this paper, we summarize the dataset and the key findings of the challenge, including the commonalities and differences between the methods developed by various participants. We have released the MoNuSAC2020 dataset to the public.
GroupViT: Semantic Segmentation Emerges from Text Supervision

Grouping and recognition are important components of visual scene understanding, e.g., for object detection and semantic segmentation. With end-to-end deep learning systems, grouping of image regions usually happens implicitly via top-down supervision from pixel-level recognition labels. Instead, in this paper, we propose to bring back the grouping mechanism into deep networks, which allows semantic segments to emerge automatically with only text supervision. We propose a hierarchical Grouping Vision Transformer (GroupViT), which goes beyond the regular grid structure representation and learns to group image regions into progressively larger arbitrary-shaped segments. We train GroupViT jointly with a text encoder on a large-scale image-text dataset via contrastive losses. With only text supervision and without any pixel-level annotations, GroupViT learns to group together semantic regions and successfully transfers to the task of semantic segmentation in a zero-shot manner, i.e., without any further fine-tuning. It achieves a zero-shot accuracy of 52.3% mIoU on the PASCAL VOC 2012 and 22.4% mIoU on PASCAL Context datasets, and performs competitively to state-of-the-art transfer-learning methods requiring greater levels of supervision. We open-source our code at https://github.com/NVlabs/GroupViT.
The intraday dynamics of bitcoin

Bitcoin has received much investor attention in recent years and following this, there has been an explosion of academic studies examining this new financial asset. We contribute to the growing literature of Bitcoin by examining the intraday variables of the leading Bitcoin exchange with the highest information share over 4 years’ worth of data to reveal the intraday stylized facts of Bitcoin and how they have developed over time. Employing GMT-timestamped tick data aggregated to the 5-mintuely frequency, we find that Bitcoin returns have increased over time, while trading volume, volatility and liquidity varied substantially over time. We also find that volume increases throughout the day and falls from around 4 pm until midnight, which is consistent with the intraday patterns found in currency markets. Realised volatility is fairly consistent throughout the day although it is highest during the opening times of the three major global stock markets. Also liquidity is highest during the opening times of the major global exchanges and the markets tend to be illiquid during the early morning. Finally, we show evidence of the mixture of distribution hypothesis of Clark (1973).
Renewable Energy-Aware Big Data Analytics in Geo-Distributed Data Centers with Reinforcement Learning

In the age of big data, companies tend to deploy their services in data centers rather than their own servers. The demands of big data analytics grow significantly, which leads to an extremely high electricity consumption at data centers. In this paper, we investigate the cost minimization problem of big data analytics on geo-distributed data centers connected to renewable energy sources with unpredictable capacity. To solve this problem, we propose a Reinforcement Learning (RL) based job scheduling algorithm by combining RL with neural network (NN). Moreover, two techniques are developed to enhance the performance of our proposal. Specifically, Random Pool Sampling (RPS) is proposed to retrain the NN via accumulated training data, and a novel Unidirectional Bridge Network (UBN) structure is designed for further enhancing the training speed by using the historical knowledge stored in the trained NN. Experiment results on real Google cluster traces and electricity price from Energy Information Administration show that our approach is able to reduce the data centers' cost significantly compared with other benchmark algorithms.
Memristor-Based Echo State Network With Online Least Mean Square

In this paper, we propose a novel computational architecture of memristor-based echo state network (MESN) with the online least mean square (LMS) algorithm. Newman and Watts small-world network is adopted for the topological structure of MESN network with memristive neural synapses. In the MESN network, the state matrix of the reservoir layer, which is obtained by raising the dimension of input data, is utilized as an input of the LMS algorithm to train the output weight matrix on chip. After certain iterations, the resistance value of memristor is adjusted to a constant. Thus, the final weight output matrix is obtained. To verify the effectiveness of the proposed MESN network, car evaluation and short-term power load forecasting are employed with the effect evaluation of the node number and the connectivity degree of the reservoir layer. The research provides a novel way to design neuromorphic computing systems.
Improving bug localization with word embedding and enhanced convolutional neural networks

Context: Automatic localization of buggy files can speed up the process of bug fixing to improve the efficiency and productivity of software quality assurance teams. Useful semantic information is available in bug reports and source code, but it is usually underutilized by existing bug localization approaches. Objective: To improve the performance of bug localization, we propose DeepLoc, a novel deep learning-based model that makes full use of semantic information. Method: DeepLoc is composed of an enhanced convolutional neural network (CNN) that considers bug-fixing recency and frequency, together with word-embedding and feature-detecting techniques. DeepLoc uses word embeddings to represent the words in bug reports and source files that retain their semantic information, and different CNNs to detect features from them. DeepLoc is evaluated on over 18,500 bug reports extracted from AspectJ, Eclipse, JDT, SWT, and Tomcat projects. Results: The experimental results show that DeepLoc achieves 10.87%–13.4% higher MAP (mean average precision) than conventional CNN. DeepLoc outperforms four current state-of-the-art approaches (DeepLocator, HyLoc, LR+WE, and BugLocator) in terms of [email protected] (the percentage of bug reports for which at least one real buggy file is located within the top k rank), MAP, and MRR (mean reciprocal rank) using less computation time. Conclusion: DeepLoc is capable of automatically connecting bug reports to the corresponding buggy files and achieves better performance than four state-of-the-art approaches based on a deep understanding of semantics in bug reports and source code.
R<sup>3</sup>-Net: A Deep Network for Multioriented Vehicle Detection in Aerial Images and Videos

Vehicle detection is a significant and challenging task in aerial remote sensing applications. Most existing methods detect vehicles with regular rectangle boxes and fail to offer the orientation of vehicles. However, the orientation information is crucial for several practical applications, such as the trajectory and motion estimation of vehicles. In this paper, we propose a novel deep network, called rotatable region-based residual network (R$^3$-Net), to detect multi-oriented vehicles in aerial images and videos. More specially, R$^3$-Net is utilized to generate rotatable rectangular target boxes in a half coordinate system. First, we use a rotatable region proposal network (R-RPN) to generate rotatable region of interests (R-RoIs) from feature maps produced by a deep convolutional neural network. Here, a proposed batch averaging rotatable anchor (BAR anchor) strategy is applied to initialize the shape of vehicle candidates. Next, we propose a rotatable detection network (R-DN) for the final classification and regression of the R-RoIs. In R-DN, a novel rotatable position sensitive pooling (R-PS pooling) is designed to keep the position and orientation information simultaneously while downsampling the feature maps of R-RoIs. In our model, R-RPN and R-DN can be trained jointly. We test our network on two open vehicle detection image datasets, namely DLR 3K Munich Dataset and VEDAI Dataset, demonstrating the high precision and robustness of our method. In addition, further experiments on aerial videos show the good generalization capability of the proposed method and its potential for vehicle tracking in aerial videos. The demo video is available at https://youtu.be/xCYD-tYudN0.
Dual-Stream Recurrent Neural Network for Video Captioning

Recent progress in using recurrent neural networks (RNNs) for video description has attracted an increasing interest, due to its capability to encode a sequence of frames for caption generation. While existing methods have studied various features (e.g., CNN, 3D CNN, and semantic attributes) for visual encoding, the representation and fusion of heterogeneous information from multi-modal spaces have not fully explored. Consider that different modalities are often asynchronous, frame-level multi-modal fusion (e.g., concatenation and linear fusion) will negatively influence each modality. In this paper, we propose a dual-stream RNN (DS-RNN) framework to jointly discover and integrate the hidden states of both visual and semantic streams for video caption generation. First, an encoding RNN is used for each stream to flexibly exploit the hidden states of respective modality. Specifically, we proposed an attentive multi-grained encoder module to enhance the local feature learning with global semantics feature. Then, a dual-stream decoder is deployed to integrate the asynchronous yet complementary sequential hidden states from both streams for caption generation. Extensive experiments on three benchmark datasets, namely, MSVD, MSR-VTT, and MPII-MD, show that DS-RNN achieves competitive performance against the state-of-the-art. Additional ablation studies were conducted on various variants of the proposed DS-RNN.
A large-group emergency risk decision method based on data mining of public attribute preferences

Aiming at the lack of public concern about the attributes of large-group emergency decision making in major emergencies, a large-group emergency risk decision method based on data mining of public attribute preferences is proposed. First, according to historical data from a similar emergency, an attribute-keyword lexicon is established using text mining and latent semantic analysis. Meanwhile, fuzzy association rule mining and a fuzzy cognitive graph are applied to obtain information regarding public opinion about the attributes. Then, a risk measurement model of degree indicators based on information entropy is presented, and according to the linguistic preference for information given by large-group members, the decision risk of large-group members is measured. On this basis, a group member clustering algorithm is constructed to cluster large groups, and then the best alternative is acquired by comparing score function and precise function of the interval intuitionistic fuzzy number. Finally, a case application using historical data is conducted to verify the method’s rationality and feasibility.
Improving the Performance of Fuzzy Rule-Based Classification Systems Based on a Nonaveraging Generalization of CC-Integrals Named $C_{F_1F_2}$-Integrals

A key component of fuzzy rule-based classification systems (FRBCS) is the fuzzy reasoning method (FRM) since it infers the class predicted for new examples. A crucial stage in any FRM is the way in which the information given by the fired rules during the inference process is aggregated. A widely used FRM is the winning rule, which applies the maximum to accomplish this aggregation. The maximum is an averaging operator, which means that its result is within the range delimited by the minimum and the maximum of the aggregated values. Recently, new averaging operators based on generalizations of the Choquet integral have been proposed to perform this aggregation process. However, the most accurate FRBCSs use the FRM known as additive combination that considers the normalized sum as the aggregation operator, which is nonaveraging. For this reason, this paper is aimed at introducing a new nonaveraging operator named C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1F2</sub> -integral, which is a generalization of the Choquet-like Copula-based integral (CC-integral). C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1F2</sub> -integrals present the desired properties of an aggregation-like operator since they satisfy appropriate boundary conditions and have some kind of increasingness property. We show that C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1F2</sub> -integrals, when used to cope with classification problems, enhance the results of the previous averaging generalizations of the Choquet integral and provide competitive results (even better) when compared with state-of-the-art FRBCSs.
Triple Verification Network for Generalized Zero-Shot Learning

Conventional Zero-shot Learning approaches often suffer from severe performance degradation in the Generalised Zero-shot Learning (GZSL) scenario, i.e. to recognise test images that are from both seen and unseen classes. This paper studies the Class-level Over-fitting (CO) and empirically shows its effects to GZSL. We then address ZSL as a Triple Verification problem and propose a unified optimisation of regression and compatibility functions, i.e. two main streams of existing ZSL approaches. The complementary losses mutually regularise the same model to mitigate the CO problem. Furthermore, we implement a deep extension paradigm to linear models and significantly outperforms state-of-the-art methods in both GZSL and ZSL scenarios on the four standard benchmarks.
Multi-view intact space clustering

Multi-view clustering is a hot research topic due to the urgent need for analyzing a vast amount of heterogeneous data. Although many multi-view clustering methods have been developed, yet they mostly neglect the view-insufficiency issue. That is, most of the existing multi-view clustering methods assume that each individual view is sufficient for discovering the cluster structure, which is however not guaranteed in real applications. In this paper, we propose a novel multi-view clustering method termed multi-view intact space clustering (MVIC), which is able to simultaneously recover the latent intact space from multiple insufficient views and discover the cluster structure from the intact space. For each view, a view generation function is designed to map the latent intact space representation into the view representation. With the view representation given, the latent intact space can be restored by mapping back from each individual view representation. Then matrix factorization is used to obtain the cluster structure in intact space by factorizing the latent intact space representation into the cluster centroids and the cluster assignments. Therefore, the proposed model is composed of two components, namely the reconstruction error of the latent intact space and the distortion error of data clustering in intact space. An alternating iterative method is designed to solve the optimization of the model, in which the latent intact space and the cluster structure are updated in an interplay manner. Experimental results conducted on a wide-range of multi-view datasets have confirmed the superiority of our method over state-of-the-art approaches.