Reviewerly/exHarmony huggingface dataset:
Non-Markovian features in semiconductor quantum optics: quantifying the role of phonons in experiment and theory

Abstract We discuss phonon-induced non-Markovian and Markovian features in QD-based quantum nanooptics. We cover lineshapes in linear absorption experiments, phonon-induced incoherence in the Heitler regime, and memory correlations in two-photon coherences. To qualitatively and quantitatively understand the underlying physics, we present several theoretical models that capture the non-Markovian properties of the electron–phonon interaction accurately in different regimes. Examples are the Heisenberg equation of motion approach, the polaron master equation, and Liouville propagator techniques in the independent boson limit and beyond via the path integral method. Phenomenological modeling overestimates typically the dephasing due to the finite memory kernel of phonons and we give instructive examples of phonon-mediated coherence such as phonon-dressed anticrossings in Mollow physics, robust quantum state preparation, cavity feeding, and the stabilization of the collapse and revival phenomenon in the strong coupling limit of cavity quantum electrodynamics.
Facial Expression Recognition Using Frequency Neural Network

Facial expression recognition has become a newly-emerging topic in recent decades, which has important value in the field of human-computer interaction. In this paper, we present a deep learning based approach, named frequency neural network (FreNet), for facial expression recognition. Different from convolutional neural network in spatial domain, FreNet inherits the advantages of processing image in frequency domain, such as efficient computation and spatial redundancy elimination. First, we propose the learnable multiplication kernel and construct multiple multiplication layers to learn features in frequency domain. Second, a summarization layer is proposed following multiplication layers to further yield high-level features. Third, based on the property of discrete cosine transform (DCT), we utilize multiplication layers and summarization layer to construct the Basic-FreNet, which can yield high-level features on the widely used DCT feature. Finally, to further achieve better performance on Basic-FreNet, we propose the Block-FreNet in which the weight-shared multiplication kernel is designed for feature learning and the block sub-sampling is designed for dimension reduction. The experimental results show that the Block-FreNet not only achieves superior performance, but also greatly reduces the computational cost. To our best knowledge, the proposed approach is the first attempt to fill in the blank of frequency based deep learning model for facial expression recognition.
Modulation format identification in fiber communications using single dynamical node-based photonic reservoir computing

We present a simple approach based on photonic reservoir computing (P-RC) for modulation format identification (MFI) in optical fiber communications. Here an optically injected semiconductor laser with self-delay feedback is trained with the representative features from the asynchronous amplitude histograms of modulation signals. Numerical simulations are conducted for three widely used modulation formats (on–off keying, differential phase-shift keying, and quadrature amplitude modulation) for various transmission situations where the optical signal-to-noise ratio varies from 12 to 26 dB, the chromatic dispersion varies from <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="m1"> <mml:mrow> <mml:mo form="prefix">−</mml:mo> <mml:mn>500</mml:mn> </mml:mrow> </mml:math> to 500 ps/nm, and the differential group delay varies from 0 to 20 ps. Under these situations, final simulation results demonstrate that this technique can efficiently identify all those modulation formats with an accuracy of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="m2"> <mml:mrow> <mml:mo form="prefix">&gt;</mml:mo> <mml:mn>95</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> </mml:math> after optimizing the control parameters of the P-RC layer such as the injection strength, feedback strength, bias current, and frequency detuning. The proposed technique utilizes very simple devices and thus offers a resource-efficient alternative approach to MFI.
RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition

Video action recognition is a complex task dependent on modeling spatial and temporal context. Standard approaches rely on 2D or 3D convolutions to process such context, resulting in expensive operations with millions of parameters. Recent efficient architectures leverage a channel-wise shift-based primitive as a replacement for temporal convolutions, but remain bottlenecked by spatial convolution operations to maintain strong accuracy and a fixed-shift scheme. Naively extending such developments to a 3D setting is a difficult, intractable goal. To this end, we introduce RubiksNet, a new efficient architecture for video action recognition which is based on a proposed learnable 3D spatiotemporal shift operation instead. We analyze the suitability of our new primitive for video action recognition and explore several novel variations of our approach to enable stronger representational flexibility while maintaining an efficient design. We benchmark our approach on several standard video recognition datasets, and observe that our method achieves comparable or better accuracy than prior work on efficient video action recognition at a fraction of the performance cost, with 2.9–5.9 $$\times $$ fewer parameters and 2.1–3.7 $$\times $$ fewer FLOPs. We also perform a series of controlled ablation studies to verify our significant boost in the efficiency-accuracy tradeoff curve is rooted in the core contributions of our RubiksNet architecture.
Challenges and research opportunities in eCommerce search and recommendations

With the rapid adoption of online shopping, academic research in the eCommerce domain has gained traction. However, significant research challenges remain, spanning from classic eCommerce search problems such as matching textual queries to multi-modal documents and ranking optimization for two-sided marketplaces to human-computer interaction and recommender systems for discovery and browsing. These research areas are important for understanding customer behavior, driving engagement, and improving product discoverability and conversion. In this article we identify the challenges and highlight research opportunities to improve the eCommerce customer experience.
A substrate-independent framework to characterize reservoir computers

The Reservoir Computing (RC) framework states that any non-linear, input-driven dynamical system (the reservoir) exhibiting properties such as a fading memory and input separability can be trained to perform computational tasks. This broad inclusion of systems has led to many new physical substrates for RC. Properties essential for reservoirs to compute are tuned through reconfiguration of the substrate, such as change in virtual topology or physical morphology. As a result, each substrate possesses a unique `quality' -- obtained through reconfiguration -- to realise different reservoirs for different tasks. Here we describe an experimental framework to characterise the quality of potentially any substrate for RC. Our framework reveals that a definition of quality is not only useful to compare substrates, but can help map the non-trivial relationship between properties and task performance. In the wider context, the framework offers a greater understanding as to what makes a dynamical system compute, helping improve the design of future substrates for RC.
Boosting Contrastive Self-Supervised Learning with False Negative Cancellation

Self-supervised representation learning has made significant leaps fueled by progress in contrastive learning, which seeks to learn transformations that embed positive input pairs nearby, while pushing negative pairs far apart. While positive pairs can be generated reliably (e.g., as different views of the same image), it is difficult to accurately establish negative pairs, defined as samples from different images regardless of their semantic content or visual features. A fundamental problem in contrastive learning is mitigating the effects of false negatives. Contrasting false negatives induces two critical issues in representation learning: discarding semantic information and slow convergence. In this paper, we propose novel approaches to identify false negatives, as well as two strategies to mitigate their effect, i.e. false negative elimination and attraction, while systematically performing rigorous evaluations to study this problem in detail. Our method exhibits consistent improvements over existing contrastive learning-based methods. Without labels, we identify false negatives with ~40% accuracy among 1000 semantic classes on ImageNet, and achieve 5.8% absolute improvement in top-1 accuracy over the previous state-of-the-art when finetuning with 1% labels. Our code is available at https://github.com/google-research/fnc
Batch normalization embeddings for deep domain generalization

Domain generalization aims at training machine learning models to perform robustly across different and unseen domains. Several methods train models from multiple datasets to extract domain-invariant features, hoping to generalize to unseen domains. Instead, first we explicitly train domain-dependent representations leveraging ad-hoc batch normalization layers to collect independent domain’s statistics. Then, we propose to use these statistics to map domains in a shared latent space, where membership to a domain is measured by means of a distance function. At test time, we project samples from an unknown domain into the same space and infer properties of their domain as a linear combination of the known ones. We apply the same mapping strategy at training and test time, learning both a latent representation and a powerful but lightweight ensemble model. We show a significant increase in classification accuracy over current state-of-the-art techniques on popular domain generalization benchmarks: PACS, Office-31 and Office-Caltech.
Two-Phase Pseudo Label Densification for Self-training Based Domain Adaptation

Recently, deep self-training approaches emerged as a powerful solution to the unsupervised domain adaptation. The self-training scheme involves iterative processing of target data; it generates target pseudo labels and retrains the network. However, since only the confident predictions are taken as pseudo labels, existing self-training approaches inevitably produce sparse pseudo labels in practice. We see this is critical because the resulting insufficient training-signals lead to a sub-optimal, error-prone model. In order to tackle this problem, we propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. For the easy samples, we now employ their full pseudo-labels. For the hard ones, we instead adopt adversarial learning to enforce hard-to-easy feature alignment. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss. We show the proposed TPLD can be easily integrated into existing self-training based approaches and improves the performance significantly. Combined with the recently proposed CRST self-training framework, we achieve new state-of-the-art results on two standard UDA benchmarks.
ScanRefer: 3D Object Localization in RGB-D Scans Using Natural Language

We introduce the task of 3D object localization in RGB-D scans using natural language descriptions. As input, we assume a point cloud of a scanned 3D scene along with a free-form description of a specified target object. To address this task, we propose ScanRefer, learning a fused descriptor from 3D object proposals and encoded sentence embeddings. This fused descriptor correlates language expressions with geometric features, enabling regression of the 3D bounding box of a target object. We also introduce the ScanRefer dataset, containing $$51,583$$ descriptions of $$11,046$$ objects from $$800$$ ScanNet [8] scenes. ScanRefer is the first large-scale effort to perform object localization via natural language expression directly in 3D (Code: https://daveredrum.github.io/ScanRefer/ ).
Thermal Prediction for Efficient Energy Management of Clouds Using Machine Learning

Thermal management in the hyper-scale cloud data centers is a critical problem. Increased host temperature creates hotspots which significantly increases cooling cost and affects reliability. Accurate prediction of host temperature is crucial for managing the resources effectively. Temperature estimation is a non-trivial problem due to thermal variations in the data center. Existing solutions for temperature estimation are inefficient due to their computational complexity and lack of accurate prediction. However, data-driven machine learning methods for temperature prediction is a promising approach. In this regard, we collect and study data from a private cloud and show the presence of thermal variations. We investigate several machine learning models to accurately predict the host temperature. Specifically, we propose a gradient boosting machine learning model for temperature prediction. The experiment results show that our model accurately predicts the temperature with the average RMSE value of 0.05 or an average prediction error of 2.38 °C, which is 6 °C less as compared to an existing theoretical model. In addition, we propose a dynamic scheduling algorithm to minimize the peak temperature of hosts. The results show that our algorithm reduces the peak temperature by 6.5 °C and consumes 34.5 percent less energy as compared to the baseline algorithm.
Deep Nets: What have They Ever Done for Vision?

This is an opinion paper about the strengths and weaknesses of Deep Nets for vision. They are at the heart of the enormous recent progress in artificial intelligence and are of growing importance in cognitive science and neuroscience. They have had many successes but also have several limitations and there is limited understanding of their inner workings. At present Deep Nets perform very well on specific visual tasks with benchmark datasets but they are much less general purpose, flexible, and adaptive than the human visual system. We argue that Deep Nets in their current form are unlikely to be able to overcome the fundamental problem of computer vision, namely how to deal with the combinatorial explosion, caused by the enormous complexity of natural images, and obtain the rich understanding of visual scenes that the human visual achieves. We argue that this combinatorial explosion takes us into a regime where “big data is not enough” and where we need to rethink our methods for benchmarking performance and evaluating vision algorithms. We stress that, as vision algorithms are increasingly used in real world applications, that performance evaluation is not merely an academic exercise but has important consequences in the real world. It is impractical to review the entire Deep Net literature so we restrict ourselves to a limited range of topics and references which are intended as entry points into the literature. The views expressed in this paper are our own and do not necessarily represent those of anybody else in the computer vision community.
An empirical analysis of the costs of clone- and platform-oriented software reuse

Software reuse lowers development costs and improves the quality of software systems. Two strategies are common: clone & own (copying and adapting a system) and platform-oriented reuse (building a configurable platform). The former is readily available, flexible, and initially cheap, but does not scale with the frequency of reuse, imposing high maintenance costs. The latter scales, but imposes high upfront investments for building the platform, and reduces flexibility. As such, each strategy has distinctive advantages and disadvantages, imposing different development activities and software architectures. Deciding for one strategy is a core decision with long-term impact on an organization’s software development. Unfortunately, the strategies’ costs are not well-understood - not surprisingly, given the lack of systematically elicited empirical data, which is difficult to collect. We present an empirical study of the development activities, costs, cost factors, and benefits associated with either reuse strategy. For this purpose, we combine quantitative and qualitative data that we triangulated from 26 interviews at a large organization and a systematic literature review covering 57 publications. Our study both confirms and refutes common hypotheses on software reuse. For instance, we confirm that developing for platform-oriented reuse is more expensive, but simultaneously reduces reuse costs; and that platform-orientation results in higher code quality compared to clone & own. Surprisingly, refuting common hypotheses, we find that change propagation can be more expensive in a platform, that platforms can facilitate the advancement into innovative markets, and that there is no strict distinction of clone & own and platform-oriented reuse in practice.
Personality-Aware Product Recommendation System Based on User Interests Mining and Metapath Discovery

A recommendation system is an integral part of any modern online shopping or social network platform. The product recommendation system as a typical example of the legacy recommendation systems suffers from two major drawbacks: recommendation redundancy and unpredictability concerning new items (cold start). These limitations take place because the legacy recommendation systems rely only on the user's previous buying behavior to recommend new items. Incorporating the user's social features, such as personality traits and topical interest, might help alleviate the cold start and remove recommendation redundancy. Therefore, in this article, we propose Meta-Interest, a personality-aware product recommendation system based on user interest mining and metapath discovery. Meta-Interest predicts the user's interest and the items associated with these interests, even if the user's history does not contain these items or similar ones. This is done by analyzing the user's topical interests and, eventually, recommending the items associated with the user's interest. The proposed system is personality-aware from two aspects; it incorporates the user's personality traits to predict his/her topics of interest and to match the user's personality facets with the associated items. The proposed system was compared against recent recommendation methods, such as deep-learning-based recommendation system and session-based recommendation systems. Experimental results show that the proposed method can increase the precision and recall of the recommendation system, especially in cold-start settings.
Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video

We address the challenging task of anticipating human-object interaction in first person videos. Most existing methods either ignore how the camera wearer interacts with objects, or simply considers body motion as a separate modality. In contrast, we observe that the intentional hand movement reveals critical information about the future activity. Motivated by this observation, we adopt intentional hand movement as a feature representation, and propose a novel deep network that jointly models and predicts the egocentric hand motion, interaction hotspots and future action. Specifically, we consider the future hand motion as the motor attention, and model this attention using probabilistic variables in our deep model. The predicted motor attention is further used to select the discriminative spatial-temporal visual features for predicting actions and interaction hotspots. We present extensive experiments demonstrating the benefit of the proposed joint model. Importantly, our model produces new state-of-the-art results for action anticipation on both EGTEA Gaze+ and the EPIC-Kitchens datasets. Our project page is available at https://aptx4869lm.github.io/ForecastingHOI/ .
Automatic Classification of Sexism in Social Networks: An Empirical Study on Twitter Data

During the last decade, hateful and sexist content towards women is being increasingly spread on social networks. The exposure to sexist speech has serious consequences to women's life and limits their freedom of speech. Previous studies have focused on identifying hatred or violence towards women. However, sexism is expressed in very different forms: it includes subtle stereotypes and attitudes that, although frequently unnoticed, are extremely harmful for both women and society. In this work, we propose a new task that aims to understand and analyze how sexism, from explicit hate or violence to subtle expressions, is expressed in online conversations. To this end, we have developed and released the first dataset of sexist expressions and attitudes in Twitter in Spanish (MeTwo) and investigate the feasibility of using machine learning techniques (both traditional and novel deep learning models) for automatically detecting different types of sexist behaviours. Our results show that sexism is frequently found in many forms in social networks, that it includes a wide range of behaviours, and that it is possible to detect them using deep learning approaches. We discuss the performance of automatic classification methods to deal with different types of sexism and the generalizability of our task to other subdomains, such as misogyny.
A deep learning model using data augmentation for detection of architectural distortion in whole and patches of images

Breast cancer is now widely known to be the second most lethal disease among women. Computer-aided detection (CAD) systems, deep learning (DL) in particular, have continued to provide significant computational solution in early detection and diagnosis of this disease. Research efforts are advancing novel approaches to improve the performance of DL-based models. Techniques such as data augmentation, varying depth of model, image quality enhancement, and choice of classifier have been proposed to improve performance in the characterization of abnormalities in mammograms. However, no significant progress has been made in applying deep learning techniques to the detection of architectural distortion – a form of abnormalities in breast images. In this research, we propose a novel convolution neural network (CNN) model for the detection of architectural distortion by enhancing its performance using data augmentation technique. We also investigate the performance of the proposed model on different operations of image augmentation. Furthermore, the new model was adapted to detect images presenting the right and left breast presented in MLO and CC views. Similarly, we investigate the performance of our model under the fixed-size region of interests (ROIs) and multi-size whole images inputs. Our method was trained on 5136 ROIs from MIAS, 410 whole images from INbreast, 322 whole images from MIAS, and 55,890 ROIs from DDSM + CBS databases. Performance evaluation of the proposed model in comparison with other state-of-the-art techniques revealed that the model achieved 93.75 % accuracy. This study has, therefore, strengthened the need to leverage data augmentation techniques to enhance the detection of architectural distortion, thereby reducing the rate of advanced cases of breast cancer.