Reviewerly/exHarmony huggingface dataset:
Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

Deciphering human behaviors to predict their future paths/trajectories and what they would do from videos is important in many applications. Motivated by this idea, this paper studies predicting a pedestrian's future path jointly with future activities. We propose an end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings. To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen. Experimental results demonstrate our state-of-the-art performance over two public benchmarks on future trajectory prediction. Moreover, our method is able to produce meaningful future activity prediction in addition to the path. The result provides the first empirical evidence that joint modeling of paths and activities benefits future path prediction.
A Novel Two-Step Registration Method for Remote Sensing Images Based on Deep and Local Features

Automatic remote sensing image registration has achieved great accomplishment. However, it is still a vital challenging problem to develop a robust and accurate registration method due to the negative effects of noise and imaging differences between images. For these images, it is difficult to guarantee the accuracy and robustness at the same time for one-step registration methods. To address this issue, we introduce an effective coarse-to-fine strategy and develop a new two-step registration method based on deep and local features in this paper. The first step is to calculate the approximate spatial relationship, which is obtained by a convolutional neural network. This step makes full use of the deep features to match and can generate stable results. For the second step, a matching strategy considering spatial relationship is applied to the local feature-based method. In addition, this step adopts more accurate features in location to adjust the results of the previous step. A variety of homologous and multimodal remote sensing images, including optical, synthetic aperture radar, and general map images, are used to evaluate the proposed method. The comparison experiments demonstrate that our method can apparently increase the correct correspondences, can improve the ratio of correct correspondences, and is highly robust and accurate.
AHCNet: An Application of Attention Mechanism and Hybrid Connection for Liver Tumor Segmentation in CT Volumes

The liver is a common site for the development of primary (i.e., originating from the liver, e.g., hepatocellular carcinoma) or secondary (i.e., spread to the liver, e.g., colorectal cancer) tumor.Due to its complex background, heterogeneous, and diffusive shape, automatic segmentation of tumor remains a challenging task.So far, only the interactive method has been adopted to obtain the acceptable segmentation results of a liver tumor.In this paper, we design an Attention Hybrid Connection Network architecture which combines soft and hard attention mechanism and long and short skip connections.We also propose a cascade network based on the liver localization network, liver segmentation network, and tumor segmentation network to cope with this challenge.Simultaneously, the joint dice loss function is proposed to train the liver localization network to obtain the accurate 3D liver bounding box, and the focal binary cross entropy is used as a loss function to fine-tune the tumor segmentation network for detecting more potentially malignant tumor and reduce false positives.Our framework is trained using the 110 cases in the LiTS dataset and extensively evaluated by the 20 cases in the 3DIRCADb dataset and the 117 cases in the Clinical dataset, which indicates that the proposed method can achieve faster network convergence and accurate semantic segmentation and further demonstrate that the proposed method has a good clinical value.
A Machine Learning Approach for Fall Detection and Daily Living Activity Recognition

The number of older people in western countries is constantly increasing. Most of them prefer to live independently and are susceptible to fall incidents. Falls often lead to serious or even fatal injuries which are the leading cause of death for elderlies. To address this problem, it is essential to develop robust fall detection systems. In this context, we develop a machine learning framework for fall detection and daily living activity recognition. We use acceleration and angular velocity data from two public databases to recognize seven different activities, including falls and activities of daily living. From the acceleration and angular velocity data, we extract time- and frequency-domain features and provide them to a classification algorithm. In this paper, we test the performance of four algorithms for classifying human activities. These algorithms are the artificial neural network (ANN), $K$ -nearest neighbors (KNN), quadratic support vector machine (QSVM), and ensemble bagged tree (EBT). New features that improve the performance of the classifier are extracted from the power spectral density of the acceleration. In the first step, only the acceleration data are used for activity recognition. Our results reveal that the KNN, ANN, QSVM, and EBT algorithms could achieve overall accuracy of 81.2%, 87.8%, 93.2%, and 94.1%, respectively. The accuracy of fall detection reaches 97.2% and 99.1% without any false alarms for the QSVM and EBT algorithms, respectively. In a second step, we extract features from the autocorrelation function and the power spectral density of both the acceleration and the angular velocity data, which improves the classification accuracy. By using the proposed features, we could achieve overall accuracy of 85.8%, 91.8%, 96.1%, and 97.7% for the KNN, ANN, QSVM, and EBT algorithms, respectively. The accuracy of fall detection reaches 100% for both the QSVM and EBT algorithms without any false alarm, which is the best achievable performance.
Adaptive image enhancement method for correcting low-illumination images

In this study, to improve the adaptability of image enhancement in images with low illumination, a colored image correction method based on nonlinear functional transformation according to the illumination-reflection model and multiscale theory is proposed. First, the original RGB image is converted to HSV color space, and the V component is used to extract the illumination component of the scene using the multiscale Gaussian function. Then, a correction function is constructed based on the Weber-Fechner law, and two images are obtained through adaptive adjustments to the image enhancement function parameters based on the distribution profiles of the illumination components. Finally, an image fusion strategy is formulated and used to extract the details from the two images. Compared with the classic algorithm, the proposed algorithm can improve the overall brightness and contrast of an image while reducing the impact of uneven illumination. The enhanced images appear clear, bright, and natural.
Verifying multipartite entangled Greenberger-Horne-Zeilinger states via multiple quantum coherences

An 18-qubit GHZ state with multipartite entanglement is prepared and measured on a 20-qubit device. The detection technique used, which is based on measuring multiple quantum coherences, is robust to noise and only requires measuring the population in the ground state.
Gmail Smart Compose

In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing. In the design and deployment of such a large-scale and complicated system, we faced several challenges including model selection, performance evaluation, serving and other practical issues. At the core of Smart Compose is a large-scale neural language model. We leveraged state-of-the-art machine learning techniques for language model training which enabled high-quality suggestion prediction, and constructed novel serving infrastructure for high-throughput and real-time inference. Experimental results show the effectiveness of our proposed system design and deployment approach. This system is currently being served in Gmail.
Sentiment Classification Using Convolutional Neural Networks

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.
Inverse Cooking: Recipe Generation From Food Images

People enjoy food photography because they appreciate food. Behind each meal there is a story described in a complex recipe and, unfortunately, by simply looking at a food image we do not have access to its preparation process. Therefore, in this paper we introduce an inverse cooking system that recreates cooking recipes given food images. Our system predicts ingredients as sets by means of a novel architecture, modeling their dependencies without imposing any order, and then generates cooking instructions by attending to both image and its inferred ingredients simultaneously. We extensively evaluate the whole system on the large-scale Recipe1M dataset and show that (1) we improve performance w.r.t. previous baselines for ingredient prediction; (2) we are able to obtain high quality recipes by leveraging both image and ingredients; (3) our system is able to produce more compelling recipes than retrieval-based approaches according to human judgment. We make code and models publicly available.
FOCNet: A Fractional Optimal Control Network for Image Denoising

Deep convolutional neural networks (DCNN) have been successfully used in many low-level vision problems such as image denoising. Recent studies on the mathematical foundation of DCNN has revealed that the forward propagation of DCNN corresponds to a dynamic system, which can be described by an ordinary differential equation (ODE) and solved by the optimal control method. However, most of these methods employ integer-order differential equation, which has local connectivity in time space and cannot describe the long-term memory of the system. Inspired by the fact that the fractional-order differential equation has long-term memory, in this paper we develop an advanced image denoising network, namely FOCNet, by solving a fractional optimal control (FOC) problem. Specifically, the network structure is designed based on the discretization of a fractional-order differential equation, which enjoys long-term memory in both forward and backward passes. Besides, multi-scale feature interactions are introduced into the FOCNet to strengthen the control of the dynamic system. Extensive experiments demonstrate the leading performance of the proposed FOCNet on image denoising. Code will be made available.
Var-CNN: A Data-Efficient Website Fingerprinting Attack Based on Deep Learning

In recent years, there have been several works that use website fingerprinting techniques to enable a local adversary to determine which website a Tor user visits. While the current state-of-the-art attack, which uses deep learning, outperforms prior art with medium to large amounts of data, it attains marginal to no accuracy improvements when both use small amounts of training data. In this work, we propose Var-CNN, a website fingerprinting attack that leverages deep learning techniques along with novel insights specific to packet sequence classification. In open-world settings with large amounts of data, Var-CNN attains over $1\%$ higher true positive rate (TPR) than state-of-the-art attacks while achieving $4\times$ lower false positive rate (FPR). Var-CNN's improvements are especially notable in low-data scenarios, where it reduces the FPR of prior art by $3.12\%$ while increasing the TPR by $13\%$. Overall, insights used to develop Var-CNN can be applied to future deep learning based attacks, and substantially reduce the amount of training data needed to perform a successful website fingerprinting attack. This shortens the time needed for data collection and lowers the likelihood of having data staleness issues.
Machine learning based concept drift detection for predictive maintenance

In this work we present a machine learning based approach for detecting drifting behavior – so-called concept drifts – in continuous data streams. The motivation for this contribution originates from the currently intensively investigated topic Predictive Maintenance (PdM), which refers to a proactive way of triggering servicing actions for industrial machinery. The aim of this maintenance strategy is to identify wear and tear, and consequent malfunctioning by analyzing condition monitoring data, recorded by sensor equipped machinery, in real-time. Recent developments in this area have shown potential to save time and material by preventing breakdowns and improving the overall predictability of industrial processes. However, due to the lack of high quality monitoring data and only little experience concerning the applicability of analysis methods, real-world implementations of Predictive Maintenance are still rare. Within this contribution, we present a method, to detect concept drift in data streams as potential indication for defective system behavior and depict initial tests on synthetic data sets. Further on, we present a real-world case study with industrial radial fans and discuss promising results gained from applying the detailed approach in this scope.
Sampling-bias-corrected neural modeling for large corpus item recommendations

Many recommendation systems retrieve and score items from a very large corpus. A common recipe to handle data sparsity and power-law item distribution is to learn item representations from its content features. Apart from many content-aware systems based on matrix factorization, we consider a modeling framework using two-tower neural net, with one of the towers (item tower) encoding a wide variety of item content features. A general recipe of training such two-tower models is to optimize loss functions calculated from in-batch negatives, which are items sampled from a random mini-batch. However, in-batch loss is subject to sampling biases, potentially hurting model performance, particularly in the case of highly skewed distribution. In this paper, we present a novel algorithm for estimating item frequency from streaming data. Through theoretical analysis and simulation, we show that the proposed algorithm can work without requiring fixed item vocabulary, and is capable of producing unbiased estimation and being adaptive to item distribution change. We then apply the sampling-bias-corrected modeling approach to build a large scale neural retrieval system for YouTube recommendations. The system is deployed to retrieve personalized suggestions from a corpus with tens of millions of videos. We demonstrate the effectiveness of sampling-bias correction through offline experiments on two real-world datasets. We also conduct live A/B testings to show that the neural retrieval system leads to improved recommendation quality for YouTube.
Word2Vec Model Analysis for Semantic Similarities in English Words

This paper examines the calculation of the similarity between words in English using word representation techniques. Word2Vec is a model used in this paper to represent words into vector form. The model in this study was formed using the 320,000 articles in the English Wikipedia as the corpus and then Cosine Similarity calculation method is used to determine the similarity value. This model then tested by the test set gold standard WordSim-353 as many as 353 pairs of words and SimLex-999 as many as 999 pairs of words, which have been labelled with similarity values according to human judgment. Pearson Correlation was used to find out the accuracy of the correlation. The results of the correlation from this study are 0.665 for WordSim-353 and 0.284 for SimLex-999 using the Windows size 9 and 300 vector dimension configurations.
A Misdirected Principle with a Catch: Explicability for AI

Abstract There is widespread agreement that there should be a principle requiring that artificial intelligence (AI) be ‘explicable’. Microsoft, Google, the World Economic Forum, the draft AI ethics guidelines for the EU commission, etc. all include a principle for AI that falls under the umbrella of ‘explicability’. Roughly, the principle states that “for AI to promote and not constrain human autonomy, our ‘decision about who should decide’ must be informed by knowledge of how AI would act instead of us” (Floridi et al. in Minds Mach 28(4):689–707, 2018). There is a strong intuition that if an algorithm decides, for example, whether to give someone a loan, then that algorithm should be explicable. I argue here, however, that such a principle is misdirected. The property of requiring explicability should attach to a particular action or decision rather than the entity making that decision. It is the context and the potential harm resulting from decisions that drive the moral need for explicability—not the process by which decisions are reached. Related to this is the fact that AI is used for many low-risk purposes for which it would be unnecessary to require that it be explicable. A principle requiring explicability would prevent us from reaping the benefits of AI used in these situations. Finally, the explanations given by explicable AI are only fruitful if we already know which considerations are acceptable for the decision at hand. If we already have these considerations, then there is no need to use contemporary AI algorithms because standard automation would be available. In other words, a principle of explicability for AI makes the use of AI redundant.
Exploring blockchain for the energy transition: Opportunities and challenges based on a case study in Japan

Under pressures to reach net zero emissions by 2050, there is an ongoing transition of energy decarbonization, decentralization and digitalization. Physical and information flows in energy systems are increasingly complex and distributed, leaving centralized structures inefficient. Blockchain technology is suggested as part of the next step in this transition. Blockchain has potential to facilitate distributed, peer-to-peer trading with reduced transaction costs, increased security via cryptography, and prosumer choice. However, there are as of yet multiple challenges to the expansion of blockchain in the energy sector. This paper argues that analysis of these challenges requires a multi-angled approach incorporating technological, economic, social, environmental, and institutional dimensions. First, each dimension is explored, substantiated based on a blockchain-based energy system case in Japan. Concrete challenges of scaling this case toward 2050 and potential opportunities in overcoming these challenges are discussed, leveraging extensive literature review. Finally, an overview of strategic indications is suggested. The findings of this paper present initial indications on challenges and opportunities to overcome them based on a multi-dimensional overview. It is suggested that the factors identified across the dimensions are interrelated. This would in turn call for coherent innovation management and multi-stakeholder innovation ecosystems. Living Labs and regulatory sandboxes are prospective foundations to support such ecosystems, and enable informed decision-making among both private and public sector actors. At large, it is suggested that a holistic and pragmatic approach can benefit the application and scalability of blockchain in the energy transition.
COCO-GAN: Generation by Parts via Conditional Coordinating

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraints, we propose COnditional COordinate GAN (COCO-GAN) of which the generator generates images by parts based on their spatial coordinates as the condition. On the other hand, the discriminator learns to justify realism across multiple assembled patches by global coherence, local appearance, and edge-crossing continuity. Despite the full images are never manipulated during training, we show that COCO-GAN can produce state-of-the-art-quality full images during inference. We further demonstrate a variety of novel applications enabled by our coordinate-aware framework. First, we perform extrapolation to the learned coordinate manifold and generate off-the-boundary patches. Combining with the originally generated full image, COCO-GAN can produce images that are larger than training samples, which we called "beyond-boundary generation". We then showcase panorama generation within a cylindrical coordinate system that inherently preserves horizontally cyclic topology. On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides high-parallelism, and can generate parts of images on-demand.
Image Generation From Small Datasets via Batch Statistics Adaptation

Thanks to the recent development of deep generative models, it is becoming possible to generate high-quality images with both fidelity and diversity. However, the training of such generative models requires a large dataset. To reduce the amount of data required, we propose a new method for transferring prior knowledge of the pre-trained generator, which is trained with a large dataset, to a small dataset in a different domain. Using such prior knowledge, the model can generate images leveraging some common sense that cannot be acquired from a small dataset. In this work, we propose a novel method focusing on the parameters for batch statistics, scale and shift, of the hidden layers in the generator. By training only these parameters in a supervised manner, we achieved stable training of the generator, and our method can generate higher quality images compared to previous methods without collapsing, even when the dataset is small (~100). Our results show that the diversity of the filters acquired in the pre-trained generator is important for the performance on the target domain. Our method makes it possible to add a new class or domain to a pre-trained generator without disturbing the performance on the original domain. Code is available at github.com/nogu-atsu/small-dataset-image-generation.