Reviewerly/exHarmony huggingface dataset:
Crosslingual Generalization through Multitask Finetuning

Multitask prompted finetuning (MTF) has been shown to help large language models generalize to new tasks in a zero-shot setting, but so far explorations of MTF have focused on English data and models.We apply MTF to the pretrained multilingual BLOOM and mT5 model families to produce finetuned variants called BLOOMZ and mT0.We find finetuning large multilingual language models on English tasks with English prompts allows for task generalization to non-English languages that appear only in the pretraining corpus.Finetuning on multilingual tasks with English prompts further improves performance on English and non-English tasks leading to various state-ofthe-art zero-shot results.We also investigate finetuning on multilingual tasks with prompts that have been machine-translated from English to match the language of each dataset.We find training on these machine-translated prompts leads to better performance on humanwritten prompts in the respective languages.Surprisingly, we find models are capable of zero-shot generalization to tasks in languages they have never intentionally seen.We conjecture that the models are learning higher-level capabilities that are both task-and languageagnostic.In addition, we introduce xP3, a composite of supervised datasets in 46 languages with English and machine-translated prompts.Our code, datasets and models are freely available at https://github.com/bigscience-workshop/xmtf.
Discriminant Functional Learning of Color Features for the Recognition of Facial Action Units and Their Intensities

Color is a fundamental image feature of facial expressions. For example, when we furrow our eyebrows in anger, blood rushes in, turning some face areas red; or when one goes white in fear as a result of the drainage of blood from the face. Surprisingly, these image properties have not been exploited to recognize the facial action units (AUs) associated with these expressions. Herein, we present the first system to do recognition of AUs and their intensities using these functional color changes. These color features are shown to be robust to changes in identity, gender, race, ethnicity, and skin color. Specifically, we identify the chromaticity changes defining the transition of an AU from inactive to active and use an innovative Gabor transform-based algorithm to gain invariance to the timing of these changes. Because these image changes are given by functions rather than vectors, we use functional classifiers to identify the most discriminant color features of an AU and its intensities. We demonstrate that, using these discriminant color features, one can achieve results superior to those of the state-of-the-art. Finally, we define an algorithm that allows us to use the learned functional color representation in still images. This is done by learning the mapping between images and the identified functional color features in videos. Our algorithm works in realtime, i.e., > 30 frames/second/CPU thread.
A Privacy-Preserving Algorithm for Clinical Decision-Support Systems Using Random Forest

Clinical decision-support systems are technology-based tools that help healthcare providers enhance the quality of their services to satisfy their patients and earn their trust. These systems are used to improve physicians’ diagnostic processes in terms of speed and accuracy. Using data-mining techniques, a clinical decision support system builds a classification model from hospital’s dataset for diagnosing new patients using their symptoms. In this work, we propose a privacy-preserving clinical decision-support system that uses a privacy-preserving random forest algorithm to diagnose new symptoms without disclosing patients’ information and exposing them to cyber and network attacks. Solving the same problem with a different methodology, the simulation results show that the proposed algorithm outperforms previous work by removing unnecessary attributes and avoiding cryptography algorithms. Moreover, our model is validated against the privacy requirements of the hospitals’ datasets and votes, and patients’ diagnosed symptoms.
R2N: A Novel Deep Learning Architecture for Rain Removal from Single Image

Visual degradation of captured images caused by rainy streaks under rainy weather can adversely affect the performance of many open-air vision systems. Hence, it is necessary to address the problem of eliminating rain streaks from the individual rainy image. In this work, a deep convolution neural network (CNN) based method is introduced, called Rain-Removal Net (R2N), to solve the single image de-raining issue. Firstly, we decomposed the rainy image into its high-frequency detail layer and low-frequency base layer. Then, we used the high-frequency detail layer to input the carefully designed CNN architecture to learn the mapping between it and its corresponding de-rained high-frequency detail layer. The CNN architecture consists of four convolution layers and four deconvolution layers, as well as three skip connections. The experiments on synthetic and real-world rainy images show that the performance of our architecture outperforms the compared state-of-the-art de-raining models with respects to the quality of de-rained images and computing efficiency.
Public Auditing for Trusted Cloud Storage Services

Cloud storage can provide on-demand outsourcing of data services for organizations and individuals. However, because customers may not fully trust that cloud service providers meet their legal expectations for data security, techniques for auditing the cloud have attracted increasing attention. Here, we present an architecture of public data auditing, review existing methods or mechanisms for various auditing objectives, and discuss trends and possible future developments.
Design and Development of a Real-Time Optimal Route Recommendation System Using Big Data for Tourists in Jeju Island

Nowadays researchers and engineers are trying to build travel route recommendation systems to guide tourists around the globe. The tourism industry is on the rise and it has attracted researchers to provide such systems for comfortable and convenient traveling. Mobile internet growth is increasing rapidly. Mobile data usage and traffic growth has increased interest in building mobile applications for tourists. This research paper aims to provide design and implementation of a travel route recommendation system based on user preference. Real-time big data is collected from Wi-Fi routers installed at more than 149 unique locations in Jeju Island, South Korea. This dataset includes tourist movement patterns collected from thousands of mobile tourists in the year 2016–2017. Data collection and analysis is necessary for a country to make public policies and development of the global travel and tourism industry. In this research paper we propose an optimal travel route recommendation system by performing statistical analysis of tourist movement patterns. Route recommendation is based on user preferences. User preference can vary over time and differ from one user to another. We have taken three main factors into consideration to the recommend optimal route i.e., time, distance, and popularity of location. Beside these factors, we have also considered weather and traffic condition using a third-party application program interfaces (APIs). We have classified regions into six major categories. Popularity of location can vary from season to season. We used a Naïve Bayes classifier to find the probability of tourists going to visit next location. Third-party APIs are used to find the longitude and latitude of the location. The Haversine formula is used to calculate the distance between unique locations. On the basis of these factors, we recommend the optimal route for tourists. The proposed system is highly responsive to mobile users. The results of this system show that the recommended route is convenient and allows tourists to visit maximum number of famous locations as compared to previous data.
Cryptanalysis of an Image Encryption Using 2D Henon-Sine Map and DNA Approach

Recently, a novel gray image encryption scheme was proposed based on 2D Henon-Sine map (2D-HSM) and DNA approach.In the original scheme, the image pixels are substituted and diffused by the DNA approach and then permutated by a 2D-HSM key sequence.The structure of the encryption scheme is equivalent to substitution-diffusion-permutation.However, it is found that the attacker can recover the encrypted image by using the chosen plaintext attack without the secret keys.First, the equivalent key matrix used for permutation of the encryption scheme can be obtained, and the effects of diffusion can also be eliminated by the proposed method.Then, the mapping matrix between the input plaintext image and the intermediate image encrypted by the DNA approach is obtained.Both theoretical analysis and the experimental results show that the encrypted images using the original scheme can be cracked successfully.The attack complexity of our scheme is O(18), which greatly reduces the number of attacks compared with the traditional method.In addition, our approach will help to improve the security of image-encryption schemes based on permutation-diffusion-scrambling structures.
Closed-loop control of nonlinear neural networks: The estimate of control time and energy cost

This paper concentrates on an estimate of the upper bounds for control time and energy cost of a class of nonlinear neural networks (NNs). By constructing the appropriate closed-loop controller uS and utilizing the inequality technique, sufficient conditions are proposed to guarantee achieving control target in finite time of the considered systems. Then, the estimate of the upper bounds for the control energy cost of the designed controller uS is proposed. Our results provide a new controller which can ensure the realization of finite time control and energy consumption control for a class of nonlinear NNs. Meanwhile, the obtained results contribute to qualitative analysis of some nonlinear systems. Finally, numerical examples are presented to demonstrate the effectiveness of our theoretical results.
Hyperspectral Light Field Stereo Matching

In this paper, we describe how scene depth can be extracted using a hyperspectral light field capture (H-LF) system. Our H-LF system consists of a <inline-formula><tex-math notation="LaTeX">$5 \times 6$</tex-math></inline-formula> array of cameras, with each camera sampling a different narrow band in the visible spectrum. There are two parts to extracting scene depth. The first part is our novel cross-spectral pairwise matching technique, which involves a new spectral-invariant feature descriptor and its companion matching metric we call bidirectional weighted normalized cross correlation (BWNCC). The second part, namely, H-LF stereo matching, uses a combination of spectral-dependent correspondence and defocus cues. These two new cost terms are integrated into a Markov Random Field (MRF) for disparity estimation. Experiments on synthetic and real H-LF data show that our approach can produce high-quality disparity maps. We also show that these results can be used to produce the complete plenoptic cube in addition to synthesizing all-focus and defocused color images under different sensor spectral responses.
Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in $O(\sqrt{n})$ iterations

We analyze the classical EM algorithm for parameter estimation in the symmetric two-component Gaussian mixtures in $d$ dimensions. We show that, even in the absence of any separation between components, provided that the sample size satisfies $n=\Omega(d \log^4 d)$, the randomly initialized EM algorithm converges to an estimate in at most $O(\sqrt{n})$ iterations with high probability, which is at most $O((d / n)^{1/4} \log n)$ in Euclidean distance from the true parameter and within logarithmic factors of the minimax rate of $(d / n)^{1/4}$. Both the nonparametric statistical rate and the sublinear convergence rate are direct consequences of the zero Fisher information in the worst case. Refined pointwise guarantees beyond worst-case analysis and convergence to the MLE are also shown under mild conditions.
Testing Smart Contracts Using Ganache

In Chapter 4 , you learned how to create your own private test Ethereum network using the Ethereum client – Geth. While Geth is not that difficult to use to set up an Ethereum blockchain, it still takes considerable effort to set one up. In addition, you also need to create accounts and perform mining in order to obtain Ethers so that you can use them to deploy Smart Contracts and for performing transactions.
End-to-End Multi-Speaker Speech Recognition Using Speaker Embeddings and Transfer Learning

This paper presents our latest investigation on end-to-end automatic speech recognition (ASR) for overlapped speech. We propose to train an end-to-end system conditioned on speaker embeddings and further improved by transfer learning from clean speech. This proposed framework does not require any parallel non-overlapped speech materials and is independent of the number of speakers. Our experimental results on overlapped speech datasets show that joint conditioning on speaker embeddings and transfer learning significantly improves the ASR performance.
Dual-Server Public-Key Authenticated Encryption with Keyword Search

In cloud storage, how to search sensitive data efficiently and securely is a challenging problem. The searchable encryption technique provides a secure storage method without loss of data confidentiality and usability. As an important branch of searchable encryption, public-key encryption with keyword search (PEKS) is widely studied by scholars. However, most of the traditional PEKS schemes are vulnerable to the inside keyword guessing attack (IKGA). Resisting the inside keyword guessing attack is likely to become an essential property of all new PEKS schemes. For a long time, mitigating IKGA has been inefficient and difficult, and most existing PEKS schemes fail in achieving their security goals. To address the above problems, we define the notion of <b>D</b>ual-server <b>P</b>ublic-key <b>A</b>uthenticated <b>E</b>ncryption with <b>K</b>eyword <b>S</b>earch (DPAEKS), which protects against IKGA by leveraging two servers that do not cooperate, and supports the authentication property. Then, we provide a construction of DPAEKS without bilinear pairings. Experimental results obtained using a real-world dataset show that our scheme is highly efficient and provides strong security, making it suitable for deployment in practical applications.
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering

Multi-hop question answering (QA) requires an information retrieval (IR) system that can find multiple supporting evidence needed to answer the question, making the retrieval process very challenging.This paper introduces an IR technique that uses information of entities present in the initially retrieved evidence to learn to 'hop' to other relevant evidence.In a setting, with more than 5 million Wikipedia paragraphs, our approach leads to significant boost in retrieval performance.The retrieved evidence also increased the performance of an existing QA model (without any training) on the HOTPOTQA benchmark by 10.59 F1.
A secure logistics model based on blockchain

Modern services of logistics has developed and expanded rapidly in China. The information of package delivery is managed by the logistics enterprises. Thus, while enjoying the modern services of logistics, consumers face the threats of personal privacy disclosure. In order to address such challenges in modern services of logistics, we present techniques to exploit blockchain to secure logistics for protecting personal privacy. The main contribution is that we propose alogistics blockchain model. We implement the logistics blockchain on the decentralised and distributed platform, which shows that it is efficient and secure.
Option Discovery using Deep Skill Chaining

Autonomously discovering temporally extended actions, or skills, is a longstanding goal of hierarchical reinforcement learning. We propose a new algorithm that combines skill chaining with deep neural networks to autonomously discover skills in high-dimensional, continuous domains. The resulting algorithm, deep skill chaining, constructs skills with the property that executing one enables the agent to execute another. We demonstrate that deep skill chaining significantly outperforms both non-hierarchical agents and other state-of-the-art skill discovery techniques in challenging continuous control tasks.
Exploring Model-based Planning with Policy Networks

Model-based reinforcement learning (MBRL) with model-predictive control or online planning has shown great potential for locomotion control tasks in both sample efficiency and asymptotic performance. Despite the successes, the existing planning methods search from candidate sequences randomly generated in the action space, which is inefficient in complex high-dimensional environments. In this paper, we propose a novel MBRL algorithm, model-based policy planning (POPLIN), that combines policy networks with online planning. More specifically, we formulate action planning at each time-step as an optimization problem using neural networks. We experiment with both optimization w.r.t. the action sequences initialized from the policy network, and also online optimization directly w.r.t. the parameters of the policy network. We show that POPLIN obtains state-of-the-art performance in the MuJoCo benchmarking environments, being about 3x more sample efficient than the state-of-the-art algorithms, such as PETS, TD3 and SAC. To explain the effectiveness of our algorithm, we show that the optimization surface in parameter space is smoother than in action space. Further more, we found the distilled policy network can be effectively applied without the expansive model predictive control during test time for some environments such as Cheetah. Code is released.
Bayesian deep matrix factorization network for multiple images denoising

This paper aims at proposing a robust and fast low rank matrix factorization model for multiple images denoising. To this end, a novel model, Bayesian deep matrix factorization network (BDMF), is presented, where a deep neural network (DNN) is designed to model the low rank components and the model is optimized via stochastic gradient variational Bayes. By the virtue of deep learning and Bayesian modeling, BDMF makes significant improvement on synthetic experiments and real-world tasks (including shadow removal and hyperspectral image denoising), compared with existing state-of-the-art models.
Compositional Language Continual Learning

Motivated by the human's ability to continually learn and gain knowledge over time, several research efforts have been pushing the limits of machines to constantly learn while alleviating catastrophic forgetting. Most of the existing methods have been focusing on continual learning of label prediction tasks, which have fixed input and output sizes. In this paper, we propose a new scenario of continual learning which handles sequence-to-sequence tasks common in language learning. We further propose an approach to use label prediction continual learning algorithm for sequence-to-sequence continual learning by leveraging compositionality. Experimental results show that the proposed method has significant improvement over state-of-the-art methods. It enables knowledge transfer and prevents catastrophic forgetting, resulting in more than 85% accuracy up to 100 stages, compared with less than 50% accuracy for baselines in instruction learning task. It also shows significant improvement in machine translation task. This is the first work to combine continual learning and compositionality for language learning, and we hope this work will make machines more helpful in various tasks.
Artificial intelligence driven next-generation renal histomorphometry

Purpose of review Successful integration of artificial intelligence into extant clinical workflows is contingent upon a number of factors including clinician comprehension and interpretation of computer vision. This article discusses how image analysis and machine learning have enabled comprehensive characterization of kidney morphology for development of automated diagnostic and prognostic renal pathology applications. Recent findings The primordial digital pathology informatics work employed classical image analysis and machine learning to prognosticate renal disease. Although this classical approach demonstrated tremendous potential, subsequent advancements in hardware technology rendered artificial neural networks ‘(ANNs) the method of choice for machine vision in computational pathology’. Offering rapid and reproducible detection, characterization and classification of kidney morphology, ANNs have facilitated the development of diagnostic and prognostic applications. In addition, modern machine learning with ANNs has revealed novel biomarkers in kidney disease, demonstrating the potential for machine vision to elucidate novel pathologic mechanisms beyond extant clinical knowledge. Summary Despite the revolutionary developments potentiated by modern machine learning, several challenges remain, including data quality control and curation, image annotation and ontology, integration of multimodal data and interpretation of machine vision or ‘opening the black box’. Resolution of these challenges will not only revolutionize diagnostic pathology but also pave the way for precision medicine and integration of artificial intelligence in the process of care.
Where are the Masks: Instance Segmentation with Image-level Supervision

A major obstacle in instance segmentation is that existing methods often need many per-pixel labels in order to be effective. These labels require large human effort and for certain applications, such labels are not readily available. To address this limitation, we propose a novel framework that can effectively train with image-level labels, which are significantly cheaper to acquire. For instance, one can do an internet search for the term and obtain many images where a car is present with minimal effort. Our framework consists of two stages: (1) train a classifier to generate pseudo masks for the objects of interest; (2) train a fully supervised Mask R-CNN on these pseudo masks. Our two main contribution are proposing a pipeline that is simple to implement and is amenable to different segmentation methods; and achieves new state-of-the-art results for this problem setup. Our results are based on evaluating our method on PASCAL VOC 2012, a standard dataset for weakly supervised methods, where we demonstrate major performance gains compared to existing methods with respect to mean average precision.
From Local SGD to Local Fixed Point Methods for Federated Learning

Most algorithms for solving optimization problems or finding saddle points of convex-concave functions are fixed-point algorithms. In this work we consider the generic problem of finding a fixed point of an average of operators, or an approximation thereof, in a distributed setting. Our work is motivated by the needs of federated learning. In this context, each local operator models the computations done locally on a mobile device. We investigate two strategies to achieve such a consensus: one based on a fixed number of local steps, and the other based on randomized computations. In both cases, the goal is to limit communication of the locally-computed variables, which is often the bottleneck in distributed frameworks. We perform convergence analysis of both methods and conduct a number of experiments highlighting the benefits of our approach.