Reviewerly/exHarmony huggingface dataset:
Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition

Multi-label image recognition is a practical and challenging task compared to single-label image classification. However, previous works may be suboptimal because of a great number of object proposals or complex attentional region generation modules. In this paper, we propose a simple but efficient two-stream framework to recognize multi-category objects from global image to local regions, similar to how human beings perceive objects. To bridge the gap between global and local streams, we propose a multi-class attentional region module which aims to make the number of attentional regions as small as possible and keep the diversity of these regions as high as possible. Our method can efficiently and effectively recognize multi-class objects with an affordable computation cost and a parameter-free region localization module. Over three benchmarks on multi-label image classification, our method achieves new state-of-the-art results with a single model only using image semantics without label dependency. In addition, the effectiveness of the proposed method is extensively demonstrated under different factors such as global pooling strategy, input size and network architecture. Code has been made available at https://github.com/gaobb/MCAR.
Adoption of mobile library apps as learning tools in higher education: a tale between Hong Kong and Japan

Purpose The purpose of this study is to examine students' perception of mobile technology adoption for library service apps as a lens to explore the factors affecting mobile learning in higher education. Design/methodology/approach With the data collected from university students from Hong Kong and Japan, the authors conducted a comparative study for gaining a better understanding of students' perceptions toward adopting mobile apps as learning tools and other socio-cultural factors affecting such adoption. In particular, the authors derive alternative models from our analysis (benchmarked on technology acceptance model and unified theory of acceptance and use of technology) for a better explanation of the formation of the intentions, and, more importantly, cultural factors would affect such formation, as informed by the Hofstede Cultural Dimensions. Findings The findings are valuable for enhancing the effectiveness and adoption of mobile services for teaching and learning purposes in the context of higher education, as the focus of promotion of services should be tailored to the different student expectations in different regions, e.g. mainly performance expectancy in Japan versus in Hong Kong facilitating condition, performance expectancy and perceived usefulness altogether. Originality/value Defined as contributing to the development of learning assistance, mobile technology for library use and information access have become a significant issue raised in many higher educational institutions over the world, as the characteristics of mobile technology, such as portability and mobility, strengthen its possible use in education. However, scant research has been done in understanding the perception of mobile library use and factors affecting the behavioral intentions through the use of mobile technologies in Hong Kong and most Asian countries have not been sufficiently studied.
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Pre-trained representations are becoming crucial for many NLP and perception tasks. While representation learning in NLP has transitioned to training on raw text without human annotations, visual and vision-language representations still rely heavily on curated training datasets that are expensive or require expert knowledge. For vision applications, representations are mostly learned using datasets with explicit class labels such as ImageNet or OpenImages. For vision-language, popular datasets like Conceptual Captions, MSCOCO, or CLIP all involve a non-trivial data collection (and cleaning) process. This costly curation process limits the size of datasets and hence hinders the scaling of trained models. In this paper, we leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps in the Conceptual Captions dataset. A simple dual-encoder architecture learns to align visual and language representations of the image and text pairs using a contrastive loss. We show that the scale of our corpus can make up for its noise and leads to state-of-the-art representations even with such a simple learning scheme. Our visual representation achieves strong performance when transferred to classification tasks such as ImageNet and VTAB. The aligned visual and language representations enables zero-shot image classification and also set new state-of-the-art results on Flickr30K and MSCOCO image-text retrieval benchmarks, even when compared with more sophisticated cross-attention models. The representations also enable cross-modality search with complex text and text + image queries.
FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations

Binary neural networks (BNNs) have 1-bit weights and activations. Such networks are well suited for FPGAs, as their dominant computations are bitwise arithmetic and the memory requirement is also significantly reduced. However, compared to start-of-the-art compact convolutional neural network (CNN) models, BNNs tend to produce a much lower accuracy on realistic datasets such as ImageNet. In addition, the input layer of BNNs has gradually become a major compute bottleneck, because it is conventionally excluded from binarization to avoid a large accuracy loss.
PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching

Supervised learning with deep convolutional neural networks (DCNNs) has seen huge adoption in stereo matching. However, the acquisition of large-scale datasets with well-labeled ground truth is cumbersome and labor-intensive, making supervised learning-based approaches often hard to implement in practice. To overcome this drawback, we propose a robust and effective self-supervised stereo matching approach, consisting of a pyramid voting module (PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically, our OptStereo first builds multi-scale cost volumes, and then adopts a recurrent unit to iteratively update disparity estimations at high resolution; while our PVM can generate reliable semi-dense disparity images, which can be employed to supervise OptStereo training. Furthermore, we publish the HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under different illumination and weather conditions for research purposes. Extensive experimental results demonstrate the effectiveness and efficiency of our self-supervised stereo matching approach on the KITTI Stereo benchmarks and our HKUST-Drive dataset. PVStereo, our best-performing implementation, greatly outperforms all other state-of-the-art self-supervised stereo matching approaches. Our project page is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sites.google.com/view/pvstereo</uri> .
An Efficient Approach Based on Privacy-Preserving Deep Learning for Satellite Image Classification

Satellite images have drawn increasing interest from a wide variety of users, including business and government, ever since their increased usage in important fields ranging from weather, forestry and agriculture to surface changes and biodiversity monitoring. Recent updates in the field have also introduced various deep learning (DL) architectures to satellite imagery as a means of extracting useful information. However, this new approach comes with its own issues, including the fact that many users utilize ready-made cloud services (both public and private) in order to take advantage of built-in DL algorithms and thus avoid the complexity of developing their own DL architectures. However, this presents new challenges to protecting data against unauthorized access, mining and usage of sensitive information extracted from that data. Therefore, new privacy concerns regarding sensitive data in satellite images have arisen. This research proposes an efficient approach that takes advantage of privacy-preserving deep learning (PPDL)-based techniques to address privacy concerns regarding data from satellite images when applying public DL models. In this paper, we proposed a partially homomorphic encryption scheme (a Paillier scheme), which enables processing of confidential information without exposure of the underlying data. Our method achieves robust results when applied to a custom convolutional neural network (CNN) as well as to existing transfer learning methods. The proposed encryption scheme also allows for training CNN models on encrypted data directly, which requires lower computational overhead. Our experiments have been performed on a real-world dataset covering several regions across Saudi Arabia. The results demonstrate that our CNN-based models were able to retain data utility while maintaining data privacy. Security parameters such as correlation coefficient (−0.004), entropy (7.95), energy (0.01), contrast (10.57), number of pixel change rate (4.86), unified average change intensity (33.66), and more are in favor of our proposed encryption scheme. To the best of our knowledge, this research is also one of the first studies that applies PPDL-based techniques to satellite image data in any capacity.
What makes people install a COVID-19 contact-tracing app? Understanding the influence of app design and individual difference on contact-tracing app adoption intention

Smartphone-based contact-tracing apps are a promising solution to help scale up the conventional contact-tracing process. However, low adoption rates have become a major issue that prevents these apps from achieving their full potential. In this paper, we present a national-scale survey experiment (N=1963) in the U.S. to investigate the effects of app design choices and individual differences on COVID-19 contact-tracing app adoption intentions. We found that individual differences such as prosocialness, COVID-19 risk perceptions, general privacy concerns, technology readiness, and demographic factors played a more important role than app design choices such as decentralized design vs. centralized design, location use, app providers, and the presentation of security risks. Certain app designs could exacerbate the different preferences in different sub-populations which may lead to an inequality of acceptance to certain app design choices (e.g., developed by state health authorities vs. a large tech company) among different groups of people (e.g., people living in rural areas vs. people living in urban areas). Our mediation analysis showed that one’s perception of the public health benefits offered by the app and the adoption willingness of other people had a larger effect in explaining the observed effects of app design choices and individual differences than one’s perception of the app’s security and privacy risks. With these findings, we discuss practical implications on the design, marketing, and deployment of COVID-19 contact-tracing apps in the U.S.
Deniable Authentication Encryption for Privacy Protection using Blockchain

Cloud applications that work on medical data using blockchain is used by managers and doctors in order to get the image data that is shared between various healthcare institutions. To ensure workability and privacy of the image data, it is important to verify the authenticity of the data, retrieve cypher data and encrypt plain image data. An effective methodology to encrypt the data is the use of a public key authenticated encryption methodology which ensures workability and privacy of the data. But, there are a number of such methodologies available that have been formulated previously. However, the drawback with those methodologies is their inadequacy in protecting the privacy of the data. In order to overcome these disadvantages, we propose a searchable encryption algorithm that can be used for sharing blockchain- based medical image data. This methodology provides traceability, unforgettable and non-tampered image data using blockhain technology, overcoming the drawbacks of blockchain such as computing power and storage. The proposed work will also sustain keyword guessing attacks apart from verification of authenticity and privacy protection of the image data. Taking these factors into consideration, it is determine that there is much work involved in providing stronger security and protecting privacy of data senders. The proposed methodology also meets the requirement of indistinguishability of trapdoor and ciphertext. The highlights of the proposed work are its capability in improving the performance of the system in terms of security and privacy protection.
Symmetry and simplicity spontaneously emerge from the algorithmic nature of evolution

Significance Why does evolution favor symmetric structures when they only represent a minute subset of all possible forms? Just as monkeys randomly typing into a computer language will preferentially produce outputs that can be generated by shorter algorithms, so the coding theorem from algorithmic information theory predicts that random mutations, when decoded by the process of development, preferentially produce phenotypes with shorter algorithmic descriptions. Since symmetric structures need less information to encode, they are much more likely to appear as potential variation. Combined with an arrival-of-the-frequent mechanism, this algorithmic bias predicts a much higher prevalence of low-complexity (high-symmetry) phenotypes than follows from natural selection alone and also explains patterns observed in protein complexes, RNA secondary structures, and a gene regulatory network.
Transfer learning for medical images analyses: A survey

The advent of deep learning has brought great change to the community of computer science and also revitalized numerous fields where traditional machine learning methods failed to make breakthroughs. Benefitted from the development of deep learning, analysis of medical images, which used to be a challenging yet exhausting task carried out manually by physicians, has experienced fast development as well. However, training deep learning models in these systems for analysis from scratch can be quite challenging. The small-scale data can’t guarantee the performance of the developed systems, while large-scale data is usually unavailable due to expensive costs in the process of collection and storage. To allow a fast transition from one domain to another for reuse, experts and researchers have extensively delved transfer learning, which turns out to be an efficient and low-cost learning technique. In this paper, we will present a comprehensive survey of transfer learning on medical image analysis. The imaging modalities include but not limits to Computed Tomography (CT), Ultrasound (US), and Magnetic Resonance Imaging (MRI). The subjects covered in this paper include the brain, breast, lung, kidney, etc. Besides, this survey provides systematic knowledge about deep learning and transfer learning for beginners. Readers with different backgrounds can easily catch up with the interdisciplinary knowledge and new trends of transfer learning via this survey.
Sketch-n-Sketch

For creative tasks, programmers face a choice: Use a GUI and sacrifice flexibility, or write code and sacrifice ergonomics?
Class-Balanced Loss Based on Effective Number of Samples

With the rapid increase of large-scale, real-world datasets, it becomes critical to address the problem of long-tailed data distribution (i.e., a few classes account for most of the data, while most classes are under-represented). Existing solutions typically adopt class re-balancing strategies such as re-sampling and re-weighting based on the number of observations for each class. In this work, we argue that as the number of samples increases, the additional benefit of a newly added data point will diminish. We introduce a novel theoretical framework to measure data overlap by associating with each sample a small neighboring region rather than a single point. The effective number of samples is defined as the volume of samples and can be calculated by a simple formula $(1-\beta^{n})/(1-\beta)$, where $n$ is the number of samples and $\beta \in [0,1)$ is a hyperparameter. We design a re-weighting scheme that uses the effective number of samples for each class to re-balance the loss, thereby yielding a class-balanced loss. Comprehensive experiments are conducted on artificially induced long-tailed CIFAR datasets and large-scale datasets including ImageNet and iNaturalist. Our results show that when trained with the proposed class-balanced loss, the network is able to achieve significant performance gains on long-tailed datasets.
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations

A rapidly growing area of work has studied the existence of adversarial examples, datapoints which have been perturbed to fool a classifier, but the vast majority of these works have focused primarily on threat models defined by $\ell_p$ norm-bounded perturbations. In this paper, we propose a new threat model for adversarial attacks based on the Wasserstein distance. In the image classification setting, such distances measure the cost of moving pixel mass, which naturally cover "standard" image manipulations such as scaling, rotation, translation, and distortion (and can potentially be applied to other settings as well). To generate Wasserstein adversarial examples, we develop a procedure for projecting onto the Wasserstein ball, based upon a modified version of the Sinkhorn iteration. The resulting algorithm can successfully attack image classification models, bringing traditional CIFAR10 models down to 3% accuracy within a Wasserstein ball with radius 0.1 (i.e., moving 10% of the image mass 1 pixel), and we demonstrate that PGD-based adversarial training can improve this adversarial accuracy to 76%. In total, this work opens up a new direction of study in adversarial robustness, more formally considering convex metrics that accurately capture the invariances that we typically believe should exist in classifiers. Code for all experiments in the paper is available at https://github.com/locuslab/projected_sinkhorn.
Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection

This report presents our method which wins the nuScenes3D Detection Challenge [17] held in Workshop on Autonomous Driving(WAD, CVPR 2019). Generally, we utilize sparse 3D convolution to extract rich semantic features, which are then fed into a class-balanced multi-head network to perform 3D object detection. To handle the severe class imbalance problem inherent in the autonomous driving scenarios, we design a class-balanced sampling and augmentation strategy to generate a more balanced data distribution. Furthermore, we propose a balanced group-ing head to boost the performance for the categories withsimilar shapes. Based on the Challenge results, our methodoutperforms the PointPillars [14] baseline by a large mar-gin across all metrics, achieving state-of-the-art detection performance on the nuScenes dataset. Code will be released at CBGS.
Optimal measurements for quantum multiparameter estimation with general states

We generalize the approach by Braunstein and Caves [Phys. Rev. Lett. 72, 3439 (1994)] to quantum multi-parameter estimation with general states. We derive a matrix bound of the classical Fisher information matrix due to each measurement operator. The saturation of all these bounds results in the saturation of the matrix Helstrom Cram\'er-Rao bound. Remarkably, the saturation of the matrix bound is equivalent to the saturation of the scalar bound with respect to any given positive definite weight matrix. Necessary and sufficient conditions are obtained for the optimal measurements that give rise to the Helstrom Cram\'er-Rao bound associated with a general quantum state. To saturate the Helstrom bound with separable measurements or collective measurement entangling only a small number of identical states, we find it is necessary for the symmetric logarithmic derivatives to commute on the support of the state. As an important application of our results, we construct several local optimal measurements for the problem of estimating the three-dimensional separation of two incoherent optical point sources.
KG-BERT: BERT for Knowledge Graph Completion

Knowledge graphs are important resources for many artificial intelligence tasks but often suffer from incompleteness. In this work, we propose to use pre-trained language models for knowledge graph completion. We treat triples in knowledge graphs as textual sequences and propose a novel framework named Knowledge Graph Bidirectional Encoder Representations from Transformer (KG-BERT) to model these triples. Our method takes entity and relation descriptions of a triple as input and computes scoring function of the triple with the KG-BERT language model. Experimental results on multiple benchmark knowledge graphs show that our method can achieve state-of-the-art performance in triple classification, link prediction and relation prediction tasks.
Credit Card Fraud Detection using Machine Learning and Data Science

It is vital that credit card companies are able to identify fraudulent credit card transactions so that customers are not charged for items that they did not purchase.Such problems can be tackled with Data Science and its importance, along with Machine Learning, cannot be overstated.This project intends to illustrate the modelling of a data set using machine learning with Credit Card Fraud Detection.The Credit Card Fraud Detection Problem includes modelling past credit card transactions with the data of the ones that turned out to be fraud.This model is then used to recognize whether a new transaction is fraudulent or not.Our objective here is to detect 100% of the fraudulent transactions while minimizing the incorrect fraud classifications.Credit Card Fraud Detection is a typical sample of classification.In this process, we have focused on analysing and pre-processing data sets as well as the deployment of multiple anomaly detection algorithms such as Local Outlier Factor and Isolation Forest algorithm on the PCA transformed Credit Card Transaction data.
Impact and Detection of Facial Beautification in Face Recognition: An Overview

Facial beautification induced by plastic surgery, cosmetics or retouching has the ability to substantially alter the appearance of face images.Such types of beautification can negatively affect the accuracy of face recognition systems.In this work, a conceptual categorisation of beautification is presented, relevant scenarios with respect to face recognition are discussed, and related publications are revisited.Additionally, technical considerations and trade-offs of the surveyed methods are summarized along with open issues and challenges in the field.This survey is targeted to provide a comprehensive point of reference for biometric researchers and practitioners working in the field of face recognition, who aim at tackling challenges caused by facial beautification.