Reviewerly/exHarmony huggingface dataset:
COTR: Correspondence Transformer for Matching Across Images

We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other. By doing so, one has the option to query only the points of interest and retrieve sparse correspondences, or to query all points in an image and obtain dense mappings. Importantly, in order to capture both local and global priors, and to let our model relate between image regions using the most relevant among said priors, we realize our network using a transformer. At inference time, we apply our correspondence network by recursively zooming in around the estimates, yielding a multiscale pipeline able to provide highly-accurate correspondences. Our method significantly outperforms the state of the art on both sparse and dense correspondence problems on multiple datasets and tasks, ranging from wide-baseline stereo to optical flow, without any retraining for a specific dataset. We commit to releasing data, code, and all the tools necessary to train from scratch and ensure reproducibility.
Focal Frequency Loss for Image Reconstruction and Synthesis

Image reconstruction and synthesis have witnessed remarkable progress thanks to the development of generative models. Nonetheless, gaps could still exist between the real and generated images, especially in the frequency domain. In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further. We propose a novel focal frequency loss, which allows a model to adaptively focus on frequency components that are hard to synthesize by down-weighting the easy ones. This objective function is complementary to existing spatial losses, offering great impedance against the loss of important frequency information due to the inherent bias of neural networks. We demonstrate the versatility and effectiveness of focal frequency loss to improve popular models, such as VAE, pix2pix, and SPADE, in both perceptual quality and quantitative performance. We further show its potential on StyleGAN2. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> , <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>
Learn to Match: Automatic Matching Network Design for Visual Tracking

Siamese tracking has achieved groundbreaking performance in recent years, where the essence is the efficient matching operator cross-correlation and its variants. Besides the remarkable success, it is important to note that the heuristic matching network design relies heavily on expert experience. Moreover, we experimentally find that one sole matching operator is difficult to guarantee stable tracking in all challenging environments. Thus, in this work, we introduce six novel matching operators from the perspective of feature fusion instead of explicit similarity learning, namely Concatenation, Pointwise-Addition, Pairwise-Relation, FiLM, Simple-Transformer and Transductive-Guidance, to explore more feasibility on matching operator selection. The analyses reveal these operators’ selective adaptability on different environment degradation types, which inspires us to combine them to explore complementary features. To this end, we propose binary channel manipulation (BCM) to search for the optimal combination of these operators. BCM determines to retrain or discard one operator by learning its contribution to other tracking steps. By inserting the learned matching networks to a strong baseline tracker Ocean [47], our model achieves favorable gains by 67.2 → 71.4, 52.6 → 58.3, 70.3 → 76.0 success on OTB100, LaSOT, and TrackingNet, respectively. Notably, Our tracker, dubbed AutoMatch, uses less than half of training data/time than the baseline tracker, and runs at 50 FPS using PyTorch. Code and model are released at https://github.com/JudasDie/SOTS.
100,000-spin coherent Ising machine

Computers based on physical systems are increasingly anticipated to overcome the impending limitations on digital computer performance. One such computer is a coherent Ising machine (CIM) for solving combinatorial optimization problems. Here, we report a CIM with 100,512 degenerate optical parametric oscillator pulses working as the Ising spins. We show that the CIM delivers fine solutions to maximum cut problems of 100,000-node graphs drastically faster than standard simulated annealing. Moreover, the CIM, when operated near the phase transition point, provides some extremely good solutions and a very broad distribution. This characteristic will be useful for applications that require fast random sampling such as machine learning.
Generalization in quantum machine learning from few training data

Modern quantum machine learning (QML) methods involve variationally optimizing a parameterized quantum circuit on a training data set, and subsequently making predictions on a testing data set (i.e., generalizing). In this work, we provide a comprehensive study of generalization performance in QML after training on a limited number $N$ of training data points. We show that the generalization error of a quantum machine learning model with $T$ trainable gates scales at worst as $\sqrt{T/N}$. When only $K \ll T$ gates have undergone substantial change in the optimization process, we prove that the generalization error improves to $\sqrt{K / N}$. Our results imply that the compiling of unitaries into a polynomial number of native gates, a crucial application for the quantum computing industry that typically uses exponential-size training data, can be sped up significantly. We also show that classification of quantum states across a phase transition with a quantum convolutional neural network requires only a very small training data set. Other potential applications include learning quantum error correcting codes or quantum dynamical simulation. Our work injects new hope into the field of QML, as good generalization is guaranteed from few training data.
FMD-Yolo: An efficient face mask detection method for COVID-19 prevention and control in public

Coronavirus disease 2019 (COVID-19) is a world-wide epidemic and efficient prevention and control of this disease has become the focus of global scientific communities. In this paper, a novel face mask detection framework FMD-Yolo is proposed to monitor whether people wear masks in a right way in public, which is an effective way to block the virus transmission. In particular, the feature extractor employs Im-Res2Net-101 which combines Res2Net module and deep residual network, where utilization of hierarchical convolutional structure, deformable convolution and non-local mechanisms enables thorough information extraction from the input. Afterwards, an enhanced path aggregation network En-PAN is applied for feature fusion, where high-level semantic information and low-level details are sufficiently merged so that the model robustness and generalization ability can be enhanced. Moreover, localization loss is designed and adopted in model training phase, and Matrix NMS method is used in the inference stage to improve the detection efficiency and accuracy. Benchmark evaluation is performed on two public databases with the results compared with other eight state-of-the-art detection algorithms. At IoU = 0.5 level, proposed FMD-Yolo has achieved the best precision AP50 of 92.0% and 88.4% on the two datasets, and AP75 at IoU = 0.75 has improved 5.5% and 3.9% respectively compared with the second one, which demonstrates the superiority of FMD-Yolo in face mask detection with both theoretical values and practical significance.
Human activity recognition in artificial intelligence framework: a narrative review

Human activity recognition (HAR) has multifaceted applications due to its worldly usage of acquisition devices such as smartphones, video cameras, and its ability to capture human activity data. While electronic devices and their applications are steadily growing, the advances in Artificial intelligence (AI) have revolutionized the ability to extract deep hidden information for accurate detection and its interpretation. This yields a better understanding of rapidly growing acquisition devices, AI, and applications, the three pillars of HAR under one roof. There are many review articles published on the general characteristics of HAR, a few have compared all the HAR devices at the same time, and few have explored the impact of evolving AI architecture. In our proposed review, a detailed narration on the three pillars of HAR is presented covering the period from 2011 to 2021. Further, the review presents the recommendations for an improved HAR design, its reliability, and stability. Five major findings were: (1) HAR constitutes three major pillars such as devices, AI and applications; (2) HAR has dominated the healthcare industry; (3) Hybrid AI models are in their infancy stage and needs considerable work for providing the stable and reliable design. Further, these trained models need solid prediction, high accuracy, generalization, and finally, meeting the objectives of the applications without bias; (4) little work was observed in abnormality detection during actions; and (5) almost no work has been done in forecasting actions. We conclude that: (a) HAR industry will evolve in terms of the three pillars of electronic devices, applications and the type of AI. (b) AI will provide a powerful impetus to the HAR industry in future.The online version contains supplementary material available at 10.1007/s10462-021-10116-x.
Learning Self-Consistency for Deepfake Detection

We propose a new method to detect deepfake images using the cue of the source feature inconsistency within the forged images. It is based on the hypothesis that images’ distinct source features can be preserved and extracted after going through state-of-the-art deepfake generation processes. We introduce a novel representation learning approach, called pair-wise self-consistency learning (PCL), for training ConvNets to extract these source features and detect deepfake images. It is accompanied by a new image synthesis approach, called inconsistency image genera-tor (I2G), to provide richly annotated training data for PCL. Experimental results on seven popular datasets show that our models improve averaged AUC over the state of the art from 96.45% to 98.05% in the in-dataset evaluation and from 86.03% to 92.18% in the cross-dataset evaluation.
Simulations of subatomic many-body physics on a quantum frequency processor

Simulating complex many-body quantum phenomena is a major scientific impetus behind the development of quantum computing, and a range of technologies are being explored to address such systems. We present the results of the largest photonics-based simulation to date, applied in the context of subatomic physics. Using an all-optical quantum frequency processor, the ground-state energies of light nuclei including the triton ($^{3}\mathrm{H}$), $^{3}\mathrm{He}$, and the alpha particle ($^{4}\mathrm{He}$) are computed. Complementing these calculations and utilizing a 68-dimensional Hilbert space, our photonic simulator is used to perform subnucleon calculations of the two- and three-body forces between heavy mesons in the Schwinger model. This work is a first step in simulating subatomic many-body physics on quantum frequency processors---augmenting classical computations that bridge scales from quarks to nuclei.
A Comprehensive Database for Benchmarking Imaging Systems

Cross-modality face recognition is an emerging topic due to the wide-spread usage of different sensors in day-to-day life applications. The development of face recognition systems relies greatly on existing databases for evaluation and obtaining training examples for data-hungry machine learning algorithms. However, currently, there is no publicly available face database that includes more than two modalities for the same subject. In this work, we introduce the Tufts Face Database that includes images acquired in various modalities: photograph images, thermal images, near infrared images, a recorded video, a computerized facial sketch, and 3D images of each volunteer's face. An Institutional Research Board protocol was obtained and images were collected from students, staff, faculty, and their family members at Tufts University. The database includes over 10,000 images from 113 individuals from more than 15 different countries, various gender identities, ages, and ethnic backgrounds. The contributions of this work are: 1) Detailed description of the content and acquisition procedure for images in the Tufts Face Database; 2) The Tufts Face Database is publicly available to researchers worldwide, which will allow assessment and creation of more robust, consistent, and adaptable recognition algorithms; 3) A comprehensive, up-to-date review on face recognition systems and face datasets.
Security and privacy issues in e-health cloud-based system: A comprehensive content analysis

The recent advancement in Information and Communication Technology (ICT) has undoubtedly improved services in all sectors in the world. Specifically, Information Technology (IT) has led to a very vital innovation in health sector called electronic health (e-Health). In order to optimize full and excellent benefits of this innovation, its implementation in a cloud-based environment is important. However, with noticeable and numerous benefits inherent from e-Health in a cloud computing, its full utilization is still being hampered by challenges of security and privacy. In this paper, we focused on extensive review of current and existing literatures of various approaches and mechanisms being used to handle security and privacy related matters in e-Health. Strengths and weaknesses of some of these approaches were enunciated. The literature review was carried out after selecting over One Hundred and Ten (1 1 0) original articles and figured out several models adopted in their solutions. After comparing models used, we arrived at the reviewed articles. Reviewed articles were narrowed down to the current number because of similarity observed in the models adopted by some researchers. Also, we give an acceptable and standard definition of e-Health. Effort was made to classify cloud-based models. Security and privacy requirements as recommended by Health Insurance Portability and Accountability Act (HIPAA) were also discussed and provided. Remarks and recommendations were made regarding the review process and future directions on security and privacy of e-Health in cloud computing was also provided. Finally, authors propose a secured and dependable architecture for electronic health that could guarantee efficiency, reliability and regulated access framework to health information. The architecture, though is currently under implementation, will guarantee absolute security and privacy between healthcare providers and the patients.
Resource scheduling algorithm with load balancing for cloud service provisioning

Cloud computing uses scheduling and load balancing for virtualized file sharing in cloud infrastructure. These two have to be performed in an optimized manner in cloud computing environment to achieve optimal file sharing. Recently, Scalable traffic management has been developed in cloud data centers for traffic load balancing and quality of service provisioning. However, latency reducing during multidimensional resource allocation still remains a challenge. Hence, there necessitates efficient resource scheduling for ensuring load optimization in cloud. The objective of this work is to introduce an integrated resource scheduling and load balancing algorithm for efficient cloud service provisioning. The method constructs a Fuzzy-based Multidimensional Resource Scheduling model to obtain resource scheduling efficiency in cloud infrastructure. Increasing utilization of Virtual Machines through effective and fair load balancing is then achieved by dynamically selecting a request from a class using Multidimensional Queuing Load Optimization algorithm. A load balancing algorithm is then implemented to avoid underutilization and overutilization of resources, improving latency time for each class of request. Simulations were conducted to evaluate the effectiveness using Cloudsim simulator in cloud data centers and results shows that the proposed method achieves better performance in terms of average success rate, resource scheduling efficiency and response time. Simulation analysis shows that the method improves the resource scheduling efficiency by 7% and also reduces the response time by 35.5 % when compared to the state-of-the-art works.
Proof of Authentication: IoT-Friendly Blockchains

This article introduces the concept of proof of authentication (PoAh) for the lightweight implementation of blockchains in the Internet of Things (IoT). The PoAh can replace existing consensus algorithms, such as proof of work (PoW), proof of stake (PoS), and proof of activity (PoA), for resource- and energy-constrained infrastructures, such as the IoT.
A Deep Scene Representation for Aerial Scene Classification

As a fundamental problem in earth observation, aerial scene classification tries to assign a specific semantic label to an aerial image. In recent years, the deep convolutional neural networks (CNNs) have shown advanced performances in aerial scene classification. The successful pretrained CNNs can be transferable to aerial images. However, global CNN activations may lack geometric invariance and, therefore, limit the improvement of aerial scene classification. To address this problem, this paper proposes a deep scene representation to achieve the invariance of CNN features and further enhance the discriminative power. The proposed method: 1) extracts CNN activations from the last convolutional layer of pretrained CNN; 2) performs multiscale pooling (MSP) on these activations; and 3) builds a holistic representation by the Fisher vector method. MSP is a simple and effective multiscale strategy, which enriches multiscale spatial information in affordable computational time. The proposed representation is particularly suited at aerial scenes and consistently outperforms global CNN activations without requiring feature adaptation. Extensive experiments on five aerial scene data sets indicate that the proposed method, even with a simple linear classifier, can achieve the state-of-the-art performance.
Will virtual reality be a double-edged sword? Exploring the moderation effects of the expected enjoyment of a destination on travel intention

Although virtual reality (VR) has been found to be more effective in tourism marketing compared to traditional methods, few scholars have explored whether VR has any negative effect on tourism marketing. Based on extrinsic and intrinsic motivation theory, tourists' perceptions of a VR device and VR content are distinguished to explore whether VR could be a double-edged sword. Using a scenario-based questionnaire survey and an experiment, the study found that perceived enjoyment of VR mediates the positive effects of tourists' perceived ease of use and the usefulness of VR on their travel intention. However, the positive relationship is moderated by tourists' expected enjoyment of the destination. When the expected enjoyment of destination is low, higher perceived enjoyment of VR actually leads to lower travel intention. The findings of this study provide evidence that VR will inhibit tourists’ travel intention under certain conditions and offer a new perspective for technology-related tourism studies.
Deep neural network for hierarchical extreme multi-label text classification

The classification of natural language texts has gained a growing importance in many real world applications due to its significant implications in relation to crucial tasks, such as Information Retrieval, Question Answering, Text Summarization, Natural Language Understanding. In this paper we present an analysis of a Deep Learning architecture devoted to text classification, considering the extreme multi-class and multi-label text classification problem, when a hierarchical label set is defined. The paper presents a methodology named Hierarchical Label Set Expansion (HLSE), used to regularize the data labels, and an analysis of the impact of different Word Embedding (WE) models that explicitly incorporate grammatical and syntactic features. We evaluate the aforementioned methodologies on the PubMed scientific articles collection, where a multi-class and multi-label text classification problem is defined with the Medical Subject Headings (MeSH) label set, a hierarchical set of 27,775 classes. The experimental assessment proves the usefulness of the proposed HLSE methodology and also provides some interesting results relating to the impact of different uses and combinations of WE models as input to the neural network in this kind of application.
Effective Aesthetics Prediction With Multi-Level Spatially Pooled Features

We propose an effective deep learning approach to aesthetics quality assessment that relies on a new type of pre-trained features, and apply it to the AVA data set, the currently largest aesthetics database. While previous approaches miss some of the information in the original images, due to taking small crops, down-scaling or warping the originals during training, we propose the first method that efficiently supports full resolution images as an input, and can be trained on variable input sizes. This allows us to significantly improve upon the state of the art, increasing the Spearman rank-order correlation coefficient (SRCC) of ground-truth mean opinion scores (MOS) from the existing best reported of 0.612 to 0.756. To achieve this performance, we extract multi-level spatially pooled (MLSP) features from all convolutional blocks of a pre-trained InceptionResNet-v2 network, and train a custom shallow Convolutional Neural Network (CNN) architecture on these new features.
Cryptocurrencies as hedges and safe-havens for US equity sectors

We uncover the hedging and safe-haven properties of eight cryptocurrencies against down movements in the S&P 500 and its 10 equity sectors. Evidence from the cross-quantilogram approach supports many cryptocurrencies as a potentially valuable digital asset class. However, a significant heterogeneity is reported in many cases. Bitcoin, Ripple and Stellar are safe-havens for all US equity indices, while Litecoin and Monero are safe-havens for the aggerate US equity index and selected sectors. As for Ethereum, Dash and Nem, they are hedges for few equity sectors. Our findings help investors in improving their ability to offset equity losses with new cryptocurrencies.