Reviewerly/exHarmony huggingface dataset:
Theory of deep convolutional neural networks: Downsampling

Establishing a solid theoretical foundation for structured deep neural networks is greatly desired due to the successful applications of deep learning in various practical domains. This paper aims at an approximation theory of deep convolutional neural networks whose structures are induced by convolutions. To overcome the difficulty in theoretical analysis of the networks with linearly increasing widths arising from convolutions, we introduce a downsampling operator to reduce the widths. We prove that the downsampled deep convolutional neural networks can be used to approximate ridge functions nicely, which hints some advantages of these structured networks in terms of approximation or modeling. We also prove that the output of any multi-layer fully-connected neural network can be realized by that of a downsampled deep convolutional neural network with free parameters of the same order, which shows that in general, the approximation ability of deep convolutional neural networks is at least as good as that of fully-connected networks. Finally, a theorem for approximating functions on Riemannian manifolds is presented, which demonstrates that deep convolutional neural networks can be used to learn manifold features of data.
Mobile-assisted language learning: A Duolingo case study

Abstract The growing availability of mobile technologies has contributed to an increase in mobile-assisted language learning in which learners can autonomously study a second language (L2) anytime or anywhere (e.g. Kukulska-Hulme, Lee &amp; Norris, 2017; Reinders &amp; Benson, 2017). Research investigating the effectiveness of such study for L2 learning, however, has been limited, especially regarding large-scale commercial L2 learning apps, such as Duolingo. Although one commissioned research study found favorable language learning outcomes (Vesselinov &amp; Grego, 2012), limited independent research has reported issues related to learner persistence, motivation, and program efficacy (Lord, 2015; Nielson, 2011). The current study investigates the semester-long learning experiences and results of nine participants learning Turkish on Duolingo. The participants showed improvement on L2 measures at the end of the study, and results indicate a positive, moderate correlation between the amount of time spent on Duolingo and learning gains. In terms of perceptions of their experiences, the participants generally viewed Duolingo’s flexibility and gamification aspects positively; however, variability in motivation to study and frustration with instructional materials were also expressed.
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.
Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks

Abstract We present a novel approach to automatic Sign Language Production using recent developments in Neural Machine Translation (NMT), Generative Adversarial Networks, and motion generation. Our system is capable of producing sign videos from spoken language sentences. Contrary to current approaches that are dependent on heavily annotated data, our approach requires minimal gloss and skeletal level annotations for training. We achieve this by breaking down the task into dedicated sub-processes. We first translate spoken language sentences into sign pose sequences by combining an NMT network with a Motion Graph. The resulting pose information is then used to condition a generative model that produces photo realistic sign language video sequences. This is the first approach to continuous sign video generation that does not use a classical graphical avatar. We evaluate the translation abilities of our approach on the PHOENIX14 T Sign Language Translation dataset. We set a baseline for text-to-gloss translation, reporting a BLEU-4 score of 16.34/15.26 on dev/test sets. We further demonstrate the video generation capabilities of our approach for both multi-signer and high-definition settings qualitatively and quantitatively using broadcast quality assessment metrics.
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media

In this paper, we describe our approach to utilize pre-trained BERT models with Convolutional Neural Networks for sub-task A of the Multilingual Offensive Language Identification shared task (OffensEval 2020), which is a part of the SemEval 2020.We show that combining CNN with BERT is better than using BERT on its own, and we emphasize the importance of utilizing pre-trained language models for downstream tasks.Our system, ranked 4 th with macro averaged F1-Score of 0.897 in Arabic, 4 th with score of 0.843 in Greek, and 3 rd with score of 0.814 in Turkish.Additionally, we present ArabicBERT, a set of pre-trained transformer language models for Arabic that we share with the community.
STC Antispoofing Systems for the ASVspoof2019 Challenge

This paper describes the Speech Technology Center (STC) antispoofing systems submitted to the ASVspoof 2019 challenge. The ASVspoof2019 is the extended version of the previous challenges and includes 2 evaluation conditions: logical access use-case scenario with speech synthesis and voice conversion attack types and physical access use-case scenario with replay attacks. During the challenge we developed anti-spoofing solutions for both scenarios. The proposed systems are implemented using deep learning approach and are based on different types of acoustic features. We enhanced Light CNN architecture previously considered by the authors for replay attacks detection and which performed high spoofing detection quality during the ASVspoof2017 challenge. In particular here we investigate the efficiency of angular margin based softmax activation for training robust deep Light CNN classifier to solve the mentioned-above tasks. Submitted systems achieved EER of 1.86% in logical access scenario and 0.54% in physical access scenario on the evaluation part of the Challenge corpora. High performance obtained for the unknown types of spoofing attacks demonstrates the stability of the offered approach in both evaluation conditions.
Content and Style Disentanglement for Artistic Style Transfer

Artists rarely paint in a single style throughout their career. More often they change styles or develop variations of it. In addition, artworks in different styles and even within one style depict real content differently: while Picasso's Blue Period displays a vase in a blueish tone but as a whole, his Cubist works deconstruct the object. To produce artistically convincing stylizations, style transfer models must be able to reflect these changes and variations. Recently many works have aimed to improve the style transfer task, but neglected to address the described observations. We present a novel approach which captures particularities of style and the variations within and separates style and content. This is achieved by introducing two novel losses: a fixpoint triplet style loss to learn subtle variations within one style or between different styles and a disentanglement loss to ensure that the stylization is not conditioned on the real input photo. In addition the paper proposes various evaluation methods to measure the importance of both losses on the validity, quality and variability of final stylizations. We provide qualitative results to demonstrate the performance of our approach.
Superconducting nanowire single-photon detectors for quantum information

Abstract The superconducting nanowire single-photon detector (SNSPD) is a quantum-limit superconducting optical detector based on the Cooper-pair breaking effect by a single photon, which exhibits a higher detection efficiency, lower dark count rate, higher counting rate, and lower timing jitter when compared with those exhibited by its counterparts. SNSPDs have been extensively applied in quantum information processing, including quantum key distribution and optical quantum computation. In this review, we present the requirements of single-photon detectors from quantum information, as well as the principle, key metrics, latest performance issues, and other issues associated with SNSPD. The representative applications of SNSPDs with respect to quantum information will also be covered.
A multi-layered gravitational search algorithm for function optimization and real-world problems

A gravitational search algorithm (GSA) uses gravitational force among individuals to evolve population. Though GSA is an effective population-based algorithm, it exhibits low search performance and premature convergence. To ameliorate these issues, this work proposes a multi-layered GSA called MLGSA. Inspired by the two-layered structure of GSA, four layers consisting of population, iteration-best, personal-best and global-best layers are constructed. Hierarchical interactions among four layers are dynamically implemented in different search stages to greatly improve both exploration and exploitation abilities of population. Performance comparison between MLGSA and nine existing GSA variants on twenty-nine CEC2017 test functions with low, medium and high dimensions demonstrates that MLGSA is the most competitive one. It is also compared with four particle swarm optimization variants to verify its excellent performance. Moreover, the analysis of hierarchical interactions is discussed to illustrate the influence of a complete hierarchy on its performance. The relationship between its population diversity and fitness diversity is analyzed to clarify its search performance. Its computational complexity is given to show its efficiency. Finally, it is applied to twenty-two CEC2011 real-world optimization problems to show its practicality.
Designing Disease Prediction Model Using Machine Learning Approach

Now-a-days, people face various diseases due to the environmental condition and their living habits. So the prediction of disease at earlier stage becomes important task. But the accurate prediction on the basis of symptoms becomes too difficult for doctor. The correct prediction of disease is the most challenging task. To overcome this problem data mining plays an important role to predict the disease. Medical science has large amount of data growth per year. Due to increase amount of data growth in medical and healthcare field the accurate analysis on medical data which has been benefits from early patient care. With the help of disease data, data mining finds hidden pattern information in the huge amount of medical data. We proposed general disease prediction based on symptoms of the patient. For the disease prediction, we use K-Nearest Neighbor (KNN) and Convolutional neural network (CNN) machine learning algorithm for accurate prediction of disease. For disease prediction required disease symptoms dataset. In this general disease prediction the living habits of person and checkup information consider for the accurate prediction. The accuracy of general disease prediction by using CNN is 84.5% which is more than KNN algorithm. And the time and the memory requirement is also more in KNN than CNN. After general disease prediction, this system able to gives the risk associated with general disease which is lower risk of general disease or higher.
Neural Sign Language Translation Based on Human Keypoint Estimation

We propose a sign language translation system based on human keypoint estimation. It is well-known that many problems in the field of computer vision require a massive dataset to train deep neural network models. The situation is even worse when it comes to the sign language translation problem as it is far more difficult to collect high-quality training data. In this paper, we introduce the KETI (Korea Electronics Technology Institute) sign language dataset, which consists of 14,672 videos of high resolution and quality. Considering the fact that each country has a different and unique sign language, the KETI sign language dataset can be the starting point for further research on the Korean sign language translation. Using the KETI sign language dataset, we develop a neural network model for translating sign videos into natural language sentences by utilizing the human keypoints extracted from the face, hands, and body parts. The obtained human keypoint vector is normalized by the mean and standard deviation of the keypoints and used as input to our translation model based on the sequence-to-sequence architecture. As a result, we show that our approach is robust even when the size of the training data is not sufficient. Our translation model achieved 93.28% (55.28%, respectively) translation accuracy on the validation set (test set, respectively) for 105 sentences that can be used in emergency situations. We compared several types of our neural sign translation models based on different attention mechanisms in terms of classical metrics for measuring the translation performance.
Addressing Security and Privacy Issues of IoT Using Blockchain Technology

Internet of Things (IoT) has been the most emerging technology in the last decade because the number of smart devices and its associated technologies has rapidly grown in both industrial and research prospectives. The applications are developed using IoT techniques for real-time monitoring. Due to low processing power and storage capacity, smart things are vulnerable to the attacks as existing security or cryptography techniques are not suitable. In this study, we initially review and identify the security and privacy issues that exist in the IoT system. Second, as per blockchain technology, we provide some security solutions. The detailed analysis, including enabling technology and integration of IoT technologies, is explained. Finally, a case study is implemented using the Ethererum-based blockchain system in a smart IoT system and the results are discussed.
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation

While accurate lip synchronization has been achieved for arbitrary-subject audio-driven talking face generation, the problem of how to efficiently drive the head pose remains. Previous methods rely on pre-estimated structural information such as landmarks and 3D parameters, aiming to generate personalized rhythmic movements. However, the inaccuracy of such estimated information under extreme conditions would lead to degradation problems. In this paper, we propose a clean yet effective framework to generate pose-controllable talking faces. We operate on non-aligned raw face images, using only a single photo as an identity reference. The key is to modularize audio-visual representations by devising an implicit low-dimension pose code. Substantially, both speech content and head pose information lie in a joint non-identity embedding space. While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.Extensive experiments show that our method generates accurately lip-synced talking faces whose poses are controllable by other videos. Moreover, our model has multiple advanced capabilities including extreme view robustness and talking face frontalization. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Spectral Embedded Adaptive Neighbors Clustering

Spectral clustering has been widely used in various aspects, especially the machine learning fields. Clustering with similarity matrix and low-dimensional representation of data is the main reason of its promising performance shown in spectral clustering. However, such similarity matrix and low-dimensional representation directly derived from input data may not always hold when the data are high dimensional and has complex distribution. First, the similarity matrix simply based on the distance measurement might not be suitable for all kinds of data. Second, the low-dimensional representation might not be able to reflect the manifold structure of the original data. In this brief, we propose a novel linear space embedded clustering method, which uses adaptive neighbors to address the above-mentioned problems. Linearity regularization is used to make the data representation a linear embedded spectral. We also use adaptive neighbors to optimize the similarity matrix and clustering results simultaneously. Extensive experimental results show promising performance compared with the other state-of-the-art algorithms.
Improving quality of teaching and learning in classes by using augmented reality video

This study contributes to enhancing students’ learning experience and increasing their understanding of complex issues by incorporating an augmented reality (AR) mobile application (app) into a sewing workshop in which a threading task was carried out to facilitate better learning relative to a conventional approach. Participants included 46 freshmen. One group of students was provided with a handout, while the other was asked to view an AR video. Questionnaires were then administered, and the results showed a significant difference between the two groups in terms of their understanding of the task. This finding was consistent with the post-test scores and the time required to learn the threading task. Several feedback items indicated higher learning efficiency with the use of AR videos.
DeepCF: A Unified Framework of Representation Learning and Matching Function Learning in Recommender System

In general, recommendation can be viewed as a matching problem, i.e., match proper items for proper users. However, due to the huge semantic gap between users and items, it’s almost impossible to directly match users and items in their initial representation spaces. To solve this problem, many methods have been studied, which can be generally categorized into two types, i.e., representation learning-based CF methods and matching function learning-based CF methods. Representation learning-based CF methods try to map users and items into a common representation space. In this case, the higher similarity between a user and an item in that space implies they match better. Matching function learning-based CF methods try to directly learn the complex matching function that maps user-item pairs to matching scores. Although both methods are well developed, they suffer from two fundamental flaws, i.e., the limited expressiveness of dot product and the weakness in capturing low-rank relations respectively. To this end, we propose a general framework named DeepCF, short for Deep Collaborative Filtering, to combine the strengths of the two types of methods and overcome such flaws. Extensive experiments on four publicly available datasets demonstrate the effectiveness of the proposed DeepCF framework.
Linearized two-layers neural networks in high dimension

We consider the problem of learning an unknown function f⋆ on the d-dimensional sphere with respect to the square loss, given i.i.d. samples {(yi,xi)}i≤n where xi is a feature vector uniformly distributed on the sphere and yi=f⋆(xi)+εi. We study two popular classes of models that can be regarded as linearizations of two-layers neural networks around a random initialization: the random features model of Rahimi–Recht (RF); the neural tangent model of Jacot–Gabriel–Hongler (NT). Both these models can also be regarded as randomized approximations of kernel ridge regression (with respect to different kernels), and enjoy universal approximation properties when the number of neurons N diverges, for a fixed dimension d. We consider two specific regimes: the infinite-sample finite-width regime, in which n=∞ while d and N are large but finite, and the infinite-width finite-sample regime in which N=∞ while d and n are large but finite. In the first regime, we prove that if dℓ+δ≤N≤dℓ+1−δ for small δ>0, then RF effectively fits a degree-ℓ polynomial in the raw features, and NT fits a degree-(ℓ+1) polynomial. In the second regime, both RF and NT reduce to kernel methods with rotationally invariant kernels. We prove that, if the sample size satisfies dℓ+δ≤n≤dℓ+1−δ, then kernel methods can fit at most a degree-ℓ polynomial in the raw features. This lower bound is achieved by kernel ridge regression, and near-optimal prediction error is achieved for vanishing ridge regularization.
Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model

Sarcasm is a subtle form of language in which people express the opposite of what is implied.Previous works of sarcasm detection focused on texts.However, more and more social media platforms like Twitter allow users to create multi-modal messages, including texts, images, and videos.It is insufficient to detect sarcasm from multi-model messages based only on texts.In this paper, we focus on multimodal sarcasm detection for tweets consisting of texts and images in Twitter.We treat text features, image features and image attributes as three modalities and propose a multi-modal hierarchical fusion model to address this task.Our model first extracts image features and attribute features, and then leverages attribute features and bidirectional LSTM network to extract text features.Features of three modalities are then reconstructed and fused into one feature vector for prediction.We create a multi-modal sarcasm detection dataset based on Twitter.Evaluation results on the dataset demonstrate the efficacy of our proposed model and the usefulness of the three modalities.
STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems

We propose a new STAcked and Reconstructed Graph Convolutional Networks (STAR-GCN) architecture to learn node representations for boosting the performance in recommender systems, especially in the cold start scenario. STAR-GCN employs a stack of GCN encoder-decoders combined with intermediate supervision to improve the final prediction performance. Unlike the graph convolutional matrix completion model with one-hot encoding node inputs, our STAR-GCN learns low-dimensional user and item latent factors as the input to restrain the model space complexity. Moreover, our STAR-GCN can produce node embeddings for new nodes by reconstructing masked input node embeddings, which essentially tackles the cold start problem. Furthermore, we discover a label leakage issue when training GCN-based models for link prediction tasks and propose a training strategy to avoid the issue. Empirical results on multiple rating prediction benchmarks demonstrate our model achieves state-of-the-art performance in four out of five real-world datasets and significant improvements in predicting ratings in the cold start scenario. The code implementation is available in https://github.com/jennyzhang0215/STAR-GCN.