Reviewerly/exHarmony huggingface dataset:
Multilabel Image Classification via Feature/Label Co-Projection

This article presents a simple and intuitive solution for multilabel image classification, which achieves the competitive performance on the popular COCO and PASCAL VOC benchmarks. The main idea is to capture how humans perform this task: we recognize both labels (i.e., objects and attributes) and the correlation of labels at the same time. Here, label recognition is performed by a standard ConvNet pipeline, whereas label correlation modeling is done by projecting both labels and image features extracted by the ConvNet to a common latent vector space. Specifically, we carefully design the loss function to ensure that: 1) labels and features that co-appear frequently are close to each other in the latent space and 2) conversely, labels/features that do not appear together are far apart. This information is then combined with the original ConvNet outputs to form the final prediction. The whole model is trained end-to-end, with no additional supervised information other than the image-level supervised information. Experiments show that the proposed method consistently outperforms previous approaches on COCO and PASCAL VOC in terms of mAP, macro/micro precision, recall, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F$ </tex-math></inline-formula> -measure. Further, our model is highly efficient at test time, with only a small number of additional weights compared to the base model for direct label recognition.
Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution

Generating non-existing frames from a consecutive video sequence has been an interesting and challenging problem in the video processing field. Typical kernel-based interpolation methods predict pixels with a single convolution process that convolves source frames with spatially adaptive local kernels, which circumvents the time-consuming, explicit motion estimation in the form of optical flow. However, when scene motion is larger than the pre-defined kernel size, these methods are prone to yield less plausible results. In addition, they cannot directly generate a frame at an arbitrary temporal position because the learned kernels are tied to the midpoint in time between the input frames. In this paper, we try to solve these problems and propose a novel non-flow kernel-based approach that we refer to as enhanced deformable separable convolution (EDSC) to estimate not only adaptive kernels, but also offsets, masks and biases to make the network obtain information from non-local neighborhood. During the learning process, different intermediate time step can be involved as a control variable by means of an extension of coord-conv trick, allowing the estimated components to vary with different input temporal information. This makes our method capable to produce multiple in-between frames. Furthermore, we investigate the relationships between our method and other typical kernel- and flow-based methods. Experimental results show that our method performs favorably against the state-of-the-art methods across a broad range of datasets. Code will be publicly available on URL: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Xianhang/EDSC-pytorch</uri> .
CRTransSar: A Visual Transformer Based on Contextual Joint Representation Learning for SAR Ship Detection

Synthetic-aperture radar (SAR) image target detection is widely used in military, civilian and other fields. However, existing detection methods have low accuracy due to the limitations presented by the strong scattering of SAR image targets, unclear edge contour information, multiple scales, strong sparseness, background interference, and other characteristics. In response, for SAR target detection tasks, this paper combines the global contextual information perception of transformers and the local feature representation capabilities of convolutional neural networks (CNNs) to innovatively propose a visual transformer framework based on contextual joint-representation learning, referred to as CRTransSar. First, this paper introduces the latest Swin Transformer as the basic architecture. Next, it introduces the CNN’s local information capture and presents the design of a backbone, called CRbackbone, based on contextual joint representation learning, to extract richer contextual feature information while strengthening SAR target feature attributes. Furthermore, the design of a new cross-resolution attention-enhancement neck, called CAENeck, is presented to enhance the characterizability of multiscale SAR targets. The mAP of our method on the SSDD dataset attains 97.0% accuracy, reaching state-of-the-art levels. In addition, based on the HISEA-1 commercial SAR satellite, which has been launched into orbit and in whose development our research group participated, we released a larger-scale SAR multiclass target detection dataset, called SMCDD, which verifies the effectiveness of our method.
Graph regularized Lp smooth non-negative matrix factorization for data representation

This paper proposes a Graph regularized L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> smooth non-negative matrix factorization (GSNMF) method by incorporating graph regularization and L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> smoothing constraint, which considers the intrinsic geometric information of a data set and produces smooth and stable solutions. The main contributions are as follows: first, graph regularization is added into NMF to discover the hidden semantics and simultaneously respect the intrinsic geometric structure information of a data set. Second, the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> smoothing constraint is incorporated into NMF to combine the merits of isotropic (L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm) and anisotropic (L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm) diffusion smoothing, and produces a smooth and more accurate solution to the optimization problem. Finally, the update rules and proof of convergence of GSNMF are given. Experiments on several data sets show that the proposed method outperforms related state-of-the-art methods.
Differential Privacy and Census Data: Implications for Social and Economic Research

The Census Bureau has announced new methods for disclosure control in public use data products. The new approach, known as differential privacy, represents a radical departure from current practice. In its pure form, differential privacy techniques may make the release of useful microdata impossible and limit the utility of tabular small-area data. Adoption of differential privacy will have far-reaching consequences for research. It is likely that scientists, planners, and the public will lose the free access we have enjoyed for six decades to reliable public Census Bureau data describing US social and economic change.
Educational Innovation in Higher Education: Use of Role Playing and Educational Video in Future Teachers’ Training

Information and communication technologies (ICTs) have led to the emergence of a variety of active and innovative teaching methods. This is the case in role-playing, which consists of simulating a real-life situation, in this case the school context, in which the student takes on a certain role and interacts with other students in a fictitious situation. Framed in this way, the present study aims to show if the application of the role-playing method promotes the improvement of attitude variables and practical skills. To this end, we advocated the use of a quasi-experimental methodology, with a control and experimental group and the application of a post-test. The sample is composed of 138 students from the Master of Teachers of Compulsory Secondary Education in Ceuta (Spain). The results showed that the students positively valued the application of the method, obtaining better scores in the set of variables studied, especially in motivation, creativity and collaboration. Therefore, it continues to be observed that the application of innovative methodologies through technology promotes the increase of multiple skills in the student body. This study aimed to prove that the use of active methods provides an increase in students’ skills, and that, therefore, we must bet on the use of sustainable pedagogies in order to promote a real innovation in the classrooms.
Industry Scale Semi-Supervised Learning for Natural Language Understanding

Luoxin Chen, Francisco Garcia, Varun Kumar, He Xie, Jianhua Lu. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers. 2021.
Technology supported learning and pedagogy in times of crisis: the case of COVID-19 pandemic

Abstract Online teaching within disciplines such as Engineering require experiential learning that equip future graduates with highly intellectual and professional skills to meet the demands of employers and the industry. The outbreak of COVID-19 however, has shifted the academic community into new landscapes that require educators and students to adapt and manage their expectations. Although literature reports on research attempts to study the implications of Covid-19 on the Higher Education curricular, little has been reported on its impact on Engineering Education. This paper therefore uses the theory of Emergency Management Life Cycle (mitigation, preparedness, response, and recover) as a lens to examine the challenges faced by students and academics and coping mechanism during the COVID period. This study adopts a mixed method approach using a case study from the College of Engineering at a Higher Education Institution in the UAE due to the sudden migration to online teaching amid COVID-19. Data is collected through interviews and surveys with both students and instructors on challenges, strategies and online delivery good practices that enhanced students’ learning experience. The results show that, Technology Supported Learning tools are capable of enhancing students’ experiential learning and associated competencies, however there were a number of pedagogical, technological and psychological challenges that faced students and instructors as a result of the sudden migration online, which are likely to play a role in the impediment of the students’ learning cycle, due to the lack of preparedness in response to the state of emergency created by Covid-19. Despite these challenges, the study found that instructors with effective communication skills and teaching style, competent use of technology, flexible, friendly and supportive attitude towards teaching, played a positive role in mitigating for the lack of preparedness in response to sudden migration online. The study also reveals that by overcoming some of the technical challenges such as slow internet connection and interruptions, lessons learnt from the sudden migration to online delivery amid COVID-19, will help create new opportunities for the use of blended learning approaches to meet the needs of the on-going COVID and future online deliveries.
No Training Required: Exploring Random Encoders for Sentence Classification

We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important observations about proper experimental protocol for sentence classification evaluation, together with recommendations for future research.
DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks

Recent advances in neural sequence-tosequence models have led to promising results for several language generation-based tasks, including dialogue response generation, summarization, and machine translation.However, these models are known to have several problems, especially in the context of chit-chat based dialogue systems: they tend to generate short and dull responses that are often too generic.Furthermore, these models do not ground conversational responses on knowledge and facts, resulting in turns that are not accurate, informative and engaging for the users.In this paper, we propose and experiment with a series of response generation models that aim to serve in the general scenario where in addition to the dialogue context, relevant unstructured external knowledge in the form of text is also assumed to be available for models to harness.Our proposed approach extends pointer-generator networks (See et al., 2017) by allowing the decoder to hierarchically attend and copy from external knowledge in addition to the dialogue context.We empirically show the effectiveness of the proposed model compared to several baselines including (Ghazvininejad et al., 2018;Zhang et al., 2018) through both automatic evaluation metrics and human evaluation on CONVAI2 dataset.
Experimental realization of arbitrary activation functions for optical neural networks

We experimentally demonstrate an on-chip electro-optic circuit for realizing arbitrary nonlinear activation functions for optical neural networks (ONNs). The circuit operates by converting a small portion of the input optical signal into an electrical signal and modulating the intensity of the remaining optical signal. Electrical signal processing allows the activation function circuit to realize any optical-to-optical nonlinearity that does not require amplification. Such line shapes are not constrained to those of conventional optical nonlinearities. Through numerical simulations, we demonstrate that the activation function improves the performance of an ONN on the MNIST image classification task. Moreover, the activation circuit allows for the realization of nonlinearities with far lower optical signal attenuation, paving the way for much deeper ONNs.
A Hybrid CNN-LSTM Model for SMS Spam Detection in Arabic and English Messages

Despite the rapid evolution of Internet protocol-based messaging services, SMS still remains an indisputable communication service in our lives until today. For example, several businesses consider that text messages are more effective than e-mails. This is because 82% of SMSs are read within 5 min., but consumers only open one in four e-mails they receive. The importance of SMS for mobile phone users has attracted the attention of spammers. In fact, the volume of SMS spam has increased considerably in recent years with the emergence of new security threats, such as SMiShing. In this paper, we propose a hybrid deep learning model for detecting SMS spam messages. This detection model is based on the combination of two deep learning methods CNN and LSTM. It is intended to deal with mixed text messages that are written in Arabic or English. For the comparative evaluation, we also tested other well-known machine learning algorithms. The experimental results that we present in this paper show that our CNN-LSTM model outperforms the other algorithms. It achieved a very good accuracy of 98.37%.
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages

Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved."Lowresourced"-ness is a complex problem going beyond data availability and reflects systemic problems in society.In this paper, we focus on the task of Machine Translation (MT), that plays a crucial role for information accessibility and communication worldwide.Despite immense improvements in MT over the past decade, MT is centered around a few highresourced languages.* ∀ to represent the whole Masakhane community.
A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection

A great deal of recent research effort on speech spoofing countermeasures has been invested into back-end neural networks and training criteria. We contribute to this effort with a comparative perspective in this study. Our comparison of countermeasure models on the ASVspoof 2019 logical access task takes into account recently proposed margin-based training criteria, widely used front ends, and common strategies to deal with varied-length input trials. We also measured intra-model differences through multiple training-evaluation rounds with random initialization. Our statistical analysis demonstrates that the performance of the same model may be significantly different when just changing the random initial seed. Thus, we recommend similar analysis or multiple training-evaluation rounds for further research on the database. Despite the intra-model differences, we observed a few promising techniques such as the average pooling to process varied-length inputs and a new hyper-parameter-free loss function. The two techniques led to the best single model in our experiment, which achieved an equal error rate of 1.92% and was significantly different in statistical sense from most of the other experimental models.
GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio

This paper introduces GigaSpeech, an evolving, multi-domain English speech recognition corpus with 10,000 hours of high quality labeled audio suitable for supervised training, and 40,000 hours of total audio suitable for semi-supervised and unsupervised training. Around 40,000 hours of transcribed audio is first collected from audiobooks, podcasts and YouTube, covering both read and spontaneous speaking styles, and a variety of topics, such as arts, science, sports, etc. A new forced alignment and segmentation pipeline is proposed to create sentence segments suitable for speech recognition training, and to filter out segments with low-quality transcription. For system training, GigaSpeech provides five subsets of different sizes, 10h, 250h, 1000h, 2500h, and 10000h. For our 10,000-hour XL training subset, we cap the word error rate at 4% during the filtering/validation stage, and for all our other smaller training subsets, we cap it at 0%. The DEV and TEST evaluation sets, on the other hand, are re-processed by professional human transcribers to ensure high transcription quality. Baseline systems are provided for popular speech recognition toolkits, namely Athena, ESPnet, Kaldi and Pika.
A Survey on blockchain for industrial Internet of Things

Blockchain and the Internet of Things (IoT) are separately regarded as highly capable popular technologies. Blockchain is a database used for decentralized transaction purposes. It provides novel directions to store and manage data, whereas IoT relates to the propagation of linked machines by providing information through the Internet. A mixture of Blockchain and the IoT appears hopeful, even though blockchain requires real-time data application, and IoT describes processes to store and manage information overloads safely and proficiently. The technology is significant to the manufacturing business, experiencing a digital revolution by incorporating equipment, developments, and data, leading to the Industrial IoT (IIoT). Such a combination of IIoT and blockchain is termed Blockchain for Industrial Internet of Things (BIIoT). The paper gives a detailed survey on BIIoT and discusses all relevant aspects of this novel concept. Firstly, IoT as a concept has been explored briefly, and relevant threats to IoT has been investigated. Furthermore, the domain of IIoT and the relevant challenges to IIoT is discussed. Additionally, an overview of blockchain technology is presented. Subsequently, a combination of IIoT and blockchain is explored, and the BIIoT structural design proposal is submitted. Finally, issues related to the use of IIoT with blockchain for industrial applications of BIIoT is explored. Thus, all relevant open research directions in Blockchain and IIoT are summarized in this paper. This survey shows that the proposed BIIoT is used to develop a redundant, traceable, and secure complex interconnected IIoT environment. Furthermore, the BIIoT system enables us to communicate with each other without the need for a trusted intermediary in a decentralized, unreliable, peer-to-peer network.
Simulated Perceptual Grouping: An Application to Human-Computer Interaction

The perceptual principles that allow people to group visually similar objects into entities, or groups, have been called the Gestalt Laws of perception. Two well known principles of perceptual grouping are proximity and similarity: objects that lie close together are perceived to fall into groups; objects of similar shape, size or color are more likely to form groups than objects differing along these dimensions. While the primary function of these “laws” is to help us perceive the world, they also enter into our communications. People can build on assumptions about each other’s perception of the world as a basis for simplifying discourse: for example, we invariably refer to collections of objects simply by gesturing in their direction and uttering “those.” The current work describes an algorithm that simulates parts of the visual grouping mechanism at the object level. The system uses feature spaces and simple ranking methods to produce object groupings. Computational aspects of this system are described in detail and its uses for enhancing multi-modal interfaces are explained.
Evaluation of metadata representations in RDF stores

The maintenance and use of metadata such as provenance and time-related information is of increasing importance in the Semantic Web, especially for Big Data applications that work on heterogeneous data from multiple sources and which require high data quality. In an RDF dataset, it is possible to s tore metadata alongside the actual RDF data and several possible metadata representation models have been proposed. However, there is still no in-depth comparative evaluation of the main representation alternatives on both the conceptual level and the implementation level using different graph backends. In order to help to close this gap, we introduce major use cases and requirements for storing and using diverse kinds of metadata. Based on these requirements, we perform a detailed comparison and benchmark study for different RDF-based metadata representations, including a new approach based on so-called companion properties. The benchmark evaluation considers two datasets and evaluates different representations for three popular RDF stores.