Reviewerly/exHarmony huggingface dataset:
Self-Normalized Importance Sampling for Neural Language Modeling

To mitigate the problem of having to traverse over the full vocabulary in the softmax normalization of a neural language model, sampling-based training criteria are proposed and investigated in the context of large vocabulary word-based neural language models. These training criteria typically enjoy the benefit of faster training and testing, at a cost of slightly degraded performance in terms of perplexity and almost no visible drop in word error rate. While noise contrastive estimation is one of the most popular choices, recently we show that other sampling-based criteria can also perform well, as long as an extra correction step is done, where the intended class posterior probability is recovered from the raw model outputs. In this work, we propose self-normalized importance sampling. Compared to our previous work, the criteria considered in this work are self-normalized and there is no need to further conduct a correction step. Through self-normalized language model training as well as lattice rescoring experiments, we show that our proposed self-normalized importance sampling is competitive in both research-oriented and production-oriented automatic speech recognition tasks.
The Effect of E-Verify Laws on Crime

E-Verify laws, which have been adopted by 23 states, require employers to verify whether new employees are eligible to legally work prior to employment. In the main, these laws are designed to reduce employment opportunities for unauthorized immigrants, reduce incentives for their immigration, and increase employment and earnings for low-skilled natives. This study explores the impact of state E-Verify laws on crime. Using agency-by-month data from the 2004 to 2015 National Incident Based Reporting System (NIBRS), we find that the enactment of E-Verify is associated with a 5 to 10 percent reduction in property crimes involving Hispanic arrestees, an effect driven by universal E-Verify mandates that extend to private employers. Supplemental analyses from the Current Population Survey (CPS) suggest that E-Verify-induced increases in employment of low-skilled natives of Hispanic descent, and outmigration of younger Hispanics are important channels. We find no evidence that crime was displaced to nearby U.S. jurisdictions without E-Verify or that violent crime was impacted by E-Verify mandates. Moreover, neither arrests nor labor market outcomes of white or African American adults were affected by E-Verify laws. The magnitudes of our estimates suggest that E-Verify mandates generated $491 million in social benefits of reduced crime to the United States.
Development of the Information System for Registering the Result of Scientific Institution’ Employees Intellectual Activity

The article describes a Web-system developed by the authors that implements services related to the formation and provision of multifaceted information about the results of scientific activities (publications, copyright certificates and reports at scientific events) of employees of an organization or a group of organizations. The system is focused both on the end user interested in obtaining specific data, and on the administrative staff, who generates reporting materials for the parent organization. The information base of the system contains metadata on the following classes of objects: persons (authors), organizations and their subdivisions; publications at analytical, monographic and summary levels; copyright certificates; scientific events (conferences, symposia, seminars); reports. The system includes two modules – an administrative one (intended for entering and editing data) and a user one, which is a special search engine that searches for information, visualizes it, provides navigation among related resources and exports data. A distinctive feature of the system is the introduced concept of “equivalent” objects. Objects are considered equivalent if they are represented in the system by different metadata, but referring to the same physical entity. Such objects are “persons” corresponding to one author with different spellings of the surname in the bibliographic descriptions of publications; organizations with different variants of names; articles published unchanged in various languages. In accordance with modern requirements for reporting on publications, the system reflects the sources of research funding, as well as the affiliations indicated in the articles for each author.
Algebraic Operations-Based Secret-Key Design for Encryption Algorithm (ASKEA) for Energy Informatics and Smart Internet of Things (IoT) Applications

This paper designs a secret key that has the ability to behave in a different way based on plaintext input. This paper proposes two schemes of secret key length which are: \(2^{i}\) and \(2^{i} + 2^{i - 1}\). The aim is to increase unpredictability and privacy of plaintext. A series of algebraic operations has been used to create a specific coefficient by which the relation between plaintext size and key length is adjustable. This design of secret key increases privacy of sensitive data used with energy informatics and smart Energy Internet-of-Things (EIoT) applications. The proposed design has been evaluated in terms of time complexity computation time to generate one secret key and encrypt plaintext. Additionally, results show that the proposed Algebraic Operations-based Secret-Key Design for Encryption Algorithm (ASKEA) has less computation time and complexity than other competitive research works. Privacy and unpredictability of plaintext and secret key could be preserved and achieved.
Colourlab Image Database: Geometric Distortions

Over the years, a high number of different objective image quality metrics have been proposed.While some image quality metrics show a high correlation with subjective scores provided in different datasets, there still exists room for improvement.Different studies have pointed to evaluating the quality of images affected by geometrical distortions as a challenge for current image quality metrics.In this work, we introduce the Colourlab Image Database: Geometric Distortions (CID:GD) with 49 different reference images made specifically to evaluate image quality metrics.CID:GD is one of the first datasets which include three different types of geometrical distortions; seam carving, lens distortion, and image rotation.35 state-ofthe-art image quality metrics are tested on this dataset, showing that apart from a handful of these objective metrics, most are not able to show a high performance.
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning

Actor-critic methods are widely used in offline reinforcement learning practice, but are not so well-understood theoretically. We propose a new offline actor-critic algorithm that naturally incorporates the pessimism principle, leading to several key advantages compared to the state of the art. The algorithm can operate when the Bellman evaluation operator is closed with respect to the action value function of the actor's policies; this is a more general setting than the low-rank MDP model. Despite the added generality, the procedure is computationally tractable as it involves the solution of a sequence of second-order programs. We prove an upper bound on the suboptimality gap of the policy returned by the procedure that depends on the data coverage of any arbitrary, possibly data dependent comparator policy. The achievable guarantee is complemented with a minimax lower bound that is matching up to logarithmic factors.
Local two-atom correlations induced by a two-mode cavity under nonlinear media: Quantum uncertainty and quantum Fisher information

The nonlinearity effect on the various formed quantum correlations between two atoms interacting locally with a bimodal cavity is explored in this work. The local quantum Fisher information, local quantum uncertainty, and negativity are used to investigate the quantum correlation’s sensitivity to the cavity modes characteristics. It is demonstrated that when the ratio of the Stark shift of the two modes increases, the atomic quantum correlations rise. During the time intervals of sudden death negativity entanglement, the three two-atom correlation quantifiers reveal that the two-atom system can contain local Fisher information and local uncertainty correlations. The Stark shift may be utilized to amplify the produced atomic correlation, while the Kerr-like medium can be used to suppress it. The regularity, the amount of generated atomic correlations, and their immunity to the Stark shift as well as the Kerr-like medium non-linearities are improved when the mean photon number is increased. As a result, the cavity modes’ nonlinear parameters might be utilized as a controller to optimize atomic quantum correlations.
DCU team at the NTCIR-15 micro-activity retrieval task

The growing attention to lifelogging research has led to the creation of many retrieval systems, most of which employed event segmentation as core functionality. While previous literature focused on splitting lifelog data into broad segments of daily living activities, less attention was paid to micro-activities which last for short periods of time, yet carry valuable information for building a high-precision retrieval engine. In this paper, we present our efforts in addressing the NTCIR-15 MART challenge, in which the participants were asked to retrieve micro-activities from a multi-modal dataset. We proposed five models which investigate imagery and sensory data, both jointly and separately using various Deep Learn- ing and Machine Learning techniques, and achieved a maximum mAP score of 0.901 using an Image Tabular Pair-wise Similarity model, and overall ranked second in the competition. Our model not only captures the information coming from the temporal visual data combined with sensor signal, but also works as a Siamese network to discriminate micro-activities.
Adaptive Risk Minimization: Learning to Adapt to Domain Shift

A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. However, this assumption is violated in almost all practical applications: machine learning systems are regularly tested under distribution shift, due to changing temporal correlations, atypical end users, or other factors. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts, corresponding to new domains or domain distributions. Most prior methods aim to learn a single robust model or invariant feature space that performs well on all domains. In contrast, we aim to learn models that adapt at test time to domain shift using unlabeled test points. Our primary contribution is to introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains. Compared to prior methods for robustness, invariance, and adaptation, ARM methods provide performance gains of 1-4% test accuracy on a number of image classification problems exhibiting domain shift.
Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data

Unsupervised domain adaptation aims to align a labeled source domain and anunlabeled target domain, but it requires to access the source data which oftenraises concerns in data privacy, data portability and data transmissionefficiency. We study unsupervised model adaptation (UMA), or calledUnsupervised Domain Adaptation without Source Data, an alternative setting thataims to adapt source-trained models towards target distributions withoutaccessing source data. To this end, we design an innovative historicalcontrastive learning (HCL) technique that exploits historical source hypothesisto make up for the absence of source data in UMA. HCL addresses the UMAchallenge from two perspectives. First, it introduces historical contrastiveinstance discrimination (HCID) that learns from target samples by contrastingtheir embeddings which are generated by the currently adapted model and thehistorical models. With the source-trained and earlier-epoch models as thehistorical models, HCID encourages UMA to learn instance-discriminative targetrepresentations while preserving the source hypothesis. Second, it introduceshistorical contrastive category discrimination (HCCD) that pseudo-labels targetsamples to learn category-discriminative target representations. Instead ofglobally thresholding pseudo labels, HCCD re-weights pseudo labels according totheir prediction consistency across the current and historical models.Extensive experiments show that HCL outperforms and complementsstate-of-the-art methods consistently across a variety of visual tasks (e.g.,segmentation, classification and detection) and setups (e.g., close-set,open-set and partial adaptation).
Fine-Grained Vehicle Classification in Urban Traffic Scenes Using Deep Learning

The increasingly dense traffic is becoming a challenge in our local settings, urging the need for a better traffic monitoring and management system. Fine-grained vehicle classification appears to be a challenging task as compared to vehicle coarse classification. Exploring a robust approach for vehicle detection and classification into fine-grained categories is therefore essentially required. Existing Vehicle Make and Model Recognition (VMMR) systems have been developed on synchronized and controlled traffic conditions. Need for robust VMMR in complex, urban, heterogeneous, and unsynchronized traffic conditions still remain an open research area. In this paper, vehicle detection and fine-grained classification are addressed using deep learning. To perform fine-grained classification with related complexities, local dataset THS-10 having high intraclass and low interclass variation is exclusively prepared. The dataset consists of 4250 vehicle images of 10 vehicle models, i.e., Honda City, Honda Civic, Suzuki Alto, Suzuki Bolan, Suzuki Cultus, Suzuki Mehran, Suzuki Ravi, Suzuki Swift, Suzuki Wagon R and Toyota Corolla. This dataset is available online. Due to having almost no design variation in some make and models over the years, vehicle models are not separated by their year of generation. Two approaches have been explored and analyzed for classification of vehicles i.e., fine-tuning, and feature extraction from deep neural networks. A comparative study is performed, and it is demonstrated that simpler approaches can produce good results in local environment to deal with complex issues such as dense occlusion and lane departures. Hence reducing computational load and time, e.g. finetuning Inception-v3 produced highest accuracy of 97.4% with lowest misclassification rate of 2.08%. Finetuning MobileNet-v2 and ResNet-18 produced 96.8% and 95.7% accuracies, respectively. Extracting features from fc6 layer of AlexNet produces an accuracy of 93.5% with a misclassification rate of 6.5%.
A NOVEL AUGMENTED REALITY SYSTEM TO SUPPORT VOLUMETRIC VISUALIZATION IN INDUSTRIAL PROCESS TOMOGRAPHY

Augmented Reality (AR), as a variation of Virtual Reality (VR), has been proved useful for decades.However, it is not widely utilized in most industries.To fill the gap between this technique and industrial usage, we propose a novel AR system in the context of industrial process tomography (IPT).As the pioneering AR deployment in IPT, this system offers a new solution to underpin the onsite data analysis regarding volumetric visualization.In our work, an endeavor to provide intelligent control for an industrial drying system is pursued by using microwave tomography (MWT), a breed of IPT, as an imaging modality.Here, the AR system is integrated with the MWT for post processing of its volumetric images, containing critical information of the industrial process.The core part of the AR system is implemented by an interactive mobile app that is supported on iOS/Android platforms.The overall system is generalized by four distinctive findings: interactivity, mobility, information richness, and mutual collaboration.Our proposed system opens the horizon of leveraging AR in IPT to benefit domain-related users regarding onsite data analysis and visualization.
Reinforcement Learning Methods to Evaluate the Impact of AI Changes in Game Design

Game development has become a long process that requires many professionals working on a project during several months or years. With this scenario the re-utilization of resources is crucial not only to alleviate the process but also to bring coherence into the final product. In this paper we focus on the reuse of NPCs and the problems it brings about. In particular it is common to have different breeds (or personalities) of NPCs that are placed on different levels on the game. The problem arises when their behaviors are fine-tuned to accommodate a specific level needs without taking into consideration that this change may alter their performance on previous already-tested levels. The paper presents the application of reinforcement learning together with behavior trees to automatically test if modifications to the AIs of a stealth game have an impact on the user experience. Our experiments reveal that this approach provides a way of diagnosing alterations in level gameplay that correspond to the effects observed by human testers.
Global Horizontal and Direct Normal Solar Irradiance Modeling by the Machine Learning Methods Xgboost and Deep Neural Networks with CNN-LSTM Layers: A Case Study Using the GOES-16 Satellite Imagery

Restrictive legislations over the usage of fossil fuels are encouraging the research on clean and renewable energies. As a drawback, renewable energies are characterized by having random behavior, which hampers its integration on the current energy base system. Thus, solar irradiation estimation is important to a better assimilation of the renewable energies into the energy matrix. In this paper, two machine learning (ML) estimation models for global (GHI) and direct normal solar irradiance (DNI) are proposed: the first using XGBoost and the second using a hybrid convolutional neural network (CNN) with a long short-term memory (LSTM), CNN-LSTM. Both models use images from the GOES-16 satellite, taken from the city of Petrolina, Brazil. The results of the proposed models are analyzed against the reference models Copernicus Atmosphere Monitoring Service (CAMS), Solcast and the Physical Solar Model (PSM) provided by the National Solar Radiation Database (NSRDB). For GHI estimation, the PSM model reached the smallest RMSE, equal to 147.23 W/m^2, while the model CNN-LSTM was the best one to estimate DNI with a RMSE equal to 238.22 W/m^2. The proposed model managed to improve RMSE when compared against the benchmarking models by 2.89 % and 1.70 % for CNN-LSTM and XGBoost, respectively.
Neuromorphic Integrated Photonics as Hardware Accelerators for Ultra-high Speed Telecom and Imaging Applications

In this paper we propose a new neuromorphic photonic integrated architecture based on recurrent optical filters as core reservoir computing building blocks forming an efficient hardware accelerator for pattern recognition in telecom and imaging applications.
The Zone of Optimised Motivation (ZOOM)

This contribution is based upon inclusive design-based research that investigates convergence of digital technologies under the concept of Virtual Interactive Space (VIS) and Virtual Reality (VR), in creating immersive interactive environments that are tailored to end-users (individuals or communities). The history of the work has been focused in/across (re)habilitation training and interactive creative/playful expression targeting inclusive well-being linked to technology-empowered physical activities. Technology for inclusive well-being goals align with investigating means to supplement traditional intervention strategies in (re)habilitation. Relative discussion is on next-generation advances including Artificial Intelligence, Machine Learning and tailored autonomous solutions posited as emerging topics of industrial revolution specific to the field, i.e. Health 4.0 and 5.0 where, respectively, keywords of digitisation and personalisation relate (Kowalkiewicz, 2017). This text shares a result, from the approximate four decades of work, by presenting the emergent model that relates to in-/on- action recursive reflection supporting emergent practice. The model is titled “The Zone of Optimised Motivation” (ZOOM) and is inspired from first-person intervention experiences linked to a synthesis of theoretical frames. It includes phases of design, intervention, evaluation and reiterative reflexive recursive refinement. It is herein offered for interdisciplinary critique and analysis beyond original focused use context, i.e. in (re)habilitation. The body of research wherein ZOOM originated is titled “SoundScapes”.
The Application of Copula Continuous Extension Technique for Bivariate Discrete Data: A Case Study on Dependence Modeling of Seismicity Data

The Copula approach for continuous variables is highly developed, while discrete ones are underdeveloped due to computational difficulties and sometimes algorithm failure to convergent. Therefore, providing an alternative method for discrete variables becomes an essential issue. In this paper, a simple method is proposed to answer the problem by applying the Continuous Extension Technique (CET). This is carried out by adding random independent perturbations in the form of either Uniform distribution U(0,1) or (U(0,1)−1), and the discrete variables are treated as continuous. Subsequently, a Copula model for resulted variables is estimated based on the Copula theory for continuous variables. This method is called a Copula continuous extension technique. Our analytic and simulation approaches show that both random perturbation forms produce the same Kendall’s Tau measure and the selected Copula bivariate model. As illustrations, the proposed method is applied to the seismicity data obtained from the annual frequencies of earthquakes that occurred in the Sumatra megathrust of Indonesia, from January 1971 to December 2018, with magnitudes ( Mw ) of at least 4.6. Based on the selected Copula models, our simulations confirm the evidence of dependence seismic activity in each of the two adjacent large earthquake sources. These results provide new information regarding the seismicity behavior in the Sumatra megathrust.
Student teachers and their attitudes towards ICT

This paper explores the attitudes that student teachers in Malta, Norway, and Spain convey to digital technologies in formal educational settings as they start the 1st semester. A number of studies look at educational inclinations and employment of digital technologies (Granić &amp; Marangunić, 2019; Ritter, 2017; Scherer &amp; Teo, 2019). We have chosen to examine student teachers’ attitudes towards the professional use of digital technologies within a pedagogical framework. In this respect, a comparative qualitative analysis of one open-ended question that forms part of a more extensive questionnaire distributed to all participants is considered. The employed analytical lens subsequently centres on four concepts: ‘adaptability’, ‘creativity’, ‘critical thinking’, and ‘understanding of technology’. In this regard, our findings support arguments for asserting ‘attitude’ as a kind of teacher-specific digital competence for guiding their practice. We conclude by suggesting our analytical framework as a potential point of initiation for further development to understand attitudes as forming part of teachers’ specific digital competencies within teacher education and professional practice.