Reviewerly/exHarmony huggingface dataset:
Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism

In this paper, we consider hybrid parallelism---a paradigm that employs both Data Parallelism (DP) and Model Parallelism (MP)---to scale distributed training of large recommendation models. We propose a compression framework called Dynamic Communication Thresholding (DCT) for communication-efficient hybrid training. DCT filters the entities to be communicated across the network through a simple hard-thresholding function, allowing only the most relevant information to pass through. For communication efficient DP, DCT compresses the parameter gradients sent to the parameter server during model synchronization. The threshold is updated only once every few thousand iterations to reduce the computational overhead of compression. For communication efficient MP, DCT incorporates a novel technique to compress the activations and gradients sent across the network during the forward and backward propagation, respectively. This is done by identifying and updating only the most relevant neurons of the neural network for each training sample in the data. We evaluate DCT on publicly available natural language processing and recommender models and datasets, as well as recommendation systems used in production at Facebook. DCT reduces communication by at least 100x and 20x during DP and MP, respectively. The algorithm has been deployed in production, and it improves end-to-end training time for a state-of-the-art industrial recommender model by 37%, without any loss in performance.
Super Resolution and Demosaicing based self learning Adaptive Dictionary Image Denoising framework

It is an effective technique that can be applied to visual diversity such as editing, visual criticism and the so-called fragmentation of painting images. Creativity, image-based optimization and high-resolution frame work were showcased based on the record-breaking success of a minority representation process. The proposed structure for reconstructing the image is learned from a complete dictionary and much of the image input used for two important problems. It was first used by creating a Gaussian noise figure and then blaming the rain. Special structures using conventional methods suggest that the production environment is grappling with artistic solutions and problems. The proposed method was implemented in rich color using ultra-resolution and demo techniques with removing sound and craftsmanship.
Explainable artificial intelligence for digital forensics

Abstract EXplainable artificial intelligence (XAI) is an emerging research area relating to the creation of machine learning algorithms from which explanations for outputs are provided. In many fields, such as law enforcement, it is necessary that decisions made by and with the assistance of artificial intelligence (AI)‐based tools can be justified and explained to a human. We seek to explore the potential of XAI to further enhance triage and analysis of digital forensic evidence, using examples of the current state of the art as a starting point. This opinion letter will discuss both practical and novel ideas as well as controversial points for leveraging XAI to improve the efficacy of digital forensic (DF) analysis and extract forensically sound pieces of evidence (also known as artifacts) that could be used to assist investigations and potentially in a court of law. This article is categorized under: Digital and Multimedia Science &gt; Artificial Intelligence Digital and Multimedia Science &gt; Cybercrime Investigation
On systematically building a controlled natural language for functional requirements

Natural language (NL) is pervasive in software requirements specifications (SRSs). However, despite its popularity and widespread use, NL is highly prone to quality issues such as vagueness, ambiguity, and incompleteness. Controlled natural languages (CNLs) have been proposed as a way to prevent quality problems in requirements documents, while maintaining the flexibility to write and communicate requirements in an intuitive and universally understood manner. In collaboration with an industrial partner from the financial domain, we systematically develop and evaluate a CNL, named Rimay, intended at helping analysts write functional requirements. We rely on Grounded Theory for building Rimay and follow well-known guidelines for conducting and reporting industrial case study research. Our main contributions are: (1) a qualitative methodology to systematically define a CNL for functional requirements; this methodology is intended to be general for use across information-system domains, (2) a CNL grammar to represent functional requirements; this grammar is derived from our experience in the financial domain, but should be applicable, possibly with adaptations, to other information-system domains, and (3) an empirical evaluation of our CNL (Rimay) through an industrial case study. Our contributions draw on 15 representative SRSs, collectively containing 3215 NL requirements statements from the financial domain. Our evaluation shows that Rimay is expressive enough to capture, on average, 88% (405 out of 460) of the NL requirements statements in four previously unseen SRSs from the financial domain.
Eye-tracking emulation software: a promising urban design tool

The human brain is designed as a social engagement system hard-wired to take in and respond to other people. This study begins to explore how this hidden brain design also directs our engagement with the built environment, in this instance, determining our behaviour around buildings. Using eye-tracking emulation software, this study tracked the unconscious responses people have to new-urbanist and more typical, car-centric suburban American house facades (elevations). It found that this kind of emulation software can help demonstrate how new urbanist homes and streetscapes were implicitly easier for people to take in and focus on, than those in car-centric subdivisions. The implications of these findings are that urban designers and architects can employ eye-tracking emulation software to explore the ways that humans unconsciously handle visual stimuli, subject to validation from alternative data sources.
Normalization of breast MRIs using cycle-consistent generative adversarial networks

Dynamic Contrast Enhanced-Magnetic Resonance Imaging (DCE-MRI) is widely used to complement ultrasound examinations and x-ray mammography during the early detection and diagnosis of breast cancer. However, images generated by various MRI scanners (e.g. GE Healthcare vs Siemens) differ both in intensity and noise distribution, preventing algorithms trained on MRIs from one scanner to generalize to data from other scanners successfully. We propose a method for image normalization to solve this problem. MRI normalization is challenging because it requires both normalizing intensity values and mapping between the noise distributions of different scanners. We utilize a cycle-consistent generative adversarial network to learn a bidirectional mapping between MRIs produced by GE Healthcare and Siemens scanners. This allows us learning the mapping between two different scanner types without matched data, which is not commonly available. To ensure the preservation of breast shape and structures within the breast, we propose two technical innovations. First, we incorporate a mutual information loss with the CycleGAN architecture to ensure that the structure of the breast is maintained. Second, we propose a modified discriminator architecture which utilizes a smaller field-of-view to ensure the preservation of finer details in the breast tissue. Quantitative and qualitative evaluations show that the second proposed method was able to consistently preserve a high level of detail in the breast structure while also performing the proper intensity normalization and noise mapping. Our results demonstrate that the proposed model can successfully learn a bidirectional mapping between MRIs produced by different vendors, potentially enabling improved accuracy of downstream computational algorithms for diagnosis and detection of breast cancer. All the data used in this study are publicly available.
Decentralized learning of energy optimal production policies using PLC-informed reinforcement learning

This paper presents a novel approach to distributed optimization in production systems using reinforcement learning (RL) with particular emphasis on energy efficient production. Unlike existing approaches in RL which learn optimal policies from scratch, we speed-up the learning process by considering available control code of the Programmable Logic Controller (PLC) as a baseline for further optimizations. For such PLC-informed RL, we propose Teacher-Student RL to distill the available control code of the individual modules into a neural network which is subsequently optimized using standard RL. The proposed general framework allows to incorporate PLC control code with different level of detail. We implement the approach on a laboratory scale testbed representing different production scenarios ranging from continuous to batch production. We compare the results for different control strategies which show that comparably simple control logic yields considerable improvements of the optimization compared to learning from scratch. The obtained results underline the applicability and potential of the approach in terms of improved production efficiency while considerably reducing the energy consumption of the production schedules.
ADCF: Attentive representation learning and deep collaborative filtering model

In this paper, we propose a deep collaborative filtering recommendation model, which consists of an attention-based representation learning component and a multi-input matching function learning component. This model takes interaction matrix based on implicit feedback as data source to construct representations of long-term user preferences and item latent features. In the representation learning, a time-aware attention network is used, which uses the embedding vectors of the predicted item, recent historical interaction items, and the interaction time of recent historical interaction items to estimate the weights of different historical interaction items to short-term user preferences modeling. Then, the dynamic user preference representation can be obtained by combining short-term preferences with long-term preferences. In the matching function learning, a multi-input deep learning model is used. Its input includes not only the dynamic user preference representation and the item latent feature representation, but also the linear interaction between the two representations, so that the model has more powerful feature interactions learning ability. Experimental results on four datasets from different domains show that our method is largely superior to the state-of-the-art collaborative filtering methods, and the novel techniques we propose in this paper are effective in improving recommendation performance.
Geo-AI to aid disaster response by memory-augmented deep reservoir computing

It is a fact that natural disasters often cause severe damage both to ecosystems and humans. Moreover, man-made disasters can have enormous moral and economic consequences for people. A typical example is the large deadly and catastrophic explosion in Beirut on 4 August 2020, which destroyed a very large area of the city. This research paper introduces a Geo-AI disaster response computer vision system, capable to map an area using material from Synthetic Aperture Radar (SAR). SAR is a unique form of radar that can penetrate the clouds and collect data day and night under any weather conditions. Specifically, the Memory-Augmented Deep Convolutional Echo State Network (MA/DCESN) is introduced for the first time in the literature, as an advanced Machine Vision (MAV) architecture. It uses a meta-learning technique, which is based on a memory-augmented approach. The target is the employment of Deep Reservoir Computing (DRC) for domain adaptation. The developed Deep Convolutional Echo State Network (DCESN) combines a classic Convolutional Neural Network (CNN), with a Deep Echo State Network (DESN), and analog neurons with sparse random connections. Its training is performed following the Recursive Least Square (RLS) method. In addition, the integration of external memory allows the storage of useful data from past processes, while facilitating the rapid integration of new information, without the need for retraining. The proposed DCESN implements a set of original modifications regarding training setting, memory retrieval mechanisms, addressing techniques, and ways of assigning attention weights to memory vectors. As it is experimentally shown, the whole approach produces remarkable stability, high generalization efficiency and significant classification accuracy, significantly extending the state-of-the-art Machine Vision methods.
NVC-Net: End-To-End Adversarial Voice Conversion

Voice conversion (VC) has gained increasing popularity in many speech synthesis applications. The idea is to change the voice identity from one speaker into another while keeping the linguistic content unchanged. Many VC approaches rely on the use of a vocoder to reconstruct the speech from acoustic features, and as a consequence, the speech quality heavily depends on such a vocoder. In this paper, we propose NVC-Net, an end-to-end adversarial network, which performs VC directly on the raw audio waveform. By disentangling the speaker identity from the speech content, NVC-Net is able to perform non-parallel traditional many-to-many VC as well as zero-shot VC from a short utterance of an unseen target speaker. Importantly, NVC-Net is non-autoregressive and fully convolutional, achieving fast inference. Objective and subjective evaluations on VC tasks show that NVC-Net obtains competitive results with significantly fewer parameters.
Efficient Conditional GAN Transfer with Knowledge Propagation across Classes

Generative adversarial networks (GANs) have shown impressive results in both unconditional and conditional image generation. In recent literature, it is shown that pre-trained GANs, on a different dataset, can be transferred to improve the image generation from a small target data. The same, however, has not been well-studied in the case of conditional GANs (cGANs), which provides new opportunities for knowledge transfer compared to unconditional setup. In particular, the new classes may borrow knowledge from the related old classes, or share knowledge among themselves to improve the training. This motivates us to study the problem of efficient conditional GAN transfer with knowledge propagation across classes. To address this problem, we introduce a new GAN transfer method to explicitly propagate the knowledge from the old classes to the new classes. The key idea is to enforce the popularly used conditional batch normalization (BN) to learn the class-specific information of the new classes from that of the old classes, with implicit knowledge sharing among the new ones. This allows for an efficient knowledge propagation from the old classes to the new ones, with the BN parameters increasing linearly with the number of new classes. The extensive evaluation demonstrates the clear superiority of the proposed method over state-of-the-art competitors for efficient conditional GAN transfer tasks. The code is available at: https://github.com/mshahbazi72/cGANTransfer
Learning to Recommend Visualizations from Data

Visualization recommendation is important for exploratory analysis and making sense of the data quickly by automatically recommending relevant visualizations to the user. In this work, we propose the first end-to-end ML-based visualization recommendation system that leverages a large corpus of datasets and their relevant visualizations to learn a visualization recommendation model automatically. Then, given a new unseen dataset from an arbitrary user, the model automatically generates visualizations for that new dataset, derives scores for the visualizations, and outputs a list of recommended visualizations to the user ordered by effectiveness. We also describe an evaluation framework to quantitatively evaluate visualization recommendation models learned from a large corpus of visualizations and datasets. Through quantitative experiments, a user study, and qualitative analysis, we show that our end-to-end ML-based system recommends more effective and useful visualizations compared to existing state-of-the-art rule-based systems.
Entrepreneurship and productivity in Africa: the role of institutions

Through an empirical analysis of selected African countries for a period of 2006–2017, this paper explores three hypotheses suggesting that African entrepreneurship does not promote productivity except there are conducive institutional environments. The study applies panel data estimation techniques on data obtained from seventeen countries in Africa. The results obtained are summarized as follows: (1) African entrepreneurship does not promote productivity. (2) We do not confirm any significant U-shaped relationship between entrepreneurship and productivity in Africa. (3) African entrepreneurship promotes productivity via conducive institutional quality at a threshold value of 4.56 on a scale of 0 to 5 point. Among the institutional dimensions, government effectiveness, regulatory quality, rule of law, and control of corruption respectively turn to be more relevant in the selected countries. The study concludes that strong institutional quality would help in promoting total factor productivity in Africa.
Secure Web development using OWASP Guidelines

Website security is a major concern for large organizations as well as individual developers, the rarer the technology used the harder it becomes to come up with secure practices for developing a website. Vulnerabilities that are not fixed during development, and are deployed as such become easy targets for hackers. This could cause the company or the individual to lose a lot of money. It is not just the developers who are affected, end users who end up on vulnerable websites may get exposed to XSS attack which could compromise their system or an unsecured configuration of database system could lead to a potential data leak and hence the password of every registered user on the website is compromised, users who use the same password on multiple websites are affected the most The motivation for this paper comes from the fact that there is an overwhelming number of vulnerabilities in any application under development and every developer, experienced or not needs a starting point to patch these vulnerabilities that might have occurred in their application, this research provides the most common vulnerabilities which should be taken care of in any application and thus provide the much-needed starting point for developers. The objective of this paper is to design and develop a secure web application according to Open Web Application Security Project (OWASP) guidelines. This paper highlights the mitigation of vulnerabilities in the web application using configuration changes, coding and applying patches. The vulnerabilities SQL injection, Broken authentication, Sensitive data exposure, Broken Access Control, and XML external entities discussed in this paper are listed under the OWASP top 10 vulnerabilities. The security of the web application is tested and proved to have defense mechanism implemented for the mentioned vulnerabilities.
Recent advances in biometrics-based user authentication for wearable devices: A contemporary survey

In recent years, wearable technology is interwoven with our everyday lives because of its commoditization and comfort. Security and privacy become a big concern as many user-sensitive data have been stored in such devices, such as personal emails and bank accounts. Traditional user authentication techniques like PIN entry are unfriendly and vulnerable to shoulder surfing attacks. To address these problems, a number of new authentication methods have been proposed. In this survey, we review and categorize recent advances in user authentication for wearable devices. We classify existing studies into physiological biometrics based and behavioral biometrics based methods. For each category, we review how signal processing techniques have been used to extract features in various wearable devices. Leveraging these extracted features, specifically designed classification methods can be used to realize user authentication. Finally, we review evaluation metrics for user authentication in wearable devices. Overall, in this survey, we systematically study assorted state-of-the-art user authentication methods for wearable devices, aiming to provide guidance and directions for future research in this area.
A Concise Yet Effective Model for Non-Aligned Incomplete Multi-View and Missing Multi-Label Learning

In reality, learning from multi-view multi-label data inevitably confronts three challenges: missing labels, incomplete views, and non-aligned views. Existing methods mainly concern the first two and commonly need multiple assumptions to attack them, making even state-of-the-arts involve at least two explicit hyper-parameters such that model selection is quite difficult. More roughly, they will fail in handling the third challenge, let alone addressing the three jointly. In this paper, we aim at meeting these under the least assumption by building a concise yet effective model with just one hyper-parameter. To ease insufficiency of available labels, we exploit not only the consensus of multiple views but also the global and local structures hidden among multiple labels. Specifically, we introduce an indicator matrix to tackle the first two challenges in a regression form while aligning the same individual labels and all labels of different views in a common label space to battle the third challenge. In aligning, we characterize the global and local structures of multiple labels to be high-rank and low-rank, respectively. Subsequently, an efficient algorithm with linear time complexity in the number of samples is established. Finally, even without view-alignment, our method substantially outperforms state-of-the-arts with view-alignment on five real datasets.
The Salted Kalman Filter: Kalman filtering on hybrid dynamical systems

Many state estimation and control algorithms require knowledge of how probability distributions propagate through dynamical systems. However, despite hybrid dynamical systems becoming increasingly important in many fields, there has been little work on utilizing the knowledge of how probability distributions map through hybrid transitions. Here, we make use of a propagation law that employs the saltation matrix (a first-order update to the sensitivity equation) to create the Salted Kalman Filter (SKF), a natural extension of the Kalman Filter and Extended Kalman Filter to hybrid dynamical systems. Away from hybrid events, the SKF is a standard Kalman filter. When a hybrid event occurs, the saltation matrix plays an analogous role as that of the system dynamics, subsequently inducing a discrete modification to both the prediction and update steps. The SKF outperforms a naive variational update – the Jacobian of the reset map – by having a reduced mean squared error in state estimation, especially immediately after a hybrid transition event. Compared against a hybrid particle filter, the particle filter outperforms the SKF in mean squared error only when a large number of particles are used, likely due to a more accurate accounting of the split distribution near a hybrid transition.