Reviewerly/exHarmony huggingface dataset:
Multiple Patch-Aware Network for Faster Real-World Image Dehazing

This paper proposes a Multiple Patch-aware Dehazing Network (MPADN), to remove haze in real-world images fast and efficiently. Firstly, we design a Multiple Patch-aware Module (MPAM), which utilizes joint decisions from multiple patch awareness to gain more stable local features. Compared with the multi-patch hierarchical strategy, MPAM extremely compresses the size of the model. Besides, we propose a novel data enhancement method called Concentration Sampling Enhancement (CSE), which generates new training samples by haze concentration sampling based on hazy images and clear images. This method makes use of the precious limited real-world paired data to generate more potentially usable data. It is easy to extend CSE to more models or tasks where the ground truth and the input information are similar. Experiments show that MPADN outperforms state-of-art fast dehazing methods, and it only takes 0.0065s to process a 1600x1200 image.
Iterative Learning for Distorted Image Restoration

Deep generative networks have achieved great success on distorted image restoration. However, existing deep learning approaches mainly focus on delicate module structure while ignoring the saturation problem. In this paper, we study the influence of different learning schemes on fitting capability and tackle the problem by proposing a novel iterative learning scheme. It accumulates weight importance from past episodes and guides the network to search for the optimal of current episodes based on obtained knowledge. Since public available datasets contain very few distortion types, we also release a new benchmark to explore this task. Extensive experimental evaluations on the benchmarks demonstrate that our learning approach significantly outperforms all other methods and achieves new state-of-the-art results.
MA-NET: Multi-Scale Attention-Aware Network for Optical Flow Estimation

Existing methods for optical flow estimation can perform well on images with small offsets of large objects. However, they could fail when there are a number of small and fast-moving objects. In particular, current deep learning methods often impose a down-sampling of the input data for a reduction of computational complexity. This practice inevitably incurs a loss of image details and causes small objects be ignored. To solve the problem, a multi-scale attention-aware network (MA-Net) is proposed, in which coarse-scale and fine-scale features are extracted in parallel and then attention is paid to them for producing the optical flow estimation. Extensive experiments conducted on the Sintel and KITTI2015 datasets show that the proposed MA-Net can capture fast-moving small objects with high accuracy and thus deliver superior performance over a number of state-of-the-art methods.
A Web-based Application for Interesting Place Recommendation

Today, the Internet has become one of the most essential media for disseminating information, particularly in the field of tourism. When it comes to organising a trip, most people will seek assistance. Existing travel websites, on the other hand, lack a suggestion feature that can recommend ideal locations based on the wants and requirements of customers. As a result, users had difficulty planning their vacations. In this article, an interesting place recommender system for travellers is proposed. The proposed system is a website application that provides a list of recommended places for users based on their background. In greater detail, this recommender system will compare the users' information to the history of other tourists and suggest a suitable trip destination for the user. The waterfall model, which is a step-by-step approach to designing an application, was used to develop this application. Thirty-one people were asked to use and test the system as part of the system evaluation. The results suggest that the system performs well, and the majority of respondents were pleased with the proposed system.
Data Efficient Support Vector Machine Training Using the Minimum Description Length Principle

Support vector machines (SVMs) are established as highly successful classifiers in a broad range of applications, including numerous medical ones. Nevertheless, their current employment is restricted by a limitation in the manner in which they are trained, most often the training-validation-test or kfold cross-validation approaches, which are wasteful both in terms of the use of the available data as well as computational resources. This is a particularly important consideration in many medical problems, in which data availability is low (be it because of the inherent difficulty in obtaining sufficient data, or because of practical reasons, e.g. pertaining to privacy and data sharing). In this paper we propose a novel approach to training SVMs which does not suffer from the aforementioned limitation, which is at the same time much more rigorous in nature, being built upon solid information theoretic grounds. Specifically, we show how the training process, that is the process of hyperparameter inference, can be formulated as a search for the optimal model under the minimum description length (MDL) criterion, allowing for theory rather than empiricism driven selection and removing the need for validation data. The effectiveness and superiority of our approach are demonstrated on the Wisconsin Diagnostic Breast Cancer Data Set.
Punctuation Prediction for Streaming On-Device Speech Recognition

Punctuation prediction is essential for automatic speech recognition (ASR). Although many works have been proposed for punctuation prediction, the on-device scenarios are rarely discussed with an end-to-end ASR. The punctuation prediction task is often treated as a post-processing of ASR outputs, but the mismatch between natural language in training input and ASR hypotheses in testing is ignored. Besides, language models built with deep neural networks are too large for edge devices. In this paper, we discuss one-pass models for both ASR and punctuation prediction to replace the conventional two-pass post-processing pipeline. Then the joint ASR-punctuation model is proposed to utilize multi-task learning to decouple the recognition and punctuation on the ASR decoder. Experimental results show that the proposed joint model not only outperforms the traditional post-processing method with limited extra parameters, but also achieves better accuracy in comparison to the direct ASR modeling on transcripts with punctuation.
Denoising-Guided Deep Reinforcement Learning For Social Recommendation

Social recommendation (SR) aims to enhance the performance of recommendations by incorporating social information. However, such information is not always reliable, e.g., some of the friends may share similar preferences with the user on a specific item, while others may be irrelevant to this item due to domain differences. Therefore, modeling all of the user’s social relationships without considering the relevance of friends will introduce noises to the social context. To address this issue, in this work, we propose a Denoisingguided deep Reinforcement Learning framework for Social recommendation (DRL4So). Specifically, the agent (i.e., social denoiser) in our framework automatically masks the user’s friends who are irrelevant to the target item; Then, the environment (i.e., recommender) is designed to give rewards to the agent for social denoising without supervised signals; Finally, the two components are jointly trained by DPG to ensure that social denoising correctly guides the recommendation. We conduct extensive experiments on three public datasets, and the results show that DRL4So outperforms existing state-of-the-art SR methods (improving 38.67% and 19.81% in terms of HR@10 and NDCG@10, respectively).
Course Recommendation using Domain-based Cluster Knowledge and Matrix Factorization

Selection of elective courses is one of the most crucial choices that a student needs to make in their academic journey. This choice also helps them shape their career path and develop their skill set in a certain domain. This is also one of the choices that students mess up due to lack of knowledge about the availability of the courses, confusion among courses, and sometimes, peer influence. This research proposes a system that will help students to take up the best available elective courses according to their domains of interest. This system creates a cluster of students based on their domains of interest and uses the matrix factorization technique to look at the historical performance records of students in those domains to make predictions for the courses that can be taken by a particular student belonging to the cluster. As a result, the system is essentially filtering the best possible candidates, for a particular student, in the cluster. Using the data of those candidates, predictions were made for particular students. The proposed system can enhance the overall learning experience for students, thereby helping them develop the foundation for their careers.
Byzantine-Robust Aggregation with Gradient Difference Compression and Stochastic Variance Reduction for Federated Learning

We investigate the problem of Byzantine-robust compressed federated learning, where the transmissions from the workers to the master node are compressed, and subject to malicious attacks from an unknown number of Byzantine workers. We show that the vanilla combination of the distributed compressed stochastic gradient descent (SGD) with geometric median-based robust aggregation suffers from the compression noise under Byzantine attacks. In light of this observation, we propose to reduce the compression noise with gradient difference compression to improve the Byzantine-robustness. We also observe the impact of the intrinsic stochastic noise from selecting random samples, and adopt the stochastic average gradient algorithm (SAGA) to gradually eliminate the inner variations of regular workers. We prove that the proposed algorithm reaches a neighborhood of the optimal solution at a linear convergence rate, and the asymptotic learning error is in the same order as that of the state-of-the-art uncompressed method. Finally, numerical experiments demonstrate the effectiveness of the proposed method.
Bert Distillation to Enhance the Performance of Machine Learning Models for Sentiment Analysis on Movie Review Data

Recently, Deep Neural Networks DNN, transformers, and ensemble models have accomplished incredible SOTA performance in numerous fields, including Computer Vision CV, Natural Language Processing NLP, etc. Drawbacks often accompany these advantages, such as heavyweight with nearly millions (and billions) of parameters and computationally too expensive, which stand as a barrier for these models to be deployed on the client-side of a web application, edge devices, or embedded devices. Distillation is one of the active model compression methods that draws the researcher&#x0027;s attention, which is used to overcome these drawbacks (big size and inference computation time). It fills the gap between learnability and expressive power, which is achieved by training a small, lightweight machine learning or deep learning model based on extra knowledge obtained from a large and high-weight, already trained model. BERT transformer is a remarkable example of these large models and one of the excellent solution choices in NLP, especially in text classification tasks such as sentiment analysis. This paper aimed to distill the BERT transformer model and use the distilled knowledge obtained to enhance lightweight traditional machine learning models (deployable models) performance on the sentiment analysis task on the benchmark movie review dataset. The results revealed that this approach could boost the performance of the lightweight ML models on testing data and increase the accuracy by more than 1&#x0025; of the chosen classifiers.
3D Texture Super Resolution via the Rendering Loss

Deep learning-based methods have made significant impact and demonstrated superior performance for the classical image and video super-resolution (SR) tasks. Yet, deep learning-based approaches to super-resolve the appearance of 3D objects are still sparse. Due to the nature of rendering 3D models, 2D SR methods applied directly to 3D object texture may not be a good approach. In this paper, we propose a rendering loss derived from the rendering of a 3D model and demonstrate its application to the SR task in the context of 3D texturing. Unlike other literature on the 3D appearance SR, no geometry information of the 3D model is required during network inference. Experimental results demonstrate that incorporating the rendering loss during network training outperforms existing state-of-the-art methods for 3D appearance SR. Furthermore, we provide a new 3D dataset consisting of 97 complete 3D models for further research in this field.
Bi-Directional Normalization and Color Attention-Guided Generative Adversarial Network for Image Enhancement

Most existing image enhancement methods require paired images, and rarely consider the aesthetic quality. This paper proposes a bi-directional normalization and color attention-guided generative adversarial network (BNCAGAN) for unsupervised image enhancement. An auxiliary attention classifier (AAC) and a bi-directional normalization residual (BNR) module are designed to assist the generator in flexibly controlling the local details with the constraint from both the low/high-quality domain. Moreover, a color attention module (CAM) is proposed to preserve the color fidelity in the discriminator. The qualitative and quantitative experimental results demonstrate that our BNCAGAN is superior to the existing methods with distinctively improved authenticity and naturalness of the enhanced images. The source code is available at https://github.com/SWU-CS-MediaLab/BNCAGAN.
A Novel Integrated IoT Framework with Classification Approach for Medical Data Analysis

In the healthcare systems, usage of advanced integrated technologies like Internet of things (IoT) and Machine learning (ML) techniques were limited. Different amalgams of IoT devices and ML mechanisms are available for medical sector but are limited to certain domains only. These models either provide patients current state data or specific domain analyzing and surveillance pre/post treatment data like heart or brain functions with corresponding medical aid. Also data available is used as clinical study for medical professionals and for better understandings of patients about their state. Specific domain gadgets&#x0027; like wrist bands or smart bands uses some sensors about vital, temperature and pulse etc., checkups are available but they were not meant for diagnosing or for treatments. In this paper, we proposed an integrated model to use IoT and ML algorithms for a healthcare system. Tracking of patients&#x0027; status can be done using some sensors such as lightweight, portable, and low-powered sensor nodes. These Sensors sense the patient&#x0027;s status and send the parametric data to the central controller, to take actions during the critical condition of the patients. The data sent to the controller always provided in secure and encryption form. At the same time, patient data is sent to doctors, so that they can provide the instructions to the caretakers of the patients with quick and proper solutions in real-time. For disease prediction, our model uses supervised machine learning algorithms, In order to get the efficient feature set and improve the better accuracy, and pre-processing techniques to eliminate features that are irrelevant, missing values and outliers from biomedical data which aids in better disease prediction. To further strengthen the proposed integrated model design is compared with various traditional classification algorithms to specify its improved accuracy and computational time for accurate prediction of the patient&#x0027;s disease and acts as a decision support system for health care assistants.
Adaptive Discounting of Implicit Language Models in RNN-Transducers

RNN-Transducer (RNN-T) models have become synonymous with streaming end-to-end ASR systems. While they perform competitively on a number of evaluation categories, rare words pose a serious challenge to RNN-T models. One main reason for the degradation in performance on rare words is that the language model (LM) internal to RNN-Ts can be-come overconfident and lead to hallucinated predictions that are acoustically inconsistent with the underlying speech. To address this issue, we propose a lightweight adaptive LM dis-counting technique ADAPTLMD, that can be used with any RNN-T architecture without requiring any external resources or additional parameters. ADAPTLMD uses a two-pronged approach: 1. Randomly mask the prediction network output to encourage the RNN-T to not be overly reliant on it’s outputs. 2. Dynamically choose when to discount the implicit LM (ILM) based on rarity of recently predicted tokens and divergence between ILM and implicit acoustic model (IAM) scores. Comparing ADAPTLMD to a competitive RNN-T baseline, we obtain up to 4% and 14% relative reductions in overall WER and rare word PER, respectively, on a conversational, code-mixed Hindi-English ASR task.
Unsupervised Hierarchical Translation-Based Model for Multi-Modal Medical Image Registration

Deformable registration of multi-modal medical images is a challenging task in medical image processing due to the differences in both appearance and structure. We propose an unsupervised hierarchical translation-based model to perform a coarse to fine registration of multi-modal medical images. The proposed model consists of three parts: a coarse registration network, a modal translation network and a fine registration network. First, the coarse registration network learns to obtain the coarse deformation field, which is applied as structure-preserving information to generate a translated image by the modal translation network. Then, the translated image as enhancing information combined with the original images are used to derive a fine deformation field in the fine registration network. Furthermore, the final deformation field is composed from the coarse and the fine deformation fields. In this way, the proposed model can learn high accurate deformation field to implement multi-modal medical image registration. Experiments on two multi-modal brain image datasets demonstrate the effectiveness of this model.
3d Cross-Scale Feature Transformer Network for Brain Mr Image Super-Resolution

High-resolution (HR) magnetic resonance (MR) images could provide reliable visual information for clinical diagnosis. Recently, super-resolution (SR) methods based on convolutional neural networks (CNNs) have shown great potential in obtaining HR MR images. However, most existing CNN-based SR methods neglect the internal priors of the MR image, which hides the performance of SR. In this work, we propose a 3D cross-scale feature transformer network (CFTN) to utilize the cross-scale priors within MR features. Specifically, we stack multiple 3D residual channel attention blocks (RCABs) as the backbone. Meanwhile, we design a plug-in mutual-projection feature enhancement module (MFEM) to extract the target-scale features with HR cues, which is able to capture the global cross-scale self-similarity within features and can be flexibly inserted into any position of the backbone. Furthermore, we propose a spatial attention fusion module (SAFM) to adaptively adjust and fuse the target-scale features and up-sampled features that are respectively extracted by the MFEM and the backbone. Experimental results show that our CFTN achieves a new state-of-the-art MR image SR performance.
LSTM-based eye-movement trajectory analysis for reading behavior classification

Studying eye movement trajectory through eye tracker can provide insights into classification mechanism of memory and reasoning. For the given text or images, memory and reasoning based observations are useful in multiple key areas of psychology, cognition and related fields. For our project, the EyeLink 1000 Plus is used to inspect and capture fixation, saccades, blinks etc. in eye tracking protocols of each participant during trials. Based on fixation sequence of each participant our trained deep recurrent neural network architecture–LSTM classifies whether a participant is performing the given text-based task by memorizing it or by inference. To set trial sentences, syllogism– deductive reasoning type is followed. Sixty university students (mature readers) participated in both memory and reasoning-based trials. Participants were then divided into two equal groups and of which one group was instructed to perform memory task first while the second group was instructed to perform reasoning based task first and then their order of performing the given task was changed. From sixty-one different sentences, sixty randomly selected sentences were presented to each participant. Sequential signals from fixation of each participant were then processed to get the results. Our trained LSTM model’s high accuracy to classify the memory and the reasoning based reading of participants ensures the significance of our work which will provide a solid base for future works on eye movements to build intelligent techniques in the field of A.I backed psychology, healthcare and neuro-marketing. Our trained model has the potential for (a) achieving very high accuracy for memory and reasoning classification, (b) for data learning, it saves enormous time, (c) Economical and comfortable for trials to record data via EyeLink 1000 plus (d) and finally, possible future work opportunities are enlisted.
Improving Fastspeech TTS with Efficient Self-Attention and Compact Feed-Forward Network

FastSpeech, as a feed-forward transformer based TTS, can avoid the slow serial, autoregressive inference to generate the target mel-spectrogram in a parallel way. As a non-autoregressive TTS, the latency and computation load in inference is shifted from vocoder to transformer where the efficiency is limited by the quadratic time and memory complexity in the self-attention mechanism, particularly for a long text sequence. To tackle this challenges, We propose two models, ProbSparseFS and LinearizedFS, which have efficient self-attention arrangements to improve the inference speed and memory complexity. LinearizedFS has achieved 3.4x memory savings and 2.1x inference speedup, compared with the those of the baseline FastSpeech. A further optimized LinearizedFS with a lightweight FFN can accelerate the inference speed by 3.6x more. We do subjective voice quality evaluations in MOS and CMOS of News report and Audiobook applications, for multi-speaker and multi-style scenarios. Test results verified that the proposed models yield a TTS quality which is on-par with that of the baseline system but with much better memory efficiency and inference speed.
Path Planning for Robotic Delivery Systems

In this work we consider a path planning problem for our university campus. In particular, we deploy a modified version of Dijkstra's algorithm to find the shortest path faster. By appropriately sampling points (nodes) throughout the campus, we develop a directed graph representation of our campus. Open Street Map (OSM) was used to obtain the GPS locations of these points. While this constructed graph has a large number of nodes, we use a node-abstraction method to compress the graph to ensure that the time complexity of our algorithm is sufficiently reduced. Simulation results report the computation time saved due to the abstraction method.