Reviewerly/exHarmony huggingface dataset:
Lightweight edge‐based <scp>kNN privacy‐preserving</scp> classification scheme in cloud computing circumstance

Summary Because mobile terminals have limited computing and storage resources, individuals tend to outsource their data generated from mobile devices to clouds to do data operations. However, utilization of the abundant computation and storage resources of clouds may pose a threat to user's private data. In this paper, we focus on the issue of encrypted k‐nearest neighbor (kNN) classification on the cloud. In the past few years, many solutions were proposed to protect the user's privacy and data security. Unfortunately, most privacy‐preserving data mining schemes are not lightweight, which are not practical in real‐world applications. To solve this issue, we proposed a lightweight edge‐based kNN (EBkNN) classification scheme over encrypted cloud database utilizing edge computing technology. Our proposed scheme can provide several security guarantees: (i) user's data security, (ii) user's query privacy, and (iii) data access patterns. We analyzed the security of our scheme utilizing the semi‐honest security model and evaluated the performance using a synthetic dataset. The experiment results indicate that our scheme is more lightweight than the state‐of‐the‐art scheme.
Vehicle Re-Identification in Multi-Camera scenarios based on Ensembling Deep Learning Features

Vehicle re-identification (ReID) across multiple cameras is one of the principal issues in Intelligent Transportation System (ITS). The main challenge that vehicle ReID presents is the large intra-class and small inter-class variability of vehicles appearance, followed by illumination changes, different viewpoints and scales, lack of labelled data and camera resolution. To address these problems, we present a vehicle ReID system that combines different ReID models, including appearance and orientation deep learning features. Additionally, for results refinement re-ranking and a post-processing step taking into account the vehicle trajectory information provided by the CityFlow-ReID dataset are applied.
SaHAN: Scale-aware hierarchical attention network for scene text recognition

Scene text recognition has become a research hotspot owing to its abundant semantic information and various applications. Recent methods of scene text recognition usually focus on handling shape distortion, attention drift, or background noise, ignoring that text recognition encounters character scale-variation problem. To address this issue, in this paper, we propose a new scale-aware hierarchical attention network (SaHAN) for scene text recognition. Inspired by feature pyramid network, we exploit the inherent pyramidal structure of a deep convolutional network to retain multi-scale features for flexible receptive fields. Then, we construct a hierarchical attention decoder that performs the attention mechanism twice on multi-scale features to collect the most fine-grained information for prediction. The SaHAN is trained in a weak supervision way, requiring only images and corresponding text labels. Extensive experiments on seven benchmarks reveal that SaHAN achieves state-of-the-art performance.
Solving arithmetic word problems: A deep learning based approach

This paper presents a novel deep learning based approach to solving arithmetic word problems. Solving different types of mathematical (math) word problems (MWP) is a very complex and challenging task as it requires Natural Language Understanding (NLU) and Commonsense knowledge. An application on this can benefit learning (education) technologies such as E-learning systems, Intelligent tutoring, Learning Management Systems (LMS), Innovative teaching/learning, etc. We propose Deep Learning based Arithmetic Word Problem Solver, DLAWPS, an intelligent MWP solver system. DLAWPS consists of a Recurrent Neural Network (RNN) based Bi-directional Long Short-Term Memory (BiLSTM) to classify operation among four basic operations {+ , - , * , /}, and a knowledge-based irrelevant information removal unit (IIRU) to identify the relevant quantities to form an equation to solve arithmetic MWPs. Our system generates state-of-the-art results on the standard arithmetic word problem datasets –AddSub, SingleOp, and a Combined dataset.
Learning to Learn Cropping Models for Different Aspect Ratio Requirements

Image cropping aims at improving the framing of an image by removing its extraneous outer areas, which is widely used in the photography and printing industry. In some cases, the aspect ratio of cropping results is specified depending on some conditions. In this paper, we propose a meta-learning (learning to learn) based aspect ratio specified image cropping method called Mars, which can generate cropping results of different expected aspect ratios. In the proposed method, a base model and two meta-learners are obtained during the training stage. Given an aspect ratio in the test stage, a new model with new parameters can be generated from the base model. Specifically, the two meta-learners predict the parameters of the base model based on the given aspect ratio. The learning process of the proposed method is learning how to learn cropping models for different aspect ratio requirements, which is a typical meta-learning process. In the experiments, the proposed method is evaluated on three datasets and outperforms most state-of-the-art methods in terms of accuracy and speed. In addition, both the intermediate and final results show that the proposed model can predict different cropping windows for an image depending on different aspect ratio requirements.
A Quantum Interference Inspired Neural Matching Model for Ad-hoc Retrieval

An essential task of information retrieval (IR) is to compute the probability of relevance of a document given a query. If we regard a query term or n-gram fragment as a relevance matching unit, most retrieval models firstly calculate the relevance evidence between the given query and the candidate document separately, and then accumulate these evidences as the final document relevance prediction. This kind of approach obeys the the classical probability, which is not fully consistent with human cognitive rules in the actual retrieval process, due to the possible existence of interference effect between relevance matching units. In our work, we propose a Quantum Interference inspired Neural Matching model (QINM), which can apply the interference effects to guide the construction of additional evidence generated by the interaction between matching units in the retrieval process. Experimental results on two benchmark collections demonstrate that our approach outperforms the quantum-inspired retrieval models, and some well-known neural retrieval models in the ad-hoc retrieval task.
Robust and Fast Vehicle Turn-counts at Intersections via an Integrated Solution from Detection, Tracking and Trajectory Modeling

In this paper, we address the problem of vehicle turn-counts by class at multiple intersections, which is greatly challenged by inaccurate detection and tracking results caused by heavy weather, occlusion, illumination variations, background clutter, etc. Therefore, the complexity of the problem calls for an integrated solution that robustly extracts as much visual information as possible and efficiently combines it through sequential feedback cycles. We propose such an algorithm, which effectively combines detection, background modeling, tracking, trajectory modeling and matching in a sequential manner. Firstly, to improve detection performances, we design a GMM like background modeling method to detect moving objects. Then, the proposed GMM like background modeling method is combined with an effective yet efficient deep learning based detector to achieve high-quality vehicle detection. Based on the detection results, a simple yet effective multi-object tracking method is proposed to generate each vehicle's movement trajectory. Conditioned on each vehicle's trajectory, we then propose a trajectory modeling and matching schema which leverages the direction and speed of a local vehicle's trajectory to improve the robustness and accuracy of vehicle turn-counts. Our method is validated on the AICity Track1 dataset A, and has achieved 91.40% in effectiveness, 95.4% in efficiency, and 92.60% S1-score, respectively. The experimental results show that our method is not only effective and efficient, but also can achieve robust counting performance in real-world scenes.
Design of Reinforced Fuzzy Radial Basis Function Neural Network Classifier Driven With the Aid of Iterative Learning Techniques and Support Vector-Based Clustering

In this article, a reinforced fuzzy radial basis function neural network (R-FRBFNN) classifier is proposed. It focuses on the development of methodologies of reinforced architecture to improve classification accuracy and enhance the robust capability based on two learning strategies. The two learning strategies are summarized: 1) R-FRBFNN designed via support vector (SV)-based fuzzy C-means (FCM) clustering and softmax-based iterative reweighted least square (IRLS), which concentrate on improving the classification performance of R-FRBFNN; and 2) R-FRBFNN designed via SV-based FCM and softmax-based iterative quadratic programming (IQP), which focus on improving the robust abilities of the R-FRBFNN and reducing the effects of noise and outliers. The essential points of the proposed R-FRBFNN classifier are summarized as follows. a) The proposed R-FRBFNN consists of three phases: condition, conclusion, and inference. b) An SV-based FCM is considered for prioritizing the classification boundary and improving the classification performance of the proposed classifier. c) Three types of polynomials construct the conclusion phase. Two learning techniques are designed to update the coefficients of the polynomials. Softmax-based IRLS is a type of iterative learning technique based on Newton's method. Softmax-based IQP is more robust and avoids the degradation of generalization capabilities caused by outliers and noisy data. d) In the concept of reinforced architecture, SV-based FCM imposes compensation (membership degrees) on learning techniques according to the data characteristics encountered in the inference phase. Experimental results reported for benchmark data and outliers/noisy datasets demonstrate that the proposed classifier shows improved classification performance compared with other previously studied methods.
Learning to Optimize on SPD Manifolds

Many tasks in computer vision and machine learning are modeled as optimization problems with constraints in the form of Symmetric Positive Definite (SPD) matrices. Solving such optimization problems is challenging due to the non-linearity of the SPD manifold, making optimization with SPD constraints heavily relying on expert knowledge and human involvement. In this paper, we propose a meta-learning method to automatically learn an iterative optimizer on SPD manifolds. Specifically, we introduce a novel recurrent model that takes into account the structure of input gradients and identifies the updating scheme of optimization. We parameterize the optimizer by the recurrent model and utilize Riemannian operations to ensure that our method is faithful to the geometry of SPD manifolds. Compared with existing SPD optimizers, our optimizer effectively exploits the underlying data distribution and learns a better optimization trajectory in a data-driven manner. Extensive experiments on various computer vision tasks including metric nearness, clustering, and similarity learning demonstrate that our optimizer outperforms existing state-of-the-art methods consistently.
Multiple Quantization Parameter Optimization in Versatile Video Coding for 360° Videos

Consumer Electronics (CE) devices have started to support 360° videos in order to improve the perceived video quality by consumers. Higher compression in 360° videos not only lead to improved consumer perception, but also efficient energy and storage usage in CE devices. Consequently, efficient compression of such video content is becoming a critical design consideration of CE devices. To this end, this paper first proposes a residual weighting algorithm that utilizes the features of Weighted Craster Parabolic Projection Peak Signal-to-Noise Ratio (WCPPPSNR), to instigate the first stage of magnitude reduction of the residuals. The model parameters associated with the residual weighting algorithm are determined in the initial experimental stage. Next, the paper proposes a Quantization Parameter (QP) optimization technique which introduces the second stage of magnitude reduction for residuals with minimal impact on the overall video quality. The proposed method is tested on the Versatile Video Coding (VVC) test model with All-Intra configuration. In this context, the empirical results demonstrate that the proposed method improves the coding efficiency by 3.34% on average and outperforms the state-of-the-art techniques.
AN EXPLORATIVE STUDY TOWARDS USING VR SKETCHING AS A TOOL FOR IDEATION AND PROTOTYPING IN PRODUCT DESIGN

Abstract The advancements in the field of VR allow designers to use VR as a technology for sketching ideas in a virtual space. In this paper, Gravity Sketch (a VR sketching software) is used as an ideation sketching tool in the process of product styling. The goal of this research is to examine the advantages and points-of-pain when using virtual reality tools in the process of product styling. 29 Master students used the VR sketching tool in their design process and all of them had feedback and insights on the tool.
Reduced-Complexity Key Equation Solvers for Generalized Integrated Interleaved BCH Decoders

Generalized integrated interleaved (GII) codes nest linear block codewords to generate codewords belonging to stronger linear block codes. They can achieve very high throughput with excellent error-correcting capability. GII codes can be based on either Reed-Solomon (RS) or BCH codes. GII-BCH codes are the most promising candidate for error correction in next-generation terabit/s Flash and storage class memories, and optical communications. On the other hand, GII decoder implementation faces many challenges. In particular, the key equation solver (KES) in the nested decoding process causes not only clock frequency bottleneck but also large area. Although techniques have been developed recently to address these issues for GII-RS codes, they cannot be directly extended for binary GII-BCH codes if every other iteration of the nested KES is skipped to reduce the latency. Two instead of one higher-order syndromes need to be incorporated into each nested KES iteration and this makes the implementation much more challenging. In this paper, algorithmic reformulations and architectural modifications are developed to eliminate the clock frequency bottleneck and reduce the area of GII-BCH nested KES. Additionally, the number of processing elements is substantially reduced by analyzing the minimum number of coefficients to keep for the involved polynomials without undesirable degradation on the error-correcting performance. The critical path of the proposed scaled nested BCH KES architecture with reduced processing elements is only one multiplier and three adders. For an example GII-BCH code over GF(2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">12</sup> ) that has a t <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">v</sub> =58$ -error-correcting code as the most powerful code generated by the nesting, our design also further reduces the area by 33%.
IP lifecycle management using blockchain and machine learning: Application to 3D printing datafiles

This paper describes a process in which Artificial Intelligence (AI) and blockchain are complementarily exploited for the lifecycle management of IP rights. A given party interacts with a blockchain to notarize IP assets and uses a machine learning pipeline to compare such assets to other assets from competitors. Analytics and statistics are additionally used to achieve meaningful comparisons and draw conclusions in terms of IP right infringement or validity. Application is made to 3D printing datafiles (Lego-like bricks). In general, such datafiles may potentially be subject to various IP rights, including copyright, trademarks, design patents, utility patents, and prior use rights. As such, they offer use cases of potential interest to many IP practitioners.
An OCR for Classical Indic Documents Containing Arbitrarily Long Words

OCR for printed classical Indic documents written in Sanskrit is a challenging research problem. It involves complexities such as image degradation, lack of datasets and long-length words. Due to these challenges, the word accuracy of available OCR systems, both academic and industrial, is not very high for such documents. To address these shortcomings, we develop a Sanskrit specific OCR system. We present an attention-based LSTM model for reading Sanskrit characters in line images. We introduce a dataset of Sanskrit document images annotated at line level. To augment real data and enable high performance for our OCR, we also generate synthetic data via curated font selection and rendering designed to incorporate crucial glyph substitution rules. Consequently, our OCR achieves a word error rate of 15.97% and a character error rate of 3.71% on challenging Indic document texts and outperforms strong baselines. Overall, our contributions set the stage for application of OCRs on large corpora of classic Sanskrit texts containing arbitrarily long and highly conjoined words.
Unsupervised Magnification of Posture Deviations Across Subjects

Analyzing human posture and precisely comparing it across different subjects is essential for accurate understanding of behavior and numerous vision applications such as medical diagnostics, sports, or surveillance. Motion magnification techniques help to see even small deviations in posture that are invisible to the naked eye. However, they fail when comparing subtle posture differences across individuals with diverse appearance. Keypoint-based posture estimation and classification techniques can handle large variations in appearance, but are invariant to subtle deviations in posture. We present an approach to unsupervised magnification of posture differences across individuals despite large deviations in appearance. We do not require keypoint annotation and visualize deviations on a sub-bodypart level. To transfer appearance across subjects onto a magnified posture, we propose a novel loss for disentangling appearance and posture in an autoencoder. Posture magnification yields exaggerated images that are different from the training set. Therefore, we incorporate magnification already into the training of the disentangled autoencoder and learn on real data and synthesized magnifications without supervision. Experiments confirm that our approach improves upon the state-of-the-art in magnification and on the application of discovering posture deviations due to impairment.
Multi-Objective Optimization of Wavelet-Packet-Based Features in Pathological Diagnosis of Alzheimer Using Spontaneous Speech Signals

Alzheimer's disease (AD) ranks among the main types of neurodegenerative disorders. Patients suffering AD should tackle serious problems since their language skills malfunction. The impact of such disorders is reflected by reduced quality and feature variation of spontaneous speech signals in speech analysis. This paper aims at assessing the variations of some specific types of these energy- and entropy-based features within the frequency range of the speech signals. In the approach followed, the wavelet-packet coefficients are utilized to extract the energy and entropy measures at every spectral sub-band in six successive levels of decomposition. However, the decomposition process conducts a set of high-dimensional feature vectors that is a challenging task for feature selection. This study suggests the application of a Non-dominated Sorting Genetic Algorithm-II (NSGA-II) for enhancing a group of the sub-band indexes of a wavelet-packet for which the extracted features lead to the highest diagnosis rate of the grouping of Alzheimer's and healthy individuals. The technique proposed here showed that the best overall classification results for both optimized entropy feature vs. energy are more noticeable in discriminating patients with AD from healthy subjects. It is also confirmed the significant impact of multi-objective feature selection on performance of classification (i.e., disease diagnosis) and, its conformity to the disordered nature of the biological signals could help diagnose AD in an efficient manner.
Evade Deep Image Retrieval by Stashing Private Images in the Hash Space

With the rapid growth of visual content, deep learning to hash is gaining popularity in the image retrieval community recently. Although it greatly facilitates search efficiency, privacy is also at risks when images on the web are retrieved at a large scale and exploited as a rich mine of personal information. An adversary can extract private images by querying similar images from the targeted category for any usable model. Existing methods based on image processing preserve privacy at a sacrifice of perceptual quality. In this paper, we propose a new mechanism based on adversarial examples to "stash'' private images in the deep hash space while maintaining perceptual similarity. We first find that a simple approach of hamming distance maximization is not robust against brute-force adversaries. Then we develop a new loss function by maximizing the hamming distance to not only the original category, but also the centers from all the classes, partitioned into clusters of various sizes. The extensive experiment shows that the proposed defense can harden the attacker's efforts by 2-7 orders of magnitude, without significant increase of computational overhead and perceptual degradation. We also demonstrate 30-60% transferability in hash space with a black-box setting. The code is available at: https://github.com/sugarruy/hashstash
FBRNN: feedback recurrent neural network for extreme image super-resolution

Single image extreme Super Resolution (SR) is a difficult task as scale factor in the order of 10X or greater is typically attempted. For instance, in the case of 16x upscale of an image, a single pixel from a low resolution image gets expanded to a 16x16 image patch. Such attempts often result fuzzy quality and loss in details in reconstructed images. To handle these difficulties, we propose a network architecture composed of a series of connected blocks in recurrent and feedback fashions for enhanced SR reconstruction. By use of recurrent network, an SR image is refined over a sequence of enhancement stages in coarse to fine manner. Additionally, each stage involves back projection of SR image to LR images for continuously being refined during the sequence. According to the preliminary results of NTIRE 2020 Perceptual Extreme SR challenge, our team (KU_ISPLB) secured 6th place by PSNR and 7th place by SSIM among all participants.
Choppy: Cut Transformer for Ranked List Truncation

Work in information retrieval has traditionally focused on ranking and relevance: given a query, return some number of results ordered by relevance to the user. However, the problem of determining how many results to return, i.e. how to optimally truncate the ranked result list, has received less attention despite being of critical importance in a range of applications. Such truncation is a balancing act between the overall relevance, or usefulness of the results, with the user cost of processing more results. In this work, we propose Choppy, an assumption-free model based on the widely successful Transformer architecture, to the ranked list truncation problem. Needing nothing more than the relevance scores of the results, the model uses a powerful multi-head attention mechanism to directly optimize any user-defined IR metric. We show Choppy improves upon recent state-of-the-art methods.