OpenRaiser/Intern-Atlas huggingface dataset:
ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network
Venue: arXiv cs.CV 2020
Year: 2020

Food recognition has received more and more attention in the multimedia community for its various real-world applications, such as diet management and self-service restaurants. A large-scale ontology of food images is urgently needed for developing advanced large-scale food recognition algorithms, as well as for providing the benchmark dataset for such algorithms. To encourage further progress in food recognition, we introduce the dataset ISIA Food- 500 with 500 categories from the list in the Wikipedia and 399,726 images, a more comprehensive food dataset that surpasses existing popular benchmark datasets by category coverage and data volume. Furthermore, we propose a stacked global-local attention network, which consists of two sub-networks for food recognition. One subnetwork first utilizes hybrid spatial-channel attention to extract more discriminative features, and then aggregates these multi-scale discriminative features from multiple layers into global-level representation (e.g., texture and shape information about food). The other one generates attentional regions (e.g., ingredient relevant regions) from different regions via cascaded spatial transformers, and further aggregates these multi-scale regional features from different layers into local-level representation. These two types of features are finally fused as comprehensive representation for food recognition. Extensive experiments on ISIA Food-500 and other two popular benchmark datasets demonstrate the effectiveness of our proposed method, and thus can be considered as one strong baseline. The dataset, code and models can be found at http://123.57.42.89/FoodComputing-Dataset/ISIA-Food500.html.
JQF: Optimal JPEG Quantization Table Fusion by Simulated Annealing on Texture Images and Predicting Textures
Venue: arXiv cs.MM 2020
Year: 2020

JPEG has been a widely used lossy image compression codec for nearly three decades. The JPEG standard allows to use customized quantization table; however, it's still a challenging problem to find an optimal quantization table within acceptable computational cost. This work tries to solve the dilemma of balancing between computational cost and image specific optimality by introducing a new concept of texture mosaic images. Instead of optimizing a single image or a collection of representative images, the simulated annealing technique is applied to texture mosaic images to search for an optimal quantization table for each texture category. We use pre-trained VGG-16 CNN model to learn those texture features and predict the new image's texture distribution, then fuse optimal texture tables to come out with an image specific optimal quantization table. On the Kodak dataset with the quality setting $Q=95$, our experiment shows a size reduction of 23.5% over the JPEG standard table with a slightly 0.35% FSIM decrease, which is visually unperceivable. The proposed JQF method achieves per image optimality for JPEG encoding with less than one second additional timing cost. The online demo is available at https://matthorn.s3.amazonaws.com/JQF/qtbl_vis.html
Area Optimized Quasi Delay Insensitive Majority Voter for TMR Applications
Venue: arXiv cs.AR 2020
Year: 2020

Mission-critical and safety-critical applications generally tend to incorporate triple modular redundancy (TMR) to embed fault tolerance in their physical implementations. In a TMR realization, an original function block, which may be a circuit or a system, and two exact copies of the function block are used to successfully overcome any temporary fault or permanent failure of an arbitrary function block during the routine operation. The corresponding outputs of the function blocks are majority voted using 3-input majority voters whose outputs define the outputs of a TMR realization. Hence, a 3-input majority voter forms an important component of a TMR realization. Many synchronous majority voters and an asynchronous non-delay insensitive majority voter have been presented in the literature. Recently, quasi delay insensitive (QDI) asynchronous majority voters for TMR applications were also discussed in the literature. In this regard, this paper presents a new QDI asynchronous majority voter for TMR applications, which is better optimized in area compared to the existing QDI majority voters. The proposed QDI majority voter requires 30.2% less area compared to the best of the existing QDI majority voters, and this could be useful for resource-constrained fault tolerance applications. The example QDI TMR circuits were implemented using a 32/28nm complementary metal oxide semiconductor (CMOS) process. The delay insensitive dual rail code was used for data encoding, and 4-phase return-to-zero and return-to-one handshake protocols were used for data communication.
Robust Image Matching By Dynamic Feature Selection
Venue: arXiv cs.CV 2020
Year: 2020

Estimating dense correspondences between images is a long-standing image under-standing task. Recent works introduce convolutional neural networks (CNNs) to extract high-level feature maps and find correspondences through feature matching. However,high-level feature maps are in low spatial resolution and therefore insufficient to provide accurate and fine-grained features to distinguish intra-class variations for correspondence matching. To address this problem, we generate robust features by dynamically selecting features at different scales. To resolve two critical issues in feature selection,i.e.,how many and which scales of features to be selected, we frame the feature selection process as a sequential Markov decision-making process (MDP) and introduce an optimal selection strategy using reinforcement learning (RL). We define an RL environment for image matching in which each individual action either requires new features or terminates the selection episode by referring a matching score. Deep neural networks are incorporated into our method and trained for decision making. Experimental results show that our method achieves comparable/superior performance with state-of-the-art methods on three benchmarks, demonstrating the effectiveness of our feature selection strategy.
Strategies for Efficient Executions of Irregular Message-Driven Parallel Applications on GPU Systems
Venue: arXiv cs.DC 2020
Year: 2020

Message-driven executions with over-decomposition of tasks constitute an important model for parallel programming and have been demonstrated for irregular applications. Supporting efficient execution of such message-driven irregular applications on GPU systems presents a number of challenges related to irregular data accesses and computations. In this work, we have developed strategies including coalescing irregular data accesses and combining with data reuse, and adaptive methods for hybrid executions to minimize idling. We have integrated these runtime strategies into our {\em G-Charm} framework for efficient execution of message-driven parallel applications on hybrid GPU systems. We demonstrate our strategies for irregular applications with an N-Body simulations and a molecular dynamics application and show that our dynamic strategies result in 8-38\% reduction in execution times for these irregular applications over the corresponding static strategies that are amenable for regular applications.
Exploration of Gender Differences in COVID-19 Discourse on Reddit
Venue: arXiv cs.CL 2020
Year: 2020

Decades of research on differences in the language of men and women have established postulates about preferences in lexical, topical, and emotional expression between the two genders, along with their sociological underpinnings. Using a novel dataset of male and female linguistic productions collected from the Reddit discussion platform, we further confirm existing assumptions about gender-linked affective distinctions, and demonstrate that these distinctions are amplified in social media postings involving emotionally-charged discourse related to COVID-19. Our analysis also confirms considerable differences in topical preferences between male and female authors in spontaneous pandemic-related discussions.
Learning Temporally Invariant and Localizable Features via Data Augmentation for Video Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Deep-Learning-based video recognition has shown promising improvements along with the development of large-scale datasets and spatiotemporal network architectures. In image recognition, learning spatially invariant features is a key factor in improving recognition performance and robustness. Data augmentation based on visual inductive priors, such as cropping, flipping, rotating, or photometric jittering, is a representative approach to achieve these features. Recent state-of-the-art recognition solutions have relied on modern data augmentation strategies that exploit a mixture of augmentation operations. In this study, we extend these strategies to the temporal dimension for videos to learn temporally invariant or temporally localizable features to cover temporal perturbations or complex actions in videos. Based on our novel temporal data augmentation algorithms, video recognition performances are improved using only a limited amount of training data compared to the spatial-only data augmentation algorithms, including the 1st Visual Inductive Priors (VIPriors) for data-efficient action recognition challenge. Furthermore, learned features are temporally localizable that cannot be achieved using spatial augmentation algorithms. Our source code is available at https://github.com/taeoh-kim/temporal_data_augmentation.
Pose Estimation for Vehicle-mounted Cameras via Horizontal and Vertical Planes
Venue: arXiv cs.CV 2020
Year: 2020

We propose two novel solvers for estimating the egomotion of a calibrated camera mounted to a moving vehicle from a single affine correspondence via recovering special homographies. For the first class of solvers, the sought plane is expected to be perpendicular to one of the camera axes. For the second class, the plane is orthogonal to the ground with unknown normal, e.g., it is a building facade. Both methods are solved via a linear system with a small coefficient matrix, thus, being extremely efficient. Both the minimal and over-determined cases can be solved by the proposed methods. They are tested on synthetic data and on publicly available real-world datasets. The novel methods are more accurate or comparable to the traditional algorithms and are faster when included in state of the art robust estimators.
MASRI-HEADSET: A Maltese Corpus for Speech Recognition
Venue: arXiv cs.CL 2020
Year: 2020

Maltese, the national language of Malta, is spoken by approximately 500,000 people. Speech processing for Maltese is still in its early stages of development. In this paper, we present the first spoken Maltese corpus designed purposely for Automatic Speech Recognition (ASR). The MASRI-HEADSET corpus was developed by the MASRI project at the University of Malta. It consists of 8 hours of speech paired with text, recorded by using short text snippets in a laboratory environment. The speakers were recruited from different geographical locations all over the Maltese islands, and were roughly evenly distributed by gender. This paper also presents some initial results achieved in baseline experiments for Maltese ASR using Sphinx and Kaldi. The MASRI-HEADSET Corpus is publicly available for research/academic purposes.
CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration
Venue: arXiv cs.CV 2020
Year: 2020

Image registration is a fundamental task in medical image analysis. Recently, deep learning based image registration methods have been extensively investigated due to their excellent performance despite the ultra-fast computational time. However, the existing deep learning methods still have limitation in the preservation of original topology during the deformation with registration vector fields. To address this issues, here we present a cycle-consistent deformable image registration. The cycle consistency enhances image registration performance by providing an implicit regularization to preserve topology during the deformation. The proposed method is so flexible that can be applied for both 2D and 3D registration problems for various applications, and can be easily extended to multi-scale implementation to deal with the memory issues in large volume registration. Experimental results on various datasets from medical and non-medical applications demonstrate that the proposed method provides effective and accurate registration on diverse image pairs within a few seconds. Qualitative and quantitative evaluations on deformation fields also verify the effectiveness of the cycle consistency of the proposed method.
Process Discovery for Structured Program Synthesis
Venue: arXiv cs.AI 2020
Year: 2020

A core task in process mining is process discovery which aims to learn an accurate process model from event log data. In this paper, we propose to use (block-) structured programs directly as target process models so as to establish connections to the field of program synthesis and facilitate the translation from abstract process models to executable processes, e.g., for robotic process automation. Furthermore, we develop a novel bottom-up agglomerative approach to the discovery of such structured program process models. In comparison with the popular top-down recursive inductive miner, our proposed agglomerative miner enjoys the similar theoretical guarantee to produce sound process models (without deadlocks and other anomalies) while exhibiting some advantages like avoiding silent activities and accommodating duplicate activities. The proposed algorithm works by iteratively applying a few graph rewriting rules to the directly-follows-graph of activities. For real-world (sparse) directly-follows-graphs, the algorithm has quadratic computational complexity with respect to the number of distinct activities. To our knowledge, this is the first process discovery algorithm that is made for the purpose of program synthesis. Experiments on the BPI-Challenge 2020 dataset and the Karel programming dataset have demonstrated that our proposed algorithm can outperform the inductive miner not only according to the traditional process discovery metrics but also in terms of the effectiveness in finding out the true underlying structured program from a small number of its execution traces.
Revealing the Hidden Patterns: A Comparative Study on Profiling Subpopulations of MOOC Students
Venue: arXiv cs.HC 2020
Year: 2020

Massive Open Online Courses (MOOCs) exhibit a remarkable heterogeneity of students. The advent of complex "big data" from MOOC platforms is a challenging yet rewarding opportunity to deeply understand how students are engaged in MOOCs. Past research, looking mainly into overall behavior, may have missed patterns related to student diversity. Using a large dataset from a MOOC offered by FutureLearn, we delve into a new way of investigating hidden patterns through both machine learning and statistical modelling. In this paper, we report on clustering analysis of student activities and comparative analysis on both behavioral patterns and demographical patterns between student subpopulations in the MOOC. Our approach allows for a deeper understanding of how MOOC students behave and achieve. Our findings may be used to design adaptive strategies towards an enhanced MOOC experience
Recurrent Deconvolutional Generative Adversarial Networks with Application to Text Guided Video Generation
Venue: arXiv cs.CV 2020
Year: 2020

This paper proposes a novel model for video generation and especially makes the attempt to deal with the problem of video generation from text descriptions, i.e., synthesizing realistic videos conditioned on given texts. Existing video generation methods cannot be easily adapted to handle this task well, due to the frame discontinuity issue and their text-free generation schemes. To address these problems, we propose a recurrent deconvolutional generative adversarial network (RD-GAN), which includes a recurrent deconvolutional network (RDN) as the generator and a 3D convolutional neural network (3D-CNN) as the discriminator. The RDN is a deconvolutional version of conventional recurrent neural network, which can well model the long-range temporal dependency of generated video frames and make good use of conditional information. The proposed model can be jointly trained by pushing the RDN to generate realistic videos so that the 3D-CNN cannot distinguish them from real ones. We apply the proposed RD-GAN to a series of tasks including conventional video generation, conditional video generation, video prediction and video classification, and demonstrate its effectiveness by achieving well performance.
Balancing Taxi Distribution in A City-Scale Dynamic Ridesharing Service: A Hybrid Solution Based on Demand Learning
Venue: arXiv cs.CY 2020
Year: 2020

In this paper, we study the challenging problem of how to balance taxi distribution across a city in a dynamic ridesharing service. First, we introduce the architecture of the dynamic ridesharing system and formally define the performance metrics indicating the efficiency of the system. Then, we propose a hybrid solution involving a series of algorithms: the Correlated Pooling collects correlated rider requests, the Adjacency Ride-Matching based on Demand Learning assigns taxis to riders and balances taxi distribution locally, the Greedy Idle Movement aims to direct taxis without a current assignment to the areas with riders in need of service. In the experiment, we apply city-scale data sets from the city of Chicago and complete a case study analyzing the threshold of correlated rider requests and the average online running time of each algorithm. We also compare our hybrid solution with multiple other methods. The results of our experiment show that our hybrid solution improves customer serving rate without increasing the number of taxis in operation, allows both drivers to earn more and riders to save more per trip, and all with a small increase in calling and extra trip time.
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Venue: arXiv cs.CV 2020
Year: 2020

We present a new, publicly-available image dataset generated by the NVIDIA Deep Learning Data Synthesizer intended for use in object detection, pose estimation, and tracking applications. This dataset contains 144k stereo image pairs that synthetically combine 18 camera viewpoints of three photorealistic virtual environments with up to 10 objects (chosen randomly from the 21 object models of the YCB dataset [1]) and flying distractors. Object and camera pose, scene lighting, and quantity of objects and distractors were randomized. Each provided view includes RGB, depth, segmentation, and surface normal images, all pixel level. We describe our approach for domain randomization and provide insight into the decisions that produced the dataset.
Zecale: Reconciling Privacy and Scalability on Ethereum
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we present Zecale, a general purpose SNARK proof aggregator that uses recursive composition of SNARKs. We start by introducing the notion of recursive composition of SNARKs, before introducing Zecale as a privacy preserving scalability solution. Then, we list application types that can emerge and be built with Zecale. Finally, we argue that such scalability solutions for privacy preserving state transitions are paramount to emulate "cash" on blockchain systems.
Sniffing for Codebase Secret Leaks with Known Production Secrets in Industry
Venue: arXiv cs.CR 2020
Year: 2020

Leaked secrets, such as passwords and API keys, in codebases were responsible for numerous security breaches. Existing heuristic techniques, such as pattern matching, entropy analysis, and machine learning, exist to detect and alert developers of such leaks. Heuristics, however, naturally exhibit false positives, which require triaging and can lead to developer frustration. We propose to use known production secrets as a source of ground truth for sniffing secret leaks in codebases. We develop techniques for using known secrets to sniff whole codebases and continuously sniff differential code revisions. We uncover different performance and security needs when sniffing for known secrets in these two situations in an industrial environment.
On the design of text editors
Venue: arXiv cs.HC 2020
Year: 2020

Text editors are written by and for developers. They come with a large set of default and implicit choices in terms of layout, typography, colorization and interaction that hardly change from one editor to the other. It is not clear if these implicit choices derive from the ignorance of alternatives or if they derive from developers' habits, reproducing what they are used to. The goal of this article is to characterize these implicit choices and to illustrate what are some alternatives without prescribing one or the other.
Towards Visually Explaining Similarity Models
Venue: arXiv cs.CV 2020
Year: 2020

We consider the problem of visually explaining similarity models, i.e., explaining why a model predicts two images to be similar in addition to producing a scalar score. While much recent work in visual model interpretability has focused on gradient-based attention, these methods rely on a classification module to generate visual explanations. Consequently, they cannot readily explain other kinds of models that do not use or need classification-like loss functions (e.g., similarity models trained with a metric learning loss). In this work, we bridge this crucial gap, presenting a method to generate gradient-based visual attention for image similarity predictors. By relying solely on the learned feature embedding, we show that our approach can be applied to any kind of CNN-based similarity architecture, an important step towards generic visual explainability. We show that our resulting attention maps serve more than just interpretability; they can be infused into the model learning process itself with new trainable constraints. We show that the resulting similarity models perform, and can be visually explained, better than the corresponding baseline models trained without these constraints. We demonstrate our approach using extensive experiments on three different kinds of tasks: generic image retrieval, person re-identification, and low-shot semantic segmentation.
Constructed emotions and superinformation: a constructor-theoretic approach
Venue: arXiv cs.AI 2020
Year: 2020

In this paper we apply the constructor-theoretic approach to the theory of constructed emotions, showing that core affect valence and knowledge can be considered as two different observables, leading to information or superinformation conditions: this depends on subject's strategy, coherently with the affect infusion model. In the second part of the article we show that additional hypotheses on the structure of information allows to study emotions in terms of the contructor-theoretic version of phase task. Quantum algorithms are presented as an example of the connection between emotions and memory tasks.