OpenRaiser/Intern-Atlas huggingface dataset:
An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with Diacritics
Venue: arXiv cs.CV 2022
Year: 2022

This research is the second phase in a series of investigations on developing an Optical Character Recognition (OCR) of Arabic historical documents and examining how different modeling procedures interact with the problem. The first research studied the effect of Transformers on our custom-built Arabic dataset. One of the downsides of the first research was the size of the training data, a mere 15000 images from our 30 million images, due to lack of resources. Also, we add an image enhancement layer, time and space optimization, and Post-Correction layer to aid the model in predicting the correct word for the correct context. Notably, we propose an end-to-end text recognition approach using Vision Transformers as an encoder, namely BEIT, and vanilla Transformer as a decoder, eliminating CNNs for feature extraction and reducing the model's complexity. The experiments show that our end-to-end model outperforms Convolutions Backbones. The model attained a CER of 4.46%.
Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer
Venue: arXiv cs.SE 2022
Year: 2022

Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems. Nevertheless, if inexperienced developers fail to describe their problems clearly, it is hard for them to attract sufficient attention and get the anticipated answers. We propose M$_3$NSCT5, a novel approach to automatically generate multiple post titles from the given code snippets. Developers may use the generated titles to find closely related posts and complete their problem descriptions. M$_3$NSCT5 employs the CodeT5 backbone, which is a pre-trained Transformer model having an excellent language understanding and generation ability. To alleviate the ambiguity issue that the same code snippets could be aligned with different titles under varying contexts, we propose the maximal marginal multiple nucleus sampling strategy to generate multiple high-quality and diverse title candidates at a time for the developers to choose from. We build a large-scale dataset with 890,000 question posts covering eight programming languages to validate the effectiveness of M$_3$NSCT5. The automatic evaluation results on the BLEU and ROUGE metrics demonstrate the superiority of M$_3$NSCT5 over six state-of-the-art baseline models. Moreover, a human evaluation with trustworthy results also demonstrates the great potential of our approach for real-world application.
ssFPN: Scale Sequence (S^2) Feature Based-Feature Pyramid Network for Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Feature Pyramid Network (FPN) has been an essential module for object detection models to consider various scales of an object. However, average precision (AP) on small objects is relatively lower than AP on medium and large objects. The reason is why the deeper layer of CNN causes information loss as feature extraction level. We propose a new scale sequence (S^2) feature extraction of FPN to strengthen feature information of small objects. We consider FPN structure as scale-space and extract scale sequence (S^2) feature by 3D convolution on the level axis of FPN. It is basically scale invariant feature and is built on high-resolution pyramid feature map for small objects. Furthermore, the proposed S^2 feature can be extended to most object detection models based on FPN. We demonstrate the proposed S2 feature can improve the performance of both one-stage and two-stage detectors on MS COCO dataset. Based on the proposed S2 feature, we achieve upto 1.3% and 1.1% of AP improvement for YOLOv4-P5 and YOLOv4-P6, respectively. For Faster RCNN and Mask R-CNN, we observe upto 2.0% and 1.6% of AP improvement with the suggested S^2 feature, respectively.
A Scalable and Energy Efficient GPU Thread Map for m-Simplex Domains
Venue: arXiv cs.DC 2022
Year: 2022

This work proposes a new GPU thread map for $m$-simplex domains, that scales its speedup with dimension and is energy efficient compared to other state of the art approaches. The main contributions of this work are i) the formulation of the new block-space map $\mathcal{H}: \mathbb{Z}^m \mapsto \mathbb{Z}^m$ for regular orthogonal simplex domains, which is analyzed in terms of resource usage, and ii) the experimental evaluation in terms of speedup over a bounding box approach and energy efficiency as elements per second per Watt. Results from the analysis show that $\mathcal{H}$ has a potential speedup of up to $2\times$ and $6\times$ for $2$ and $3$-simplices, respectively. Experimental evaluation shows that $\mathcal{H}$ is competitive for $2$-simplices, reaching $1.2\times \sim 2.0\times$ of speedup for different tests, which is on par with the fastest state of the art approaches. For $3$-simplices $\mathcal{H}$ reaches up to $1.3\times \sim 6.0\times$ of speedup making it the fastest of all. The extension of $\mathcal{H}$ to higher dimensional $m$-simplices is feasible and has a potential speedup that scales as $m!$ given a proper selection of parameters $r, β$ which are the scaling and replication factors, respectively. In terms of energy consumption, although $\mathcal{H}$ is among the highest in power consumption, it compensates by its short duration, making it one of the most energy efficient approaches. Lastly, further improvements with Tensor and Ray Tracing Cores are analyzed, giving insights to leverage each one of them. The results obtained in this work show that $\mathcal{H}$ is a scalable and energy efficient map that can contribute to the efficiency of GPU applications when they need to process $m$-simplex domains, such as Cellular Automata or PDE simulations.
On a Built-in Conflict between Deep Learning and Systematic Generalization
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we hypothesize that internal function sharing is one of the reasons to weaken o.o.d. or systematic generalization in deep learning for classification tasks. Under equivalent prediction, a model partitions an input space into multiple parts separated by boundaries. The function sharing prefers to reuse boundaries, leading to fewer parts for new outputs, which conflicts with systematic generalization. We show such phenomena in standard deep learning models, such as fully connected, convolutional, residual networks, LSTMs, and (Vision) Transformers. We hope this study provides novel insights into systematic generalization and forms a basis for new research directions.
Towards Sparsified Federated Neuroimaging Models via Weight Pruning
Venue: arXiv cs.LG 2022
Year: 2022

Federated training of large deep neural networks can often be restrictive due to the increasing costs of communicating the updates with increasing model sizes. Various model pruning techniques have been designed in centralized settings to reduce inference times. Combining centralized pruning techniques with federated training seems intuitive for reducing communication costs -- by pruning the model parameters right before the communication step. Moreover, such a progressive model pruning approach during training can also reduce training times/costs. To this end, we propose FedSparsify, which performs model pruning during federated training. In our experiments in centralized and federated settings on the brain age prediction task (estimating a person's age from their brain MRI), we demonstrate that models can be pruned up to 95% sparsity without affecting performance even in challenging federated learning environments with highly heterogeneous data distributions. One surprising benefit of model pruning is improved model privacy. We demonstrate that models with high sparsity are less susceptible to membership inference attacks, a type of privacy attack.
ForestEyes Project: Conception, Enhancements, and Challenges
Venue: arXiv cs.CV 2022
Year: 2022

Rainforests play an important role in the global ecosystem. However, significant regions of them are facing deforestation and degradation due to several reasons. Diverse government and private initiatives were created to monitor and alert for deforestation increases from remote sensing images, using different ways to deal with the notable amount of generated data. Citizen Science projects can also be used to reach the same goal. Citizen Science consists of scientific research involving nonprofessional volunteers for analyzing, collecting data, and using their computational resources to outcome advancements in science and to increase the public's understanding of problems in specific knowledge areas such as astronomy, chemistry, mathematics, and physics. In this sense, this work presents a Citizen Science project called ForestEyes, which uses volunteer's answers through the analysis and classification of remote sensing images to monitor deforestation regions in rainforests. To evaluate the quality of those answers, different campaigns/workflows were launched using remote sensing images from Brazilian Legal Amazon and their results were compared to an official groundtruth from the Amazon Deforestation Monitoring Project PRODES. In this work, the first two workflows that enclose the State of Rondônia in the years 2013 and 2016 received more than $35,000$ answers from $383$ volunteers in the $2,050$ created tasks in only two and a half weeks after their launch. For the other four workflows, even enclosing the same area (Rondônia) and different setups (e.g., image segmentation method, image resolution, and detection target), they received $51,035$ volunteers' answers gathered from $281$ volunteers in $3,358$ tasks. In the performed experiments...
Hyperparameter Optimization for Unsupervised Outlier Detection
Venue: arXiv cs.LG 2022
Year: 2022

Given an unsupervised outlier detection (OD) algorithm, how can we optimize its hyperparameter(s) (HP) on a new dataset, without any labels? In this work, we address this challenging hyperparameter optimization for unsupervised OD problem, and propose the first systematic approach called HPOD that is based on meta-learning. HPOD capitalizes on the prior performance of a large collection of HPs on existing OD benchmark datasets, and transfers this information to enable HP evaluation on a new dataset without labels. Moreover, HPOD adapts a prominent sampling paradigm to identify promising HPs efficiently. Extensive experiments show that HPOD works with both deep (e.g., Robust AutoEncoder) and shallow (e.g., Local Outlier Factor (LOF) and Isolation Forest (iForest)) OD algorithms on discrete and continuous HP spaces, and outperforms a wide range of baselines with on average 58% and 66% performance improvement over the default HPs of LOF and iForest.
Maximum Likelihood on the Joint (Data, Condition) Distribution for Solving Ill-Posed Problems with Conditional Flow Models
Venue: arXiv cs.LG 2022
Year: 2022

I describe a trick for training flow models using a prescribed rule as a surrogate for maximum likelihood. The utility of this trick is limited for non-conditional models, but an extension of the approach, applied to maximum likelihood of the joint probability distribution of data and conditioning information, can be used to train sophisticated \textit{conditional} flow models. Unlike previous approaches, this method is quite simple: it does not require explicit knowledge of the distribution of conditions, auxiliary networks or other specific architecture, or additional loss terms beyond maximum likelihood, and it preserves the correspondence between latent and data spaces. The resulting models have all the properties of non-conditional flow models, are robust to unexpected inputs, and can predict the distribution of solutions conditioned on a given input. They come with guarantees of prediction representativeness and are a natural and powerful way to solve highly uncertain problems. I demonstrate these properties on easily visualized toy problems, then use the method to successfully generate class-conditional images and to reconstruct highly degraded images via super-resolution.
Sampling and Optimal Preference Elicitation in Simple Mechanisms
Venue: arXiv cs.GT 2022
Year: 2022

In this work we are concerned with the design of efficient mechanisms while eliciting limited information from the agents. First, we study the performance of sampling approximations in facility location games. Our key result is to show that for any $ε> 0$, a sample of size $c(ε) = Θ(1/ε^2)$ yields in expectation a $1 + ε$ approximation with respect to the optimal social cost of the generalized median mechanism on the metric space $(\mathbb{R}^d, \| \cdot \|_1)$, while the number of agents $n \to \infty$. Moreover, we study a series of exemplar environments from auction theory through a communication complexity framework, measuring the expected number of bits elicited from the agents; we posit that any valuation can be expressed with $k$ bits, and we mainly assume that $k$ is independent of the number of agents $n$. In this context, we show that Vickrey's rule can be implemented with an expected communication of $1 + ε$ bits from an average bidder, for any $ε> 0$, asymptotically matching the trivial lower bound. As a corollary, we provide a compelling method to increment the price in an English auction. We also leverage our single-item format with an efficient encoding scheme to prove that the same communication bound can be recovered in the domain of additive valuations through simultaneous ascending auctions, assuming that the number of items is a constant. Finally, we propose an ascending-type multi-unit auction under unit demand bidders; our mechanism announces at every round two separate prices and is based on a sampling algorithm that performs approximate selection with limited communication, leading again to asymptotically optimal communication. Our results do not require any prior knowledge on the agents' valuations, and mainly follow from natural sampling techniques.
Orthogonal Time Frequency Space (OTFS) Modulation for Wireless Communications
Venue: arXiv cs.IT 2022
Year: 2022

Orthogonal time frequency space (OTFS) modulation is a recently proposed multi-carrier transmission scheme, which innovatively multiplexes the information symbols in the delay-Doppler (DD) domain instead of the conventional time-frequency (TF) domain. The DD domain symbol multiplexing gives rise to a direct interaction between the DD domain information symbols and DD domain channel responses, which are usually quasi-static, compact, separable, and potentially sparse. Therefore, OTFS modulation enjoys appealing advantages over the conventional orthogonal frequency-division multiplexing (OFDM) modulation for wireless communications. In this thesis, we investigate the related subjects of OTFS modulation for wireless communications, specifically focusing on its signal detection, performance analysis, and applications. These important aspects are discussed based on the review of the state-of-the-art and a detailed derivation of OTFS modulation from the discrete Zak transform (DZT). Finally, a summary of future research directions on OTFS modulation are also provided.
Are We Making Progress In Visualization Research?
Venue: arXiv cs.HC 2022
Year: 2022

In this work, I use a survey of senior visualization researchers and thinkers to ideate about the notion of progress in visualization research: how are we growing as a field, what are we building towards, and are our existing methods sufficient to get us there? My respondents discussed several potential challenges for visualization research in terms of knowledge formation: a lack of rigor in the methods used, a lack of applicability to actual communities of practice, and a lack of theoretical structures that incorporate everything that happens to people and to data both before and after the few seconds when a viewer looks at a value in a chart. Orienting the field around progress (if such a thing is even desirable, which is another point of contention) I believe will require drastic re-conceptions of what the field is, what it values, and how it is taught.
A Perturbation Resistant Transformation and Classification System for Deep Neural Networks
Venue: arXiv cs.CV 2022
Year: 2022

Deep convolutional neural networks accurately classify a diverse range of natural images, but may be easily deceived when designed, imperceptible perturbations are embedded in the images. In this paper, we design a multi-pronged training, input transformation, and image ensemble system that is attack agnostic and not easily estimated. Our system incorporates two novel features. The first is a transformation layer that computes feature level polynomial kernels from class-level training data samples and iteratively updates input image copies at inference time based on their feature kernel differences to create an ensemble of transformed inputs. The second is a classification system that incorporates the prediction of the undefended network with a hard vote on the ensemble of filtered images. Our evaluations on the CIFAR10 dataset show our system improves the robustness of an undefended network against a variety of bounded and unbounded white-box attacks under different distance metrics, while sacrificing little accuracy on clean images. Against adaptive full-knowledge attackers creating end-to-end attacks, our system successfully augments the existing robustness of adversarially trained networks, for which our methods are most effectively applied.
An Empirical Analysis of the Efficacy of Different Sampling Techniques for Imbalanced Classification
Venue: arXiv cs.LG 2022
Year: 2022

Learning from imbalanced data is a challenging task. Standard classification algorithms tend to perform poorly when trained on imbalanced data. Some special strategies need to be adopted, either by modifying the data distribution or by redesigning the underlying classification algorithm to achieve desirable performance. The prevalence of imbalance in real-world datasets has led to the creation of a multitude of strategies for the class imbalance issue. However, not all the strategies are useful or provide good performance in different imbalance scenarios. There are numerous approaches to dealing with imbalanced data, but the efficacy of such techniques or an experimental comparison among those techniques has not been conducted. In this study, we present a comprehensive analysis of 26 popular sampling techniques to understand their effectiveness in dealing with imbalanced data. Rigorous experiments have been conducted on 50 datasets with different degrees of imbalance to thoroughly investigate the performance of these techniques. A detailed discussion of the advantages and limitations of the techniques, as well as how to overcome such limitations, has been presented. We identify some critical factors that affect the sampling strategies and provide recommendations on how to choose an appropriate sampling technique for a particular application.
A Privacy-Preserving and End-to-End-Based Encrypted Image Retrieval Scheme
Venue: arXiv cs.MM 2022
Year: 2022

Applying encryption technology to image retrieval can ensure the security and privacy of personal images. The related researches in this field have focused on the organic combination of encryption algorithm and artificial feature extraction. Many existing encrypted image retrieval schemes cannot prevent feature leakage and file size increase or cannot achieve satisfied retrieval performance. In this paper, A new end-to-end encrypted image retrieval scheme is presented. First, images are encrypted by using block rotation, new orthogonal transforms and block permutation during the JPEG compression process. Second, we combine the triplet loss and the cross entropy loss to train a network model, which contains gMLP modules, by end-to-end learning for extracting cipher-images' features. Compared with manual features extraction such as extracting color histogram, the end-to-end mechanism can economize on manpower. Experimental results show that our scheme has good retrieval performance, while can ensure compression friendly and no feature leakage.
Multi-Hop Beam Routing for Hybrid Active/Passive IRS Aided Wireless Communications
Venue: arXiv cs.IT 2022
Year: 2022

Prior studies on intelligent reflecting surface (IRS) have mostly considered wireless communication systems aided by a single passive IRS, which, however, has limited control over wireless propagation environment and suffers from product-distance path-loss. To address these issues, we propose in this paper a new hybrid active/passive IRS aided wireless communication system, where an active IRS and multiple passive IRSs are deployed to assist the communication between a base station (BS) and a remote user in complex environment, by establishing a multihop reflection path across active/passive IRSs. In particular, the active IRS enables signal reflection with power amplification, thus effectively compensating the severe path-loss in the multi-reflection path. To maximize the achievable rate at the user, we first design the optimal beamforming of the BS and selected (active/passive) IRSs for a given multi-reflection path, and then propose an efficient algorithm to obtain the optimal multi-reflection path by using the path decomposition method and graph theory. We show that the active IRS should be selected to establish the beam routing path when its amplification power and/or number of active reflecting elements are sufficiently large. Last, numerical results demonstrate the effectiveness of the proposed hybrid active/passive IRS beam routing design as compared to the benchmark scheme with passive IRSs only.
Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we propose a probabilistic continuous-time visual-inertial odometry (VIO) for rolling shutter cameras. The continuous-time trajectory formulation naturally facilitates the fusion of asynchronized high-frequency IMU data and motion-distorted rolling shutter images. To prevent intractable computation load, the proposed VIO is sliding-window and keyframe-based. We propose to probabilistically marginalize the control points to keep the constant number of keyframes in the sliding window. Furthermore, the line exposure time difference (line delay) of the rolling shutter camera can be online calibrated in our continuous-time VIO. To extensively examine the performance of our continuous-time VIO, experiments are conducted on publicly-available WHU-RSVI, TUM-RSVI, and SenseTime-RSVI rolling shutter datasets. The results demonstrate the proposed continuous-time VIO significantly outperforms the existing state-of-the-art VIO methods. The codebase of this paper will also be open-sourced at \url{https://github.com/APRIL-ZJU/Ctrl-VIO}.