OpenRaiser/Intern-Atlas huggingface dataset:
Importance of Textlines in Historical Document Classification
Venue: arXiv cs.CV 2022
Year: 2022

This paper describes a system prepared at Brno University of Technology for ICDAR 2021 Competition on Historical Document Classification, experiments leading to its design, and the main findings. The solved tasks include script and font classification, document origin localization, and dating. We combined patch-level and line-level approaches, where the line-level system utilizes an existing, publicly available page layout analysis engine. In both systems, neural networks provide local predictions which are combined into page-level decisions, and the results of both systems are fused using linear or log-linear interpolation. We propose loss functions suitable for weakly supervised classification problem where multiple possible labels are provided, and we propose loss functions suitable for interval regression in the dating task. The line-level system significantly improves results in script and font classification and in the dating task. The full system achieved 98.48 %, 88.84 %, and 79.69 % accuracy in the font, script, and location classification tasks respectively. In the dating task, our system achieved a mean absolute error of 21.91 years. Our system achieved the best results in all tasks and became the overall winner of the competition.
Describe me if you can! Characterized Instance-level Human Parsing
Venue: arXiv cs.CV 2022
Year: 2022

Several computer vision applications such as person search or online fashion rely on human description. The use of instance-level human parsing (HP) is therefore relevant since it localizes semantic attributes and body parts within a person. But how to characterize these attributes? To our knowledge, only some single-HP datasets describe attributes with some color, size and/or pattern characteristics. There is a lack of dataset for multi-HP in the wild with such characteristics. In this article, we propose the dataset CCIHP based on the multi-HP dataset CIHP, with 20 new labels covering these 3 kinds of characteristics. In addition, we propose HPTR, a new bottom-up multi-task method based on transformers as a fast and scalable baseline. It is the fastest method of multi-HP state of the art while having precision comparable to the most precise bottom-up method. We hope this will encourage research for fast and accurate methods of precise human descriptions.
Team-Optimal MMSE Combining for Cell-Free Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Cell-free (CF) massive multiple-input multiple-output (MIMO) systems are expected to implement advanced cooperative communication techniques to let geographically distributed access points jointly serve user equipments. Building on the \emph{Team Theory}, we design the uplink team minimum mean-squared error (TMMSE) combining under limited data and flexible channel state information (CSI) sharing. Taking into account the effect of both channel estimation errors and pilot contamination, a minimum MSE problem is formulated to derive unidirectional TMMSE, centralized TMMSE and statistical TMMSE combining functions, where CF massive MIMO systems operate in unidirectional CSI, centralized CSI and statistical CSI sharing schemes, respectively. We then derive the uplink spectral efficiency (SE) of the considered system. The results show that, compared to centralized TMMSE, the unidirectional TMMSE only needs nearly half the cost of CSI sharing burden with neglectable SE performance loss. Moreover, the performance gap between unidirectional and centralized TMMSE combining schemes can be effectively reduced by increasing the number of APs and antennas per AP.
On the Complexity of Attacking Elliptic Curve Based Authentication Chips
Venue: arXiv cs.CR 2022
Year: 2022

In this paper we discuss the difficulties of mounting successful attack against crypto implementations when essential information is missing. We start with a detailed description of our attack against our own design, to highlight which information is needed to increase the success of an attack, i.e. we use it as a blueprint to the following attack against commercially available crypto chips. We would like to stress that our attack against our own design is very similar to what happens during certification e.g. according to Common Criteria Standard as in those cases the manufacturer needs to provide detailed information. When attacking the commercial designs without signing NDAs, we needed to intensively search the Internet for information about the designs. We cannot to reveal the private keys used by the attacked commercial authentication chips 100% correctly. Moreover, the missing knowledge of the used keys does not allow us to evaluate the success of our attack. We were able to reveal information on the processing sequence during the authentication process even as detailed as identifying the clock cycles in which the individual key bits are processed. To summarize the effort of such an attack is significantly higher than the one of attacking a well-known implementation.
DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations
Venue: arXiv cs.LG 2022
Year: 2022

AI-aided drug discovery (AIDD) is gaining increasing popularity due to its promise of making the search for new pharmaceuticals quicker, cheaper and more efficient. In spite of its extensive use in many fields, such as ADMET prediction, virtual screening, protein folding and generative chemistry, little has been explored in terms of the out-of-distribution (OOD) learning problem with \emph{noise}, which is inevitable in real world AIDD applications. In this work, we present DrugOOD, a systematic OOD dataset curator and benchmark for AI-aided drug discovery, which comes with an open-source Python package that fully automates the data curation and OOD benchmarking processes. We focus on one of the most crucial problems in AIDD: drug target binding affinity prediction, which involves both macromolecule (protein target) and small-molecule (drug compound). In contrast to only providing fixed datasets, DrugOOD offers automated dataset curator with user-friendly customization scripts, rich domain annotations aligned with biochemistry knowledge, realistic noise annotations and rigorous benchmarking of state-of-the-art OOD algorithms. Since the molecular data is often modeled as irregular graphs using graph neural network (GNN) backbones, DrugOOD also serves as a valuable testbed for \emph{graph OOD learning} problems. Extensive empirical studies have shown a significant performance gap between in-distribution and out-of-distribution experiments, which highlights the need to develop better schemes that can allow for OOD generalization under noise for AIDD.
Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN
Venue: arXiv cs.CV 2022
Year: 2022

The semantically disentangled latent subspace in GAN provides rich interpretable controls in image generation. This paper includes two contributions on semantic latent subspace analysis in the scenario of face generation using StyleGAN2. First, we propose a novel approach to disentangle latent subspace semantics by exploiting existing face analysis models, e.g., face parsers and face landmark detectors. These models provide the flexibility to construct various criterions with very concrete and interpretable semantic meanings (e.g., change face shape or change skin color) to restrict latent subspace disentanglement. Rich latent space controls unknown previously can be discovered using the constructed criterions. Second, we propose a new perspective to explain the behavior of a CNN classifier by generating counterfactuals in the interpretable latent subspaces we discovered. This explanation helps reveal whether the classifier learns semantics as intended. Experiments on various disentanglement criterions demonstrate the effectiveness of our approach. We believe this approach contributes to both areas of image manipulation and counterfactual explainability of CNNs. The code is available at \url{https://github.com/prclibo/ice}.
Multidimensional Manhattan Preferences
Venue: arXiv cs.MA 2022
Year: 2022

A preference profile with $m$ alternatives and $n$ voters is $d$-Manhattan (resp. $d$-Euclidean) if both the alternatives and the voters can be placed into the $d$-dimensional space such that between each pair of alternatives, every voter prefers the one which has a shorter Manhattan (resp. Euclidean) distance to the voter. Following Bogomolnaia and Laslier [Journal of Mathematical Economics, 2007] and Chen and Grottke [Social Choice and Welfare, 2021] who look at $d$-Euclidean preference profiles, we study which preference profiles are $d$-Manhattan depending on the values $m$ and $n$. First, we show that each preference profile with $m$ alternatives and $n$ voters is $d$-Manhattan whenever $d$ $\geq$ min($n$, $m$-$1$). Second, for $d = 2$, we show that the smallest non $d$-Manhattan preference profile has either three voters and six alternatives, or four voters and five alternatives, or five voters and four alternatives. This is more complex than the case with $d$-Euclidean preferences (see [Bogomolnaia and Laslier, 2007] and [Bulteau and Chen, 2020].
Feature transforms for image data augmentation
Venue: arXiv cs.CV 2022
Year: 2022

A problem with Convolutional Neural Networks (CNNs) is that they require large datasets to obtain adequate robustness; on small datasets, they are prone to overfitting. Many methods have been proposed to overcome this shortcoming with CNNs. In cases where additional samples cannot easily be collected, a common approach is to generate more data points from existing data using an augmentation technique. In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we build ensembles on the data level by adding images generated by combining fourteen augmentation approaches, three of which are proposed here for the first time. These novel methods are based on the Fourier Transform (FT), the Radon Transform (RT) and the Discrete Cosine Transform (DCT). Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method. These networks and several fusions are evaluated and compared across eleven benchmarks. Results show that building ensembles on the data level by combining different data augmentation methods produce classifiers that not only compete competitively against the state-of-the-art but often surpass the best approaches reported in the literature.
Exploration of Hyperdimensional Computing Strategies for Enhanced Learning on Epileptic Seizure Detection
Venue: arXiv cs.NE 2022
Year: 2022

Wearable and unobtrusive monitoring and prediction of epileptic seizures has the potential to significantly increase the life quality of patients, but is still an unreached goal due to challenges of real-time detection and wearable devices design. Hyperdimensional (HD) computing has evolved in recent years as a new promising machine learning approach, especially when talking about wearable applications. But in the case of epilepsy detection, standard HD computing is not performing at the level of other state-of-the-art algorithms. This could be due to the inherent complexity of the seizures and their signatures in different biosignals, such as the electroencephalogram (EEG), the highly personalized nature, and the disbalance of seizure and non-seizure instances. In the literature, different strategies for improved learning of HD computing have been proposed, such as iterative (multi-pass) learning, multi-centroid learning and learning with sample weight ("OnlineHD"). Yet, most of them have not been tested on the challenging task of epileptic seizure detection, and it stays unclear whether they can increase the HD computing performance to the level of the current state-of-the-art algorithms, such as random forests. Thus, in this paper, we implement different learning strategies and assess their performance on an individual basis, or in combination, regarding detection performance and memory and computational requirements. Results show that the best-performing algorithm, which is a combination of multi-centroid and multi-pass, can indeed reach the performance of the random forest model on a highly unbalanced dataset imitating a real-life epileptic seizure detection application.
Aerial Energy Provisioning for Massive Energy-Constrained IoT by UAVs
Venue: arXiv cs.NI 2022
Year: 2022

Autonomy of devices is a major challenge in many Internet of Things (IoT) applications, in particular when the nodes are deployed remotely or difficult to assess places. In this paper we present an approach to provide energy to these devices by Unmanned Aerial Vehicles (UAVs). Therefore, the two major challenges, finding and charging the node are presented. We propose a model to give the energy constrained node an unlimited autonomy by taken the Wireless Power Transfer (WPT) link and battery capacity into account. Selecting the most suitable battery technology allows a reduction in battery capacity and waste. Moreover, an upgrade of existing IoT nodes is feasible with a limited impact on the design and form factor.
Using Computational Grounded Theory to Understand Tutors' Experiences in the Gig Economy
Venue: arXiv cs.HC 2022
Year: 2022

The introduction of online marketplace platforms has led to the advent of new forms of flexible, on-demand (or 'gig') work. Yet, most prior research concerning the experience of gig workers examines delivery or crowdsourcing platforms, while the experience of the large numbers of workers who undertake educational labour in the form of tutoring gigs remains understudied. To address this, we use a computational grounded theory approach to analyse tutors' discussions on Reddit. This approach consists of three phases including data exploration, modelling and human-centred interpretation. We use both validation and human evaluation to increase the trustworthiness and reliability of the computational methods. This paper is a work in progress and reports on the first of the three phases of this approach.
Spectral-PQ: A Novel Spectral Sensitivity-Orientated Perceptual Compression Technique for RGB 4:4:4 Video Data
Venue: arXiv cs.CV 2022
Year: 2022

There exists an intrinsic relationship between the spectral sensitivity of the Human Visual System (HVS) and colour perception; these intertwined phenomena are often overlooked in perceptual compression research. In general, most previously proposed visually lossless compression techniques exploit luminance (luma) masking including luma spatiotemporal masking, luma contrast masking and luma texture/edge masking. The perceptual relevance of color in a picture is often overlooked, which constitutes a gap in the literature. With regard to the spectral sensitivity phenomenon of the HVS, the color channels of raw RGB 4:4:4 data contain significant color-based psychovisual redundancies. These perceptual redundancies can be quantized via color channel-level perceptual quantization. In this paper, we propose a novel spatiotemporal visually lossless coding method named Spectral Perceptual Quantization (Spectral-PQ). With application for RGB 4:4:4 video data, Spectral-PQ exploits HVS spectral sensitivity-related color masking in addition to spatial masking and temporal masking; the proposed method operates at the Coding Block (CB) level and the Prediction Unit (PU) level in the HEVC standard. Spectral-PQ perceptually adjusts the Quantization Step Size (QStep) at the CB level if high variance spatial data in G, B and R CBs is detected and also if high motion vector magnitudes in PUs are detected. Compared with anchor 1 (HEVC HM 16.17 RExt), Spectral-PQ considerably reduces bitrates with a maximum reduction of approximately 81%. The Mean Opinion Score (MOS) in the subjective evaluations show that Spectral-PQ successfully achieves perceptually lossless quality.
GlobalWalk: Learning Global-aware Node Embeddings via Biased Sampling
Venue: arXiv cs.LG 2022
Year: 2022

Popular node embedding methods such as DeepWalk follow the paradigm of performing random walks on the graph, and then requiring each node to be proximate to those appearing along with it. Though proved to be successful in various tasks, this paradigm reduces a graph with topology to a set of sequential sentences, thus omitting global information. To produce global-aware node embeddings, we propose GlobalWalk, a biased random walk strategy that favors nodes with similar semantics. Empirical evidence suggests GlobalWalk can generally enhance global awareness of the generated embeddings.
What is the cost of adding a constraint in linear least squares?
Venue: arXiv cs.CV 2022
Year: 2022

Although the theory of constrained least squares (CLS) estimation is well known, it is usually applied with the view that the constraints to be imposed are unavoidable. However, there are cases in which constraints are optional. For example, in camera color calibration, one of several possible color processing systems is obtained if a constraint on the row sums of a desired color correction matrix is imposed; in this example, it is not clear a priori whether imposing the constraint leads to better system performance. In this paper, we derive an exact expression connecting the constraint to the increase in fitting error obtained from imposing it. As another contribution, we show how to determine projection matrices that separate the measured data into two components: the first component drives up the fitting error due to imposing a constraint, and the second component is unaffected by the constraint. We demonstrate the use of these results in the color calibration problem.
ReLSO: A Transformer-based Model for Latent Space Optimization and Generation of Proteins
Venue: arXiv cs.LG 2022
Year: 2022

The development of powerful natural language models have increased the ability to learn meaningful representations of protein sequences. In addition, advances in high-throughput mutagenesis, directed evolution, and next-generation sequencing have allowed for the accumulation of large amounts of labeled fitness data. Leveraging these two trends, we introduce Regularized Latent Space Optimization (ReLSO), a deep transformer-based autoencoder which features a highly structured latent space that is trained to jointly generate sequences as well as predict fitness. Through regularized prediction heads, ReLSO introduces a powerful protein sequence encoder and novel approach for efficient fitness landscape traversal. Using ReLSO, we explicitly model the sequence-function landscape of large labeled datasets and generate new molecules by optimizing within the latent space using gradient-based methods. We evaluate this approach on several publicly-available protein datasets, including variant sets of anti-ranibizumab and GFP. We observe a greater sequence optimization efficiency (increase in fitness per optimization step) by ReLSO compared to other approaches, where ReLSO more robustly generates high-fitness sequences. Furthermore, the attention-based relationships learned by the jointly-trained ReLSO models provides a potential avenue towards sequence-level fitness attribution information.
Generating Clarifying Questions for Query Refinement in Source Code Search
Venue: arXiv cs.SE 2022
Year: 2022

In source code search, a common information-seeking strategy involves providing a short initial query with a broad meaning, and then iteratively refining the query using terms gleaned from the results of subsequent searches. This strategy requires programmers to spend time reading search results that are irrelevant to their development needs. In contrast, when programmers seek information from other humans, they typically refine queries by asking and answering clarifying questions. Clarifying questions have been shown to benefit general-purpose search engines, but have not been examined in the context of code search. We present a method for generating natural-sounding clarifying questions using information extracted from function names and comments. Our method outperformed a keyword-based method for single-turn refinement in synthetic studies, and was associated with shorter search duration in human studies.
Learning Task-Parameterized Skills from Few Demonstrations
Venue: arXiv cs.RO 2022
Year: 2022

Moving away from repetitive tasks, robots nowadays demand versatile skills that adapt to different situations. Task-parameterized learning improves the generalization of motion policies by encoding relevant contextual information in the task parameters, hence enabling flexible task executions. However, training such a policy often requires collecting multiple demonstrations in different situations. To comprehensively create different situations is non-trivial thus renders the method less applicable to real-world problems. Therefore, training with fewer demonstrations/situations is desirable. This paper presents a novel concept to augment the original training dataset with synthetic data for policy improvements, thus allows learning task-parameterized skills with few demonstrations.
Automatic Recognition and Digital Documentation of Cultural Heritage Hemispherical Domes using Images
Venue: arXiv cs.CV 2022
Year: 2022

Advancements in optical metrology has enabled documentation of dense 3D point clouds of cultural heritage sites. For large scale and continuous digital documentation, processing of dense 3D point clouds becomes computationally cumbersome, and often requires additional hardware for data management, increasing the time cost, and complexity of projects. To this end, this manuscript presents an original approach to generate fast and reliable semantic digital models of heritage hemispherical domes using only two images. New closed formulations were derived to establish the relationships between spheres and their projected ellipses onto images, which fostered the development of a new automatic framework for as-built generation of spheres. The effectiveness of the proposed method was evaluated under both laboratory and real-world datasets. The results revealed that the proposed method achieved as-built modeling accuracy of around 6mm, while improving the computation time by a factor of 7, when compared to established point cloud processing methods.
Learning Resource Allocation Policies from Observational Data with an Application to Homeless Services Delivery
Venue: arXiv cs.LG 2022
Year: 2022

We study the problem of learning, from observational data, fair and interpretable policies that effectively match heterogeneous individuals to scarce resources of different types. We model this problem as a multi-class multi-server queuing system where both individuals and resources arrive stochastically over time. Each individual, upon arrival, is assigned to a queue where they wait to be matched to a resource. The resources are assigned in a first come first served (FCFS) fashion according to an eligibility structure that encodes the resource types that serve each queue. We propose a methodology based on techniques in modern causal inference to construct the individual queues as well as learn the matching outcomes and provide a mixed-integer optimization (MIO) formulation to optimize the eligibility structure. The MIO problem maximizes policy outcome subject to wait time and fairness constraints. It is very flexible, allowing for additional linear domain constraints. We conduct extensive analyses using synthetic and real-world data. In particular, we evaluate our framework using data from the U.S. Homeless Management Information System (HMIS). We obtain wait times as low as an FCFS policy while improving the rate of exit from homelessness for underserved or vulnerable groups (7% higher for the Black individuals and 15% higher for those below 17 years old) and overall.