OpenRaiser/Intern-Atlas huggingface dataset:
Diverse Instance Discovery: Vision-Transformer for Instance-Aware Multi-Label Image Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Previous works on multi-label image recognition (MLIR) usually use CNNs as a starting point for research. In this paper, we take pure Vision Transformer (ViT) as the research base and make full use of the advantages of Transformer with long-range dependency modeling to circumvent the disadvantages of CNNs limited to local receptive field. However, for multi-label images containing multiple objects from different categories, scales, and spatial relations, it is not optimal to use global information alone. Our goal is to leverage ViT's patch tokens and self-attention mechanism to mine rich instances in multi-label images, named diverse instance discovery (DiD). To this end, we propose a semantic category-aware module and a spatial relationship-aware module, respectively, and then combine the two by a re-constraint strategy to obtain instance-aware attention maps. Finally, we propose a weakly supervised object localization-based approach to extract multi-scale local features, to form a multi-view pipeline. Our method requires only weakly supervised information at the label level, no additional knowledge injection or other strongly supervised information is required. Experiments on three benchmark datasets show that our method significantly outperforms previous works and achieves state-of-the-art results under fair experimental comparisons.
Embracing AWKWARD! Real-time Adjustment of Reactive Plans Using Social Norms
Venue: arXiv cs.MA 2022
Year: 2022

This paper presents the AWKWARD architecture for the development of hybrid agents in Multi-Agent Systems. AWKWARD agents can have their plans re-configured in real time to align with social role requirements under changing environmental and social circumstances. The proposed hybrid architecture makes use of Behaviour Oriented Design (BOD) to develop agents with reactive planning and of the well-established OperA framework to provide organisational, social, and interaction definitions in order to validate and adjust agents' behaviours. Together, OperA and BOD can achieve real-time adjustment of agent plans for evolving social roles, while providing the additional benefit of transparency into the interactions that drive this behavioural change in individual agents. We present this architecture to motivate the bridging between traditional symbolic- and behaviour-based AI communities, where such combined solutions can help MAS researchers in their pursuit of building stronger, more robust intelligent agent teams. We use DOTA2, a game where success is heavily dependent on social interactions, as a medium to demonstrate a sample implementation of our proposed hybrid architecture.
A New Polar Code Design Based on Reciprocal Channel Approximation
Venue: arXiv cs.IT 2022
Year: 2022

This paper revisits polar code design for a binary-input additive white Gaussian noise (BI-AWGN) channel when successive cancellation (SC) decoding is applied at the receiver. We focus on the reciprocal channel approximation (RCA), which is often adopted in the design of low-density parity-check (LDPC) codes. In order to apply RCA to polar code design for various codeword lengths, we derive rigorous closed-form approximations that are valid over a wide range of SNR over an AWGN channel, for both the mutual information of BPSK signaling and the corresponding reciprocal channel mapping. As a result, the computational complexity required for evaluating channel polarization is thus equivalent to that based on the popular Gaussian approximation (GA) approach. Simulation results show that the proposed polar code design based on RCA outperforms those based on GA as well as the so-called improved GA (IGA) approach, especially as the codeword length is increased. Furthermore, the RCA-based design yields a better block error rate (BLER) estimate compared to GA-based approaches.
Reference Vector Adaptation and Mating Selection Strategy via Adaptive Resonance Theory-based Clustering for Many-objective Optimization
Venue: arXiv cs.NE 2022
Year: 2022

Decomposition-based multiobjective evolutionary algorithms (MOEAs) with clustering-based reference vector adaptation show good optimization performance for many-objective optimization problems (MaOPs). Especially, algorithms that employ a clustering algorithm with a topological structure (i.e., a network composed of nodes and edges) show superior optimization performance to other MOEAs for MaOPs with irregular Pareto optimal fronts (PFs). These algorithms, however, do not effectively utilize information of the topological structure in the search process. Moreover, the clustering algorithms typically used in conventional studies have limited clustering performance, inhibiting the ability to extract useful information for the search process. This paper proposes an adaptive reference vector-guided evolutionary algorithm using an adaptive resonance theory-based clustering with a topological structure. The proposed algorithm utilizes the information of the topological structure not only for reference vector adaptation but also for mating selection. The proposed algorithm is compared with 8 state-of-the-art MOEAs on 78 test problems. Experimental results reveal the outstanding optimization performance of the proposed algorithm over the others on MaOPs with various properties.
MCBeth: A Measurement Based Quantum Programming Language
Venue: arXiv cs.PL 2022
Year: 2022

Gate-based quantum programming languages are ubiquitous but measurement-based languages currently exist only on paper. This work introduces MCBeth, a quantum programming language which allows programmers to directly represent, program, and simulate measurement-based and cluster state computation by building upon the measurement calculus. While MCBeth programs are meant to be executed directly on hardware, to take advantage of current machines we also provide a compiler to gate-based instructions. We argue that there are clear advantages to measurement-based quantum computation compared to gate-based when it comes to implementing common quantum algorithms and distributed quantum computation.
DACSR: Decoupled-Aggregated End-to-End Calibrated Sequential Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Sequential recommendations have made great strides in accurately predicting the future behavior of users. However, seeking accuracy alone may bring side effects such as unfair and overspecialized recommendation results. In this work, we focus on the calibrated recommendations for sequential recommendation, which is connected to both fairness and diversity. On the one hand, it aims to provide fairer recommendations whose preference distributions are consistent with users' historical behaviors. On the other hand, it can improve the diversity of recommendations to a certain degree. But existing methods for calibration have mainly relied on the post-processing on the candidate lists, which require more computation time in generating recommendations. In addition, they fail to establish the relationship between accuracy and calibration, leading to the limitation of accuracy. To handle these problems, we propose an end-to-end framework to provide both accurate and calibrated recommendations for sequential recommendation. We design an objective function to calibrate the interests between recommendation lists and historical behaviors. We also provide distribution modification approaches to improve the diversity and mitigate the effect of imbalanced interests. In addition, we design a decoupled-aggregated model to improve the recommendation. The framework assigns two objectives to two individual sequence encoders, and aggregates the outputs by extracting useful information. Experiments on benchmark datasets validate the effectiveness of our proposed model.
Federated Learning Enables Big Data for Rare Cancer Boundary Detection
Venue: arXiv cs.LG 2022
Year: 2022

Although machine learning (ML) has shown promise in numerous domains, there are concerns about generalizability to out-of-sample data. This is currently addressed by centrally sharing ample, and importantly diverse, data from multiple sites. However, such centralization is challenging to scale (or even not feasible) due to various limitations. Federated ML (FL) provides an alternative to train accurate and generalizable ML models, by only sharing numerical model updates. Here we present findings from the largest FL study to-date, involving data from 71 healthcare institutions across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, utilizing the largest dataset of such patients ever used in the literature (25,256 MRI scans from 6,314 patients). We demonstrate a 33% improvement over a publicly trained model to delineate the surgically targetable tumor, and 23% improvement over the tumor's entire extent. We anticipate our study to: 1) enable more studies in healthcare informed by large and diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further quantitative analyses for glioblastoma via performance optimization of our consensus model for eventual public release, and 3) demonstrate the effectiveness of FL at such scale and task complexity as a paradigm shift for multi-site collaborations, alleviating the need for data sharing.
An active approach towards monitoring and enhancing drivers' capabilities -- the ADAM cogtec solution
Venue: arXiv cs.HC 2022
Year: 2022

Driver's cognitive ability at a given moment is the most elusive variable in assessing driver's safety. In contrast to other physical conditions, such as short-sight, or manual disability cognitive ability is transient. Safety regulations attempt to reduce risk related to driver's cognitive ability by removing risk factors such as alcohol or drug consumption, forbidding secondary tasks such as texting, and urging drivers to take breaks when feeling tired. However, one cannot regulate all factors that affect driver's cognition, furthermore, the driver's momentary cognitive ability in most cases is covert even to driver. Here, we introduce an active approach aiming at monitoring a specific cognitive process that is affected by all these forementioned causes and directly affects the driver's performance in the driving task. We lean on the scientific approach that was framed by Karl Friston (Friston, 2010). We developed a closed loop-method in which driver's ocular responses to visual probing were recorded. Machine-learning-algorithms were trained on ocular responses of vigilant condition and were able to detect decrease in capability due fatigue and substance abuse. Our results show that we manage to correctly classify subjects with impaired and unimpaired cognitive process regardless of the cause of impairment (77% accuracy, 5% false alarms).
ParticLS: Object-oriented software for discrete element methods and peridynamics
Venue: arXiv cs.MS 2022
Year: 2022

ParticLS (\emph{Partic}le \emph{L}evel \emph{S}ets) is a software library that implements the discrete element method (DEM) and meshfree methods. ParticLS tracks the interaction between individual particles whose geometries are defined by level sets capable of capturing complex shapes. These particles either represent rigid bodies or material points within a continuum. Particle-particle interactions using various contact laws numerically approximate solutions to energy and mass conservation equations, simulating rigid body dynamics or deformation/fracture. By leveraging multiple contact laws, ParticLS can simulate interacting bodies that deform, fracture, and are composed of many particles. In the continuum setting, we numerically solve the peridynamic equations -- integro-differential equations capable of modeling objects with discontinuous displacement fields and complex fracture dynamics. We show that the discretized peridynamic equations can be solved using the same software infrastructure that implements the DEM. Therefore, we design a unique software library where users can easily add particles with arbitrary geometries and new contact laws that model either rigid-body interaction or peridynamic constitutive relationships. We demonstrate ParticLS' versatility on test problems meant to showcase features applicable to a broad selection of fields such as tectonics, granular media, multiscale simulations, glacier calving, and sea ice.
Blockchain and Cryptocurrency in Human Computer Interaction: A Systematic Literature Review and Research Agenda
Venue: arXiv cs.HC 2022
Year: 2022

We present a systematic literature review of cryptocurrency and blockchain research in Human-Computer Interaction (HCI) published between 2014 and 2021. We aim to provide an overview of the field, consolidate existing knowledge, and chart paths for future research. Our analysis of 99 articles identifies six major themes: (1) the role of trust, (2) understanding motivation, risk, and perception of cryptocurrencies, (3) cryptocurrency wallets, (4) engaging users with blockchain, (5) using blockchain for application-specific use cases, and (6) support tools for blockchain. We discuss the focus of the existing research body and juxtapose it to the changing landscape of emerging blockchain technologies to highlight future research avenues for HCI and interaction design. With this review, we identify key aspects where interaction design is critical for the adoption of blockchain systems. Doing so, we provide a starting point for new scholars and designers and help them position future contributions.
Roofline Model for UAVs: A Bottleneck Analysis Tool for Onboard Compute Characterization of Autonomous Unmanned Aerial Vehicles
Venue: arXiv cs.RO 2022
Year: 2022

We introduce an early-phase bottleneck analysis and characterization model called the F-1 for designing computing systems that target autonomous Unmanned Aerial Vehicles (UAVs). The model provides insights by exploiting the fundamental relationships between various components in the autonomous UAV, such as sensor, compute, and body dynamics. To guarantee safe operation while maximizing the performance (e.g., velocity) of the UAV, the compute, sensor, and other mechanical properties must be carefully selected or designed. The F-1 model provides visual insights that can aid a system architect in understanding the optimal compute design or selection for autonomous UAVs. The model is experimentally validated using real UAVs, and the error is between 5.1\% to 9.5\% compared to real-world flight tests. An interactive web-based tool for the F-1 model called Skyline is available for free of cost use at: ~\url{https://bit.ly/skyline-tool}
Nested Papercrafts for Anatomical and Biological Edutainment
Venue: arXiv cs.GR 2022
Year: 2022

In this paper, we present a new workflow for the computer-aided generation of physicalizations, addressing nested configurations in anatomical and biological structures. Physicalizations are an important component of anatomical and biological education and edutainment. However, existing approaches have mainly revolved around creating data sculptures through digital fabrication. Only a few recent works proposed computer-aided pipelines for generating sculptures, such as papercrafts, with affordable and readily available materials. Papercraft generation remains a challenging topic by itself. Yet, anatomical and biological applications pose additional challenges, such as reconstruction complexity and insufficiency to account for multiple, nested structures--often present in anatomical and biological structures. Our workflow comprises the following steps: (i) define the nested configuration of the model and detect its levels, (ii) calculate the viewpoint that provides optimal, unobstructed views on inner levels, (iii) perform cuts on the outer levels to reveal the inner ones based on the viewpoint selection, (iv) estimate the stability of the cut papercraft to ensure a reliable outcome, (v) generate textures at each level, as a smart visibility mechanism that provides additional information on the inner structures, and (vi) unfold each textured mesh guaranteeing reconstruction. Our novel approach exploits the interactivity of nested papercraft models for edutainment purposes.
Listing, Verifying and Counting Lowest Common Ancestors in DAGs: Algorithms and Fine-Grained Lower Bounds
Venue: arXiv cs.DS 2022
Year: 2022

The AP-LCA problem asks, given an $n$-node directed acyclic graph (DAG), to compute for every pair of vertices $u$ and $v$ in the DAG a lowest common ancestor (LCA) of $u$ and $v$ if one exists. In this paper we study several interesting variants of AP-LCA, providing both algorithms and fine-grained lower bounds for them. The lower bounds we obtain are the first conditional lower bounds for LCA problems higher than $n^{ω-o(1)}$, where $ω$ is the matrix multiplication exponent. Some of our results include: - In any DAG, we can detect all vertex pairs that have at most two LCAs and list all of their LCAs in $O(n^ω)$ time. This algorithm extends a result of [Kowaluk and Lingas ESA'07] which showed an $\tilde{O}(n^ω)$ time algorithm that detects all pairs with a unique LCA in a DAG and outputs their corresponding LCAs. - Listing $7$ LCAs per vertex pair in DAGs requires $n^{3-o(1)}$ time under the popular assumption that 3-uniform 5-hyperclique detection requires $n^{5-o(1)}$ time. This is surprising since essentially cubic time is sufficient to list all LCAs (if $ω=2$). - Counting the number of LCAs for every vertex pair in a DAG requires $n^{3-o(1)}$ time under the Strong Exponential Time Hypothesis, and $n^{ω(1,2,1)-o(1)}$ time under the $4$-Clique hypothesis. This shows that the algorithm of [Echkardt, Mühling and Nowak ESA'07] for listing all LCAs for every pair of vertices is likely optimal. - Given a DAG and a vertex $w_{u,v}$ for every vertex pair $u,v$, verifying whether all $w_{u,v}$ are valid LCAs requires $n^{2.5-o(1)}$ time assuming 3-uniform 4-hyperclique requires $n^{4 - o(1)}$ time. This defies the common intuition that verification is easier than computation since returning some LCA per vertex pair can be solved in $O(n^{2.447})$ time [Grandoni et al. SODA'21].
Unified Pretraining Framework for Document Understanding
Venue: arXiv cs.CL 2022
Year: 2022

Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabeled document datasets have opened up promising directions towards reducing annotation efforts by training models with self-supervised objectives. However, most of the existing document pretraining methods are still language-dominated. We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. Each input element is composed of words and visual features from a semantic region of the input document image. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses, encouraging the representation to model sentences, learn similarities, and align modalities. Extensive empirical analysis demonstrates that the pretraining procedure learns better joint representations and leads to improvements in downstream tasks.
Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN
Venue: arXiv cs.CV 2022
Year: 2022

Existing approaches for restoring weather-degraded images follow a fully-supervised paradigm and they require paired data for training. However, collecting paired data for weather degradations is extremely challenging, and existing methods end up training on synthetic data. To overcome this issue, we describe an approach for supervising deep networks that are based on CycleGAN, thereby enabling the use of unlabeled real-world data for training. Specifically, we introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. These new losses are obtained by jointly modeling the latent space embeddings of predicted clean images and original clean images through Deep Gaussian Processes. This enables the CycleGAN architecture to transfer the knowledge from one domain (weather-degraded) to another (clean) more effectively. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing and it outperforms other unsupervised techniques (that leverage weather-based characteristics) by a considerable margin.
Training and challenging models for text-guided fashion image retrieval
Venue: arXiv cs.CV 2022
Year: 2022

Retrieving relevant images from a catalog based on a query image together with a modifying caption is a challenging multimodal task that can particularly benefit domains like apparel shopping, where fine details and subtle variations may be best expressed through natural language. We introduce a new evaluation dataset, Challenging Fashion Queries (CFQ), as well as a modeling approach that achieves state-of-the-art performance on the existing Fashion IQ (FIQ) dataset. CFQ complements existing benchmarks by including relative captions with positive and negative labels of caption accuracy and conditional image similarity, where others provided only positive labels with a combined meaning. We demonstrate the importance of multimodal pretraining for the task and show that domain-specific weak supervision based on attribute labels can augment generic large-scale pretraining. While previous modality fusion mechanisms lose the benefits of multimodal pretraining, we introduce a residual attention fusion mechanism that improves performance. We release CFQ and our code to the research community.
Federated Geometric Monte Carlo Clustering to Counter Non-IID Datasets
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning allows clients to collaboratively train models on datasets that are acquired in different locations and that cannot be exchanged because of their size or regulations. Such collected data is increasingly non-independent and non-identically distributed (non-IID), negatively affecting training accuracy. Previous works tried to mitigate the effects of non-IID datasets on training accuracy, focusing mainly on non-IID labels, however practical datasets often also contain non-IID features. To address both non-IID labels and features, we propose FedGMCC, a novel framework where a central server aggregates client models that it can cluster together. FedGMCC clustering relies on a Monte Carlo procedure that samples the output space of client models, infers their position in the weight space on a loss manifold and computes their geometric connection via an affine curve parametrization. FedGMCC aggregates connected models along their path connectivity to produce a richer global model, incorporating knowledge of all connected client models. FedGMCC outperforms FedAvg and FedProx in terms of convergence rates on the EMNIST62 and a genomic sequence classification datasets (by up to +63%). FedGMCC yields an improved accuracy (+4%) on the genomic dataset with respect to CFL, in high non-IID feature space settings and label incongruency.