OpenRaiser/Intern-Atlas huggingface dataset:
Synthetic Latent Fingerprint Generator
Venue: arXiv cs.CV 2022
Year: 2022

Given a full fingerprint image (rolled or slap), we present CycleGAN models to generate multiple latent impressions of the same identity as the full print. Our models can control the degree of distortion, noise, blurriness and occlusion in the generated latent print images to obtain Good, Bad and Ugly latent image categories as introduced in the NIST SD27 latent database. The contributions of our work are twofold: (i) demonstrate the similarity of synthetically generated latent fingerprint images to crime scene latents in NIST SD27 and MSP databases as evaluated by the NIST NFIQ 2 quality measure and ROC curves obtained by a SOTA fingerprint matcher, and (ii) use of synthetic latents to augment small-size latent training databases in the public domain to improve the performance of DeepPrint, a SOTA fingerprint matcher designed for rolled to rolled fingerprint matching on three latent databases (NIST SD27, NIST SD302, and IIITD-SLF). As an example, with synthetic latent data augmentation, the Rank-1 retrieval performance of DeepPrint is improved from 15.50% to 29.07% on challenging NIST SD27 latent database. Our approach for generating synthetic latent fingerprints can be used to improve the recognition performance of any latent matcher and its individual components (e.g., enhancement, segmentation and feature extraction).
ProspectNet: Weighted Conditional Attention for Future Interaction Modeling in Behavior Prediction
Venue: arXiv cs.AI 2022
Year: 2022

Behavior prediction plays an important role in integrated autonomous driving software solutions. In behavior prediction research, interactive behavior prediction is a less-explored area, compared to single-agent behavior prediction. Predicting the motion of interactive agents requires initiating novel mechanisms to capture the joint behaviors of the interactive pairs. In this work, we formulate the end-to-end joint prediction problem as a sequential learning process of marginal learning and joint learning of vehicle behaviors. We propose ProspectNet, a joint learning block that adopts the weighted attention score to model the mutual influence between interactive agent pairs. The joint learning block first weighs the multi-modal predicted candidate trajectories, then updates the ego-agent's embedding via cross attention. Furthermore, we broadcast the individual future predictions for each interactive agent into a pair-wise scoring module to select the top $K$ prediction pairs. We show that ProspectNet outperforms the Cartesian product of two marginal predictions, and achieves comparable performance on the Waymo Interactive Motion Prediction benchmarks.
AMR-MUL: An Approximate Maximally Redundant Signed Digit Multiplier
Venue: arXiv cs.AR 2022
Year: 2022

In this paper, we present an energy-efficient, yet high-speed approximate maximally redundant signed digit (MRSD) multiplier (called AMR-MUL) based on a parallel structure. For the reduction stage, we suggest several approximate Full-Adder (FA) reduction cells with average positive and negative errors obtained by simplifying the structure of an exact FA cell. The optimum selection of these cells for each partial product reduction stage provides the lowest possible error, turning this task into a design space exploration problem. We also provide a branch-and-bound design space exploration algorithm to find the optimal assignment of reduction cells based on a predefined constraint (i.e., the width of the approximate part) by the user. The effectiveness of the proposed (Radix-16) multiplier design is assessed under different digit counts and approximate border column. The results show that the energy consumption of the MRSD multiplier is reduced by 7x at the cost of a 1.6% accuracy loss.
"I was Confused by It; It was Confused by Me:" Exploring the Experiences of People with Visual Impairments around Mobile Service Robots
Venue: arXiv cs.HC 2022
Year: 2022

Mobile service robots have become increasingly ubiquitous. However, these robots can pose potential accessibility issues and safety concerns to people with visual impairments (PVI). We sought to explore the challenges faced by PVI around mainstream mobile service robots and identify their needs. Seventeen PVI were interviewed about their experiences with three emerging robots: vacuum robots, delivery robots, and drones. We comprehensively investigated PVI's robot experiences by considering their different roles around robots -- direct users and bystanders. Our study highlighted participants' challenges and concerns about the accessibility, safety, and privacy issues around mobile service robots. We found that the lack of accessible feedback made it difficult for PVI to precisely control, locate, and track the status of the robots. Moreover, encountering mobile robots as bystanders confused and even scared the participants, presenting safety and privacy barriers. We further distilled design considerations for more accessible and safe robots for PVI.
Video-based Cross-modal Auxiliary Network for Multimodal Sentiment Analysis
Venue: arXiv cs.CV 2022
Year: 2022

Multimodal sentiment analysis has a wide range of applications due to its information complementarity in multimodal interactions. Previous works focus more on investigating efficient joint representations, but they rarely consider the insufficient unimodal features extraction and data redundancy of multimodal fusion. In this paper, a Video-based Cross-modal Auxiliary Network (VCAN) is proposed, which is comprised of an audio features map module and a cross-modal selection module. The first module is designed to substantially increase feature diversity in audio feature extraction, aiming to improve classification accuracy by providing more comprehensive acoustic representations. To empower the model to handle redundant visual features, the second module is addressed to efficiently filter the redundant visual frames during integrating audiovisual data. Moreover, a classifier group consisting of several image classification networks is introduced to predict sentiment polarities and emotion categories. Extensive experimental results on RAVDESS, CMU-MOSI, and CMU-MOSEI benchmarks indicate that VCAN is significantly superior to the state-of-the-art methods for improving the classification accuracy of multimodal sentiment analysis.
MRL: Learning to Mix with Attention and Convolutions
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present a new neural architectural block for the vision domain, named Mixing Regionally and Locally (MRL), developed with the aim of effectively and efficiently mixing the provided input features. We bifurcate the input feature mixing task as mixing at a regional and local scale. To achieve an efficient mix, we exploit the domain-wide receptive field provided by self-attention for regional-scale mixing and convolutional kernels restricted to local scale for local-scale mixing. More specifically, our proposed method mixes regional features associated with local features within a defined region, followed by a local-scale features mix augmented by regional features. Experiments show that this hybridization of self-attention and convolution brings improved capacity, generalization (right inductive bias), and efficiency. Under similar network settings, MRL outperforms or is at par with its counterparts in classification, object detection, and segmentation tasks. We also show that our MRL-based network architecture achieves state-of-the-art performance for H&E histology datasets. We achieved DICE of 0.843, 0.855, and 0.892 for Kumar, CoNSep, and CPM-17 datasets, respectively, while highlighting the versatility offered by the MRL framework by incorporating layers like group convolutions to improve dataset-specific generalization.
HiGNN: Hierarchical Informative Graph Neural Networks for Molecular Property Prediction Equipped with Feature-Wise Attention
Venue: arXiv cs.LG 2022
Year: 2022

Elucidating and accurately predicting the druggability and bioactivities of molecules plays a pivotal role in drug design and discovery and remains an open challenge. Recently, graph neural networks (GNN) have made remarkable advancements in graph-based molecular property prediction. However, current graph-based deep learning methods neglect the hierarchical information of molecules and the relationships between feature channels. In this study, we propose a well-designed hierarchical informative graph neural networks framework (termed HiGNN) for predicting molecular property by utilizing a co-representation learning of molecular graphs and chemically synthesizable BRICS fragments. Furthermore, a plug-and-play feature-wise attention block is first designed in HiGNN architecture to adaptively recalibrate atomic features after the message passing phase. Extensive experiments demonstrate that HiGNN achieves state-of-the-art predictive performance on many challenging drug discovery-associated benchmark datasets. In addition, we devise a molecule-fragment similarity mechanism to comprehensively investigate the interpretability of HiGNN model at the subgraph level, indicating that HiGNN as a powerful deep learning tool can help chemists and pharmacists identify the key components of molecules for designing better molecules with desired properties or functions. The source code is publicly available at https://github.com/idruglab/hignn.
A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters
Venue: arXiv cs.LG 2022
Year: 2022

Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.
Experimental Performance Evaluation of Cell-free Massive MIMO Systems Using COTS RRU with OTA Reciprocity Calibration and Phase Synchronization
Venue: arXiv cs.IT 2022
Year: 2022

Downlink coherent multiuser transmission is an essential technique for cell-free massive multiple-input multiple output (MIMO) systems, and the availability of channel state information (CSI) at the transmitter is a basic requirement. To avoid CSI feedback in a time-division duplex system, the uplink channel parameters should be calibrated to obtain the downlink CSI due to the radio frequency circuit mismatch of the transceiver. In this paper, a design of a reference signal for over-the-air reciprocity calibration is proposed. The frequency domain generated reference signals can make full use of the flexible frame structure of the fifth generation (5G) new radio, which can be completely transparent to commercial off-the-shelf (COTS) remote radio units (RRUs) and commercial user equipments. To further obtain the calibration of multiple RRUs, an interleaved RRU grouping with a genetic algorithm is proposed, and an averaged Argos calibration algorithm is also presented. We develop a cell-free massive MIMO prototype system with COTS RRUs, demonstrate the statistical characteristics of the calibration error and the effectiveness of the calibration algorithm, and evaluate the impact of the calibration delay on the different cooperative transmission schemes.
Towards making the most of NLP-based device mapping optimization for OpenCL kernels
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, we are living in an era of extreme device heterogeneity. Despite the high variety of conventional CPU architectures, accelerator devices, such as GPUs and FPGAs, also appear in the foreground exploding the pool of available solutions to execute applications. However, choosing the appropriate device per application needs is an extremely challenging task due to the abstract relationship between hardware and software. Automatic optimization algorithms that are accurate are required to cope with the complexity and variety of current hardware and software. Optimal execution has always relied on time-consuming trial and error approaches. Machine learning (ML) and Natural Language Processing (NLP) has flourished over the last decade with research focusing on deep architectures. In this context, the use of natural language processing techniques to source code in order to conduct autotuning tasks is an emerging field of study. In this paper, we extend the work of Cummins et al., namely Deeptune, that tackles the problem of optimal device selection (CPU or GPU) for accelerated OpenCL kernels. We identify three major limitations of Deeptune and, based on these, we propose four different DNN models that provide enhanced contextual information of source codes. Experimental results show that our proposed methodology surpasses that of Cummins et al. work, providing up to 4\% improvement in prediction accuracy.
Vector Perturbation Channel Inversion for SWIPT MU-MISO Systems
Venue: arXiv cs.IT 2022
Year: 2022

This letter investigates the employment of vector-perturbation (VP) precoding to convey simultaneously information and energy in multiple-user multiple-input single-output (MU-MISO) downlink channel. We show that the conventional VP in addition to the information capacity benefits that provides to linear channel inversion techniques, it enhances the harvested energy at the receivers due to the extended symbol constellation. To further boost harvesting performance, the proposed modified VP technique (named VP-EH) designs the VP integer offsets in order to maximize the delivered power. The proposed scheme incorporates an integer least square problem to find the closest lattice point to a point which is given by a Rayleigh quotient optimization problem. Finally, a convex combination between conventional VP and VP-EH is proposed to achieve a trade-off between maximizing information or energy. Theoretical and simulations results validate that VP is a promising technique to simultaneously convey information and energy in MU-MISO systems.
MODNet: Multi-offset Point Cloud Denoising Network Customized for Multi-scale Patches
Venue: arXiv cs.CV 2022
Year: 2022

The intricacy of 3D surfaces often results cutting-edge point cloud denoising (PCD) models in surface degradation including remnant noise, wrongly-removed geometric details. Although using multi-scale patches to encode the geometry of a point has become the common wisdom in PCD, we find that simple aggregation of extracted multi-scale features can not adaptively utilize the appropriate scale information according to the geometric information around noisy points. It leads to surface degradation, especially for points close to edges and points on complex curved surfaces. We raise an intriguing question -- if employing multi-scale geometric perception information to guide the network to utilize multi-scale information, can eliminate the severe surface degradation problem? To answer it, we propose a Multi-offset Denoising Network (MODNet) customized for multi-scale patches. First, we extract the low-level feature of three scales patches by patch feature encoders. Second, a multi-scale perception module is designed to embed multi-scale geometric information for each scale feature and regress multi-scale weights to guide a multi-offset denoising displacement. Third, a multi-offset decoder regresses three scale offsets, which are guided by the multi-scale weights to predict the final displacement by weighting them adaptively. Experiments demonstrate that our method achieves new state-of-the-art performance on both synthetic and real-scanned datasets.
A fundamental Game Theoretic model and approximate global Nash Equilibria computation for European Spot Power Markets
Venue: arXiv cs.GT 2022
Year: 2022

Spot electricity markets are considered under a Game-Theoretic framework, where risk averse players submit orders to the market clearing mechanism to maximise their own utility. Consistent with the current practice in Europe, the market clearing mechanism is modelled as a Social Welfare Maximisation problem, with zonal pricing, and we consider inflexible demand, physical constraints of the electricity grid, and capacity-constrained producers. A novel type of non-parametric risk aversion based on a defined worst case scenario is introduced, and this reduces the dimensionality of the strategy variables and ensures boundedness of prices. By leveraging these properties we devise Jacobi and Gauss-Seidel iterative schemes for computation of approximate global Nash Equilibria, which are in contrast to derivative based local equilibria. Our methodology is applied to the real world data of Central Western European (CWE) Spot Market during the 2019-2020 period, and offers a good representation of the historical time series of prices. By also solving for the assumption of truthful bidding, we devise a simple method based on hypothesis testing to infer if and when producers are bidding strategically (instead of truthfully), and we find evidence suggesting that strategic bidding may be fairly pronounced in the CWE region.
A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies
Venue: arXiv cs.CE 2022
Year: 2022

As an emerging technology in the era of Industry 4.0, digital twin is gaining unprecedented attention because of its promise to further optimize process design, quality control, health monitoring, decision and policy making, and more, by comprehensively modeling the physical world as a group of interconnected digital models. In a two-part series of papers, we examine the fundamental role of different modeling techniques, twinning enabling technologies, and uncertainty quantification and optimization methods commonly used in digital twins. This first paper presents a thorough literature review of digital twin trends across many disciplines currently pursuing this area of research. Then, digital twin modeling and twinning enabling technologies are further analyzed by classifying them into two main categories: physical-to-virtual, and virtual-to-physical, based on the direction in which data flows. Finally, this paper provides perspectives on the trajectory of digital twin technology over the next decade, and introduces a few emerging areas of research which will likely be of great use in future digital twin research. In part two of this review, the role of uncertainty quantification and optimization are discussed, a battery digital twin is demonstrated, and more perspectives on the future of digital twin are shared.
Tackling Multimodal Device Distributions in Inverse Photonic Design using Invertible Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Inverse design, the process of matching a device or process parameters to exhibit a desired performance, is applied in many disciplines ranging from material design over chemical processes and to engineering. Machine learning has emerged as a promising approach to overcome current limitations imposed by the dimensionality of the parameter space and multimodal parameter distributions. Most traditional optimization routines assume an invertible one-to-one mapping between the design parameters and the target performance. However, comparable or even identical performance may be realized by different designs, yielding a multimodal distribution of possible solutions to the inverse design problem which confuses the optimization algorithm. Here, we show how a generative modeling approach based on invertible neural networks can provide the full distribution of possible solutions to the inverse design problem and resolve the ambiguity of nanodevice inverse design problems featuring multimodal distributions. We implement a Conditional Invertible Neural Network (cINN) and apply it to a proof-of-principle nanophotonic problem, consisting in tailoring the transmission spectrum of a metallic film milled by subwavelength indentations. We compare our approach with the commonly used conditional Variational Autoencoder (cVAE) framework and show the superior flexibility and accuracy of the proposed cINNs when dealing with multimodal device distributions. Our work shows that invertible neural networks provide a valuable and versatile toolkit for advancing inverse design in nanoscience and nanotechnology.
Impact of Turbulence Modeling on the Simulation of Blood Flow in Aortic Coarctation
Venue: arXiv cs.CE 2022
Year: 2022

Numerical simulations of pulsatile blood flow in an aortic coarctation require the use of turbulence modeling. This paper considers three models from the class of large eddy simulation (LES) models (Smagorinsky, Vreman, $\boldsymbolσ$-model) and one model from the class of variational multiscale models (residual-based) within a finite element framework. The influence of these models on the estimation of clinically relevant biomarkers used to assess the degree of severity of the pathological condition (pressure difference, secondary flow degree, normalized flow displacement, wall shear stress) is investigated in detail. The simulations show that most methods are consistent in terms of severity indicators such as pressure difference and stenotic velocity. The numerical results indicate that second order velocity elements outperform first order elements in terms of accuracy. Moreover, using second order velocity finite elements, different turbulence models might lead to considerably different results concerning other clinically relevant quantities such as wall shear stresses. These differences may be attributed to differences in numerical dissipation introduced by the turbulence models.
Self-support topology optimization considering distortion for metal additive manufacturing
Venue: arXiv cs.CE 2022
Year: 2022

This paper proposes a self-support topology optimization method that considers distortion to improve the manufacturability of additive manufacturing. First, a self-support constraint is proposed that combines an overhang angle constraint with an adjustable degree of the dripping effect and a thermal constraint for heat dissipation in the building process. Next, we introduce a mechanical model based on the inherent strain method in the building process and propose a constraint that can suppress distortion. An optimization problem is formulated to satisfy all constraints, and an optimization algorithm based on level-set-based topology optimization is constructed. Finally, two- and three-dimensional optimization examples are presented to validate the effectiveness of the proposed topology optimization method.