OpenRaiser/Intern-Atlas huggingface dataset:
Attack Analysis of Face Recognition Authentication Systems Using Fast Gradient Sign Method
Venue: arXiv cs.CV 2022
Year: 2022

Biometric authentication methods, representing the "something you are" scheme, are considered the most secure approach for gaining access to protected resources. Recent attacks using Machine Learning techniques demand a serious systematic reevaluation of biometric authentication. This paper analyzes and presents the Fast Gradient Sign Method (FGSM) attack using face recognition for biometric authentication. Machine Learning techniques have been used to train and test the model, which can classify and identify different people's faces and which will be used as a target for carrying out the attack. Furthermore, the case study will analyze the implementation of the FGSM and the level of performance reduction that the model will have by applying this method in attacking. The test results were performed with the change of parameters both in terms of training and attacking the model, thus showing the efficiency of applying the FGSM.
Deep Multimodal Guidance for Medical Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Medical imaging is a cornerstone of therapy and diagnosis in modern medicine. However, the choice of imaging modality for a particular theranostic task typically involves trade-offs between the feasibility of using a particular modality (e.g., short wait times, low cost, fast acquisition, reduced radiation/invasiveness) and the expected performance on a clinical task (e.g., diagnostic accuracy, efficacy of treatment planning and guidance). In this work, we aim to apply the knowledge learned from the less feasible but better-performing (superior) modality to guide the utilization of the more-feasible yet under-performing (inferior) modality and steer it towards improved performance. We focus on the application of deep learning for image-based diagnosis. We develop a light-weight guidance model that leverages the latent representation learned from the superior modality, when training a model that consumes only the inferior modality. We examine the advantages of our method in the context of two clinical applications: multi-task skin lesion classification from clinical and dermoscopic images and brain tumor classification from multi-sequence magnetic resonance imaging (MRI) and histopathology images. For both these scenarios we show a boost in diagnostic performance of the inferior modality without requiring the superior modality. Furthermore, in the case of brain tumor classification, our method outperforms the model trained on the superior modality while producing comparable results to the model that uses both modalities during inference.
How to Train a (Bad) Algorithmic Caseworker: A Quantitative Deconstruction of Risk Assessments in Child-Welfare
Venue: arXiv cs.HC 2022
Year: 2022

Child welfare (CW) agencies use risk assessment tools as a means to achieve evidence-based, consistent, and unbiased decision-making. These risk assessments act as data collection mechanisms and have further evolved into algorithmic systems in recent years. Moreover, several of these algorithms have reinforced biased theoretical constructs and predictors because of the easy availability of structured assessment data. In this study, we critically examine the Washington Assessment of Risk Model (WARM), a prominent risk assessment tool that has been adopted by over 30 states in the United States and has been repurposed into more complex algorithms. We compared WARM against the narrative coding of casenotes written by caseworkers who used WARM. We found significant discrepancies between the casenotes and WARM data where WARM scores did not not mirror caseworkers' notes about family risk. We provide the SIGCHI community with some initial findings from the quantitative de-construction of a child-welfare algorithm.
Semi-constraint Optimal Transport for Entity Alignment with Dangling Cases
Venue: arXiv cs.CL 2022
Year: 2022

Entity alignment (EA) merges knowledge graphs (KGs) by identifying the equivalent entities in different graphs, which can effectively enrich knowledge representations of KGs. However, in practice, different KGs often include dangling entities whose counterparts cannot be found in the other graph, which limits the performance of EA methods. To improve EA with dangling entities, we propose an unsupervised method called Semi-constraint Optimal Transport for Entity Alignment in Dangling cases (SoTead). Our main idea is to model the entity alignment between two KGs as an optimal transport problem from one KG's entities to the others. First, we set pseudo entity pairs between KGs based on pretrained word embeddings. Then, we conduct contrastive metric learning to obtain the transport cost between each entity pair. Finally, we introduce a virtual entity for each KG to "align" the dangling entities from the other KGs, which relaxes the optimization constraints and leads to a semi-constraint optimal transport. In the experimental part, we first show the superiority of SoTead on a commonly-used entity alignment dataset. Besides, to analyze the ability for dangling entity detection with other baselines, we construct a medical cross-lingual knowledge graph dataset, MedED, where our SoTead also reaches state-of-the-art performance.
An Efficient Accelerator for Deep Learning-based Point Cloud Registration on FPGAs
Venue: arXiv cs.RO 2022
Year: 2022

Point cloud registration is the basis for many robotic applications such as odometry and Simultaneous Localization And Mapping (SLAM), which are increasingly important for autonomous mobile robots. Computational resources and power budgets are limited on these robots, thereby motivating the development of resource-efficient registration method on low-cost FPGAs. In this paper, we propose a novel approach for FPGA-based 3D point cloud registration built upon a recent deep learning-based method, PointNetLK. A highly-efficient FPGA accelerator for PointNet-based feature extraction is designed and implemented on both low-cost and mid-range FPGAs (Avnet Ultra96v2 and Xilinx ZCU104). Our accelerator design is evaluated in terms of registration speed, accuracy, resource usage, and power consumption. Experimental results show that PointNetLK with our accelerator achieves up to 21.34x and 69.60x faster registration speed than the CPU counterpart and ICP, respectively, while only consuming 722mW and maintaining the same level of accuracy.
Research on Parallel SVM Algorithm Based on Cascade SVM
Venue: arXiv cs.LG 2022
Year: 2022

Cascade SVM (CSVM) can group datasets and train subsets in parallel, which greatly reduces the training time and memory consumption. However, the model accuracy obtained by using this method has some errors compared with direct training. In order to reduce the error, we analyze the causes of error in grouping training, and summarize the grouping without error under ideal conditions. A Balanced Cascade SVM (BCSVM) algorithm is proposed, which balances the sample proportion in the subset after grouping to ensure that the sample proportion in the subset is the same as the original dataset. At the same time, it proves that the accuracy of the model obtained by BCSVM algorithm is higher than that of CSVM. Finally, two common datasets are used for experimental verification, and the results show that the accuracy error obtained by using BCSVM algorithm is reduced from 1% of CSVM to 0.1%, which is reduced by an order of magnitude.
Redistribution in Public Project Problems via Neural Networks
Venue: arXiv cs.GT 2022
Year: 2022

Many important problems in multiagent systems involve resource allocations. Self-interested agents may lie about their valuations if doing so increases their own utilities. Therefore, it is necessary to design mechanisms (collective decision-making rules) with desired properties and objectives. The VCG redistribution mechanisms are efficient (the agents who value the resources the most will be allocated), strategy-proof (the agents have no incentives to lie about their valuations), and weakly budget-balanced (no deficits). We focus on the VCG redistribution mechanisms for the classic public project problem, where a group of agents needs to decide whether or not to build a non-excludable public project. We design mechanisms via neural networks with two welfare-maximizing objectives: optimal in the worst case and optimal in expectation. Previous studies showed two worst-case optimal mechanisms for 3 agents, but worst-case optimal mechanisms have not been identified for more than 3 agents. For maximizing expected welfare, there are no existing results. We use neural networks to design VCG redistribution mechanisms. Neural networks have been used to design the redistribution mechanisms for multi-unit auctions with unit demand. We show that for the public project problem, the previously proposed neural networks, which led to optimal/near-optimal mechanisms for multi-unit auctions with unit demand, perform abysmally for the public project problem. We significantly improve the existing networks on multiple fronts: We conduct a GAN network to generate worst-case type profiles and feed prior distribution into loss function to provide quality gradients for the optimal-in-expectation objective......
CBlockSim: A Modular High-Performance Blockchain Simulator
Venue: arXiv cs.DC 2022
Year: 2022

Blockchain has attracted much attention from both academia and industry since emerging in 2008. Due to the inconvenience of the deployment of large-scale blockchains, blockchain simulators are used to facilitate blockchain design and implementation. We evaluate state-of-the-art simulators applied to both Bitcoin and Ethereum and find that they suffer from low performance and scalability which are significant limitations. To build a more general and faster blockchain simulator, we extend an existing blockchain simulator, i.e. BlockSim. We add a network module integrated with a network topology generation algorithm and a block propagation algorithm to generate a realistic blockchain network and simulate the block propagation efficiently. We design a binary transaction pool structure and migrate BlockSim from Python to C++ so that bitwise operations can be used to accelerate the simulation and reduce memory usage. Moreover, we modularize the simulator based on five primary blockchain processes. Significant blockchain elements including consensus protocols (PoW and PoS), information propagation algorithms (Gossip) and finalization rules (Longest rule and GHOST rule) are implemented in individual modules and can be combined flexibly to simulate different types of blockchains. Experiments demonstrate that the new simulator reduces the simulation time by an order of magnitude and improves scalability, enabling us to simulate more than ten thousand nodes, roughly the size of the Bitcoin and Ethereum networks. Two typical use cases are proposed to investigate network-related issues which are not covered by most other simulators.
Democratizing Contrastive Language-Image Pre-training: A CLIP Benchmark of Data, Model, and Supervision
Venue: arXiv cs.CV 2022
Year: 2022

Contrastive Language-Image Pretraining (CLIP) has emerged as a novel paradigm to learn visual models from language supervision. While researchers continue to push the frontier of CLIP, reproducing these works remains challenging. This is because researchers do not choose consistent training recipes and even use different data, hampering the fair comparison between different methods. In this work, we propose CLIP-benchmark, a first attempt to evaluate, analyze, and benchmark CLIP and its variants. We conduct a comprehensive analysis of three key factors: data, supervision, and model architecture. We find considerable intuitive or counter-intuitive insights: (1). Data quality has a significant impact on performance. (2). Certain supervision has different effects for Convolutional Networks (ConvNets) and Vision Transformers (ViT). Applying more proper supervision can effectively improve the performance of CLIP. (3). Curtailing the text encoder reduces the training cost but not much affect the final performance. Moreover, we further combine DeCLIP with FILIP, bringing us the strongest variant DeFILIP. The CLIP-benchmark would be released at: https://github.com/Sense-GVT/DeCLIP for future CLIP research.
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory
Venue: arXiv cs.CL 2022
Year: 2022

Most of the open-domain dialogue models tend to perform poorly in the setting of long-term human-bot conversations. The possible reason is that they lack the capability of understanding and memorizing long-term dialogue history information. To address this issue, we present a novel task of Long-term Memory Conversation (LeMon) and then build a new dialogue dataset DuLeMon and a dialogue generation framework with Long-Term Memory (LTM) mechanism (called PLATO-LTM). This LTM mechanism enables our system to accurately extract and continuously update long-term persona memory without requiring multiple-session dialogue datasets for model training. To our knowledge, this is the first attempt to conduct real-time dynamic management of persona information of both parties, including the user and the bot. Results on DuLeMon indicate that PLATO-LTM can significantly outperform baselines in terms of long-term dialogue consistency, leading to better dialogue engagingness.
Improve Convolutional Neural Network Pruning by Maximizing Filter Variety
Venue: arXiv cs.CV 2022
Year: 2022

Neural network pruning is a widely used strategy for reducing model storage and computing requirements. It allows to lower the complexity of the network by introducing sparsity in the weights. Because taking advantage of sparse matrices is still challenging, pruning is often performed in a structured way, i.e. removing entire convolution filters in the case of ConvNets, according to a chosen pruning criteria. Common pruning criteria, such as l1-norm or movement, usually do not consider the individual utility of filters, which may lead to: (1) the removal of filters exhibiting rare, thus important and discriminative behaviour, and (2) the retaining of filters with redundant information. In this paper, we present a technique solving those two issues, and which can be appended to any pruning criteria. This technique ensures that the criteria of selection focuses on redundant filters, while retaining the rare ones, thus maximizing the variety of remaining filters. The experimental results, carried out on different datasets (CIFAR-10, CIFAR-100 and CALTECH-101) and using different architectures (VGG-16 and ResNet-18) demonstrate that it is possible to achieve similar sparsity levels while maintaining a higher performance when appending our filter selection technique to pruning criteria. Moreover, we assess the quality of the found sparse sub-networks by applying the Lottery Ticket Hypothesis and find that the addition of our method allows to discover better performing tickets in most cases
Reprogramming FairGANs with Variational Auto-Encoders: A New Transfer Learning Model
Venue: arXiv cs.LG 2022
Year: 2022

Fairness-aware GANs (FairGANs) exploit the mechanisms of Generative Adversarial Networks (GANs) to impose fairness on the generated data, freeing them from both disparate impact and disparate treatment. Given the model's advantages and performance, we introduce a novel learning framework to transfer a pre-trained FairGAN to other tasks. This reprogramming process has the goal of maintaining the FairGAN's main targets of data utility, classification utility, and data fairness, while widening its applicability and ease of use. In this paper we present the technical extensions required to adapt the original architecture to this new framework (and in particular the use of Variational Auto-Encoders), and discuss the benefits, trade-offs, and limitations of the new model.
Deep Learning for Wireless Dynamics
Venue: arXiv cs.IT 2022
Year: 2022

This paper aims to predict radio channel variations over time by deep learning from channel observations without knowledge of the underlying channel dynamics. In next-generation wideband cellular systems, multicarrier transmission for higher data rate leads to the high-resolution predicting problem. By leveraging recent advances of deep learning in high-resolution image processing, we propose a purely data-driven deep learning (DL) approach to predicting high-resolution temporal evolution of wideband radio channels. In order to investigate the effect of architectural design choices, we develop and study three deep learning prediction models, namely, baseline, image completion, and next-frame prediction models using UNet. Numerical results show that the proposed DL approach achieves a 52% lower prediction error than the traditional approach based on the Kalman filter (KF) in mean absolute errors. To quantify impact of channel aging and prediction on precoding performance, we also evaluate the performance degradation due to outdated and predicted channel state information (CSI) compared to perfect CSI. Our simulations show that the proposed DL approach can reduce the performance loss due to channel aging by 71% through adapting precoding vector to changes in radio channel while the traditional KF approach only shows a 27% reduction.
Visualizing and Understanding Patch Interactions in Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Vision Transformer (ViT) has become a leading tool in various computer vision tasks, owing to its unique self-attention mechanism that learns visual representations explicitly through cross-patch information interactions. Despite having good success, the literature seldom explores the explainability of vision transformer, and there is no clear picture of how the attention mechanism with respect to the correlation across comprehensive patches will impact the performance and what is the further potential. In this work, we propose a novel explainable visualization approach to analyze and interpret the crucial attention interactions among patches for vision transformer. Specifically, we first introduce a quantification indicator to measure the impact of patch interaction and verify such quantification on attention window design and indiscriminative patches removal. Then, we exploit the effective responsive field of each patch in ViT and devise a window-free transformer architecture accordingly. Extensive experiments on ImageNet demonstrate that the exquisitely designed quantitative method is shown able to facilitate ViT model learning, leading the top-1 accuracy by 4.28% at most. Moreover, the results on downstream fine-grained recognition tasks further validate the generalization of our proposal.
Flat Delaunay Complexes for Homeomorphic Manifold Reconstruction
Venue: arXiv cs.CG 2022
Year: 2022

Given a smooth submanifold of the Euclidean space, a finite point cloud and a scale parameter, we introduce a construction which we call the flat Delaunay complex (FDC). This is a variant of the tangential Delaunay complex (TDC) introduced by Boissonnat et al.. Building on their work, we provide a short and direct proof that when the point cloud samples sufficiently nicely the submanifold and is sufficiently safe (a notion which we define in the paper), our construction is homeomorphic to the submanifold. Because the proof works even when data points are noisy, this allows us to propose a perturbation scheme that takes as input a point cloud sufficiently nice and returns a point cloud which in addition is sufficiently safe. Equally importantly, our construction provides the framework underlying a variational formulation of the reconstruction problem which we present in a companion paper.
Saliency-Driven Versatile Video Coding for Neural Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Saliency-driven image and video coding for humans has gained importance in the recent past. In this paper, we propose such a saliency-driven coding framework for the video coding for machines task using the latest video coding standard Versatile Video Coding (VVC). To determine the salient regions before encoding, we employ the real-time-capable object detection network You Only Look Once~(YOLO) in combination with a novel decision criterion. To measure the coding quality for a machine, the state-of-the-art object segmentation network Mask R-CNN was applied to the decoded frame. From extensive simulations we find that, compared to the reference VVC with a constant quality, up to 29 % of bitrate can be saved with the same detection accuracy at the decoder side by applying the proposed saliency-driven framework. Besides, we compare YOLO against other, more traditional saliency detection methods.
Solving Multi-Structured Problems by Introducing Linkage Kernels into GOMEA
Venue: arXiv cs.NE 2022
Year: 2022

Model-Based Evolutionary Algorithms (MBEAs) can be highly scalable by virtue of linkage (or variable interaction) learning. This requires, however, that the linkage model can capture the exploitable structure of a problem. Usually, a single type of linkage structure is attempted to be captured using models such as a linkage tree. However, in practice, problems may exhibit multiple linkage structures. This is for instance the case in multi-objective optimization when the objectives have different linkage structures. This cannot be modelled sufficiently well when using linkage models that aim at capturing a single type of linkage structure, deteriorating the advantages brought by MBEAs. Therefore, here, we introduce linkage kernels, whereby a linkage structure is learned for each solution over its local neighborhood. We implement linkage kernels into the MBEA known as GOMEA that was previously found to be highly scalable when solving various problems. We further introduce a novel benchmark function called Best-of-Traps (BoT) that has an adjustable degree of different linkage structures. On both BoT and a worst-case scenario-based variant of the well-known MaxCut problem, we experimentally find a vast performance improvement of linkage-kernel GOMEA over GOMEA with a single linkage tree as well as the MBEA known as DSMGA-II.
On The Scale Dependence and Spacetime Dimension of the Internet with Causal Sets
Venue: arXiv cs.SI 2022
Year: 2022

A statistical measure of dimension is used to compute the effective average space dimension for the Internet and other graphs, based on typed edges (links) from an ensemble of starting points. The method is applied to CAIDA's ITDK data for the Internet. The effective dimension at different scales is calibrated to the conventional Euclidean dimension using low dimensional hypercubes. Internet spacetime has a 'foamy' multi-scale containment hierarchy, with interleaving semantic types. There is an emergent scale for approximate long range order in the device node spectrum, but this is not evident at the AS level, where there is finite distance containment. Statistical dimension is thus a locally varying measure, which is scale-dependent, giving an visual analogy for the hidden scale-dependent dimensions of Kaluza-Klein theories. The characteristic exterior dimensions of the Internet lie between 1.66 +- 0.00 and 6.12 +- 0.00, and maximal interior dimensions rise to 7.7.