OpenRaiser/Intern-Atlas huggingface dataset:
Hierarchical Attention Network for Few-Shot Object Detection via Meta-Contrastive Learning
Venue: arXiv cs.CV 2022
Year: 2022

Few-shot object detection (FSOD) aims to classify and detect few images of novel categories. Existing meta-learning methods insufficiently exploit features between support and query images owing to structural limitations. We propose a hierarchical attention network with sequentially large receptive fields to fully exploit the query and support images. In addition, meta-learning does not distinguish the categories well because it determines whether the support and query images match. In other words, metric-based learning for classification is ineffective because it does not work directly. Thus, we propose a contrastive learning method called meta-contrastive learning, which directly helps achieve the purpose of the meta-learning strategy. Finally, we establish a new state-of-the-art network, by realizing significant margins. Our method brings 2.3, 1.0, 1.3, 3.4 and 2.4% AP improvements for 1-30 shots object detection on COCO dataset. Our code is available at: https://github.com/infinity7428/hANMCL
On the Expressiveness of Mixed Choice Sessions (Technical Report)
Venue: arXiv cs.LO 2022
Year: 2022

Session types provide a flexible programming style for structuring interaction, and are used to guarantee a safe and consistent composition of distributed processes. Traditional session types include only one-directional input (external) and output (internal) guarded choices. This prevents the session-processes to explore the full expressive power of the pi-calculus where the mixed choices are proved more expressive than the (non-mixed) guarded choices. To account this issue, recently Casal, Mordido, and Vasconcelos proposed the binary session types with mixed choices (CMV+). This paper carries a surprising, unfortunate result on CMV+: in spite of an inclusion of unrestricted channels with mixed choice, CMV+'s mixed choice is rather separate and not mixed. We prove this negative result using two methodologies (using either the leader election problem or a synchronisation pattern as distinguishing feature), showing that there exists no good encoding from the pi-calculus into CMV+, preserving distribution. We then close their open problem on the encoding from CMV+ into CMV (without mixed choice), proving its soundness and thereby that the encoding is good up to coupled similarity. This technical report extends a paper presented at the workshop EXPRESS/SOS'22.
A Pipeline for DNS-Based Software Fingerprinting
Venue: arXiv cs.NI 2022
Year: 2022

In this paper, we present the modular design and implementation of DONUT, a novel tool for identifying software running on a device. Our tool uses a rule-based approach to detect software-specific DNS fingerprints (stored in an easily extendable database) in passively monitored DNS traffic. We automated the rule extraction process for DONUT with the help of ATLAS, a novel tool we developed for labeling network traffic by the software that created it. We demonstrate the functionality of our pipeline by generating rules for a number of applications, evaluate the performance as well as scalability of the analysis, and confirm the functional correctness of DONUT using an artificial data set for which the ground-truth is known. In addition, we evaluate DONUT's analysis results on a large real-world data set with unknown ground truth.
Fair Incentivization of Bandwidth Sharing in Decentralized Storage Networks
Venue: arXiv cs.NI 2022
Year: 2022

Peer-to-peer (p2p) networks are not independent of their peers, and the network efficiency depends on peers contributing resources. Because shared resources are not free, this contribution must be rewarded. Peers across the network may share computation power, storage capacity, and bandwidth. This paper looks at how bandwidth incentive encourages peers to share bandwidth and rewards them for their contribution. With the advent of blockchain technology, many p2p networks attempt to reward contributions by crypto-assets. We conduct simulations to better understand current incentive mechanisms, assess the fairness of these mechanisms, and to look for ways to make those incentives more equitable. The following are the primary contributions of this study: (i) We investigate and simulate bandwidth incentives within Swarm, a cutting-edge p2p storage network; (ii) We demonstrate one approach to make the current bandwidth incentives more equitable; (iii) We use the Gini coefficient to define two quantifiable fairness characteristics to evaluate reward sharing in a decentralized p2p storage network.
Global Consistent Point Cloud Registration Based on Lie-algebraic Cohomology
Venue: arXiv cs.CV 2022
Year: 2022

We present a novel, effective method for global point cloud registration problems by geometric topology. Based on many point cloud pairwise registration methods (e.g ICP), we focus on the problem of accumulated error for the composition of transformations along any loops. The major technical contribution of this paper is a linear method for the elimination of errors, using only solving a Poisson equation. We demonstrate the consistency of our method from Hodge-Helmhotz decomposition theorem and experiments on multiple RGBD datasets of real-world scenes. The experimental results also demonstrate that our global registration method runs quickly and provides accurate reconstructions.
Xscope: Hunting for Cross-Chain Bridge Attacks
Venue: arXiv cs.SE 2022
Year: 2022

Cross-Chain bridges have become the most popular solution to support asset interoperability between heterogeneous blockchains. However, while providing efficient and flexible cross-chain asset transfer, the complex workflow involving both on-chain smart contracts and off-chain programs causes emerging security issues. In the past year, there have been more than ten severe attacks against cross-chain bridges, causing billions of loss. With few studies focusing on the security of cross-chain bridges, the community still lacks the knowledge and tools to mitigate this significant threat. To bridge the gap, we conduct the first study on the security of cross-chain bridges. We document three new classes of security bugs and propose a set of security properties and patterns to characterize them. Based on those patterns, we design Xscope, an automatic tool to find security violations in cross-chain bridges and detect real-world attacks. We evaluate Xscope on four popular cross-chain bridges. It successfully detects all known attacks and finds suspicious attacks unreported before. A video of Xscope is available at https://youtu.be/vMRO_qOqtXY.
WiFi Based Distance Estimation Using Supervised Machine Learning
Venue: arXiv cs.LG 2022
Year: 2022

In recent years WiFi became the primary source of information to locate a person or device indoor. Collecting RSSI values as reference measurements with known positions, known as WiFi fingerprinting, is commonly used in various positioning methods and algorithms that appear in literature. However, measuring the spatial distance between given set of WiFi fingerprints is heavily affected by the selection of the signal distance function used to model signal space as geospatial distance. In this study, the authors proposed utilization of machine learning to improve the estimation of geospatial distance between fingerprints. This research examined data collected from 13 different open datasets to provide a broad representation aiming for general model that can be used in any indoor environment. The proposed novel approach extracted data features by examining a set of commonly used signal distance metrics via feature selection process that includes feature analysis and genetic algorithm. To demonstrate that the output of this research is venue independent, all models were tested on datasets previously excluded during the training and validation phase. Finally, various machine learning algorithms were compared using wide variety of evaluation metrics including ability to scale out the test bed to real world unsolicited datasets.
Predicting Pedestrian Crosswalk Behavior Using Convolutional Neural Networks
Venue: arXiv cs.CV 2022
Year: 2022

A common yet potentially dangerous task is the act of crossing the street. Pedestrian accidents contribute a significant amount to the high number of annual traffic casualties, which is why it is crucial for pedestrians to use safety measures such as a crosswalk. However, people often forget to activate a crosswalk light or are unable to do so -- such as those who are visually impaired or have occupied hands. Other pedestrians are simply careless and find the crosswalk signals a hassle, which can result in an accident where a car hits them. In this paper, we consider an improvement to the crosswalk system by designing a system that can detect pedestrians and triggering the crosswalk signal automatically. We collect a dataset of images that we then use to train a convolutional neural network to distinguish between pedestrians (including bicycle riders) and various false alarms. The resulting system can capture and evaluate images in real time, and the result can be used to automatically activate systems a crosswalk light. After extensive testing of our system in real-world environments, we conclude that it is feasible as a back-up system that can compliment existing crosswalk buttons, and thereby improve the overall safety of crossing the street.
Training-Time Attacks against k-Nearest Neighbors
Venue: arXiv cs.LG 2022
Year: 2022

Nearest neighbor-based methods are commonly used for classification tasks and as subroutines of other data-analysis methods. An attacker with the capability of inserting their own data points into the training set can manipulate the inferred nearest neighbor structure. We distill this goal to the task of performing a training-set data insertion attack against $k$-Nearest Neighbor classification ($k$NN). We prove that computing an optimal training-time (a.k.a. poisoning) attack against $k$NN classification is NP-Hard, even when $k = 1$ and the attacker can insert only a single data point. We provide an anytime algorithm to perform such an attack, and a greedy algorithm for general $k$ and attacker budget. We provide theoretical bounds and empirically demonstrate the effectiveness and practicality of our methods on synthetic and real-world datasets. Empirically, we find that $k$NN is vulnerable in practice and that dimensionality reduction is an effective defense. We conclude with a discussion of open problems illuminated by our analysis.
Introducing RISK
Venue: arXiv cs.RO 2022
Year: 2022

This extended abstract introduces the initial steps taken to develop a system for Rapid Internal Simulation of Knowledge (RISK). RISK aims to enable more transparency in artificial intelligence systems, especially those created by deep learning networks by allowing real-time simulation of what the system knows. By looking at hypothetical situations based on these simulations a system may make more informed decisions, and produce them for non-expert observers to understand the reasoning behind a given action.
Placement Laundering and the Complexities of Attribution in Online Advertising
Venue: arXiv cs.CR 2022
Year: 2022

A basic assumption in online advertising is that it is possible to attribute a view of a particular ad creative (i.e., an impression) to a particular web page. In practice, however, the seemingly simple task of ad attribution is challenging due to the scale, complexity and diversity of ad delivery systems. In this paper, we describe a new form of fraud that we call placement laundering, which exploits vulnerabilities in attribution mechanisms. Placement laundering allows malicious actors to inflate revenue by making ad calls that appear to originate from high quality publishers. We describe the basic aspects of placement laundering and give details of two instances found in the wild. One of the instances that we describe abuses the intended functionality of the widely-deployed SafeFrame environment. We describe a placement laundering detection method that is capable of identifying a general class of laundering schemes, and provide results on tests with that method.
SYN-MAD 2022: Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 International Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 different countries. In the end, seven valid submissions were submitted by the participating teams and evaluated by the organizers. The competition was held to present and attract solutions that deal with detecting face morphing attacks while protecting people's privacy for ethical and legal reasons. To ensure this, the training data was limited to synthetic data provided by the organizers. The submitted solutions presented innovations that led to outperforming the considered baseline in many experimental settings. The evaluation benchmark is now available at: https://github.com/marcohuber/SYN-MAD-2022.
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Venue: arXiv cs.LG 2022
Year: 2022

Large language models have been widely adopted but require significant GPU memory for inference. We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance. With our method, a 175B parameter 16/32-bit checkpoint can be loaded, converted to Int8, and used immediately without performance degradation. This is made possible by understanding and working around properties of highly systematic emergent features in transformer language models that dominate attention and transformer predictive performance. To cope with these features, we develop a two-part quantization procedure, LLM.int8(). We first use vector-wise quantization with separate normalization constants for each inner product in the matrix multiplication, to quantize most of the features. However, for the emergent outliers, we also include a new mixed-precision decomposition scheme, which isolates the outlier feature dimensions into a 16-bit matrix multiplication while still more than 99.9% of values are multiplied in 8-bit. Using LLM.int8(), we show empirically it is possible to perform inference in LLMs with up to 175B parameters without any performance degradation. This result makes such models much more accessible, for example making it possible to use OPT-175B/BLOOM on a single server with consumer GPUs. We open-source our software.
Dürfen Maschinen denken (können)? Warum Künstliche Intelligenz eine Ethik braucht. (Are Machines Allowed to (be able to) Think? Why Artificial Intelligence Needs Ethics)
Venue: arXiv cs.CY 2022
Year: 2022

Speech manuscript (German + English) of the impulse lecture for the panel discussion "May machines (be able to) think?" at the 102nd Katholikentag on May 28, 2022 in Stuttgart. Panel: Winfried Kretschmann (MdL, Prime Minister Baden-Württemberg, Stuttgart), Ursula Nothelle-Wildfeuer (Freiburg), Michael Resch (Stuttgart),Karsten Wendland (Aalen). Moderation: Stefanie Rentsch (Fulda). Advocate of the audience: Verena Neuhausen (Stuttgart).
Comments on SPD-41 software licensing requirements
Venue: arXiv cs.SE 2022
Year: 2022

The proposed changes to Science Missions Directorate (SMD) Policy Document 41 (SPD-41) include requirements that SMD-funded software should be released using a permissive software license and following best practices from the open source software community. This is a welcome change that will lead to greater acceptance and adoption of SMD-funded software. However, ambiguities exist in the policy text around what licenses will be allowed. This could lead to problems in the solicitation process and friction with the open source software community. Moreover, some additional clarifications are warranted around what exceptions might be allowed to the software licensing policy.
A Library for Representing Python Programs as Graphs for Machine Learning
Venue: arXiv cs.LG 2022
Year: 2022

Graph representations of programs are commonly a central element of machine learning for code research. We introduce an open source Python library python_graphs that applies static analysis to construct graph representations of Python programs suitable for training machine learning models. Our library admits the construction of control-flow graphs, data-flow graphs, and composite ``program graphs'' that combine control-flow, data-flow, syntactic, and lexical information about a program. We present the capabilities and limitations of the library, perform a case study applying the library to millions of competitive programming submissions, and showcase the library's utility for machine learning research.
BoW3D: Bag of Words for Real-Time Loop Closing in 3D LiDAR SLAM
Venue: arXiv cs.CV 2022
Year: 2022

Loop closing is a fundamental part of simultaneous localization and mapping (SLAM) for autonomous mobile systems. In the field of visual SLAM, bag of words (BoW) has achieved great success in loop closure. The BoW features for loop searching can also be used in the subsequent 6-DoF loop correction. However, for 3D LiDAR SLAM, the state-of-the-art methods may fail to effectively recognize the loop in real time, and usually cannot correct the full 6-DoF loop pose. To address this limitation, we present a novel Bag of Words for real-time loop closing in 3D LiDAR SLAM, called BoW3D. Our method not only efficiently recognizes the revisited loop places, but also corrects the full 6-DoF loop pose in real time. BoW3D builds the bag of words based on the 3D LiDAR feature LinK3D, which is efficient, pose-invariant and can be used for accurate point-to-point matching. We furthermore embed our proposed method into 3D LiDAR odometry system to evaluate loop closing performance. We test our method on public dataset, and compare it against other state-of-the-art algorithms. BoW3D shows better performance in terms of F1 max and extended precision scores on most scenarios. It is noticeable that BoW3D takes an average of 48 ms to recognize and correct the loops on KITTI 00 (includes 4K+ 64-ray LiDAR scans), when executed on a notebook with an Intel Core i7 @2.2 GHz processor. We release the implementation of our method here: https://github.com/YungeCui/BoW3D.
HetVis: A Visual Analysis Approach for Identifying Data Heterogeneity in Horizontal Federated Learning
Venue: arXiv cs.HC 2022
Year: 2022

Horizontal federated learning (HFL) enables distributed clients to train a shared model and keep their data privacy. In training high-quality HFL models, the data heterogeneity among clients is one of the major concerns. However, due to the security issue and the complexity of deep learning models, it is challenging to investigate data heterogeneity across different clients. To address this issue, based on a requirement analysis we developed a visual analytics tool, HetVis, for participating clients to explore data heterogeneity. We identify data heterogeneity through comparing prediction behaviors of the global federated model and the stand-alone model trained with local data. Then, a context-aware clustering of the inconsistent records is done, to provide a summary of data heterogeneity. Combining with the proposed comparison techniques, we develop a novel set of visualizations to identify heterogeneity issues in HFL. We designed three case studies to introduce how HetVis can assist client analysts in understanding different types of heterogeneity issues. Expert reviews and a comparative study demonstrate the effectiveness of HetVis.
SGM-Net: Semantic Guided Matting Net
Venue: arXiv cs.CV 2022
Year: 2022

Human matting refers to extracting human parts from natural images with high quality, including human detail information such as hair, glasses, hat, etc. This technology plays an essential role in image synthesis and visual effects in the film industry. When the green screen is not available, the existing human matting methods need the help of additional inputs (such as trimap, background image, etc.), or the model with high computational cost and complex network structure, which brings great difficulties to the application of human matting in practice. To alleviate such problems, most existing methods (such as MODNet) use multi-branches to pave the way for matting through segmentation, but these methods do not make full use of the image features and only utilize the prediction results of the network as guidance information. Therefore, we propose a module to generate foreground probability map and add it to MODNet to obtain Semantic Guided Matting Net (SGM-Net). Under the condition of only one image, we can realize the human matting task. We verify our method on the P3M-10k dataset. Compared with the benchmark, our method has significantly improved in various evaluation indicators.
Dependency Tasks Offloading and Communication Resource Allocation in Collaborative UAVs Networks: A Meta-Heuristic Approach
Venue: arXiv cs.NI 2022
Year: 2022

In recent years, unmanned aerial vehicles (UAVs) assisted mobile edge computing systems have been exploited by researchers as a promising solution for providing computation services to mobile users outside of terrestrial infrastructure coverage. However, it remains challenging for the standalone MEC-enabled UAVs in order to meet the computation requirement of numerous mobile users due to the limited computation capacity of their onboard servers and battery lives. Therefore, we propose a collaborative scheme among UAVs so that UAVs can share the workload with idle UAVs. Moreover, current task offloading strategies frequently overlook task topology, which may result in poor performance or even system failure. To address the problem, we consider offloading tasks consisting of a set of sub-tasks, and each sub-task has dependencies on other sub-tasks, which is practical in the real world. Sub-tasks with dependencies need to wait for the resulting signal from preceding sub-tasks before being executed. This mechanism has serious effects on the offloading strategy. Then, we formulate an optimization problem to minimize the average latency experienced by users by jointly controlling the offloading decision for dependent tasks and allocating the communication resources of UAVs. The formulated problem appears to be NP-hard and cannot be solved in polynomial time. Therefore, we divide the problem into two sub-problems: the offloading decision problem and the communication resource allocation problem. Then a meta-heuristic method is proposed to find the sub-optimal solution of the task offloading problem, while the communication resource allocation problem is solved by using convex optimization. Finally, we perform substantial simulation experiments, and the result shows that the proposed offloading technique effectively minimizes the average latency of users, compared with other benchmark schemes.