OpenRaiser/Intern-Atlas huggingface dataset:
IH-ViT: Vision Transformer-based Integrated Circuit Appear-ance Defect Detection
Venue: arXiv cs.CV 2023
Year: 2023

For the problems of low recognition rate and slow recognition speed of traditional detection methods in IC appearance defect detection, we propose an IC appearance defect detection algo-rithm IH-ViT. Our proposed model takes advantage of the respective strengths of CNN and ViT to acquire image features from both local and global aspects, and finally fuses the two features for decision making to determine the class of defects, thus obtaining better accuracy of IC defect recognition. To address the problem that IC appearance defects are mainly reflected in the dif-ferences in details, which are difficult to identify by traditional algorithms, we improved the tra-ditional ViT by performing an additional convolution operation inside the batch. For the problem of information imbalance of samples due to diverse sources of data sets, we adopt a dual-channel image segmentation technique to further improve the accuracy of IC appearance defects. Finally, after testing, our proposed hybrid IH-ViT model achieved 72.51% accuracy, which is 2.8% and 6.06% higher than ResNet50 and ViT models alone. The proposed algorithm can quickly and accurately detect the defect status of IC appearance and effectively improve the productivity of IC packaging and testing companies.
Explaining with Greater Support: Weighted Column Sampling Optimization for q-Consistent Summary-Explanations
Venue: arXiv cs.AI 2023
Year: 2023

Machine learning systems have been extensively used as auxiliary tools in domains that require critical decision-making, such as healthcare and criminal justice. The explainability of decisions is crucial for users to develop trust on these systems. In recent years, the globally-consistent rule-based summary-explanation and its max-support (MS) problem have been proposed, which can provide explanations for particular decisions along with useful statistics of the dataset. However, globally-consistent summary-explanations with limited complexity typically have small supports, if there are any. In this paper, we propose a relaxed version of summary-explanation, i.e., the $q$-consistent summary-explanation, which aims to achieve greater support at the cost of slightly lower consistency. The challenge is that the max-support problem of $q$-consistent summary-explanation (MSqC) is much more complex than the original MS problem, resulting in over-extended solution time using standard branch-and-bound solvers. To improve the solution time efficiency, this paper proposes the weighted column sampling~(WCS) method based on solving smaller problems by sampling variables according to their simplified increase support (SIS) values. Experiments verify that solving MSqC with the proposed SIS-based WCS method is not only more scalable in efficiency, but also yields solutions with greater support and better global extrapolation effectiveness.
GMConv: Modulating Effective Receptive Fields for Convolutional Kernels
Venue: arXiv cs.CV 2023
Year: 2023

In convolutional neural networks, the convolutions are conventionally performed using a square kernel with a fixed N $\times$ N receptive field (RF). However, what matters most to the network is the effective receptive field (ERF) that indicates the extent with which input pixels contribute to an output pixel. Inspired by the property that ERFs typically exhibit a Gaussian distribution, we propose a Gaussian Mask convolutional kernel (GMConv) in this work. Specifically, GMConv utilizes the Gaussian function to generate a concentric symmetry mask that is placed over the kernel to refine the RF. Our GMConv can directly replace the standard convolutions in existing CNNs and can be easily trained end-to-end by standard back-propagation. We evaluate our approach through extensive experiments on image classification and object detection tasks. Over several tasks and standard base models, our approach compares favorably against the standard convolution. For instance, using GMConv for AlexNet and ResNet-50, the top-1 accuracy on ImageNet classification is boosted by 0.98% and 0.85%, respectively.
Lorentz Equivariant Model for Knowledge-Enhanced Hyperbolic Collaborative Filtering
Venue: arXiv cs.IR 2023
Year: 2023

Introducing prior auxiliary information from the knowledge graph (KG) to assist the user-item graph can improve the comprehensive performance of the recommender system. Many recent studies show that the ensemble properties of hyperbolic spaces fit the scale-free and hierarchical characteristics exhibited in the above two types of graphs well. However, existing hyperbolic methods ignore the consideration of equivariance, thus they cannot generalize symmetric features under given transformations, which seriously limits the capability of the model. Moreover, they cannot balance preserving the heterogeneity and mining the high-order entity information to users across two graphs. To fill these gaps, we propose a rigorously Lorentz group equivariant knowledge-enhanced collaborative filtering model (LECF). Innovatively, we jointly update the attribute embeddings (containing the high-order entity signals from the KG) and hyperbolic embeddings (the distance between hyperbolic embeddings reveals the recommendation tendency) by the LECF layer with Lorentz Equivariant Transformation. Moreover, we propose Hyperbolic Sparse Attention Mechanism to sample the most informative neighbor nodes. Lorentz equivariance is strictly maintained throughout the entire model, and enforcing equivariance is proven necessary experimentally. Extensive experiments on three real-world benchmarks demonstrate that LECF remarkably outperforms state-of-the-art methods.
Incorporating Total Variation Regularization in the design of an intelligent Query by Humming system
Venue: arXiv cs.SD 2023
Year: 2023

A Query-By-Humming (QBH) system constitutes a particular case of music information retrieval where the input is a user-hummed melody and the output is the original song which contains that melody. A typical QBH system consists of melody extraction and candidate melody retrieval. For melody extraction, accurate note transcription is the key enabling technology. However, current transcription methods are unable to definitively capture the melody and address inaccuracies in user-hummed queries. In this paper, we incorporate Total Variation Regularization (TVR) to denoise queries. This approach accounts for user error in humming without loss of meaningful data and reliably captures the underlying melody. For candidate melody retrieval, we employ a deep learning approach to time series classification using a Fully Convolutional Neural Network. The trained network classifies the incoming query as belonging to one of the target songs. For our experiments, we use Roger Jang's MIR-QBSH dataset which is the standard MIREX dataset. We demonstrate that inclusion of TVR denoised queries in the training set enhances the overall accuracy of the system to 93% which is higher than other state-of-the-art QBH systems.
Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute
Venue: arXiv cs.HC 2023
Year: 2023

Local governments increasingly use artificial intelligence (AI) for automated decision-making. Contestability, making systems responsive to dispute, is a way to ensure they respect human rights to autonomy and dignity. We investigate the design of public urban AI systems for contestability through the example of camera cars: human-driven vehicles equipped with image sensors. Applying a provisional framework for contestable AI, we use speculative design to create a concept video of a contestable camera car. Using this concept video, we then conduct semi-structured interviews with 17 civil servants who work with AI employed by a large northwestern European city. The resulting data is analyzed using reflexive thematic analysis to identify the main challenges facing the implementation of contestability in public AI. We describe how civic participation faces issues of representation, public AI systems should integrate with existing democratic practices, and cities must expand capacities for responsible AI development and operation.
Deep Intra-Image Contrastive Learning for Weakly Supervised One-Step Person Search
Venue: arXiv cs.CV 2023
Year: 2023

Weakly supervised person search aims to perform joint pedestrian detection and re-identification (re-id) with only person bounding-box annotations. Recently, the idea of contrastive learning is initially applied to weakly supervised person search, where two common contrast strategies are memory-based contrast and intra-image contrast. We argue that current intra-image contrast is shallow, which suffers from spatial-level and occlusion-level variance. In this paper, we present a novel deep intra-image contrastive learning using a Siamese network. Two key modules are spatial-invariant contrast (SIC) and occlusion-invariant contrast (OIC). SIC performs many-to-one contrasts between two branches of Siamese network and dense prediction contrasts in one branch of Siamese network. With these many-to-one and dense contrasts, SIC tends to learn discriminative scale-invariant and location-invariant features to solve spatial-level variance. OIC enhances feature consistency with the masking strategy to learn occlusion-invariant features. Extensive experiments are performed on two person search datasets CUHK-SYSU and PRW, respectively. Our method achieves a state-of-the-art performance among weakly supervised one-step person search approaches. We hope that our simple intra-image contrastive learning can provide more paradigms on weakly supervised person search. The source code is available at \url{https://github.com/jiabeiwangTJU/DICL}.
Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment
Venue: arXiv cs.LG 2023
Year: 2023

Self-supervised node representation learning aims to learn node representations from unlabelled graphs that rival the supervised counterparts. The key towards learning informative node representations lies in how to effectively gain contextual information from the graph structure. In this work, we present simple-yet-effective self-supervised node representation learning via aligning the hidden representations of nodes and their neighbourhood. Our first idea achieves such node-to-neighbourhood alignment by directly maximizing the mutual information between their representations, which, we prove theoretically, plays the role of graph smoothing. Our framework is optimized via a surrogate contrastive loss and a Topology-Aware Positive Sampling (TAPS) strategy is proposed to sample positives by considering the structural dependencies between nodes, which enables offline positive selection. Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing. The GSD constraint unifies some of the existing constraints and can be used to derive new implementations to combat representation collapse. By applying our methods on top of simple MLP-based node representation encoders, we learn node representations that achieve promising node classification performance on a set of graph-structured datasets from small- to large-scale.
Prefixes of the Fibonacci word
Venue: arXiv cs.FL 2023
Year: 2023

Mignosi, Restivo, and Salemi (1998) proved that for all $ε> 0$ there exists an integer $N$ such that all prefixes of the Fibonacci word of length $\geq N$ contain a suffix of exponent $α^2-ε$, where $α= (1+\sqrt{5})/2$ is the golden ratio. In this note we show how to prove an explicit version of this theorem with tools from automata theory and logic. Along the way we gain a better understanding of the repetitive structure of the Fibonacci word.
A Nonlinear Model Predictive Control Strategy for Autonomous Racing of Scale Vehicles
Venue: arXiv cs.RO 2023
Year: 2023

A Nonlinear Model Predictive Control (NMPC) strategy aimed at controlling a small-scale car model for autonomous racing competitions is presented in this paper. The proposed control strategy is concerned with minimizing the lap time while keeping the vehicle within track boundaries. The optimization problem considers both the vehicle's actuation limits and the lateral and longitudinal forces acting on the car modeled through the Pacejka's magic formula and a simple drivetrain model. Furthermore, the approach allows to safely race on a track populated by static obstacles generating collision-free trajectories and tracking them while enhancing the lap timing performance. Gazebo simulations using the F1/10 simulator showcase the feasibility and validity of the proposed control strategy. The code is released as open-source making it possible to replicate the obtained results.
Optimum Trajectory Planning for Multi-Rotor UAV Relays with Tilt and Antenna Orientation Variations
Venue: arXiv cs.RO 2023
Year: 2023

Multi-rotor Unmanned Aerial Vehicles (UAVs) need to tilt in order to move; this modifies the UAV's antenna orientation. We consider the scenario where a multi-rotor UAV serves as a communication relay between a Base Station (BS) and another UAV. We propose a framework to generate feasible trajectories for the multi-rotor UAV relay while considering its motion dynamics and the motion-induced changes of the antenna orientation. The UAV relay's trajectory is optimized to maximize the end-to-end number of bits transmitted. Numerical simulations in MATLAB and Gazebo show the benefits of accounting for the antenna orientation variations due to the UAV tilt.
A Benchmark on Uncertainty Quantification for Deep Learning Prognostics
Venue: arXiv cs.LG 2023
Year: 2023

Reliable uncertainty quantification on RUL prediction is crucial for informative decision-making in predictive maintenance. In this context, we assess some of the latest developments in the field of uncertainty quantification for prognostics deep learning. This includes the state-of-the-art variational inference algorithms for Bayesian neural networks (BNN) as well as popular alternatives such as Monte Carlo Dropout (MCD), deep ensembles (DE) and heteroscedastic neural networks (HNN). All the inference techniques share the same inception deep learning architecture as a functional model. We performed hyperparameter search to optimize the main variational and learning parameters of the algorithms. The performance of the methods is evaluated on a subset of the large NASA NCMAPSS dataset for aircraft engines. The assessment includes RUL prediction accuracy, the quality of predictive uncertainty, and the possibility to break down the total predictive uncertainty into its aleatoric and epistemic parts. The results show no method clearly outperforms the others in all the situations. Although all methods are close in terms of accuracy, we find differences in the way they estimate uncertainty. Thus, DE and MCD generally provide more conservative predictive uncertainty than BNN. Surprisingly, HNN can achieve strong results without the added training complexity and extra parameters of the BNN. For tasks like active learning where a separation of epistemic and aleatoric uncertainty is required, radial BNN and MCD seem the best options.
MetaExplorer: Facilitating Reasoning with Epistemic Uncertainty in Meta-analysis
Venue: arXiv cs.HC 2023
Year: 2023

Scientists often use meta-analysis to characterize the impact of an intervention on some outcome of interest across a body of literature. However, threats to the utility and validity of meta-analytic estimates arise when scientists average over potentially important variations in context like different research designs. Uncertainty about quality and commensurability of evidence casts doubt on results from meta-analysis, yet existing software tools for meta-analysis do not necessarily emphasize addressing these concerns in their workflows. We present MetaExplorer, a prototype system for meta-analysis that we developed using iterative design with meta-analysis experts to provide a guided process for eliciting assessments of uncertainty and reasoning about how to incorporate them during statistical inference. Our qualitative evaluation of MetaExplorer with experienced meta-analysts shows that imposing a structured workflow both elevates the perceived importance of epistemic concerns and presents opportunities for tools to engage users in dialogue around goals and standards for evidence aggregation.
A Vision-Based Algorithm for a Path Following Problem
Venue: arXiv cs.RO 2023
Year: 2023

A novel prize-winner algorithm designed for a path following problem within the Unmanned Aerial Vehicle (UAV) field is presented in this paper. The proposed approach exploits the advantages offered by the pure pursuing algorithm to set up an intuitive and simple control framework. A path fora quad-rotor UAV is obtained by using downward facing camera images implementing an Image-Based Visual Servoing (IBVS) approach. Numerical simulations in MATLAB together with the MathWorks Virtual Reality (VR) toolbox demonstrate the validity and the effectiveness of the proposed solution. The code is released as open-source making it possible to go through any part of the system and to replicate the obtained results.
Massively Multilingual Language Models for Cross Lingual Fact Extraction from Low Resource Indian Languages
Venue: arXiv cs.CL 2023
Year: 2023

Massive knowledge graphs like Wikidata attempt to capture world knowledge about multiple entities. Recent approaches concentrate on automatically enriching these KGs from text. However a lot of information present in the form of natural text in low resource languages is often missed out. Cross Lingual Information Extraction aims at extracting factual information in the form of English triples from low resource Indian Language text. Despite its massive potential, progress made on this task is lagging when compared to Monolingual Information Extraction. In this paper, we propose the task of Cross Lingual Fact Extraction(CLFE) from text and devise an end-to-end generative approach for the same which achieves an overall F1 score of 77.46.
Using Language Models for Enhancing the Completeness of Natural-language Requirements
Venue: arXiv cs.SE 2023
Year: 2023

[Context and motivation] Incompleteness in natural-language requirements is a challenging problem. [Question/problem] A common technique for detecting incompleteness in requirements is checking the requirements against external sources. With the emergence of language models such as BERT, an interesting question is whether language models are useful external sources for finding potential incompleteness in requirements. [Principal ideas/results] We mask words in requirements and have BERT's masked language model (MLM) generate contextualized predictions for filling the masked slots. We simulate incompleteness by withholding content from requirements and measure BERT's ability to predict terminology that is present in the withheld content but absent in the content disclosed to BERT. [Contribution] BERT can be configured to generate multiple predictions per mask. Our first contribution is to determine how many predictions per mask is an optimal trade-off between effectively discovering omissions in requirements and the level of noise in the predictions. Our second contribution is devising a machine learning-based filter that post-processes predictions made by BERT to further reduce noise. We empirically evaluate our solution over 40 requirements specifications drawn from the PURE dataset [1]. Our results indicate that: (1) predictions made by BERT are highly effective at pinpointing terminology that is missing from requirements, and (2) our filter can substantially reduce noise from the predictions, thus making BERT a more compelling aid for improving completeness in requirements.
Fair Notification Optimization: An Auction Approach
Venue: arXiv cs.GT 2023
Year: 2023

Notifications are important for the user experience in mobile apps and can influence their engagement. However, too many notifications can be disruptive for users. A typical mobile app usually has several types of notification, managed by distinct teams with objectives that are possibly conflicting with each other, or even with the overall platform objective. Therefore, there is a need for careful curation of notifications sent to users of these different types. In this work, we study a novel centralized approach for notification optimization, where we view the opportunities to send user notifications as items and types of notifications as buyers in an auction market. Furthermore, the auction setup is unique, and the platform has the ability to subsidize the bids from the notification types. Using tools from fair division, we study the application of competitive equilibrium for addressing this problem. We show that an Eisenberg-Gale-style convex program allows us to find an allocation that is fair to all notification types in hindsight. Using the dual of the formulation, we present an online algorithm that allocates notifications via first-price auctions using a pacing-multiplier approach. Secondly, we introduce an approach based on second-price auctions and pacing, which has the benefit of working well with existing advertising systems built for second-price auctions. Through an A/B test in production, we show that the second price-based auction system improves over a decentralized notification optimization system, leading to its launch in production for some Instagram notifications. Further, through simulations on Instagram notification data and a subsequent production A/B test, we compare the outcomes of first-price and second-price auctions and show that the former has more stable pacing multipliers.
PETIoT: PEnetration Testing the Internet of Things
Venue: arXiv cs.CR 2023
Year: 2023

Attackers may attempt exploiting Internet of Things (IoT) devices to operate them unduly as well as to gather personal data of the legitimate device owners'. Vulnerability Assessment and Penetration Testing (VAPT) sessions help to verify the effectiveness of the adopted security measures. However, VAPT over IoT devices, namely VAPT targeted at IoT devices, is an open research challenge due to the variety of target technologies and to the creativity it may require. Therefore, this article aims at guiding penetration testers to conduct VAPT sessions over IoT devices by means of a new cyber Kill Chain (KC) termed PETIoT. Several practical applications of PETIoT confirm that it is general, while its main novelty lies in the combination of attack and defence steps. PETIoT is demonstrated on a relevant example, the best-selling IP camera on Amazon Italy, the TAPO C200 by TP-Link, assuming an attacker who sits on the same network as the device's in order to assess all the network interfaces of the device. Additional knowledge is generated in terms of three zero-day vulnerabilities found and practically exploited on the camera, one of these with High severity and the other two with Medium severity by the CVSS standard. These are camera Denial of Service (DoS), motion detection breach and video stream breach. The application of PETIoT culminates with the proof-of-concept of a home-made fix, based on an inexpensive Raspberry Pi 4 Model B device, for the last vulnerability. Ultimately, our responsible disclosure with the camera vendor led to the release of a firmware update that fixes all found vulnerabilities, confirming that PetIoT has valid impact in real-world scenarios.