OpenRaiser/Intern-Atlas huggingface dataset:
Phase-shifted Adversarial Training
Venue: arXiv cs.LG 2023
Year: 2023

Adversarial training has been considered an imperative component for safely deploying neural network-based applications to the real world. To achieve stronger robustness, existing methods primarily focus on how to generate strong attacks by increasing the number of update steps, regularizing the models with the smoothed loss function, and injecting the randomness into the attack. Instead, we analyze the behavior of adversarial training through the lens of response frequency. We empirically discover that adversarial training causes neural networks to have low convergence to high-frequency information, resulting in highly oscillated predictions near each data. To learn high-frequency contents efficiently and effectively, we first prove that a universal phenomenon of frequency principle, i.e., \textit{lower frequencies are learned first}, still holds in adversarial training. Based on that, we propose phase-shifted adversarial training (PhaseAT) in which the model learns high-frequency components by shifting these frequencies to the low-frequency range where the fast convergence occurs. For evaluations, we conduct the experiments on CIFAR-10 and ImageNet with the adaptive attack carefully designed for reliable evaluation. Comprehensive results show that PhaseAT significantly improves the convergence for high-frequency information. This results in improved adversarial robustness by enabling the model to have smoothed predictions near each data.
Adaptive Context Selection for Polyp Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Accurate polyp segmentation is of great significance for the diagnosis and treatment of colorectal cancer. However, it has always been very challenging due to the diverse shape and size of polyp. In recent years, state-of-the-art methods have achieved significant breakthroughs in this task with the help of deep convolutional neural networks. However, few algorithms explicitly consider the impact of the size and shape of the polyp and the complex spatial context on the segmentation performance, which results in the algorithms still being powerless for complex samples. In fact, segmentation of polyps of different sizes relies on different local and global contextual information for regional contrast reasoning. To tackle these issues, we propose an adaptive context selection based encoder-decoder framework which is composed of Local Context Attention (LCA) module, Global Context Module (GCM) and Adaptive Selection Module (ASM). Specifically, LCA modules deliver local context features from encoder layers to decoder layers, enhancing the attention to the hard region which is determined by the prediction map of previous layer. GCM aims to further explore the global context features and send to the decoder layers. ASM is used for adaptive selection and aggregation of context features through channel-wise attention. Our proposed approach is evaluated on the EndoScene and Kvasir-SEG Datasets, and shows outstanding performance compared with other state-of-the-art methods. The code is available at https://github.com/ReaFly/ACSNet.
Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Biomedical image segmentation plays a significant role in computer-aided diagnosis. However, existing CNN based methods rely heavily on massive manual annotations, which are very expensive and require huge human resources. In this work, we adopt a coarse-to-fine strategy and propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. Specifically, we design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting, respectively. In the first phase, only the segmentation branch is used to obtain a relatively rough segmentation result. In the second step, we mask the detected lesion regions on the original image based on the initial segmentation map, and send it together with the original image into the network again to simultaneously perform inpainting and segmentation separately. For labeled data, this process is supervised by the segmentation annotations, and for unlabeled data, it is guided by the inpainting loss of masked lesion regions. Since the two tasks rely on similar feature information, the unlabeled data effectively enhances the representation of the network to the lesion regions and further improves the segmentation performance. Moreover, a gated feature fusion (GFF) module is designed to incorporate the complementary features from the two tasks. Experiments on three medical image segmentation datasets for different tasks including polyp, skin lesion and fundus optic disc segmentation well demonstrate the outstanding performance of our method compared with other semi-supervised approaches. The code is available at https://github.com/ReaFly/SemiMedSeg.
ZScribbleSeg: Zen and the Art of Scribble Supervised Medical Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Curating a large scale fully-annotated dataset can be both labour-intensive and expertise-demanding, especially for medical images. To alleviate this problem, we propose to utilize solely scribble annotations for weakly supervised segmentation. Existing solutions mainly leverage selective losses computed solely on annotated areas and generate pseudo gold standard segmentation by propagating labels to adjacent areas. However, these methods could suffer from the inaccurate and sometimes unrealistic pseudo segmentation due to the insufficient supervision and incomplete shape features. Different from previous efforts, we first investigate the principle of ''good scribble annotations'', which leads to efficient scribble forms via supervision maximization and randomness simulation. Furthermore, we introduce regularization terms to encode the spatial relationship and shape prior, where a new formulation is developed to estimate the mixture ratios of label classes. These ratios are critical in identifying the unlabeled pixels for each class and correcting erroneous predictions, thus the accurate estimation lays the foundation for the incorporation of spatial prior. Finally, we integrate the efficient scribble supervision with the prior into a unified framework, denoted as ZScribbleSeg, and apply the method to multiple scenarios. Leveraging only scribble annotations, ZScribbleSeg set new state-of-the-arts on four segmentation tasks using ACDC, MSCMRseg, MyoPS and PPSS datasets.
Code-based Cryptography in IoT: A HW/SW Co-Design of HQC
Venue: arXiv cs.CR 2023
Year: 2023

Recent advances in quantum computing pose a serious threat on the security of widely used public-key cryptosystems. Thus, new post-quantum cryptographic algorithms have been proposed as part of the associated US NIST process to enable secure, encrypted communication in the age of quantum computing. Many hardware accelerators for structured lattice-based algorithms have already been published to meet the strict power, area and latency requirements of low-power IoT edge devices. However, the security of these algorithms is still uncertain. Currently, many new attacks against the lattice structure are investigated to judge on their security. In contrast, code-based algorithms, which rely on deeply explored security metrics and are appealing candidates in the NIST process, have not yet been investigated to the same depth in the context of IoT due to the computational complexity and memory footprint of state-of-the-art software implementations. In this paper, we present to the best of our knowledge the first HW/SW co-design based implementation of the code-based Hamming Quasi Cyclic Key-Encapsulation Mechanism. We profile and evaluate this algorithm in order to explore the trade-off between software optimizations, tightly coupled hardware acceleration by instruction set extension and modular, loosely coupled accelerators. We provide detailed results on the energy consumption and performance of our design and compare it to existing implementations of lattice- and code-based algorithms. The design was implemented in two technologies: FPGA and ASIC. Our results show that code-based algorithms are valid alternatives in low-power IoT from an implementation perspective.
Hierarchical Dynamic Masks for Visual Explanation of Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

Saliency methods generating visual explanatory maps representing the importance of image pixels for model classification is a popular technique for explaining neural network decisions. Hierarchical dynamic masks (HDM), a novel explanatory maps generation method, is proposed in this paper to enhance the granularity and comprehensiveness of saliency maps. First, we suggest the dynamic masks (DM), which enables multiple small-sized benchmark mask vectors to roughly learn the critical information in the image through an optimization method. Then the benchmark mask vectors guide the learning of large-sized auxiliary mask vectors so that their superimposed mask can accurately learn fine-grained pixel importance information and reduce the sensitivity to adversarial perturbations. In addition, we construct the HDM by concatenating DM modules. These DM modules are used to find and fuse the regions of interest in the remaining neural network classification decisions in the mask image in a learning-based way. Since HDM forces DM to perform importance analysis in different areas, it makes the fused saliency map more comprehensive. The proposed method outperformed previous approaches significantly in terms of recognition and localization capabilities when tested on natural and medical datasets.
Against Algorithmic Exploitation of Human Vulnerabilities
Venue: arXiv cs.AI 2023
Year: 2023

Decisions such as which movie to watch next, which song to listen to, or which product to buy online, are increasingly influenced by recommender systems and user models that incorporate information on users' past behaviours, preferences, and digitally created content. Machine learning models that enable recommendations and that are trained on user data may unintentionally leverage information on human characteristics that are considered vulnerabilities, such as depression, young age, or gambling addiction. The use of algorithmic decisions based on latent vulnerable state representations could be considered manipulative and could have a deteriorating impact on the condition of vulnerable individuals. In this paper, we are concerned with the problem of machine learning models inadvertently modelling vulnerabilities, and want to raise awareness for this issue to be considered in legislation and AI ethics. Hence, we define and describe common vulnerabilities, and illustrate cases where they are likely to play a role in algorithmic decision-making. We propose a set of requirements for methods to detect the potential for vulnerability modelling, detect whether vulnerable groups are treated differently by a model, and detect whether a model has created an internal representation of vulnerability. We conclude that explainable artificial intelligence methods may be necessary for detecting vulnerability exploitation by machine learning-based recommendation systems.
Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly
Venue: arXiv cs.CV 2023
Year: 2023

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.
Improving Inference Performance of Machine Learning with the Divide-and-Conquer Principle
Venue: arXiv cs.LG 2023
Year: 2023

Many popular machine learning models scale poorly when deployed on CPUs. In this paper we explore the reasons why and propose a simple, yet effective approach based on the well-known Divide-and-Conquer Principle to tackle this problem of great practical importance. Given an inference job, instead of using all available computing resources (i.e., CPU cores) for running it, the idea is to break the job into independent parts that can be executed in parallel, each with the number of cores according to its expected computational cost. We implement this idea in the popular OnnxRuntime framework and evaluate its effectiveness with several use cases, including the well-known models for optical character recognition (PaddleOCR) and natural language processing (BERT).
Folding interpretations
Venue: arXiv cs.LO 2023
Year: 2023

We study the polyregular string-to-string functions, which are certain functions of polynomial output size that can be described using automata and logic. We describe a system of combinators that generates exactly these functions. Unlike previous systems, the present system includes an iteration mechanism, namely fold. Although unrestricted fold can define all primitive recursive functions, we identify a type system (inspired by linear logic) that restricts fold so that it defines exactly the polyregular functions. We also present related systems, for quantifier-free functions as well as for linear regular functions on both strings and trees.
Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models
Venue: arXiv cs.CL 2023
Year: 2023

Recent work studies the cognitive capabilities of language models through psychological tests designed for humans. While these studies are helpful for understanding the general capabilities of these models, there is no guarantee that a model possessing sufficient capabilities to pass those tests would actually use those capabilities in performing real-life tasks. In this work, we formulate task-oriented cognitive capabilities, which are human-like cognitive capabilities that language models leverage to perform tasks. These capabilities are (i) the ability to quickly generate good candidate utterances (the search capability) (ii) the ability to predict how a listener interprets those utterances and choose the most appropriate one (the pragmatic capability). We design an evaluation scheme for comparing these capabilities of a language model with those of a human. Applying this scheme to examine various models in a navigation instruction generation problem, we find that their pragmatic capability is severely lacking. This insight leads us to augment them with better models of the listener and obtain a significant boost of 11% in success rate in guiding real humans. Our work advocates for having a principled procedure for aligning language models with humans that involves (i) formulating task-oriented capabilities, (ii) devising a method to quantify their deficiency, and (iii) iteratively improving them.
Computing m-Eternal Domination Number of Cactus Graphs in Linear Time
Venue: arXiv cs.DS 2023
Year: 2023

In m-eternal domination attacker and defender play on a graph. Initially, the defender places guards on vertices. In each round, the attacker chooses a vertex to attack. Then, the defender can move each guard to a neighboring vertex and must move a guard to the attacked vertex. The m-eternal domination number is the minimum number of guards such that the graph can be defended indefinitely. In this paper, we study the m-eternal domination number of cactus graphs. We consider two variants of the m-eternal domination number: one allows multiple guards to occupy a single vertex, the second variant requires the guards to occupy distinct vertices. We develop several tools for obtaining lower and upper bounds on these problems and we use them to obtain an algorithm which computes the minimum number of required guards of cactus graphs for both variants of the problem.
Accidental Light Probes
Venue: arXiv cs.CV 2023
Year: 2023

Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs) -- common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We propose a physically-based approach to model ALPs and estimate lighting from their appearances in single images. The main idea is to model the appearance of ALPs by photogrammetrically principled shading and to invert this process via differentiable rendering to recover incidental illumination. We demonstrate that we can put an ALP into a scene to allow high-fidelity lighting estimation. Our model can also recover lighting for existing images that happen to contain an ALP.
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning
Venue: arXiv cs.CV 2023
Year: 2023

The state of neural network pruning has been noticed to be unclear and even confusing for a while, largely due to "a lack of standardized benchmarks and metrics" [3]. To standardize benchmarks, first, we need to answer: what kind of comparison setup is considered fair? This basic yet crucial question has barely been clarified in the community, unfortunately. Meanwhile, we observe several papers have used (severely) sub-optimal hyper-parameters in pruning experiments, while the reason behind them is also elusive. These sub-optimal hyper-parameters further exacerbate the distorted benchmarks, rendering the state of neural network pruning even more obscure. Two mysteries in pruning represent such a confusing status: the performance-boosting effect of a larger finetuning learning rate, and the no-value argument of inheriting pretrained weights in filter pruning. In this work, we attempt to explain the confusing state of network pruning by demystifying the two mysteries. Specifically, (1) we first clarify the fairness principle in pruning experiments and summarize the widely-used comparison setups; (2) then we unveil the two pruning mysteries and point out the central role of network trainability, which has not been well recognized so far; (3) finally, we conclude the paper and give some concrete suggestions regarding how to calibrate the pruning benchmarks in the future. Code: https://github.com/mingsun-tse/why-the-state-of-pruning-so-confusing.
Blind Judgement: Agent-Based Supreme Court Modelling With GPT
Venue: arXiv cs.CL 2023
Year: 2023

We present a novel Transformer-based multi-agent system for simulating the judicial rulings of the 2010-2016 Supreme Court of the United States. We train nine separate models with the respective authored opinions of each supreme justice active ca. 2015 and test the resulting system on 96 real-world cases. We find our system predicts the decisions of the real-world Supreme Court with better-than-random accuracy. We further find a correlation between model accuracy with respect to individual justices and their alignment between legal conservatism & liberalism. Our methods and results hold significance for researchers interested in using language models to simulate politically-charged discourse between multiple agents.
Experimental System Identification and Disturbance Observer-based Control for a Monolithic $Zθ_{x}θ_{y}$ Precision Positioning System
Venue: arXiv cs.RO 2023
Year: 2023

A compliant parallel micromanipulator is a mechanism in which the moving platform is connected to the base through a number of flexural components. Utilizing parallel-kinematics configurations and flexure joints, the monolithic micromanipulators can achieve extremely high motion resolution and accuracy. In this work, the focus was towards the experimental evaluation of a 3-DOF ($Zθ_{x}θ_{y}$) monolithic flexure-based piezo-driven micromanipulator for precise out-of-plane micro/nano positioning applications. The monolithic structure avoids the deficiencies of non-monolithic designs such as backlash, wear, friction, and improves the performance of micromanipulator in terms of high resolution, accuracy, and repeatability. A computational study was conducted to investigate and obtain the inverse kinematics of the proposed micromanipulator. As a result of computational analysis, the developed prototype of the micromanipulator is capable of executing large motion range of $\pm$238.5$μ$m $\times$ $\pm$4830.5$μ$rad $\times$ $\pm$5486.2$μ$rad. Finally, a sliding mode control strategy with nonlinear disturbance observer (SMC-NDO) was designed and implemented on the proposed micromanipulator to obtain system behaviors during experiments. The obtained results from different experimental tests validated the fine micromanipulator's positioning ability and the efficiency of the control methodology for precise micro/nano manipulation applications. The proposed micromanipulator achieved very fine spatial and rotational resolutions of $\pm$4nm, $\pm$250nrad, and $\pm$230nrad throughout its workspace.
Do the Defect Prediction Models Really Work?
Venue: arXiv cs.SE 2023
Year: 2023

You may develop a potential prediction model, but how can I trust your model that it will benefit my software?. Using a software defect prediction (SDP) model as a tool, we address this fundamental problem in machine learning research. This is a preliminary work targeted at providing an analysis of the developed binary SDP model in real-time working environments.
HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

An efficient solution to the large-scale recommender system is to represent users and items as binary hash codes in the Hamming space. Towards this end, existing methods tend to code users by modeling their Hamming similarities with the items they historically interact with, which are termed as the first-order similarities in this work. Despite their efficiency, these methods suffer from the suboptimal representative capacity, since they forgo the correlation established by connecting multiple first-order similarities, i.e., the relation among the indirect instances, which could be defined as the high-order similarity. To tackle this drawback, we propose to model both the first- and the high-order similarities in the Hamming space through the user-item bipartite graph. Therefore, we develop a novel learning to hash framework, namely Hamming Spatial Graph Convolutional Networks (HS-GCN), which explicitly models the Hamming similarity and embeds it into the codes of users and items. Extensive experiments on three public benchmark datasets demonstrate that our proposed model significantly outperforms several state-of-the-art hashing models, and obtains performance comparable with the real-valued recommendation models.