OpenRaiser/Intern-Atlas huggingface dataset:
STFAR: Improving Object Detection Robustness at Test-Time by Self-Training with Feature Alignment Regularization
Venue: arXiv cs.CV 2023
Year: 2023

Domain adaptation helps generalizing object detection models to target domain data with distribution shift. It is often achieved by adapting with access to the whole target domain data. In a more realistic scenario, target distribution is often unpredictable until inference stage. This motivates us to explore adapting an object detection model at test-time, a.k.a. test-time adaptation (TTA). In this work, we approach test-time adaptive object detection (TTAOD) from two perspective. First, we adopt a self-training paradigm to generate pseudo labeled objects with an exponential moving average model. The pseudo labels are further used to supervise adapting source domain model. As self-training is prone to incorrect pseudo labels, we further incorporate aligning feature distributions at two output levels as regularizations to self-training. To validate the performance on TTAOD, we create benchmarks based on three standard object detection datasets and adapt generic TTA methods to object detection task. Extensive evaluations suggest our proposed method sets the state-of-the-art on test-time adaptive object detection task.
DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Training
Venue: arXiv cs.AR 2023
Year: 2023

On-chip DNN inference and training at the Extreme-Edge (TinyML) impose strict latency, throughput, accuracy and flexibility requirements. Heterogeneous clusters are promising solutions to meet the challenge, combining the flexibility of DSP-enhanced cores with the performance and energy boost of dedicated accelerators. We present DARKSIDE, a System-on-Chip with a heterogeneous cluster of 8 RISC-V cores enhanced with 2-b to 32-b mixed-precision integer arithmetic. To boost performance and efficiency on key compute-intensive Deep Neural Network (DNN) kernels, the cluster is enriched with three digital accelerators: a specialized engine for low-data-reuse depthwise convolution kernels (up to 30 MAC/cycle); a minimal overhead datamover to marshal 1-b to 32-b data on-the-fly; a 16-b floating point Tensor Product Engine (TPE) for tiled matrix-multiplication acceleration. DARKSIDE is implemented in 65nm CMOS technology. The cluster achieves a peak integer performance of 65 GOPS and a peak efficiency of 835 GOPS/W when working on 2-b integer DNN kernels. When targeting floating-point tensor operations, the TPE provides up to 18.2 GFLOPS of performance or 300 GFLOPS/W of efficiency - enough to enable on-chip floating-point training at competitive speed coupled with ultra-low power quantized inference.
MARTSIA: Enabling Data Confidentiality for Blockchain-based Process Execution
Venue: arXiv cs.CR 2023
Year: 2023

Multi-party business processes rely on the collaboration of various players in a decentralized setting. Blockchain technology can facilitate the automation of these processes, even in cases where trust among participants is limited. Transactions are stored in a ledger, a replica of which is retained by every node of the blockchain network. The operations saved thereby are thus publicly accessible. While this enhances transparency, reliability, and persistence, it hinders the utilization of public blockchains for process automation as it violates typical confidentiality requirements in corporate settings. In this paper, we propose MARTSIA: A Multi-Authority Approach to Transaction Systems for Interoperating Applications. MARTSIA enables precise control over process data at the level of message parts. Based on Multi-Authority Attribute-Based Encryption (MA-ABE), MARTSIA realizes a number of desirable properties, including confidentiality, transparency, and auditability. We implemented our approach in proof-of-concept prototypes, with which we conduct a case study in the area of supply chain management. Also, we show the integration of MARTSIA with a state-of-the-art blockchain-based process execution engine to secure the data flow.
Unsupervised crack detection on complex stone masonry surfaces
Venue: arXiv cs.CV 2023
Year: 2023

Computer vision for detecting building pathologies has interested researchers for quite some time. Vision-based crack detection is a non-destructive assessment technique, which can be useful especially for Cultural Heritage (CH) where strict regulations apply and, even simple, interventions are not permitted. Recently, shallow and deep machine learning architectures applied on various types of imagery are gaining ground. In this article a crack detection methodology for stone masonry walls is presented. In the proposed approach, crack detection is approached as an unsupervised anomaly detection problem on RGB (Red Green Blue) image patches. Towards this direction, some of the most popular state of the art CNN (Convolutional Neural Network) architectures are deployed and modified to binary classify the images or image patches by predicting a specific class for the tested imagery; 'Crack' or 'No crack', and detect and localize those cracks on the RGB imagery with high accuracy. Testing of the model was performed on various test sites and random images retrieved from the internet and collected by the authors and results suggested the high performance of specific networks compared to the rest, considering also the small numbers of epochs required for training. Those results met the accuracy delivered by more complex and computationally heavy approaches, requiring a large amount of data for training. Source code is available on GitHub https://github.com/pagraf/Crack-detection while datasets are available on Zenodo https://doi.org/10.5281/zenodo.6516913 .
Augmented Collective Intelligence in Collaborative Ideation: Agenda and Challenges
Venue: arXiv cs.CY 2023
Year: 2023

AI systems may be better thought of as peers than as tools. This paper explores applications of augmented collective intelligence (ACI) beneficial to collaborative ideation. Design considerations are offered for an experiment that evaluates the performance of hybrid human- AI collectives. The investigation described combines humans and large language models (LLMs) to ideate on increasingly complex topics. A promising real-time collection tool called Polis is examined to facilitate ACI, including case studies from citizen engagement projects in Taiwan and Bowling Green, Kentucky. The authors discuss three challenges to consider when designing an ACI experiment: topic selection, participant selection, and evaluation of results. The paper concludes that researchers should address these challenges to conduct empirical studies of ACI in collaborative ideation.
Blockchain-based Immutable Evidence and Decentralized Loss Adjustment for Autonomous Vehicle Accidents in Insurance
Venue: arXiv cs.CR 2023
Year: 2023

In case of an accident between two autonomous vehicles equipped with emerging technologies, how do we apportion liability among the various players? A special liability regime has not even yet been established for damages that may arise due to the accidents of autonomous vehicles. Would the immutable, time-stamped sensor records of vehicles on distributed ledger help define the intertwined relations of liability subjects right through the accident? What if the synthetic media created through deepfake gets involved in the insurance claims? While integrating AI-powered anomaly or deepfake detection into automated insurance claims processing helps to prevent insurance fraud, it is only a matter of time before deepfake becomes nearly undetectable even to elaborate forensic tools. This paper proposes a blockchain-based insurtech decentralized application to check the authenticity and provenance of the accident footage and also to decentralize the loss-adjusting process through a hybrid of decentralized and centralized databases using smart contracts.
Shirakami: A Hybrid Concurrency Control Protocol for Tsurugi Relational Database System
Venue: arXiv cs.DB 2023
Year: 2023

Modern real-world transactional workloads such as bills of materials or telecommunication billing need to process both short transactions and long transactions. Recent concurrency control protocols do not cope with such workloads since they assume only classical workloads (i.e., YCSB and TPC-C) that have relatively short transactions. To this end, we proposed a new concurrency control protocol Shirakami. Shirakami has two sub-protocols. Shirakami-LTX protocol is for long transactions based on multiversion concurrency control and Shirakami-OCC protocol is for short transactions based on Silo. Shirakami naturally integrates them with write preservation method and epoch-based synchronization. Shirakami is a module in Tsurugi system, which is a production-purpose relational database system.
MAGNNETO: A Graph Neural Network-based Multi-Agent system for Traffic Engineering
Venue: arXiv cs.NI 2023
Year: 2023

Current trends in networking propose the use of Machine Learning (ML) for a wide variety of network optimization tasks. As such, many efforts have been made to produce ML-based solutions for Traffic Engineering (TE), which is a fundamental problem in ISP networks. Nowadays, state-of-the-art TE optimizers rely on traditional optimization techniques, such as Local search, Constraint Programming, or Linear programming. In this paper, we present MAGNNETO, a distributed ML-based framework that leverages Multi-Agent Reinforcement Learning and Graph Neural Networks for distributed TE optimization. MAGNNETO deploys a set of agents across the network that learn and communicate in a distributed fashion via message exchanges between neighboring agents. Particularly, we apply this framework to optimize link weights in OSPF, with the goal of minimizing network congestion. In our evaluation, we compare MAGNNETO against several state-of-the-art TE optimizers in more than 75 topologies (up to 153 nodes and 354 links), including realistic traffic loads. Our experimental results show that, thanks to its distributed nature, MAGNNETO achieves comparable performance to state-of-the-art TE optimizers with significantly lower execution times. Moreover, our ML-based solution demonstrates a strong generalization capability to successfully operate in new networks unseen during training.
Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection
Venue: arXiv cs.CV 2023
Year: 2023

Despite the fact that DeepFake forgery detection algorithms have achieved impressive performance on known manipulations, they often face disastrous performance degradation when generalized to an unseen manipulation. Some recent works show improvement in generalization but rely on features fragile to image distortions such as compression. To this end, we propose Diff-ID, a concise and effective approach that explains and measures the identity loss induced by facial manipulations. When testing on an image of a specific person, Diff-ID utilizes an authentic image of that person as a reference and aligns them to the same identity-insensitive attribute feature space by applying a face-swapping generator. We then visualize the identity loss between the test and the reference image from the image differences of the aligned pairs, and design a custom metric to quantify the identity loss. The metric is then proved to be effective in distinguishing the forgery images from the real ones. Extensive experiments show that our approach achieves high detection performance on DeepFake images and state-of-the-art generalization ability to unknown forgery methods, while also being robust to image distortions.
SemHint-MD: Learning from Noisy Semantic Labels for Self-Supervised Monocular Depth Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Without ground truth supervision, self-supervised depth estimation can be trapped in a local minimum due to the gradient-locality issue of the photometric loss. In this paper, we present a framework to enhance depth by leveraging semantic segmentation to guide the network to jump out of the local minimum. Prior works have proposed to share encoders between these two tasks or explicitly align them based on priors like the consistency between edges in the depth and segmentation maps. Yet, these methods usually require ground truth or high-quality pseudo labels, which may not be easily accessible in real-world applications. In contrast, we investigate self-supervised depth estimation along with a segmentation branch that is supervised with noisy labels provided by models pre-trained with limited data. We extend parameter sharing from the encoder to the decoder and study the influence of different numbers of shared decoder parameters on model performance. Also, we propose to use cross-task information to refine current depth and segmentation predictions to generate pseudo-depth and semantic labels for training. The advantages of the proposed method are demonstrated through extensive experiments on the KITTI benchmark and a downstream task for endoscopic tissue deformation tracking.
The challenge of redundancy on multi-agent value factorisation
Venue: arXiv cs.AI 2023
Year: 2023

In the field of cooperative multi-agent reinforcement learning (MARL), the standard paradigm is the use of centralised training and decentralised execution where a central critic conditions the policies of the cooperative agents based on a central state. It has been shown, that in cases with large numbers of redundant agents these methods become less effective. In a more general case, there is likely to be a larger number of agents in an environment than is required to solve the task. These redundant agents reduce performance by enlarging the dimensionality of both the state space and and increasing the size of the joint policy used to solve the environment. We propose leveraging layerwise relevance propagation (LRP) to instead separate the learning of the joint value function and generation of local reward signals and create a new MARL algorithm: relevance decomposition network (RDN). We find that although the performance of both baselines VDN and Qmix degrades with the number of redundant agents, RDN is unaffected.
What Makes for Effective Few-shot Point Cloud Classification?
Venue: arXiv cs.CV 2023
Year: 2023

Due to the emergence of powerful computing resources and large-scale annotated datasets, deep learning has seen wide applications in our daily life. However, most current methods require extensive data collection and retraining when dealing with novel classes never seen before. On the other hand, we humans can quickly recognize new classes by looking at a few samples, which motivates the recent popularity of few-shot learning (FSL) in machine learning communities. Most current FSL approaches work on 2D image domain, however, its implication in 3D perception is relatively under-explored. Not only needs to recognize the unseen examples as in 2D domain, 3D few-shot learning is more challenging with unordered structures, high intra-class variances, and subtle inter-class differences. Moreover, different architectures and learning algorithms make it difficult to study the effectiveness of existing 2D methods when migrating to the 3D domain. In this work, for the first time, we perform systematic and extensive studies of recent 2D FSL and 3D backbone networks for benchmarking few-shot point cloud classification, and we suggest a strong baseline and learning architectures for 3D FSL. Then, we propose a novel plug-and-play component called Cross-Instance Adaptation (CIA) module, to address the high intra-class variances and subtle inter-class differences issues, which can be easily inserted into current baselines with significant performance improvement. Extensive experiments on two newly introduced benchmark datasets, ModelNet40-FS and ShapeNet70-FS, demonstrate the superiority of our proposed network for 3D FSL.
Accelerating exploration and representation learning with offline pre-training
Venue: arXiv cs.LG 2023
Year: 2023

Sequential decision-making agents struggle with long horizon tasks, since solving them requires multi-step reasoning. Most reinforcement learning (RL) algorithms address this challenge by improved credit assignment, introducing memory capability, altering the agent's intrinsic motivation (i.e. exploration) or its worldview (i.e. knowledge representation). Many of these components could be learned from offline data. In this work, we follow the hypothesis that exploration and representation learning can be improved by separately learning two different models from a single offline dataset. We show that learning a state representation using noise-contrastive estimation and a model of auxiliary reward separately from a single collection of human demonstrations can significantly improve the sample efficiency on the challenging NetHack benchmark. We also ablate various components of our experimental setting and highlight crucial insights.
A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners
Venue: arXiv cs.SE 2023
Year: 2023

Incorporating machine learning (ML) components into software products raises new software-engineering challenges and exacerbates existing challenges. Many researchers have invested significant effort in understanding the challenges of industry practitioners working on building products with ML components, through interviews and surveys with practitioners. With the intention to aggregate and present their collective findings, we conduct a meta-summary study: We collect 50 relevant papers that together interacted with over 4758 practitioners using guidelines for systematic literature reviews. We then collected, grouped, and organized the over 500 mentions of challenges within those papers. We highlight the most commonly reported challenges and hope this meta-summary will be a useful resource for the research community to prioritize research and education in this field.
On the Price of Transparency: A Comparison between Overt Persuasion and Covert Signaling
Venue: arXiv cs.GT 2023
Year: 2023

Transparency of information disclosure has always been considered an instrumental component of effective governance, accountability, and ethical behavior in any organization or system. However, a natural question follows: \emph{what is the cost or benefit of being transparent}, as one may suspect that transparency imposes additional constraints on the information structure, decreasing the maneuverability of the information provider. This work proposes and quantitatively investigates the \emph{price of transparency} (PoT) in strategic information disclosure by comparing the perfect Bayesian equilibrium payoffs under two representative information structures: overt persuasion and covert signaling models. PoT is defined as the ratio between the payoff outcomes in covert and overt interactions. As the main contribution, this work develops a bilevel-bilinear programming approach, called $Z$-programming, to solve for non-degenerate perfect Bayesian equilibria of dynamic incomplete information games with finite states and actions. Using $Z$-programming, we show that it is always in the information provider's interest to choose the transparent information structure, as $0\leq \textrm{PoT}\leq 1$. The upper bound is attainable for any strictly Bayesian-posterior competitive games, of which zero-sum games are a particular case. For continuous games, the PoT, still upper-bounded by $1$, can be arbitrarily close to $0$, indicating the tightness of the lower bound. This tight lower bound suggests that the lack of transparency can result in significant loss for the provider. We corroborate our findings using quadratic games and numerical examples.
End-to-end deep learning-based framework for path planning and collision checking: bin picking application
Venue: arXiv cs.RO 2023
Year: 2023

Real-time and efficient path planning is critical for all robotic systems. In particular, it is of greater importance for industrial robots since the overall planning and execution time directly impact the cycle time and automation economics in production lines. While the problem may not be complex in static environments, classical approaches are inefficient in high-dimensional environments in terms of planning time and optimality. Collision checking poses another challenge in obtaining a real-time solution for path planning in complex environments. To address these issues, we propose an end-to-end learning-based framework viz., Path Planning and Collision checking Network (PPCNet). The PPCNet generates the path by computing waypoints sequentially using two networks: the first network generates a waypoint, and the second one determines whether the waypoint is on a collision-free segment of the path. The end-to-end training process is based on imitation learning that uses data aggregation from the experience of an expert planner to train the two networks, simultaneously. We utilize two approaches for training a network that efficiently approximates the exact geometrical collision checking function. Finally, the PPCNet is evaluated in two different simulation environments and a practical implementation on a robotic arm for a bin-picking application. Compared to the state-of-the-art path planning methods, our results show significant improvement in performance by greatly reducing the planning time with comparable success rates and path lengths.
On the Relationships between Graph Neural Networks for the Simulation of Physical Systems and Classical Numerical Methods
Venue: arXiv cs.LG 2023
Year: 2023

Recent developments in Machine Learning approaches for modelling physical systems have begun to mirror the past development of numerical methods in the computational sciences. In this survey, we begin by providing an example of this with the parallels between the development trajectories of graph neural network acceleration for physical simulations and particle-based approaches. We then give an overview of simulation approaches, which have not yet found their way into state-of-the-art Machine Learning methods and hold the potential to make Machine Learning approaches more accurate and more efficient. We conclude by presenting an outlook on the potential of these approaches for making Machine Learning models for science more efficient.
Soft-Bellman Equilibrium in Affine Markov Games: Forward Solutions and Inverse Learning
Venue: arXiv cs.GT 2023
Year: 2023

Markov games model interactions among multiple players in a stochastic, dynamic environment. Each player in a Markov game maximizes its expected total discounted reward, which depends upon the policies of the other players. We formulate a class of Markov games, termed affine Markov games, where an affine reward function couples the players' actions. We introduce a novel solution concept, the soft-Bellman equilibrium, where each player is boundedly rational and chooses a soft-Bellman policy rather than a purely rational policy as in the well-known Nash equilibrium concept. We provide conditions for the existence and uniqueness of the soft-Bellman equilibrium and propose a nonlinear least-squares algorithm to compute such an equilibrium in the forward problem. We then solve the inverse game problem of inferring the players' reward parameters from observed state-action trajectories via a projected-gradient algorithm. Experiments in a predator-prey OpenAI Gym environment show that the reward parameters inferred by the proposed algorithm outperform those inferred by a baseline algorithm: they reduce the Kullback-Leibler divergence between the equilibrium policies and observed policies by at least two orders of magnitude.