OpenRaiser/Intern-Atlas huggingface dataset:
SILT: Self-supervised Lighting Transfer Using Implicit Image Decomposition
Venue: arXiv cs.CV 2022
Year: 2022

We present SILT, a Self-supervised Implicit Lighting Transfer method. Unlike previous research on scene relighting, we do not seek to apply arbitrary new lighting configurations to a given scene. Instead, we wish to transfer the lighting style from a database of other scenes, to provide a uniform lighting style regardless of the input. The solution operates as a two-branch network that first aims to map input images of any arbitrary lighting style to a unified domain, with extra guidance achieved through implicit image decomposition. We then remap this unified input domain using a discriminator that is presented with the generated outputs and the style reference, i.e. images of the desired illumination conditions. Our method is shown to outperform supervised relighting solutions across two different datasets without requiring lighting supervision.
What Would Jiminy Cricket Do? Towards Agents That Behave Morally
Venue: arXiv cs.LG 2022
Year: 2022

When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong. By contrast, artificial agents are currently not endowed with a moral sense. As a consequence, they may learn to behave immorally when trained on environments that ignore moral concerns, such as violent video games. With the advent of generally capable agents that pretrain on many environments, it will become necessary to mitigate inherited biases from environments that teach immoral behavior. To facilitate the development of agents that avoid causing wanton harm, we introduce Jiminy Cricket, an environment suite of 25 text-based adventure games with thousands of diverse, morally salient scenarios. By annotating every possible game state, the Jiminy Cricket environments robustly evaluate whether agents can act morally while maximizing reward. Using models with commonsense moral knowledge, we create an elementary artificial conscience that assesses and guides agents. In extensive experiments, we find that the artificial conscience approach can steer agents towards moral behavior without sacrificing performance.
Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications
Venue: arXiv cs.LG 2022
Year: 2022

Continual learning approaches help deep neural network models adapt and learn incrementally by trying to solve catastrophic forgetting. However, whether these existing approaches, applied traditionally to image-based tasks, work with the same efficacy to the sequential time series data generated by mobile or embedded sensing systems remains an unanswered question. To address this void, we conduct the first comprehensive empirical study that quantifies the performance of three predominant continual learning schemes (i.e., regularization, replay, and replay with examples) on six datasets from three mobile and embedded sensing applications in a range of scenarios having different learning complexities. More specifically, we implement an end-to-end continual learning framework on edge devices. Then we investigate the generalizability, trade-offs between performance, storage, computational costs, and memory footprint of different continual learning methods. Our findings suggest that replay with exemplars-based schemes such as iCaRL has the best performance trade-offs, even in complex scenarios, at the expense of some storage space (few MBs) for training examples (1% to 5%). We also demonstrate for the first time that it is feasible and practical to run continual learning on-device with a limited memory budget. In particular, the latency on two types of mobile and embedded devices suggests that both incremental learning time (few seconds - 4 minutes) and training time (1 - 75 minutes) across datasets are acceptable, as training could happen on the device when the embedded device is charging thereby ensuring complete data privacy. Finally, we present some guidelines for practitioners who want to apply a continual learning paradigm for mobile sensing tasks.
Polynomial Integrality Gap of Flow LP for Directed Steiner Tree
Venue: arXiv cs.DS 2022
Year: 2022

In the Directed Steiner Tree (DST) problem, we are given a directed graph $G=(V,E)$ on $n$ vertices with edge-costs $c \in \mathbb{R}_{\geq 0}^E$, a root vertex $r \in V$, and a set $K \subseteq V \setminus \{r\}$ of $k$ terminals. The goal is to find a minimum-cost subgraph of $G$ that contains a path from $r$ to every terminal $t \in K$. DST has been a notorious problem for decades as there is a large gap between the best-known polynomial-time approximation ratio of $O(k^ε)$ for any constant $ε> 0$, and the best quasi-polynomial-time approximation ratio of $O\left(\frac{\log^2 k}{\log \log k}\right)$. Towards understanding this gap, we study the integrality gap of the standard flow LP relaxation for the problem. We show that the LP has an integrality gap of $Ω(n^{0.0418})$. Previously, the integrality gap of the LP is only known to be $Ω\left(\frac{\log^2n}{\log\log n}\right)$ [Halperin~et~al., SODA'03 \& SIAM J.~Comput.] and $Ω(\sqrt{k})$ [Zosin-Khuller, SODA'02] in some instance with $\sqrt{k}=O\left(\frac{\log n}{\log \log n}\right)$. Our result gives the first known lower bound on the integrality gap of this standard LP that is polynomial in $n$, the number of vertices. Consequently, we rule out the possibility of developing a poly-logarithmic approximation algorithm for the problem based on the flow LP relaxation.
Defining Blockchain Governance Principles: A Comprehensive Framework
Venue: arXiv cs.SE 2022
Year: 2022

Blockchain eliminates the need for trusted third-party intermediaries in business by enabling decentralised architecture design in software applications. However, the vulnerabilities in on-chain autonomous decision-makings and cumbersome off-chain coordination lead to serious concerns about blockchain's ability to behave in a trustworthy and efficient way. Blockchain governance has received considerable attention to support the decision-making process during the use and evolution of blockchain. Nevertheless, the conventional governance frameworks do not apply to blockchain due to its distributed architecture and decentralised decision process. These inherent features lead to the absence of a clear source of authority in blockchain ecosystem. Currently, there is a lack of systematic guidance on the governance of blockchain. Therefore, in this paper, we present a comprehensive blockchain governance framework, which elucidates an integrated view of the degree of decentralisation, decision rights, incentives, accountability, ecosystem, and legal and ethical responsibilities. The above aspects are formulated as six high-level principles for blockchain governance. We demonstrate a qualitative analysis of the proposed framework, including case studies on five extant blockchain platforms, and comparison with existing blockchain governance frameworks. The results show that our proposed framework is feasible and applicable in a real-world context.
Instant Response Few-shot Object Detection with Meta Strategy and Explicit Localization Inference
Venue: arXiv cs.CV 2022
Year: 2022

Aiming at recognizing and localizing the object of novel categories by a few reference samples, few-shot object detection (FSOD) is a quite challenging task. Previous works often depend on the fine-tuning process to transfer their model to the novel category and rarely consider the defect of fine-tuning, resulting in many application drawbacks. For example, these methods are far from satisfying in the episode-changeable scenarios due to excessive fine-tuning times, and their performance on low-quality (e.g., low-shot and class-incomplete) support sets degrades severely. To this end, this paper proposes an instant response few-shot object detector (IR-FSOD) that can accurately and directly detect the objects of novel categories without the fine-tuning process. To accomplish the objective, we carefully analyze the defects of individual modules in the Faster R-CNN framework under the FSOD setting and then extend it to IR-FSOD by improving these defects. Specifically, we first propose two simple but effective meta-strategies for the box classifier and RPN module to enable the object detection of novel categories with instant response. Then, we introduce two explicit inferences into the localization module to alleviate its over-fitting to the base categories, including explicit localization score and semi-explicit box regression. Extensive experiments show that the IR-FSOD framework not only achieves few-shot object detection with the instant response but also reaches state-of-the-art performance in precision and recall under various FSOD settings.
A Normalized Gaussian Wasserstein Distance for Tiny Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Detecting tiny objects is a very challenging problem since a tiny object only contains a few pixels in size. We demonstrate that state-of-the-art detectors do not produce satisfactory results on tiny objects due to the lack of appearance information. Our key observation is that Intersection over Union (IoU) based metrics such as IoU itself and its extensions are very sensitive to the location deviation of the tiny objects, and drastically deteriorate the detection performance when used in anchor-based detectors. To alleviate this, we propose a new evaluation metric using Wasserstein distance for tiny object detection. Specifically, we first model the bounding boxes as 2D Gaussian distributions and then propose a new metric dubbed Normalized Wasserstein Distance (NWD) to compute the similarity between them by their corresponding Gaussian distributions. The proposed NWD metric can be easily embedded into the assignment, non-maximum suppression, and loss function of any anchor-based detector to replace the commonly used IoU metric. We evaluate our metric on a new dataset for tiny object detection (AI-TOD) in which the average object size is much smaller than existing object detection datasets. Extensive experiments show that, when equipped with NWD metric, our approach yields performance that is 6.7 AP points higher than a standard fine-tuning baseline, and 6.0 AP points higher than state-of-the-art competitors. Codes are available at: https://github.com/jwwangchn/NWD.
MarS-FL: Enabling Competitors to Collaborate in Federated Learning
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) is rapidly gaining popularity and enables multiple data owners ({\em a.k.a.} FL participants) to collaboratively train machine learning models in a privacy-preserving way. A key unaddressed scenario is that these FL participants are in a competitive market, where market shares represent their competitiveness. Although they are interested to enhance the performance of their respective models through FL, market leaders (who are often data owners who can contribute significantly to building high performance FL models) want to avoid losing their market shares by enhancing their competitors' models. Currently, there is no modeling tool to analyze such scenarios and support informed decision-making. In this paper, we bridge this gap by proposing the \underline{mar}ket \underline{s}hare-based decision support framework for participation in \underline{FL} (MarS-FL). We introduce {\em two notions of $δ$-stable market} and {\em friendliness} to measure the viability of FL and the market acceptability of FL. The FL participants' behaviours can then be predicted using game theoretic tools (i.e., their optimal strategies concerning participation in FL). If the market $δ$-stability is achievable, the final model performance improvement of each FL-PT shall be bounded, which relates to the market conditions of FL applications. We provide tight bounds and quantify the friendliness, $κ$, of given market conditions to FL. Experimental results show the viability of FL in a wide range of market conditions. Our results are useful for identifying the market conditions under which collaborative FL model training is viable among competitors, and the requirements that have to be imposed while applying FL under these conditions.
Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
Venue: arXiv cs.AI 2022
Year: 2022

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues.
Automating Control of Overestimation Bias for Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Overestimation bias control techniques are used by the majority of high-performing off-policy reinforcement learning algorithms. However, most of these techniques rely on pre-defined bias correction policies that are either not flexible enough or require environment-specific tuning of hyperparameters. In this work, we present a general data-driven approach for the automatic selection of bias control hyperparameters. We demonstrate its effectiveness on three algorithms: Truncated Quantile Critics, Weighted Delayed DDPG, and Maxmin Q-learning. The proposed technique eliminates the need for an extensive hyperparameter search. We show that it leads to a significant reduction of the actual number of interactions while preserving the performance.
AQP: An Open Modular Python Platform for Objective Speech and Audio Quality Metrics
Venue: arXiv cs.SD 2022
Year: 2022

Audio quality assessment has been widely researched in the signal processing area. Full-reference objective metrics (e.g., POLQA, ViSQOL) have been developed to estimate the audio quality relying only on human rating experiments. To evaluate the audio quality of novel audio processing techniques, researchers constantly need to compare objective quality metrics. Testing different implementations of the same metric and evaluating new datasets are fundamental and ongoing iterative activities. In this paper, we present AQP - an open-source, node-based, light-weight Python pipeline for audio quality assessment. AQP allows researchers to test and compare objective quality metrics helping to improve robustness, reproducibility and development speed. We introduce the platform, explain the motivations, and illustrate with examples how, using AQP, objective quality metrics can be (i) compared and benchmarked; (ii) prototyped and adapted in a modular fashion; (iii) visualised and checked for errors. The code has been shared on GitHub to encourage adoption and contributions from the community.
Subsampling Spectral Clustering for Large-Scale Social Networks
Venue: arXiv cs.SI 2022
Year: 2022

Online social network platforms such as Twitter and Sina Weibo have been extremely popular over the past 20 years. Identifying the network community of a social platform is essential to exploring and understanding the users' interests. However, the rapid development of science and technology has generated large amounts of social network data, creating great computational challenges for community detection in large-scale social networks. Here, we propose a novel subsampling spectral clustering algorithm to identify community structures in large-scale social networks with limited computing resources. More precisely, spectral clustering is conducted using only the information of a small subsample of the network nodes, resulting in a huge reduction in computational time. As a result, for large-scale datasets, the method can be realized even using a personal computer. Specifically, we introduce two different sampling techniques, namely simple random subsampling and degree corrected subsampling. The methodology is applied to the dataset collected from Sina Weibo, which is one of the largest Twitter-type social network platforms in China. Our method can very effectively identify the community structure of registered users. This community structure information can be applied to help Sina Weibo promote advertisements to target users and increase user activity.
Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Venue: arXiv cs.CV 2022
Year: 2022

Bounding box (bbox) regression is a fundamental task in computer vision. So far, the most commonly used loss functions for bbox regression are the Intersection over Union (IoU) loss and its variants. In this paper, we generalize existing IoU-based losses to a new family of power IoU losses that have a power IoU term and an additional power regularization term with a single power parameter $α$. We call this new family of losses the $α$-IoU losses and analyze properties such as order preservingness and loss/gradient reweighting. Experiments on multiple object detection benchmarks and models demonstrate that $α$-IoU losses, 1) can surpass existing IoU-based losses by a noticeable performance margin; 2) offer detectors more flexibility in achieving different levels of bbox regression accuracy by modulating $α$; and 3) are more robust to small datasets and noisy bboxes.
A Scalable Architecture for Electronic Payments
Venue: arXiv cs.CY 2022
Year: 2022

We present a scalable architecture for electronic retail payments via central bank digital currency and offer a solution to the perceived conflict between robust regulatory oversight and consumer affordances such as privacy and control. Our architecture combines existing work in payment systems and digital currency with a new approach to digital asset design for managing unforgeable, stateful, and oblivious assets without relying on either a central authority or a monolithic consensus system. Regulated financial institutions have a role in every transaction, and the consumer affordances are achieved through the use of non-custodial wallets that unlink the sender from the recipient in the transaction channel. This approach is fully compatible with the existing two-tiered banking system and can complement and extend the roles of existing money services businesses and asset custodians.
A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

The field of computational pathology has witnessed great advancements since deep neural networks have been widely applied. These networks usually require large numbers of annotated data to train vast parameters. However, it takes significant effort to annotate a large histopathology dataset. We introduce a light-weight and interpretable model for nuclei detection and weakly-supervised segmentation. It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset. Besides, it is a generative compositional model that first locates parts of nucleus, then learns the spatial correlation of the parts to further locate the nucleus. This process brings interpretability in its prediction. Empirical results on an in-house dataset show that in detection, the proposed method achieved comparable or better performance than its deep network counterparts, especially when the annotated data is limited. It also outperforms popular weakly-supervised segmentation methods. The proposed method could be an alternative solution for the data-hungry problem of deep learning methods.
Interpretable Decision Trees Through MaxSAT
Venue: arXiv cs.AI 2022
Year: 2022

We present an approach to improve the accuracy-interpretability trade-off of Machine Learning (ML) Decision Trees (DTs). In particular, we apply Maximum Satisfiability technology to compute Minimum Pure DTs (MPDTs). We improve the runtime of previous approaches and, show that these MPDTs can outperform the accuracy of DTs generated with the ML framework sklearn.
Video-based fully automatic assessment of open surgery suturing skills
Venue: arXiv cs.CV 2022
Year: 2022

The goal of this study was to develop new reliable open surgery suturing simulation system for training medical students in situation where resources are limited or in the domestic setup. Namely, we developed an algorithm for tools and hands localization as well as identifying the interactions between them based on simple webcam video data, calculating motion metrics for assessment of surgical skill. Twenty-five participants performed multiple suturing tasks using our simulator. The YOLO network has been modified to a multi-task network, for the purpose of tool localization and tool-hand interaction detection. This was accomplished by splitting the YOLO detection heads so that they supported both tasks with minimal addition to computer run-time. Furthermore, based on the outcome of the system, motion metrics were calculated. These metrics included traditional metrics such as time and path length as well as new metrics assessing the technique participants use for holding the tools. The dual-task network performance was similar to that of two networks, while computational load was only slightly bigger than one network. In addition, the motion metrics showed significant differences between experts and novices. While video capture is an essential part of minimally invasive surgery, it is not an integral component of open surgery. Thus, new algorithms, focusing on the unique challenges open surgery videos present, are required. In this study, a dual-task network was developed to solve both a localization task and a hand-tool interaction task. The dual network may be easily expanded to a multi-task network, which may be useful for images with multiple layers and for evaluating the interaction between these different layers.