OpenRaiser/Intern-Atlas huggingface dataset:
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Venue: arXiv cs.CL 2023
Year: 2023

As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop, a 1-year international and multidisciplinary initiative, was formed with the goal of researching and training large language models as a values-driven undertaking, putting issues of ethics, harm, and governance in the foreground. This paper documents the data creation and curation efforts undertaken by BigScience to assemble the Responsible Open-science Open-collaboration Text Sources (ROOTS) corpus, a 1.6TB dataset spanning 59 languages that was used to train the 176-billion-parameter BigScience Large Open-science Open-access Multilingual (BLOOM) language model. We further release a large initial subset of the corpus and analyses thereof, and hope to empower large-scale monolingual and multilingual modeling projects with both the data and the processing tools, as well as stimulate research around this large multilingual corpus.
ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identification
Venue: arXiv cs.CL 2023
Year: 2023

ChatGPT has shown strong capabilities in natural language generation tasks, which naturally leads researchers to explore where its abilities end. In this paper, we examine whether ChatGPT can be used for zero-shot text classification, more specifically, automatic genre identification. We compare ChatGPT with a multilingual XLM-RoBERTa language model that was fine-tuned on datasets, manually annotated with genres. The models are compared on test sets in two languages: English and Slovenian. Results show that ChatGPT outperforms the fine-tuned model when applied to the dataset which was not seen before by either of the models. Even when applied on Slovenian language as an under-resourced language, ChatGPT's performance is no worse than when applied to English. However, if the model is fully prompted in Slovenian, the performance drops significantly, showing the current limitations of ChatGPT usage on smaller languages. The presented results lead us to questioning whether this is the beginning of an end of laborious manual annotation campaigns even for smaller languages, such as Slovenian.
The special case of cyclotomic fields in quantum algorithms for unit groups
Venue: arXiv cs.CR 2023
Year: 2023

Unit group computations are a cryptographic primitive for which one has a fast quantum algorithm, but the required number of qubits is $\tilde O(m^5)$. In this work we propose a modification of the algorithm for which the number of qubits is $\tilde O(m^2)$ in the case of cyclotomic fields. Moreover, under a recent conjecture on the size of the class group of $\mathbb{Q}(ζ_m + ζ_m^{-1})$, the quantum algorithms is much simpler because it is a hidden subgroup problem (HSP) algorithm rather than its error estimation counterpart: continuous hidden subgroup problem (CHSP). We also discuss the (minor) speed-up obtained when exploiting Galois automorphisms thanks to the Buchmann-Pohst algorithm over $\mathcal{O}_K$-lattices.
"If I Had All the Time in the World": Ophthalmologists' Perceptions of Anchoring Bias Mitigation in Clinical AI Support
Venue: arXiv cs.HC 2023
Year: 2023

Clinical needs and technological advances have resulted in increased use of Artificial Intelligence (AI) in clinical decision support. However, such support can introduce new and amplify existing cognitive biases. Through contextual inquiry and interviews, we set out to understand the use of an existing AI support system by ophthalmologists. We identified concerns regarding anchoring bias and a misunderstanding of the AI's capabilities. Following, we evaluated clinicians' perceptions of three bias mitigation strategies as integrated into their existing decision support system. While clinicians recognised the danger of anchoring bias, we identified a concern around the impact of bias mitigation on procedure time. Our participants were divided in their expectations of any positive impact on diagnostic accuracy, stemming from varying reliance on the decision support. Our results provide insights into the challenges of integrating bias mitigation into AI decision support.
Near-Field Communications: What Will Be Different?
Venue: arXiv cs.IT 2023
Year: 2023

The design dilemma of "What will be different between near-field communications (NFC) and far-field communications (FFC)?" is addressed from four perspectives. 1) From the channel modelling perspective, the differences between near-field and far-field channel models are discussed. A novel Green's function-based channel model is proposed for continuous-aperture antennas, which is contrasted to conventional channel models tailored for spatially-discrete antennas. 2) From the performance analysis perspective, analytical results for characterizing the degrees of freedom and the power scaling laws in the near-field region are provided for both spatially-discrete and continuous-aperture antennas. 3) From the beamforming perspective, far-field beamforming is analogous to a "flashlight" that enables beamsteering, while near-field beamforming can be likened to a "spotlight" that facilitates beamfocusing. As a further advance, a couple of new beamforming structures are proposed for exploiting the new characteristics of NFC. 4) From the application perspective, new designs are discussed in the context of promising next-generation technologies in NFC, where our preliminary numerical results demonstrate that distance-aware target sensing and enhanced physical layer security can be realized in NFC. Finally, several future research directions of NFC are discussed.
Hausdorff and Gromov-Hausdorff stable subsets of the medial axis
Venue: arXiv cs.CG 2023
Year: 2023

In this paper we introduce a pruning of the medial axis called the $(λ,α)$-medial axis ($\textrm{ax}_λ^α$). We prove that the $(λ,α)$-medial axis of a set $K$ is stable in a Gromov-Hausdorff sense under weak assumptions. More formally we prove that if $K$ and $K'$ are close in the Hausdorff ($d_H$) sense then the $(λ,α)$-medial axes of $K$ and $K'$ are close as metric spaces, that is the Gromov-Hausdorff distance ($d_{GH}$) between the two is $\frac{1}{4}$-H{ö}lder in the sense that $d_{GH} (\textrm{ax}_λ^α(K),\textrm{ax}_λ^α(K')) \lesssim d_H(K,K')^{1/4}$. The Hausdorff distance between the two medial axes is also bounded, by $d_{H} (\textrm{ax}_λ^α(K),\textrm{ax}_λ^α(K')) \lesssim d_H(K,K')^{1/2}$. These quantified stability results provide guarantees for practical computations of medial axes from approximations. Moreover, they provide key ingredients for studying the computability of the medial axis in the context of computable analysis.
Root Cause Identification for Collective Anomalies in Time Series given an Acyclic Summary Causal Graph with Loops
Venue: arXiv cs.AI 2023
Year: 2023

This paper presents an approach for identifying the root causes of collective anomalies given observational time series and an acyclic summary causal graph which depicts an abstraction of causal relations present in a dynamic system at its normal regime. The paper first shows how the problem of root cause identification can be divided into many independent subproblems by grouping related anomalies using d-separation. Further, it shows how, under this setting, some root causes can be found directly from the graph and from the time of appearance of anomalies. Finally, it shows, how the rest of the root causes can be found by comparing direct effects in the normal and in the anomalous regime. To this end, an adjustment set for identifying direct effects is introduced. Extensive experiments conducted on both simulated and real-world datasets demonstrate the effectiveness of the proposed method.
Exploiting Trust for Resilient Hypothesis Testing with Malicious Robots (evolved version)
Venue: arXiv cs.RO 2023
Year: 2023

We develop a resilient binary hypothesis testing framework for decision making in adversarial multi-robot crowdsensing tasks. This framework exploits stochastic trust observations between robots to arrive at tractable, resilient decision making at a centralized Fusion Center (FC) even when i) there exist malicious robots in the network and their number may be larger than the number of legitimate robots, and ii) the FC uses one-shot noisy measurements from all robots. We derive two algorithms to achieve this. The first is the Two Stage Approach (2SA) that estimates the legitimacy of robots based on received trust observations, and provably minimizes the probability of detection error in the worst-case malicious attack. Here, the proportion of malicious robots is known but arbitrary. For the case of an unknown proportion of malicious robots, we develop the Adversarial Generalized Likelihood Ratio Test (A-GLRT) that uses both the reported robot measurements and trust observations to estimate the trustworthiness of robots, their reporting strategy, and the correct hypothesis simultaneously. We exploit special problem structure to show that this approach remains computationally tractable despite several unknown problem parameters. We deploy both algorithms in a hardware experiment where a group of robots conducts crowdsensing of traffic conditions on a mock-up road network similar in spirit to Google Maps, subject to a Sybil attack. We extract the trust observations for each robot from actual communication signals which provide statistical information on the uniqueness of the sender. We show that even when the malicious robots are in the majority, the FC can reduce the probability of detection error to 30.5% and 29% for the 2SA and the A-GLRT respectively.
Developers' Perception of GitHub Actions: A Survey Analysis
Venue: arXiv cs.SE 2023
Year: 2023

GitHub introduced "Actions" in 2019 to increase workflow velocity and add customized automation to the repositories. Any individual can develop Actions for automating workflow on GitHub repositories and others can reuse them whenever open source. GitHub introduced its marketplace for commercializing and sharing these automation tools, which currently hosts 16,730 Actions. Yet, there are numerous Actions that are developed and distributed in local repositories and outside the Marketplace. So far, the research community conducted mining studies to understand Actions with a significant focus on CI/CD. We conducted a survey study with 90 Action developers and users to understand the motivations and best practices in using, developing, and debugging Actions, and the challenges associated with these tasks. We found that developers prefer Actions with verified creators and more stars when choosing between similar Actions, and often switch to an alternative Action when facing bugs or a lack of documentation. 60.87% of the developers consider the composition of YAML files, which are essential for Action integration, challenging and error-prone. They mainly check Q\&A forums to fix issues with these YAML files. Finally, developers tend to avoid using Actions (and hence automation) to reduce complexity, and security risk, or when the benefits of Actions are not worth the cost/effort of setting up Actions for automation.
A Strategy-Oriented Bayesian Soft Actor-Critic Model
Venue: arXiv cs.AI 2023
Year: 2023

Adopting reasonable strategies is challenging but crucial for an intelligent agent with limited resources working in hazardous, unstructured, and dynamic environments to improve the system's utility, decrease the overall cost, and increase mission success probability. This paper proposes a novel hierarchical strategy decomposition approach based on the Bayesian chain rule to separate an intricate policy into several simple sub-policies and organize their relationships as Bayesian strategy networks (BSN). We integrate this approach into the state-of-the-art DRL method -- soft actor-critic (SAC) and build the corresponding Bayesian soft actor-critic (BSAC) model by organizing several sub-policies as a joint policy. We compare the proposed BSAC method with the SAC and other state-of-the-art approaches such as TD3, DDPG, and PPO on the standard continuous control benchmarks -- Hopper-v2, Walker2d-v2, and Humanoid-v2 -- in MuJoCo with the OpenAI Gym environment. The results demonstrate that the promising potential of the BSAC method significantly improves training efficiency.
Diversity Embeddings and the Hypergraph Sparsest Cut
Venue: arXiv cs.DS 2023
Year: 2023

Good approximations have been attained for the sparsest cut problem by rounding solutions to convex relaxations via low-distortion metric embeddings. Recently, Bryant and Tupper showed that this approach extends to the hypergraph setting by formulating a linear program whose solutions are so-called diversities which are rounded via diversity embeddings into $\ell_1$. Diversities are a generalization of metric spaces in which the nonnegative function is defined on all subsets as opposed to only on pairs of elements. We show that this approach yields a polytime $O(\log{n})$-approximation when either the supply or demands are given by a graph. This result improves upon Plotkin et al.'s $O(\log{(kn)}\log{n})$-approximation, where $k$ is the number of demands, for the setting where the supply is given by a graph and the demands are given by a hypergraph. Additionally, we provide a polytime $O(\min{\{r_G,r_H\}}\log{r_H}\log{n})$-approximation for when the supply and demands are given by hypergraphs whose hyperedges are bounded in cardinality by $r_G$ and $r_H$ respectively. To establish these results we provide an $O(\log{n})$-distortion $\ell_1$ embedding for the class of diversities known as diameter diversities. This improves upon Bryant and Tupper's $O(\log\^2{n})$-distortion embedding. The smallest known distortion with which an arbitrary diversity can be embedded into $\ell_1$ is $O(n)$. We show that for any $ε> 0$ and any $p>0$, there is a family of diversities which cannot be embedded into $\ell_1$ in polynomial time with distortion smaller than $O(n^{1-ε})$ based on querying the diversities on sets of cardinality at most $O(\log^p{n})$, unless $P=NP$. This disproves (an algorithmic refinement of) Bryant and Tupper's conjecture that there exists an $O(\sqrt{n})$-distortion $\ell_1$ embedding based off a diversity's induced metric.
Comparing PSDNet, pretrained networks, and traditional feature extraction for predicting the particle size distribution of granular materials from photographs
Venue: arXiv cs.CV 2023
Year: 2023

This study aims to evaluate PSDNet, a series of convolutional neural networks (ConvNets) trained with photographs to predict the particle size distribution of granular materials. Nine traditional feature extraction methods and 15 pretrained ConvNets were also evaluated and compared. A dataset including 9600 photographs of 15 different granular materials was used. The influence of image size and color band was verified by using six image sizes between 32 and 160 pixels, and both grayscale and color images as PSDNet inputs. In addition to random training, validation, and testing datasets, a material removal method was also used to evaluate the performances of each image analysis method. With this method, each material was successively removed from the training and validation datasets and used as the testing dataset. Results show that a combination of all PSDNet color and grayscale features can lead to a root mean square error (RMSE) on the percentages passing as low as 1.8 % with a random testing dataset and 9.1% with the material removal method. For the random datasets, a combination of all traditional features, and the features extracted from InceptionResNetV2 led to RMSE on the percentages passing of 2.3 and 1.7 %, respectively.
Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning
Venue: arXiv cs.LG 2023
Year: 2023

While preserving the privacy of federated learning (FL), differential privacy (DP) inevitably degrades the utility (i.e., accuracy) of FL due to model perturbations caused by DP noise added to model updates. Existing studies have considered exclusively noise with persistent root-mean-square amplitude and overlooked an opportunity of adjusting the amplitudes to alleviate the adverse effects of the noise. This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of FL and retain the capability of adjusting the learning performance. Specifically, we propose a geometric series form for the noise amplitude and reveal analytically the dependence of the series on the number of global aggregations and the $(ε,δ)$-DP requirement. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. Another important aspect is an upper bound developed for the loss function of a multi-layer perceptron (MLP) trained by FL running the new DP mechanism. Accordingly, the optimal number of global aggregations is obtained, balancing the learning and privacy. Extensive experiments are conducted using MLP, supporting vector machine, and convolutional neural network models on four public datasets. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design
Venue: arXiv cs.CV 2023
Year: 2023

Semantic image and video segmentation stand among the most important tasks in computer vision nowadays, since they provide a complete and meaningful representation of the environment by means of a dense classification of the pixels in a given scene. Recently, Deep Learning, and more precisely Convolutional Neural Networks, have boosted semantic segmentation to a new level in terms of performance and generalization capabilities. However, designing Deep Semantic Segmentation models is a complex task, as it may involve application-dependent aspects. Particularly, when considering autonomous driving applications, the robustness-efficiency trade-off, as well as intrinsic limitations - computational/memory bounds and data-scarcity - and constraints - real-time inference - should be taken into consideration. In this respect, the use of additional data modalities, such as depth perception for reasoning on the geometry of a scene, and temporal cues from videos to explore redundancy and consistency, are promising directions yet not explored to their full potential in the literature. In this paper, we conduct a survey on the most relevant and recent advances in Deep Semantic Segmentation in the context of vision for autonomous vehicles, from three different perspectives: efficiency-oriented model development for real-time operation, RGB-Depth data integration (RGB-D semantic segmentation), and the use of temporal information from videos in temporally-aware models. Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective, so that the reader can not only get started, but also be up to date in respect to recent advances in this exciting and challenging research field.
M-EBM: Towards Understanding the Manifolds of Energy-Based Models
Venue: arXiv cs.CV 2023
Year: 2023

Energy-based models (EBMs) exhibit a variety of desirable properties in predictive tasks, such as generality, simplicity and compositionality. However, training EBMs on high-dimensional datasets remains unstable and expensive. In this paper, we present a Manifold EBM (M-EBM) to boost the overall performance of unconditional EBM and Joint Energy-based Model (JEM). Despite its simplicity, M-EBM significantly improves unconditional EBMs in training stability and speed on a host of benchmark datasets, such as CIFAR10, CIFAR100, CelebA-HQ, and ImageNet 32x32. Once class labels are available, label-incorporated M-EBM (M-JEM) further surpasses M-EBM in image generation quality with an over 40% FID improvement, while enjoying improved accuracy. The code can be found at https://github.com/sndnyang/mebm.
Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraint
Venue: arXiv cs.LG 2023
Year: 2023

Soft actor-critic (SAC) in reinforcement learning is expected to be one of the next-generation robot control schemes. Its ability to maximize policy entropy would make a robotic controller robust to noise and perturbation, which is useful for real-world robot applications. However, the priority of maximizing the policy entropy is automatically tuned in the current implementation, the rule of which can be interpreted as one for equality constraint, binding the policy entropy into its specified lower bound. The current SAC is therefore no longer maximize the policy entropy, contrary to our expectation. To resolve this issue in SAC, this paper improves its implementation with a learnable state-dependent slack variable for appropriately handling the inequality constraint to maximize the policy entropy by reformulating it as the corresponding equality constraint. The introduced slack variable is optimized by a switching-type loss function that takes into account the dual objectives of satisfying the equality constraint and checking the lower bound. In Mujoco and Pybullet simulators, the modified SAC statistically achieved the higher robustness for adversarial attacks than before while regularizing the norm of action. A real-robot variable impedance task was demonstrated for showing the applicability of the modified SAC to real-world robot control. In particular, the modified SAC maintained adaptive behaviors for physical human-robot interaction, which had no experience at all during training. https://youtu.be/EH3xVtlVaJw
Towards Practical Autonomous Flight Simulation for Flapping Wing Biomimetic Robots with Experimental Validation
Venue: arXiv cs.RO 2023
Year: 2023

Tried-and-true flapping wing robot simulation is essential in developing flapping wing mechanisms and algorithms. This paper presents a novel application-oriented flapping wing platform, highly compatible with various mechanical designs and adaptable to different robotic tasks. First, the blade element theory and the quasi-steady model are put forward to compute the flapping wing aerodynamics based on wing kinematics. Translational lift, translational drag, rotational lift, and added mass force are all considered in the computation. Then we use the proposed simulation platform to investigate the passive wing rotation and the wing-tail interaction phenomena of a particular flapping-wing robot. With the help of the simulation tool and a novel statistic based on dynamic differences from the averaged system, several behaviors display their essence by investigating the flapping wing robot dynamic characteristics. After that, the attitude tracking control problem and the positional trajectory tracking problem are both overcome by robust control techniques. Further comparison simulations reveal that the proposed control algorithms compared with other existing ones show apparent superiority. What is more, with the same control algorithm and parameters tuned in simulation, we conduct real flight experiments on a self-made flapping wing robot, and obtain similar results from the proposed simulation platform. In contrast to existing simulation tools, the proposed one is compatible with most existing flapping wing robots, and can inherently drill into each subtle behavior in corresponding applications by observing aerodynamic forces and torques on each blade element.