OpenRaiser/Intern-Atlas huggingface dataset:
Learning to Compress Unmanned Aerial Vehicle (UAV) Captured Video: Benchmark and Analysis
Venue: arXiv cs.CV 2023
Year: 2023

During the past decade, the Unmanned-Aerial-Vehicles (UAVs) have attracted increasing attention due to their flexible, extensive, and dynamic space-sensing capabilities. The volume of video captured by UAVs is exponentially growing along with the increased bitrate generated by the advancement of the sensors mounted on UAVs, bringing new challenges for on-device UAV storage and air-ground data transmission. Most existing video compression schemes were designed for natural scenes without consideration of specific texture and view characteristics of UAV videos. In this work, we first contribute a detailed analysis of the current state of the field of UAV video coding. Then we propose to establish a novel task for learned UAV video coding and construct a comprehensive and systematic benchmark for such a task, present a thorough review of high quality UAV video datasets and benchmarks, and contribute extensive rate-distortion efficiency comparison of learned and conventional codecs after. Finally, we discuss the challenges of encoding UAV videos. It is expected that the benchmark will accelerate the research and development in video coding on drone platforms.
Improving Reliability of Fine-tuning with Block-wise Optimisation
Venue: arXiv cs.LG 2023
Year: 2023

Finetuning can be used to tackle domain-specific tasks by transferring knowledge. Previous studies on finetuning focused on adapting only the weights of a task-specific classifier or re-optimizing all layers of the pre-trained model using the new task data. The first type of methods cannot mitigate the mismatch between a pre-trained model and the new task data, and the second type of methods easily cause over-fitting when processing tasks with limited data. To explore the effectiveness of fine-tuning, we propose a novel block-wise optimization mechanism, which adapts the weights of a group of layers of a pre-trained model. In our work, the layer selection can be done in four different ways. The first is layer-wise adaptation, which aims to search for the most salient single layer according to the classification performance. The second way is based on the first one, jointly adapting a small number of top-ranked layers instead of using an individual layer. The third is block based segmentation, where the layers of a deep network is segmented into blocks by non-weighting layers, such as the MaxPooling layer and Activation layer. The last one is to use a fixed-length sliding window to group layers block by block. To identify which group of layers is the most suitable for finetuning, the search starts from the target end and is conducted by freezing other layers excluding the selected layers and the classification layers. The most salient group of layers is determined in terms of classification performance. In our experiments, the proposed approaches are tested on an often-used dataset, Tf_flower, by finetuning five typical pre-trained models, VGG16, MobileNet-v1, MobileNet-v2, MobileNet-v3, and ResNet50v2, respectively. The obtained results show that the use of our proposed block-wise approaches can achieve better performances than the two baseline methods and the layer-wise method.
#EndSARS Protest: Discourse and Mobilisation on Twitter
Venue: arXiv cs.CY 2023
Year: 2023

Using the @NGRPresident Twitter handle, the Government of Nigeria issued a special directive banning Special Anti-Robbery Squad (SARS) with immediate effect. The SARS is a special police unit under the Nigeria Police Force tasked with the responsibility of fighting violent crimes. However, the unit has been accused of waves of human rights abuse across the nation. According to a report by Amnesty International, between January 2017 and May 2020, 82 cases of police brutality have been committed. This has led to one of the major protests demanding more measures to be taken. The #EndSARS hashtag was widely used by the protesters to amplify their messages and reach out to wider communities on Twitter. In this study, we present a critical analysis of how the online protest unfolded. Essentially, we examine how the protest evolves on Twitter, the nature of engagement with the protest themes, the factors influencing the protest and public perceptions about the online movement. We found that the mobilisation strategies include direct and indirect engagements with influential users, sharing direct stories and vicarious experiences. Also, there is evidence that suggests the deployment of automated accounts to promote the course of the protest. In terms of participation, over 70% of the protest is confined within a few states in Nigeria, and the diaspora communities also lent their voices to the movement. The most active users are not those with high followership, and the majority of the protesters utilised mobile devices, accounting for 88% to mobilise and report on the protest. We also examined how social media users interact with the movement and the response from the wider online communities. Needless to say, the themes in the online discourse are mostly about #EndSARS and vicarious experiences with the police, however, there are topics around police reform and demand for regime change.
Who Should Predict? Exact Algorithms For Learning to Defer to Humans
Venue: arXiv cs.LG 2023
Year: 2023

Automated AI classifiers should be able to defer the prediction to a human decision maker to ensure more accurate predictions. In this work, we jointly train a classifier with a rejector, which decides on each data point whether the classifier or the human should predict. We show that prior approaches can fail to find a human-AI system with low misclassification error even when there exists a linear classifier and rejector that have zero error (the realizable setting). We prove that obtaining a linear pair with low error is NP-hard even when the problem is realizable. To complement this negative result, we give a mixed-integer-linear-programming (MILP) formulation that can optimally solve the problem in the linear setting. However, the MILP only scales to moderately-sized problems. Therefore, we provide a novel surrogate loss function that is realizable-consistent and performs well empirically. We test our approaches on a comprehensive set of datasets and compare to a wide range of baselines.
Automated Software Testing Starting from Static Analysis: Current State of the Art
Venue: arXiv cs.SE 2023
Year: 2023

The mass production of complex software has made it impossible to manually test it for security vulnerabilities. Automated security testing tools come in a variety of flavors, function at various stages of software development, and target different categories of software vulnerabilities. It is great that we have a plethora of automated tools to choose from, but it is a problem that their adoption and recognition are not prominent. The purpose of this study is to explore the possibilities of existing techniques while also broadening the horizon by exploring the future of security testing tools and related techniques.
Neuro-Symbolic World Models for Adapting to Open World Novelty
Venue: arXiv cs.AI 2023
Year: 2023

Open-world novelty--a sudden change in the mechanics or properties of an environment--is a common occurrence in the real world. Novelty adaptation is an agent's ability to improve its policy performance post-novelty. Most reinforcement learning (RL) methods assume that the world is a closed, fixed process. Consequentially, RL policies adapt inefficiently to novelties. To address this, we introduce WorldCloner, an end-to-end trainable neuro-symbolic world model for rapid novelty adaptation. WorldCloner learns an efficient symbolic representation of the pre-novelty environment transitions, and uses this transition model to detect novelty and efficiently adapt to novelty in a single-shot fashion. Additionally, WorldCloner augments the policy learning process using imagination-based adaptation, where the world model simulates transitions of the post-novelty environment to help the policy adapt. By blending ''imagined'' transitions with interactions in the post-novelty environment, performance can be recovered with fewer total environment interactions. Using environments designed for studying novelty in sequential decision-making problems, we show that the symbolic world model helps its neural policy adapt more efficiently than model-based and model-based neural-only reinforcement learning methods.
PyExperimenter: Easily distribute experiments and track results
Venue: arXiv cs.DC 2023
Year: 2023

PyExperimenter is a tool to facilitate the setup, documentation, execution, and subsequent evaluation of results from an empirical study of algorithms and in particular is designed to reduce the involved manual effort significantly. It is intended to be used by researchers in the field of artificial intelligence, but is not limited to those.
Quantifying the dynamics of peak innovation in scientific careers
Venue: arXiv cs.DL 2023
Year: 2023

We examine the innovation of researchers with long-lived careers in Computer Science and Physics. Despite the epistemological differences between such disciplines, we consistently find that a researcher's most innovative publication occurs earlier than expected if innovation were distributed at random across the sequence of publications in their career, and is accompanied by a peak year in which researchers publish other work which is more innovative than average. Through a series of linear models, we show that the innovation achieved by a researcher during their peak year is higher when it is preceded by a long period of low productivity. These findings are in stark contrast with the dynamics of academic impact, which researchers are incentivised to pursue through high productivity and incremental - less innovative - work by the currently prevalent paradigms of scientific evaluation.
Metrics for Software Process Simulation Modeling
Venue: arXiv cs.SE 2023
Year: 2023

Background: Software Process Simulation (SPS) has become an effective tool for software process management and improvement. However, its adoption in industry is less than what the research community expected due to the burden of measurement cost and the high demand for domain knowledge. The difficulty of extracting appropriate metrics with real data from process enactment is one of the great challenges. Objective: We aim to provide evidence-based support of the process metrics for software process (simulation) modeling. Method: A systematic literature review was performed by extending our previous review series to draw a comprehensive understanding of the metrics for process modeling following a meta-model of ontology of metrics in SPS. Results: We identified 145 process modeling studies that collectively involve 2130 metrics and classified them using the coding technique. Two diagrams which illustrate the high frequency causal relationships used between metrics are proposed in terms of two hierarchical levels of modeling purposes. We revisited the data issues encountered in SPS data preparing phases, as well as identified the corresponding strategies. Conclusion: The results of this study provide process modelers with an evidence-based reference of the identification and the use of metrics in SPS modeling, and further contribute to the development of the body of knowledge on software metrics in the context of process modeling. Furthermore, this study is not limited to process simulation but can be extended to software process modeling, in general. Taking simulation metrics as standards and references can further motivate and guide software developers to improve the collection, governance, and application of process data in practice.
JKU-ITS Automobile for Research on Autonomous Vehicles
Venue: arXiv cs.RO 2023
Year: 2023

In this paper, we present our brand-new platform for Automated Driving research. The chosen vehicle is a RAV4 hybrid SUV from TOYOTA provided with exteroceptive sensors such as a multilayer LIDAR, a monocular camera, Radar and GPS; and proprioceptive sensors such as encoders and a 9-DOF IMU. These sensors are integrated in the vehicle via a main computer running ROS1 under Linux 20.04. Additionally, we installed an open-source ADAS called Comma Two, that runs Openpilot to control the vehicle. The platform is currently being used to research in the field of autonomous vehicles, human and autonomous vehicles interaction, human factors, and energy consumption.
Linguistic Query-Guided Mask Generation for Referring Image Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Referring image segmentation aims to segment the image region of interest according to the given language expression, which is a typical multi-modal task. Existing methods either adopt the pixel classification-based or the learnable query-based framework for mask generation, both of which are insufficient to deal with various text-image pairs with a fix number of parametric prototypes. In this work, we propose an end-to-end framework built on transformer to perform Linguistic query-Guided mask generation, dubbed LGFormer. It views the linguistic features as query to generate a specialized prototype for arbitrary input image-text pair, thus generating more consistent segmentation results. Moreover, we design several cross-modal interaction modules (\eg, vision-language bidirectional attention module, VLBA) in both encoder and decoder to achieve better cross-modal alignment.
Wobble control of a pendulum actuated spherical robot
Venue: arXiv cs.RO 2023
Year: 2023

Spherical robots can conduct surveillance in hostile, cluttered environments without being damaged, as their protective shell can safely house sensors such as cameras. However, lateral oscillations, also known as wobble, occur when these sphere-shaped robots operate at low speeds, leading to shaky camera feedback. These oscillations in a pendulum-actuated spherical robot are caused by the coupling between the forward and steering motions due to nonholonomic constraints. Designing a controller to limit wobbling in these robots is challenging due to their underactuated nature. We propose a model-based controller to navigate a pendulum-actuated spherical robot using wobble-free turning maneuvers consisting of circular arcs and straight lines. The model is developed using Lagrange-D'Alembert equations and accounts for the coupled forward and steering motions. The model is further analyzed to derive expressions for radius of curvature, precession rate, wobble amplitude, and wobble frequency during circular motions. Finally, we design an input-output feedback linearization-based controller to control the robot's heading direction and wobble. Overall, the proposed controller enables a teleoperator to command a specific forward velocity and pendulum angle as per the desired turning radius while limiting the robot's lateral oscillations to enhance the quality of camera feedback.
Do I Belong? Modeling Sense of Virtual Community Among Linux Kernel Contributors
Venue: arXiv cs.SE 2023
Year: 2023

The sense of belonging to a community is a basic human need that impacts an individuals behavior, long-term engagement, and job satisfaction, as revealed by research in disciplines such as psychology, healthcare, and education. Despite much research on how to retain developers in Open Source Software projects and other virtual, peer-production communities, there is a paucity of research investigating what might contribute to a sense of belonging in these communities. To that end, we develop a theoretical model that seeks to understand the link between OSS developer motives and a Sense of Virtual Community. We test the model with a dataset collected in the Linux Kernel developer community, using structural equation modeling techniques. Our results for this case study show that intrinsic motivations - social or hedonic motives - are positively associated with a sense of virtual community, but living in an authoritative country and being paid to contribute can reduce the sense of virtual community. Based on these results, we offer suggestions for open source projects to foster a sense of virtual community, with a view to retaining contributors and improving projects sustainability.
Modeling Uncertain Feature Representation for Domain Generalization
Venue: arXiv cs.CV 2023
Year: 2023

Though deep neural networks have achieved impressive success on various vision tasks, obvious performance degradation still exists when models are tested in out-of-distribution scenarios. In addressing this limitation, we ponder that the feature statistics (mean and standard deviation), which carry the domain characteristics of the training data, can be properly manipulated to improve the generalization ability of deep learning models. Existing methods commonly consider feature statistics as deterministic values measured from the learned features and do not explicitly model the uncertain statistics discrepancy caused by potential domain shifts during testing. In this paper, we improve the network generalization ability by modeling domain shifts with uncertainty (DSU), i.e., characterizing the feature statistics as uncertain distributions during training. Specifically, we hypothesize that the feature statistic, after considering the potential uncertainties, follows a multivariate Gaussian distribution. During inference, we propose an instance-wise adaptation strategy that can adaptively deal with the unforeseeable shift and further enhance the generalization ability of the trained model with negligible additional cost. We also conduct theoretical analysis on the aspects of generalization error bound and the implicit regularization effect, showing the efficacy of our method. Extensive experiments demonstrate that our method consistently improves the network generalization ability on multiple vision tasks, including image classification, semantic segmentation, instance retrieval, and pose estimation. Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints. Code will be released in https://github.com/lixiaotong97/DSU.
Sparse resultant based minimal solvers in computer vision and their connection with the action matrix
Venue: arXiv cs.CV 2023
Year: 2023

Many computer vision applications require robust and efficient estimation of camera geometry from a minimal number of input data measurements, i.e., solving minimal problems in a RANSAC framework. Minimal problems are usually formulated as complex systems of sparse polynomials. The systems usually are overdetermined and consist of polynomials with algebraically constrained coefficients. Most state-of-the-art efficient polynomial solvers are based on the action matrix method that has been automated and highly optimized in recent years. On the other hand, the alternative theory of sparse resultants and Newton polytopes has been less successful for generating efficient solvers, primarily because the polytopes do not respect the constraints on the coefficients. Therefore, in this paper, we propose a simple iterative scheme to test various subsets of the Newton polytopes and search for the most efficient solver. Moreover, we propose to use an extra polynomial with a special form to further improve the solver efficiency via a Schur complement computation. We show that for some camera geometry problems our extra polynomial-based method leads to smaller and more stable solvers than the state-of-the-art Grobner basis-based solvers. The proposed method can be fully automated and incorporated into existing tools for automatic generation of efficient polynomial solvers. It provides a competitive alternative to popular Grobner basis-based methods for minimal problems in computer vision. We also study the conditions under which the minimal solvers generated by the state-of-the-art action matrix-based methods and the proposed extra polynomial resultant-based method, are equivalent. Specifically we consider a step-by-step comparison between the approaches based on the action matrix and the sparse resultant, followed by a set of substitutions, which would lead to equivalent minimal solvers.
DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a novel learning-based control policy with strong generalizability to new environments that enables a mobile robot to navigate autonomously through spaces filled with both static obstacles and dense crowds of pedestrians. The policy uses a unique combination of input data to generate the desired steering angle and forward velocity: a short history of lidar data, kinematic data about nearby pedestrians, and a sub-goal point. The policy is trained in a reinforcement learning setting using a reward function that contains a novel term based on velocity obstacles to guide the robot to actively avoid pedestrians and move towards the goal. Through a series of 3D simulated experiments with up to 55 pedestrians, this control policy is able to achieve a better balance between collision avoidance and speed (i.e., higher success rate and faster average speed) than state-of-the-art model-based and learning-based policies, and it also generalizes better to different crowd sizes and unseen environments. An extensive series of hardware experiments demonstrate the ability of this policy to directly work in different real-world environments with different crowd sizes with zero retraining. Furthermore, a series of simulated and hardware experiments show that the control policy also works in highly constrained static environments on a different robot platform without any additional training. Lastly, several important lessons that can be applied to other robot learning systems are summarized. Multimedia demonstrations are available at https://www.youtube.com/watch?v=KneELRT8GzU&list=PLouWbAcP4zIvPgaARrV223lf2eiSR-eSS.
An efficient implementation and a strengthening of Alon-Tarsi list coloring method
Venue: arXiv cs.DM 2023
Year: 2023

As one of the first applications of the polynomial method in combinatorics, Alon and Tarsi gave a way to prove that a graph is choosable (colorable from any lists of prescribed size). We describe an efficient way to implement this approach, making it feasible to test choosability of graphs with around 70 edges. We also show that in case that Alon-Tarsi method fails to show that the graph is choosable, further coefficients of the graph polynomial provide constraints on the list assignments from which the graph cannot be colored. This often enables us to confirm colorability from a given list assignment, or to decide choosability by testing just a few list assignments. The implementation can be found at https://gitlab.mff.cuni.cz/dvorz9am/alon-tarsi-method.
A Dataset of Coordinated Cryptocurrency-Related Social Media Campaigns
Venue: arXiv cs.HC 2023
Year: 2023

The rise in adoption of cryptoassets has brought many new and inexperienced investors in the cryptocurrency space. These investors can be disproportionally influenced by information they receive online, and particularly from social media. This paper presents a dataset of crypto-related bounty events and the users that participate in them. These events coordinate social media campaigns to create artificial "hype" around a crypto project in order to influence the price of its token. The dataset consists of information about 15.8K cross-media bounty events, 185K participants, 10M forum comments and 82M social media URLs collected from the Bounties(Altcoins) subforum of the BitcoinTalk online forum from May 2014 to December 2022. We describe the data collection and the data processing methods employed and we present a basic characterization of the dataset. Furthermore, we discuss potential research opportunities afforded by the dataset across many disciplines and we highlight potential novel insights into how the cryptocurrency industry operates and how it interacts with its audience.