OpenRaiser/Intern-Atlas huggingface dataset:
Wheelless Soft Robotic Snake Locomotion: Study on Sidewinding and Helical Rolling Gaits
Venue: arXiv cs.RO 2023
Year: 2023

Soft robotic snakes (SRSs) have a unique combination of continuous and compliant properties that allow them to imitate the complex movements of biological snakes. Despite the previous attempts to develop SRSs, many have been limited to planar movements or use wheels to achieve locomotion, which restricts their ability to imitate the full range of biological snake movements. We propose a new design for the SRSs that is wheelless and powered by pneumatics, relying solely on spatial bending to achieve its movements. We derive a kinematic model of the proposed SRS and utilize it to achieve two snake locomotion trajectories, namely sidewinding and helical rolling. These movements are experimentally evaluated under different gait parameters on our SRS prototype. The results demonstrate that the SRS can successfully mimic the proposed spatial locomotion trajectories. This is a significant improvement over the previous designs, which were either limited to planar movements or relied on wheels for locomotion. The ability of the SRS to effectively mimic the complex movements of biological snakes opens up new possibilities for its use in various applications.
OASIS: Automated Assessment of Urban Pedestrian Paths at Scale
Venue: arXiv cs.CV 2023
Year: 2023

The inspection of the Public Right of Way (PROW) for accessibility barriers is necessary for monitoring and maintaining the built environment for communities' walkability, rollability, safety, active transportation, and sustainability. However, an inspection of the PROW, by surveyors or crowds, is laborious, inconsistent, costly, and unscalable. The core of smart city developments involves the application of information technologies toward municipal assets assessment and management. Sidewalks, in comparison to automobile roads, have not been regularly integrated into information systems to optimize or inform civic services. We develop an Open Automated Sidewalks Inspection System (OASIS), a free and open-source automated mapping system, to extract sidewalk network data using mobile physical devices. OASIS leverages advances in neural networks, image sensing, location-based methods, and compact hardware to perform sidewalk segmentation and mapping along with the identification of barriers to generate a GIS pedestrian transportation layer that is available for routing as well as analytic and operational reports. We describe a prototype system trained and tested with imagery collected in real-world settings, alongside human surveyors who are part of the local transit pathway review team. Pilots show promising precision and recall for path mapping (0.94, 0.98 respectively). Moreover, surveyor teams' functional efficiency increased in the field. By design, OASIS takes adoption aspects into consideration to ensure the system could be easily integrated with governmental pathway review teams' workflows, and that the outcome data would be interoperable with public data commons.
Optimizing Fuel-Constrained UAV-UGV Routes for Large Scale Coverage: Bilevel Planning in Heterogeneous Multi-Agent Systems
Venue: arXiv cs.RO 2023
Year: 2023

Fast moving unmanned aerial vehicles (UAVs) are well suited for aerial surveillance, but are limited by their battery capacity. To increase their endurance UAVs can be refueled on slow moving unmanned ground vehicles (UGVs). The cooperative routing of UAV-UGV multi-agent system to survey vast regions within their speed and fuel constraints is a computationally challenging problem, but can be simplified with heuristics. Here we present multiple heuristics to enable feasible and sufficiently optimal solutions to the problem. Using the UAV fuel limits and the minimum set cover algorithm, the UGV refueling stops are determined. These refueling stops enable the allocation of mission points to the UAV and UGV. A standard traveling salesman formulation and a vehicle routing formulation with time windows, dropped visits, and capacity constraints is used to solve for the UGV and UAV route, respectively. Experimental validation on a small-scale testbed (http://tiny.cc/8or8vz) underscores the effectiveness of our multi-agent approach.
Real-Time Hand Gesture Identification in Thermal Images
Venue: arXiv cs.CV 2023
Year: 2023

Hand gesture-based human-computer interaction is an important problem that is well explored using color camera data. In this work we proposed a hand gesture detection system using thermal images. Our system is capable of handling multiple hand regions in a frame and process it fast for real-time applications. Our system performs a series of steps including background subtraction-based hand mask generation, k-means based hand region identification, hand segmentation to remove the forearm region, and a Convolutional Neural Network (CNN) based gesture classification. Our work introduces two novel algorithms, bubble growth and bubble search, for faster hand segmentation. We collected a new thermal image data set with 10 gestures and reported an end-to-end hand gesture recognition accuracy of 97%.
Modular Safety-Critical Control of Legged Robots
Venue: arXiv cs.RO 2023
Year: 2023

Safety concerns during the operation of legged robots must be addressed to enable their widespread use. Machine learning-based control methods that use model-based constraints provide promising means to improve robot safety. This study presents a modular safety filter to improve the safety of a legged robot, i.e., reduce the chance of a fall. The prerequisite is the availability of a robot that is capable of locomotion, i.e., a nominal controller exists. During locomotion, terrain properties around the robot are estimated through machine learning which uses a minimal set of proprioceptive signals. A novel deep-learning model utilizing an efficient transformer architecture is used for the terrain estimation. A quadratic program combines the terrain estimations with inverse dynamics and a novel exponential control barrier function constraint to filter and certify nominal control signals. The result is an optimal controller that acts as a filter. The filtered control signal allows safe locomotion of the robot. The resulting approach is generalizable, and could be transferred with low effort to any other legged system.
Estimating Age of Information Using Finite Order Moments
Venue: arXiv cs.IT 2023
Year: 2023

Age of information (AoI) has been proposed as a more suitable metric for characterizing the freshness of information than traditional metrics like delay and throughput. However, the calculation of AoI requires complex analysis and strict end-to-end synchronization. Most existential AoI-related works have assumed that the statistical characterizations of the arrival process and the service process are known. In fact, due to the randomness of the sources and the channel noises, these processes are often unavailable in reality. To this end, we propose a method to estimate the average AoI on a point-to-point wireless Rayleigh channel, which uses the available finite order statistical moments of the arrival process. Based on this method, we explicitly present the upper and lower bounds on the average AoI of the system. Our results show that 1) with the increase of the traffic intensity, the absolute error of the estimated average AoI bounds is first increasing and then decreasing, while the average AoI is monotonically increasing; 2) the average AoI can be effectively approximated by using the first two order moment estimation bounds, especially when traffic intensity is small or approaches unity; 3) tighter bounds can be obtained by using more moments.
Building a Modal-balanced BlockChain with Semantic Reconstruction
Venue: arXiv cs.MM 2023
Year: 2023

The current large blockchain systems (BTC Lightning network, Ethereum, etc.) are generally facing the problems of low persistence rates and high storage costs. Therefore, users tend to store single modal (textual) information on the existing blockchain systems. Inspired by semantic communication algorithms, this paper presents a new algorithm to solve the serious imbalance between textual and visual modals on blockchains. After semantic sampling of the original visual image, the resulting semantic text will be stored on the chain, and the end users can reconstruct a semantically similar image using the \textbf{R}elative \textbf{O}ptimal \textbf{S}emantic \textbf{I}sotope \textbf{S}election algorithm. Experiments on the DIV2K dataset show that the blockchain with our algorithm can achieve 430,000 times the storage capacity and 550,000 times the persistence rate for the original visual data with acceptable semantic information loss.
Variational Quantum Classifiers for Natural-Language Text
Venue: arXiv cs.CL 2023
Year: 2023

As part of the recent research effort on quantum natural language processing (QNLP), variational quantum sentence classifiers (VQSCs) have been implemented and supported in lambeq / DisCoPy, based on the DisCoCat model of sentence meaning. We discuss in some detail VQSCs, including category theory, DisCoCat for modeling sentence as string diagram, and DisCoPy for encoding string diagram as parameterized quantum circuit. Many NLP tasks, however, require the handling of text consisting of multiple sentences, which is not supported in lambeq / DisCoPy. A good example is sentiment classification of customer feedback or product review. We discuss three potential approaches to variational quantum text classifiers (VQTCs), in line with VQSCs. The first is a weighted bag-of-sentences approach which treats text as a group of independent sentences with task-specific sentence weighting. The second is a coreference resolution approach which treats text as a consolidation of its member sentences with coreferences among them resolved. Both approaches are based on the DisCoCat model and should be implementable in lambeq / DisCoCat. The third approach, on the other hand, is based on the DisCoCirc model which considers both ordering of sentences and interaction of words in composing text meaning from word and sentence meanings. DisCoCirc makes fundamental modification of DisCoCat since a sentence in DisCoCirc updates meanings of words, whereas all meanings are static in DisCoCat. It is not clear if DisCoCirc can be implemented in lambeq / DisCoCat without breaking DisCoCat.
Optimizing Low Dimensional Functions over the Integers
Venue: arXiv cs.DS 2023
Year: 2023

We consider box-constrained integer programs with objective $g(Wx) + c^T x$, where $g$ is a "complicated" function with an $m$ dimensional domain. Here we assume we have $n \gg m$ variables and that $W \in \mathbb Z^{m \times n}$ is an integer matrix with coefficients of absolute value at most $Δ$. We design an algorithm for this problem using only the mild assumption that the objective can be optimized efficiently when all but $m$ variables are fixed, yielding a running time of $n^m(m Δ)^{O(m^2)}$. Moreover, we can avoid the term $n^m$ in several special cases, in particular when $c = 0$. Our approach can be applied in a variety of settings, generalizing several recent results. An important application are convex objectives of low domain dimension, where we imply a recent result by Hunkenschröder et al. [SIOPT'22] for the 0-1-hypercube and sharp or separable convex $g$, assuming $W$ is given explicitly. By avoiding the direct use of proximity results, which only holds when $g$ is separable or sharp, we match their running time and generalize it for arbitrary convex functions. In the case where the objective is only accessible by an oracle and $W$ is unknown, we further show that their proximity framework can be implemented in $n (m Δ)^{O(m^2)}$-time instead of $n (m Δ)^{O(m^3)}$. Lastly, we extend the result by Eisenbrand and Weismantel [SODA'17, TALG'20] for integer programs with few constraints to a mixed-integer linear program setting where integer variables appear in only a small number of different constraints.
Investigating Group Distributionally Robust Optimization for Deep Imbalanced Learning: A Case Study of Binary Tabular Data Classification
Venue: arXiv cs.LG 2023
Year: 2023

One of the most studied machine learning challenges that recent studies have shown the susceptibility of deep neural networks to is the class imbalance problem. While concerted research efforts in this direction have been notable in recent years, findings have shown that the canonical learning objective, empirical risk minimization (ERM), is unable to achieve optimal imbalance learning in deep neural networks given its bias to the majority class. An alternative learning objective, group distributionally robust optimization (gDRO), is investigated in this study for imbalance learning, focusing on tabular imbalanced data as against image data that has dominated deep imbalance learning research. Contrary to minimizing average per instance loss as in ERM, gDRO seeks to minimize the worst group loss over the training data. Experimental findings in comparison with ERM and classical imbalance methods using four popularly used evaluation metrics in imbalance learning across several benchmark imbalance binary tabular data of varying imbalance ratios reveal impressive performance of gDRO, outperforming other compared methods in terms of g-mean and roc-auc.
Chasing Low-Carbon Electricity for Practical and Sustainable DNN Training
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning has experienced significant growth in recent years, resulting in increased energy consumption and carbon emission from the use of GPUs for training deep neural networks (DNNs). Answering the call for sustainability, conventional solutions have attempted to move training jobs to locations or time frames with lower carbon intensity. However, moving jobs to other locations may not always be feasible due to large dataset sizes or data regulations. Moreover, postponing training can negatively impact application service quality because the DNNs backing the service are not updated in a timely fashion. In this work, we present a practical solution that reduces the carbon footprint of DNN training without migrating or postponing jobs. Specifically, our solution observes real-time carbon intensity shifts during training and controls the energy consumption of GPUs, thereby reducing carbon footprint while maintaining training performance. Furthermore, in order to proactively adapt to shifting carbon intensity, we propose a lightweight machine learning algorithm that predicts the carbon intensity of the upcoming time frame. Our solution, Chase, reduces the total carbon footprint of training ResNet-50 on ImageNet by 13.6% while only increasing training time by 2.5%.
Visual Saliency-Guided Channel Pruning for Deep Visual Detectors in Autonomous Driving
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural network (DNN) pruning has become a de facto component for deploying on resource-constrained devices since it can reduce memory requirements and computation costs during inference. In particular, channel pruning gained more popularity due to its structured nature and direct savings on general hardware. However, most existing pruning approaches utilize importance measures that are not directly related to the task utility. Moreover, few in the literature focus on visual detection models. To fill these gaps, we propose a novel gradient-based saliency measure for visual detection and use it to guide our channel pruning. Experiments on the KITTI and COCO traffic datasets demonstrate our pruning method's efficacy and superiority over state-of-the-art competing approaches. It can even achieve better performance with fewer parameters than the original model. Our pruning also demonstrates great potential in handling small-scale objects.
Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection
Venue: arXiv cs.CL 2023
Year: 2023

Hate speech in social media is a growing phenomenon, and detecting such toxic content has recently gained significant traction in the research community. Existing studies have explored fine-tuning language models (LMs) to perform hate speech detection, and these solutions have yielded significant performance. However, most of these studies are limited to detecting hate speech only in English, neglecting the bulk of hateful content that is generated in other languages, particularly in low-resource languages. Developing a classifier that captures hate speech and nuances in a low-resource language with limited data is extremely challenging. To fill the research gap, we propose HateMAML, a model-agnostic meta-learning-based framework that effectively performs hate speech detection in low-resource languages. HateMAML utilizes a self-supervision strategy to overcome the limitation of data scarcity and produces better LM initialization for fast adaptation to an unseen target language (i.e., cross-lingual transfer) or other hate speech datasets (i.e., domain generalization). Extensive experiments are conducted on five datasets across eight different low-resource languages. The results show that HateMAML outperforms the state-of-the-art baselines by more than 3% in the cross-domain multilingual transfer setting. We also conduct ablation studies to analyze the characteristics of HateMAML.
Physics-informed neural network for friction-involved nonsmooth dynamics problems
Venue: arXiv cs.CE 2023
Year: 2023

Friction-induced vibration (FIV) is very common in engineering areas. Analysing the dynamic behaviour of systems containing a multiple-contact point frictional interface is an important topic. However, accurately simulating nonsmooth/discontinuous dynamic behaviour due to friction is challenging. This paper presents a new physics-informed neural network approach for solving nonsmooth friction-induced vibration or friction-involved vibration problems. Compared with schemes of the conventional time-stepping methodology, in this new computational framework, the theoretical formulations of nonsmooth multibody dynamics are transformed and embedded in the training process of the neural network. Major findings include that the new framework not only can perform accurate simulation of nonsmooth dynamic behaviour, but also eliminate the need for extremely small time steps typically associated with the conventional time-stepping methodology for multibody systems, thus saving much computation work while maintaining high accuracy. Specifically, four kinds of high-accuracy PINN-based methods are proposed: (1) single PINN; (2) dual PINN; (3) advanced single PINN; (4) advanced dual PINN. Two typical dynamics problems with nonsmooth contact are tested: one is a 1-dimensional contact problem with stick-slip, and the other is a 2-dimensional contact problem considering separation-reattachment and stick-slip oscillation. Both single and dual PINN methods show their advantages in dealing with the 1-dimensional stick-slip problem, which outperforms conventional methods across friction models that are difficult to simulate by the conventional time-stepping method. For the 2-dimensional problem, the capability of the advanced single and advanced dual PINN on accuracy improvement is shown, and they provide good results even in the cases when conventional methods fail.
Composing Mood Board with User Feedback in Concept Space
Venue: arXiv cs.HC 2023
Year: 2023

We propose the Mood Board Composer (MBC), which supports concept designers in retrieving and composing images on a 2-D concept space to communicate design concepts. The MBC allows users to iterate adaptive image retrievals intuitively. Our new contribution to the mood board tool is to adapt the query vector for the next iteration according to the user's rearrangement of images on the 2-D space. The algorithm emphasizes the meaning of the labels on the x- and y-axes by calculating the mean vector of the images on the mood board multiplied by the weights assigned to each cell of the 3 x 3 grid. The next image search is performed by obtaining the most similar words from the mean vector thus obtained and using them as a new query. In addition to the algorithm described above, we conducted the participant experiment with two other interaction algorithms to compare. The first allows users to delete unwanted images and go on to the next searches. The second utilizes the semantic labels on each image, on which users can provide negative feedback for query modification for the next searches. Although we did not observe significant differences among the three proposed algorithms, our experiment with 420 cases of mood board creation confirmed the effectiveness of adaptive iterations by the Creativity Support Index (CSI) score.
Minimize Web Applications vulnerabilities through the early Detection of CRLF Injection
Venue: arXiv cs.CR 2023
Year: 2023

Carriage return (CR) and line feed (LF), also known as CRLF injection is a type of vulnerability that allows a hacker to enter special characters into a web application, altering its operation or confusing the administrator. Log poisoning and HTTP response splitting are two prominent harmful uses of this technique. Additionally, CRLF injection can be used by an attacker to exploit other vulnerabilities, such as cross-site scripting (XSS). According to Open Web Application Security Project (OWASP), CRLF vulnerabilities are among the top 10 vulnerabilities and are a type of injection attack. Automated testing can help to quickly identify CRLF vulnerabilities, and is particularly useful for companies to test their applications before releasing them. However, CRLF vulnerabilities foster a better approach to mitigate CRLF vulnerabilities in the early stage and help secure applications against high-risk known vulnerabilities. There has been less research on CRLF vulnerabilities and how to detect them with automated testing. There is room for further research to be done on this subject matter in order to develop creative solutions to problems. It will also help to reduce false positive alerts by checking the header response of each request. Security automation is an important issue for companies trying to protect themselves against security threats. Automated alerts from security systems can provide a quicker and more accurate understanding of potential vulnerabilities and can help to reduce false positive alerts. Despite the extensive research on various types of vulnerabilities in web applications, CRLF vulnerabilities have only recently been included in the research. Utilizing automated testing as a recurring task can assist companies in receiving consistent updates about their systems and enhance their security.
MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition
Venue: arXiv cs.CV 2023
Year: 2023

We present a novel approach for action recognition in UAV videos. Our formulation is designed to handle occlusion and viewpoint changes caused by the movement of a UAV. We use the concept of mutual information to compute and align the regions corresponding to human action or motion in the temporal domain. This enables our recognition model to learn from the key features associated with the motion. We also propose a novel frame sampling method that uses joint mutual information to acquire the most informative frame sequence in UAV videos. We have integrated our approach with X3D and evaluated the performance on multiple datasets. In practice, we achieve 18.9% improvement in Top-1 accuracy over current state-of-the-art methods on UAV-Human(Li et al., 2021), 7.3% improvement on Drone-Action(Perera et al., 2019), and 7.16% improvement on NEC Drones(Choi et al., 2020).