OpenRaiser/Intern-Atlas huggingface dataset:
A Cloud-based Deep Learning Framework for Early Detection of Pushing at Crowded Event Entrances
Venue: arXiv cs.LG 2023
Year: 2023

Crowding at the entrances of large events may lead to critical and life-threatening situations, particularly when people start pushing each other to reach the event faster. Automatic and timely identification of pushing behavior would help organizers and security forces to intervene early and mitigate dangerous situations. In this paper, we propose a cloud-based deep learning framework for automatic early detection of pushing in crowded event entrances. The proposed framework initially modifies and trains the EfficientNetV2B0 Convolutional Neural Network model. Subsequently, it integrates the adapted model with an accurate and fast pre-trained deep optical flow model with the color wheel method to analyze video streams and identify pushing patches in real-time. Moreover, the framework uses live capturing technology and a cloud-based environment to collect video streams of crowds in real-time and provide early-stage results. A novel dataset is generated based on five real-world experiments and their associated ground truth data to train the adapted EfficientNetV2B0 model. The experimental setups simulated a crowded event entrance, while the ground truths for each video experiment was generated manually by social psychologists. Several experiments on the videos and the generated dataset are carried out to evaluate the accuracy and annotation delay time of the proposed framework. The experimental results show that the proposed framework identified pushing behaviors with an accuracy rate of 87% within a reasonable delay time.
ClaPIM: Scalable Sequence CLAssification using Processing-In-Memory
Venue: arXiv cs.LG 2023
Year: 2023

DNA sequence classification is a fundamental task in computational biology with vast implications for applications such as disease prevention and drug design. Therefore, fast high-quality sequence classifiers are significantly important. This paper introduces ClaPIM, a scalable DNA sequence classification architecture based on the emerging concept of hybrid in-crossbar and near-crossbar memristive processing-in-memory (PIM). We enable efficient and high-quality classification by uniting the filter and search stages within a single algorithm. Specifically, we propose a custom filtering technique that drastically narrows the search space and a search approach that facilitates approximate string matching through a distance function. ClaPIM is the first PIM architecture for scalable approximate string matching that benefits from the high density of memristive crossbar arrays and the massive computational parallelism of PIM. Compared with Kraken2, a state-of-the-art software classifier, ClaPIM provides significantly higher classification quality (up to 20x improvement in F1 score) and also demonstrates a 1.8x throughput improvement. Compared with EDAM, a recently-proposed SRAM-based accelerator that is restricted to small datasets, we observe both a 30.4x improvement in normalized throughput per area and a 7% increase in classification precision.
Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection
Venue: arXiv cs.HC 2023
Year: 2023

Social visual behavior, as a type of non-verbal communication, plays a central role in studying social cognitive processes in interactive and complex settings of autism therapy interventions. However, for social visual behavior analytics in children with autism, it is challenging to collect gaze data manually and evaluate them because it costs a lot of time and effort for human coders. In this paper, we introduce a social visual behavior analytics approach by quantifying the mutual gaze performance of children receiving play-based autism interventions using an automated mutual gaze detection framework. Our analysis is based on a video dataset that captures and records social interactions between children with autism and their therapy trainers (N=28 observations, 84 video clips, 21 Hrs duration). The effectiveness of our framework was evaluated by comparing the mutual gaze ratio derived from the mutual gaze detection framework with the human-coded ratio values. We analyzed the mutual gaze frequency and duration across different therapy settings, activities, and sessions. We created mutual gaze-related measures for social visual behavior score prediction using multiple machine learning-based regression models. The results show that our method provides mutual gaze measures that reliably represent (or even replace) the human coders' hand-coded social gaze measures and effectively evaluates and predicts ASD children's social visual performance during the intervention. Our findings have implications for social interaction analysis in small-group behavior assessments in numerous co-located settings in (special) education and in the workplace.
QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversion
Venue: arXiv cs.SD 2023
Year: 2023

With the development of automatic speech recognition (ASR) and text-to-speech (TTS) technology, high-quality voice conversion (VC) can be achieved by extracting source content information and target speaker information to reconstruct waveforms. However, current methods still require improvement in terms of inference speed. In this study, we propose a lightweight VITS-based VC model that uses the HuBERT-Soft model to extract content information features without speaker information. Through subjective and objective experiments on synthesized speech, the proposed model demonstrates competitive results in terms of naturalness and similarity. Importantly, unlike the original VITS model, we use the inverse short-time Fourier transform (iSTFT) to replace the most computationally expensive part. Experimental results show that our model can generate samples at over 5000 kHz on the 3090 GPU and over 250 kHz on the i9-10900K CPU, achieving competitive speed for the same hardware configuration.
An Implementation of a Dual-Processor System on FPGA
Venue: arXiv cs.AR 2023
Year: 2023

In recent years, Field-Programmable Gate Arrays (FPGA) have evolved rapidly paving the way for a whole new range of computing paradigms. On the other hand, computer applications are evolving. There is a rising demand for a system that is general-purpose and yet has the processing abilities to accommodate current trends in application processing. This work proposes a design and implementation of a tightly-coupled FPGA-based dual-processor platform. We architect a platform that optimizes the utilization of FPGA resources and allows for the investigation of practical implementation issues such as cache design. The performance of the proposed prototype is then evaluated, as different configurations of a uniprocessor and a dual-processor system are studied and compared against each other and against published results for common industry-standard CPU platforms. The proposed implementation utilizes the Nios II 32-bit embedded soft-core processor architecture designed for the Altera Cyclone III family of FPGAs.
Cybersecurity of COSPAS-SARSAT and EPIRB: threat and attacker models, exploits, future research
Venue: arXiv cs.CR 2023
Year: 2023

COSPAS-SARSAT is an International programme for "Search and Rescue" (SAR) missions based on the "Satellite Aided Tracking" system (SARSAT). It is designed to provide accurate, timely, and reliable distress alert and location data to help SAR authorities of participating countries to assist persons and vessels in distress. Two types of satellite constellations serve COSPAS-SARSAT, low earth orbit search and rescue (LEOSAR) and geostationary orbiting search and rescue (GEOSAR). Despite its nearly-global deployment and critical importance, unfortunately enough, we found that COSPAS-SARSAT protocols and standard 406 MHz transmissions lack essential means of cybersecurity. In this paper, we investigate the cybersecurity aspects of COSPAS-SARSAT space-/satellite-based systems. In particular, we practically and successfully implement and demonstrate the first (to our knowledge) attacks on COSPAS-SARSAT 406 MHz protocols, namely replay, spoofing, and protocol fuzzing on EPIRB protocols. We also identify a set of core research challenges preventing more effective cybersecurity research in the field and outline the main cybersecurity weaknesses and possible mitigations to increase the system's cybersecurity level.
dump1030: open-source plug-and-play demodulator/decoder for 1030MHz uplink
Venue: arXiv cs.SE 2023
Year: 2023

Automatic Dependent Surveillance (ADS), Automatic Dependent Surveillance-Broadcast (ADS-B), Secondary Surveillance Radars (SSR), and Mode S are key air surveillance technologies representing a critical component of next-generation air transportation systems. However, compared to 1090MHz demodulators and decoders, which have plenty of implementations, the 1030MHz uplink receivers are, in general, scarcely, if at all, represented. In this paper, we present the development and evaluation of dump1030 - cross-platform plug-and-play open-source implementation for decoding 1030MHz uplink Mode A/C/S interrogations. We demonstrate and detail an agile development process of building dump1030 by adapting a state-of-the-art dump1090 design and implementation. In our repeated experiments, dump1030 achieves a high detection accuracy of 1030MHz interrogation signals based on lab evaluation using synthetically-generated interrogation signals. We also discuss a handful of practical use cases where dump1030 can find immediate application and implementation, both in research and industrial settings.
Towards Additively Manufactured Metamaterials with Powder Inclusions for Controllable Dissipation: The Critical Influence of Packing Density
Venue: arXiv cs.CE 2023
Year: 2023

Particle dampers represent a simple yet effective means to reduce unwanted oscillations when attached to structural components. Powder bed fusion additive manufacturing of metals allows to integrate particle inclusions of arbitrary shape, size and spatial distribution directly into bulk material, giving rise to novel metamaterials with controllable dissipation without the need for additional external damping devices. At present, however, it is not well understood how the degree of dissipation is influenced by the properties of the enclosed powder packing. In the present work, a two-way coupled discrete element - finite element model is proposed allowing for the first time to consistently describe the interaction between oscillating deformable structures and enclosed powder packings. As fundamental test case, the free oscillations of a hollow cantilever beam filled with various powder packings differing in packing density, particle size, and surface properties are considered to systematically study these factors of influence. Critically, it is found that the damping characteristics strongly depend on the packing density of the enclosed powder and that an optimal packing density exists at which the dissipation is maximized. Moreover, it is found that the influence of (absolute) particle size on dissipation is rather small. First-order analytical models for different deformation modes of such powder cavities are derived to shed light on this observation.
Probabilistic Shaping for the AWGN Channel
Venue: arXiv cs.IT 2023
Year: 2023

In this report, we study communication over an additive white Gaussian noise channel with a fixed signal constellation. We measure how much information we can send through this channel and how to improve the rate of communication by changing the input probability distribution. More precisely, we study the mutual information obtained from the Maxwell-Boltzmann distribution, the Blahut-Arimoto algorithm and a constrained version of the Blahut-Arimoto algorithm. We emphasise the fact that the Maxwell--Boltzmann distribution is not optimal. We also observe that the Blahut-Arimoto algorithm does not give us the best mutual information over SNR. To get the optimal distribution for a fixed SNR, we have to implement a constrained version of the Blahut-Arimoto algorithm.
Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses
Venue: arXiv cs.AI 2023
Year: 2023

In the vehicular mixed reality (MR) Metaverse, the distance between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous driving systems. Assisted by digital twin (DT) technologies, connected autonomous vehicles (AVs), roadside units (RSU), and virtual simulators can maintain the vehicular MR Metaverse via digital simulations for sharing data and making driving decisions collaboratively. However, large-scale traffic and driving simulation via realistic data collection and fusion from the physical world for online prediction and offline training in autonomous driving systems are difficult and costly. In this paper, we propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations for improving driving safety and traffic efficiency. First, we propose a multi-task DT offloading model for the reliable execution of heterogeneous DT tasks with different requirements at RSUs. Then, based on the preferences of AV's DTs and collected realistic data, virtual simulators can synthesize unlimited conditioned driving and traffic datasets to further improve robustness. Finally, we propose a multi-task enhanced auction-based mechanism to provide fine-grained incentives for RSUs in providing resources for autonomous driving. The property analysis and experimental results demonstrate that the proposed mechanism and architecture are strategy-proof and effective, respectively.
Incremental $(1-ε)$-approximate dynamic matching in $O(poly(1/ε))$ update time
Venue: arXiv cs.DS 2023
Year: 2023

In the dynamic approximate maximum bipartite matching problem we are given bipartite graph $G$ undergoing updates and our goal is to maintain a matching of $G$ which is large compared the maximum matching size $μ(G)$. We define a dynamic matching algorithm to be $α$ (respectively $(α, β)$)-approximate if it maintains matching $M$ such that at all times $|M | \geq μ(G) \cdot α$ (respectively $|M| \geq μ(G) \cdot α- β$). We present the first deterministic $(1-ε)$-approximate dynamic matching algorithm with $O(poly(ε^{-1}))$ amortized update time for graphs undergoing edge insertions. Previous solutions either required super-constant [Gupta FSTTCS'14, Bhattacharya-Kiss-Saranurak SODA'23] or exponential in $1/ε$ [Grandoni-Leonardi-Sankowski-Schwiegelshohn-Solomon SODA'19] update time. Our implementation is arguably simpler than the mentioned algorithms and its description is self contained. Moreover, we show that if we allow for additive $(1, ε\cdot n)$-approximation our algorithm seamlessly extends to also handle vertex deletions, on top of edge insertions. This makes our algorithm one of the few small update time algorithms for $(1-ε)$-approximate dynamic matching allowing for updates both increasing and decreasing the maximum matching size of $G$ in a fully dynamic manner.
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
Venue: arXiv cs.CV 2023
Year: 2023

The incredible generative ability of large-scale text-to-image (T2I) models has demonstrated strong power of learning complex structures and meaningful semantics. However, relying solely on text prompts cannot fully take advantage of the knowledge learned by the model, especially when flexible and accurate controlling (e.g., color and structure) is needed. In this paper, we aim to ``dig out" the capabilities that T2I models have implicitly learned, and then explicitly use them to control the generation more granularly. Specifically, we propose to learn simple and lightweight T2I-Adapters to align internal knowledge in T2I models with external control signals, while freezing the original large T2I models. In this way, we can train various adapters according to different conditions, achieving rich control and editing effects in the color and structure of the generation results. Further, the proposed T2I-Adapters have attractive properties of practical value, such as composability and generalization ability. Extensive experiments demonstrate that our T2I-Adapter has promising generation quality and a wide range of applications.
Deep Multi-stream Network for Video-based Calving Sign Detection
Venue: arXiv cs.CV 2023
Year: 2023

We have designed a deep multi-stream network for automatically detecting calving signs from video. Calving sign detection from a camera, which is a non-contact sensor, is expected to enable more efficient livestock management. As large-scale, well-developed data cannot generally be assumed when establishing calving detection systems, the basis for making the prediction needs to be presented to farmers during operation, so black-box modeling (also known as end-to-end modeling) is not appropriate. For practical operation of calving detection systems, the present study aims to incorporate expert knowledge into a deep neural network. To this end, we propose a multi-stream calving sign detection network in which multiple calving-related features are extracted from the corresponding feature extraction networks designed for each attribute with different characteristics, such as a cow's posture, rotation, and movement, known as calving signs, and are then integrated appropriately depending on the cow's situation. Experimental comparisons conducted using videos of 15 cows demonstrated that our multi-stream system yielded a significant improvement over the end-to-end system, and the multi-stream architecture significantly contributed to a reduction in detection errors. In addition, the distinctive mixture weights we observed helped provide interpretability of the system's behavior.
Sanity checks and improvements for patch visualisation in prototype-based image classification
Venue: arXiv cs.CV 2023
Year: 2023

In this work, we perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes - ProtoPNet and ProtoTree. Using two fine-grained datasets (CUB-200-2011 and Stanford Cars), we first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. Secondly, using a deletion metric, we demonstrate quantitatively that saliency methods such as Smoothgrads or PRP provide more faithful image patches. We also propose a new relevance metric based on the segmentation of the object provided in some datasets (e.g. CUB-200-2011) and show that the imprecise patch visualisations generated by ProtoPNet and ProtoTree can create a false sense of bias that can be mitigated by the use of more faithful methods. Finally, we discuss the implications of our findings for other prototype-based models sharing the same visualisation method.
A Field Guide for Pacing Budget and ROS Constraints
Venue: arXiv cs.GT 2023
Year: 2023

Budget pacing is a popular service that has been offered by major internet advertising platforms since their inception. Budget pacing systems seek to optimize advertiser returns subject to budget constraints by smoothly spending advertiser budgets. In the past few years, autobidding products that provide real-time bidding as a service to advertisers have seen a prominent rise in adoption. A popular autobidding strategy is value maximization subject to return-on-spend (ROS) constraints. For historical/business reasons, the systems that govern these two services, namely budget pacing and ROS pacing, are not always a unified and coordinated entity that optimizes a global objective subject to both constraints. The purpose of this work is to theoretically and empirically compare algorithms with different degrees of coordination between these two pacing systems. In particular, we compare (a) a fully-decoupled sequential algorithm that first constructs the advertiser's ROS-pacing bid and then lowers that bid for budget pacing; (b) a minimally-coupled min-pacing algorithm that runs these two services independently, obtains the bid multipliers from both of them and applies the minimum of the two multipliers as the effective multiplier; and (c) a fully-coupled dual-based algorithm that optimally combines the dual variables from both the systems. Our main contribution is to theoretically analyze the min-pacing algorithm and show that it attains similar guarantees to the fully-coupled canonical dual-based algorithm. On the other hand, we show that the sequential algorithm, even though appealing by virtue of being fully decoupled, could badly violate the constraints. We validate our theoretical findings empirically by showing that the min-pacing algorithm performs almost as well as the canonical dual-based algorithm on a semi-synthetic dataset based on a large online advertising platform's data.
Knowledge-based Intent Modeling for Next Generation Cellular Networks
Venue: arXiv cs.NI 2023
Year: 2023

Intent-based networking (IBN) facilitates the representation of consumer expectations in a declarative and domain-independent form. However, mapping intents to service and resource models remains an open challenge. IBN requires handling existing system data in a structured yet flexible structure way. Knowledge graphs provide an efficient conceptual framework for constructing contexts and organizing known information. We utilize knowledge graphs to construct a knowledge-based for modeling of intents in the networking domain. In addition, this work also proposes a knowledge-based intent modeling and processing methodology, extending the standardized intent common model proposed by TM Forum for next-generation cellular networks and services. The proposed knowledge-based IBN approach is demonstrated for next-generation cellular services, validating its potential.
Subjective Vertical Conflict Model with Visual Vertical: Predicting Motion Sickness on Autonomous Personal Mobility Vehicles
Venue: arXiv cs.HC 2023
Year: 2023

Passengers of level 3-5 autonomous personal mobility vehicles (APMV) can perform non-driving tasks, such as reading books and smartphones, while driving. It has been pointed out that such activities may increase motion sickness, especially when frequently avoiding pedestrians or obstacles in shared spaces. Many studies have been conducted to build countermeasures, of which various computational motion sickness models have been developed. Among them, models based on subjective vertical conflict (SVC) theory, which describes vertical changes in direction sensed by human sensory organs v.s. those expected by the central nervous system, have been actively developed. However, no current computational model can integrate visual vertical information with vestibular sensations. We proposed a 6 DoF SVC-VV model which added a visually perceived vertical block into a conventional 6 DoF SVC model to predict visual vertical directions from image data simulating the visual input of a human. In a driving experiment, 27 participants experienced an APMV with two visual conditions: looking ahead (LAD) and working with a tablet device (WAD). We verified that passengers got motion sickness while riding the APMV, and the symptom were severer when especially working on it, by simulating the frequent pedestrian avoidance scenarios of the APMV in the experiment. In addition, the results of the experiment demonstrated that the proposed 6 DoF SVC-VV model could describe the increased motion sickness experienced when the visual vertical and gravitational acceleration directions were different.