Reviewerly/exHarmony huggingface dataset:
Modality Registration and Object Search Framework for UAV-Based Unregistered RGB-T Image Salient Object Detection

UAVs are widely used in various industries, and various visual tasks under the perspective of the UAV have been widely studied. In particular, the RGB-T detection method based on UAVs has shown significant advantages. However, existing RGB-T methods are designed based on registration image pairs rather than detecting images directly acquired by UAVs. This detection process is limited by the accuracy of image registration. And image registration wastes a lot of time. To solve the above problems, we construct an unregistered RGB-T image salient object detection (SOD) dataset under the UAV perspective, known as UAV RGB-T 2400. The dataset includes many challenging scenes, and the images are not manually registered. Further, we construct a modality registration and object search (MROS) framework for unregistered RGB-T SOD. Firstly, a modality registration scheme is proposed to solve the unregistration problem of modal features. We successively perform pixel-level registration from a local perspective and semantic-level registration from a global perspective for different modal features. And we carry out the channel and spatial interaction for the different modal features in modality registration. Aiming at the interference problem in the UAV detection environment, we propose an object search scheme. The two high-level features are used to search the object location, and the three low-level features are used to refine the object and produce prediction results. Experimental results on the UAV RGB-T 2400 dataset show that MROS is effective compared with state-of-the-art methods. The code is available at: https://github.com/VDT-2048/UAV-RGB-T-2400.
Deep Image Inpainting using Partial Convolutions

From random selfies to preserving the most beautiful part of a person's memory, images play a vital role in an individual's life and have various applications in computer vision. Images are most prone to damage, and it gets easily deteriorated. We use an amazing technique called image inpainting to deal with the preservation and reconstruction of damaged images. This technique refers to the process of reconstructing an image that is deteriorated due to noise in the image or corrupted or lost information. Image inpainting also has its major use in preserving the images in the form of paintings in the museum which preserves the culture and the heritage in form of art. There are various methods to perform this technique, but the most accurate ones hold onto the deep learning methodology. Here we predict the absence of the image elements using image inpainting to make the image more visually and semantically coherent. Partial convolution is one of the methods that have a major effect to focus on the restoration of damaged images. Partial convolutions are majorly used to deal with filling of missing data such as holes in images. We will be witnessing the total procedure in this paper.
Two Stage Path Planning Method for Co-Worked Double Industrial Robots

It is a significant capability for multi-industrial robots to plan an optimal collision-free path for both end-effectors and robotic arms. However, the path planning methods for co-worked multi-industrial robots, especially for closely co-worked industrial robots is still very limited. In this paper, to tackle the planning problem that has specified distance constraint of end-effector and complex collision avoidance, a two stage path planning method is proposed for the co-worked double industrial robot. In this two stage path planning method, the dual path planning with the distance constraint and the joint space planning of double robots are integrated sequentially. For the first stage, an algorithm named random sampling particle swarm optimization (RSPSO) is developed to plan the path for each end-effector, which can plan an optimal path with the collision avoidance and specified distance constraint. For the second stage, the joint space planning that combines the inverse kinematics, D-H method and collision detection is performed to find the angular displacements with collision avoidance for dual robotic arms. Two simulation examples and an experiment are used to verify the proposed method.
A Comparison of Re-Sampling Techniques for Detection of Multi-Step Attacks on Deep Learning Models

The increasing dependence on data analytics and artificial intelligence (AI) methodologies across various domains has prompted the emergence of apprehensions over data security and integrity.There exists a consensus among scholars and experts that the identification and mitigation of Multi-step attacks pose significant challenges due to the intricate nature of the diverse approaches utilized.This study aims to address the issue of imbalanced datasets within the domain of Multi-step attack detection.To achieve this objective, the research explores three distinct re-sampling strategies, namely over-sampling, undersampling, and hybrid re-sampling techniques.The study offers a comprehensive assessment of several re-sampling techniques utilized in the detection of Multi-step attacks on deep learning (DL) models.The efficacy of the solution is evaluated using a Multi-step cyber attack dataset that emulates attacks across six attack classes.Furthermore, the performance of several re-sampling approaches with numerous traditional machine learning (ML) and deep learning (DL) models are compared, based on performance metrics such as accuracy, precision, recall, F-1 score, and G-mean.In contrast to preliminary studies, the research focuses on Multi-step attack detection.The results indicate that the combination of Convolutional Neural Networks (CNN) with Deep Belief Networks (DBN), Long Short-Term Memory (LSTM), and Recurrent Neural Networks (RNN) provides optimal results as compared to standalone ML/DL models.Moreover, the results also depict that SMOTEENN, a hybrid re-sampling technique, demonstrates superior effectiveness in enhancing detection performance across various models and evaluation metrics.The findings indicate the significance of appropriate re-sampling techniques to improve the efficacy of Multi-step attack detection on DL models.
Statistical Data Analysis Models for Determining the Relevance of Structural Image Descriptions

The aim of the research is to improve the effectiveness of image recognition methods according to the description in the form of a set of keypoint descriptors. The focus is on increasing the speed of analysis and processing of description data while maintaining the required level of classification efficiency. The class of the image is provided as a description of the etalon. It is proposed to transform the description by implementing a statistical system of features for non-intersecting data fragments. The developed method is based on the aggregation of data distribution values within the description, the basis of which is the bit representation of the descriptors. Statistical features are calculated as the frequency of occurrence of the fixed value of a fragment on a set of description data and thus reflect the individual properties of images. Three main classifier models are analyzed: calculating the measure of data relevance in the form of distributions; assigning each of the descriptors to defined classes (voting); using the apparatus of statistical data analysis to decide on the significance of the difference between the distributions of the object and etalons. The results of software modeling of methods and calculations of statistical significance of differences based on distributions for training sets of images are represented. Using distributions instead of a set of descriptors increases the processing speed by hundreds of times, while the classification accuracy is maintained at a sufficient level and does not deteriorate compared to traditional voting.
Deep Learning Based Information Concealing in RGB Images

Hiding a secret image inside the cover image is a method applied in image steganography. The current methods use extracting the features manually and hide secretly in binary payloads and embed with cover images. This increase the payloads of the cover image and creates distortions of image and hidden image can be viewed. This study provides a detailed study on the image steganography existing methods and contributes by proposing the deep conceal neural network based on the customized encoder and decoder mechanism for image steganography. Experimented with standard benchmark dataset tiny imagenet dataset and comparative study made on state of art models. The performance of the proposed model is compared with metric such as Peak signal to noise ratio (PSNR) and structural index similarity (SSIM) values. From the obtained results the proposed model out performs than the state of art models.
Toward a Low-Resource Non-Latin-Complete Baseline: An Exploration of Khmer Optical Character Recognition

Many existing text recognition methods rely on the structure of Latin characters and words. Such methods may not be able to deal with non-Latin scripts that have highly complex features, such as character stacking, diacritics, ligatures, non-uniform character widths, and writing without explicit word boundaries. In addition, from a natural language processing (NLP) perspective, most non-Latin languages are considered low-resource due to the scarcity of large-scale data. This paper presents a convolutional Transformer-based text recognition method for low-resource non-Latin scripts, which uses local two-dimensional (2D) feature maps. The proposed method can handle images of arbitrarily long textlines, which may occur with non-Latin writing without explicit word boundaries, without resizing them to a fixed size by using an improved image chunking and merging strategy. It has a low time complexity in self-attention layers and allows efficient training. The Khmer script is used as the representative of non-Latin scripts because it shares many features with other non-Latin scripts, which makes the construction of an optical character recognition (OCR) method for Khmer as hard as that for other non-Latin scripts. Thus, by analogy with the AI-complete concept, a Khmer OCR method can be considered as one of the non-Latin-complete methods and can be used as a low-resource non-Latin baseline method. The proposed 2D method was trained on synthetic datasets and outperformed the baseline models on both synthetic and real datasets. Fine-tuning experiments using Khmer handwritten palm leaf manuscripts and other non-Latin scripts demonstrated the feasibility of transfer learning from the Khmer OCR method. To contribute to the low-resource language community, the training and evaluation datasets will be made publicly available.
SFAO: Sign-Flipping-Aware Optimization for Early-Stopping of Binarized Neural Networks

One of the vital challenges for the binary neural networks (BNNs) is improving their inference performance by expanding their data representation capabilities for figuring out delicate patterns and nuances in the data. Addressing the explosive computational demands on neural network training is essential to guarantee sustainable development and scalable deployment. However, mitigating the increase in the computational cost during the training phase is critical for ensuring sustainability and scalability during deployment. In this study, an advanced sign-flipping-aware optimizer (SFAO) that focuses on BNNs was introduced to diminish the computational burden. SFAO balanced the model performance and computational cost through sign-flipping-aware updating rules throughout the training of BNNs. SFAO optimizer, tailored for BNNs with binary weight-specific updating rules, considerably reduced the computing resources needed for training on the CIFAR-10 dataset. Specifically, it surpassed the conventional full-precision updating rule by reducing the total instruction count by 21.89%. In contrast, SFAO showed a marginal 0.44% decline in the image classification accuracy relative to the updating rules for the full-precision parameters. Furthermore, the implementation of early stopping using the sign flip rate led to a notable reduction of 9.37% in the average computation time per network for the ImageNet dataset.
JazzTok: Creativity, Community, and Improvisation on TikTok

Abstract Digital platforms provide many avenues for musicians to organize, share information, and collaborate on projects remotely, though some crucial aspects, like jazz improvisation, are harder to substitute online. Amid various other methods to facilitate digital performances, some musicians are experimenting with improvisation on short video platforms, such as TikTok. This article presents a case study of JazzTok, a digital collective of musicians who create and share jazz arrangements, recorded and edited by members of the community remotely on TikTok. Building on previous research, this article examines how this community of musicians formed in 2020 through repeated interactions improvising musical performances together using the TikTok Duet feature. Duet allows users to create new videos side by side with existing videos posted by other users, and they can be chained together to simulate the experience of musical jamming. This article relies on seventeen qualitative interviews with members of the JazzTok community to illustrate how the Duet feature offers a similar experience to the unpredictable and spontaneous nature of jazz improvisation. This article also explores how algorithmic recommendations influence the formation of digital musical communities of practice and how short video platforms are shifting modes of performance for jazz musicians. This contributes to studies of jazz and culture by suggesting future directions for researching the platformization of musical performances and jazz improvisation.
Scott Joplin in the Overworld: <i>Super Mario</i>'s Rag-Inflected Score

Abstract A rolling banjo, walking bass, steel drum, and syncopated piano. What strange music to accompany your trials and tribulations throughout the Mushroom Kingdom. Eclecticism pervades the Mario franchise and manifests in its digital places, the ways players interact within them, and, indeed, the sounds emanating from them. Mario and his jumping adventures have long been in the hearts of players, the minds of scholars, and the ears of ludomusicologists. By engaging with Koji Kondo's scores for early Mario games, this article highlights an overlooked component in the iconic scores of early Super Mario games: ragtime music. Furthermore, this research contributes examples of African American and Latino musical creation and innovation to ludomusicological discourse. I argue that, as a means to connect, contextualize, and enhance Mario's movement with music, composer Koji Kondo draws from ragtime and early jazz styles because of their distinctive rhythmic qualities. His rag-inflected score establishes ragtime as fundamental to early Super Mario soundscapes. While a substantial ludomusicological body of literature focuses on the Western classical tradition in games, jazz-centered research is scarce. This cross-disciplinary research draws on a wide range of scholarly perspectives from jazz studies and ludomusicology, including work by Andrew Schartmann, Edward Berlin, Ingrid Monson, Neil Lerner, David Butler, and Charles Hiroshi Garrett. As game audio is indispensable in facilitating a player's sense of verisimilitude of Mario's imaginative and, at times, nonsensical worlds, an inquiry into how jazz participates in this unique audiovisual relationship is long overdue.
PROSPER

We explore the application of Large Language Models (LLMs) (specifically GPT-3.5-turbo) to extract specifications and automating understanding of networking protocols from Internet Request for Comments (RFC) documents. LLMs have proven successful in specialized domains like medical and legal text understanding, and this work investigates their potential in automatically comprehending RFCs. We develop Artifact Miner, a tool to extract diagram artifacts from RFCs. We then couple extracted artifacts with natural language text to extract protocol automata using GPT-turbo 3.5 (chatGPT) and present our zero-shot and few-shot extraction results. We call this framework for FSM extraction 'PROSPER: Protocol Specification Miner'. We compare PROSPER with existing state-of-the-art techniques for protocol FSM state and transition extraction. Our experiments indicate that employing artifacts along with text for extraction can lead to lower false positives and better accuracy for both extracted states and transitions. Finally, we discuss efficient prompt engineering techniques, the errors we encountered, and pitfalls of using LLMs for knowledge extraction from specialized domains such as RFC documents.
Enriching Telepresence with Semantic-driven Holographic Communication

Achieving the optimal balance of minimizing bandwidth consumption and end-to-end latency while preserving a satisfactory level of visual quality becomes the ultimate goal of live, interactive holographic communication, a fundamental building block of immersive telepresence envisioned for 6G. Nevertheless, achieving this ambitious goal poses significant challenges for mobile devices with limited computing power, considering the substantial amount of 3D data to stream, the demanding latency requirements, and the high computation workload involved. Instead of distributing immersive content bit by bit, in this position paper, we propose to deliver semantic information extracted from telepresence participants to drastically reduce Internet bandwidth usage for task-oriented applications such as remote collaboration. We contribute a taxonomy by categorizing related semantics into three different types (i.e., keypoints, 2D images, and text), pinpoint the open research challenges associated with developing a practical system for each category in our comprehensive research agenda, and delve into the potential solutions for overcoming these challenges. The preliminary results from our proof-of-concept implementation that harnesses keypoint-based semantics (partially) validate the feasibility of our research agenda.
Logical Memory Pools

We propose logical memory pools, a memory disaggregation architecture for the emerging Compute Express Link (CXL) technology in datacenters. The key idea is to create a memory pool by carving out parts of the local memory in each server, rather than using a physical memory pool that is separate from servers. Logical pools provide significant benefits over physical pools, namely, lower cost, support for near-memory computing without extra hardware, and flexibility on designating whether memory is part of the memory pool or not. We demonstrate that logical pools can execute workloads that are unfeasible in physical pools, and that its faster access leads to better performance. Realizing logical memory pools poses five major challenges, which we believe can be overcome. Given the benefits of logical pools, we believe the CXL community should refocus efforts on logical, rather than physical memory pools.
Lightweb

This paper proposes lightweb, a new system for private browsing. A lightweb client can browse a web of text-based pages without revealing to anyone---not the network, not the servers hosting the pages---which pages it is reading. Unlike Tor and other anonymizing web proxies, which are inherently vulnerable to traffic-analysis attacks, lightweb's design protects against traffic-analysis attacks by design. While lightweb is expensive in relative terms (hundreds of core-seconds of server computation per page load), we show with microbenchmarks that the total system cost can be inexpensive in absolute terms (comparable to the cost of a Netflix membership). This paper does not present a polished system, but instead aims to spark discussion on radical approaches to a privacy-first web.
Learning a QoE Metric from Social Media and Gaming Footage

Defining a universal metric for Quality of Experience (QoE) is notoriously hard due to the complex relationship between low-level performance metrics and user satisfaction. The most common metric, the Mean Opinion Score (MOS), has well-known biases and inconsistency issues. We propose an alternative that leverages (a) social-media comments on network performance and (b) streaming footage that includes performance numbers. We argue that our proposal is feasible for online gaming, and it may apply to other applications in the near future. We discuss its potential to enable a direct mapping from low-level performance metrics to accurate QoE scores---the golden standard for assessing user satisfaction.
Boosting Application Performance using Heterogeneous Virtual Channels

Interactive networked applications require high throughput, low latency, and high reliability from the network to provide a seamless user experience. While meeting these three requirements simultaneously is difficult, there has been an emergence of heterogeneous virtual channels (HVCs) which support some subset of them at the expense of the others. For instance, URLLC sacrifices throughput to achieve low latency and reliability in 5G NR, and Wi-Fi 7 and other novel Internet architectures provide similar disparate types of service. Prior work either focuses on aggregating the bandwidth of these channels whilst neglecting their unique properties or fails to generalize in the sense of achieving high performance across different applications and channels. To utilize HVCs to their fullest, we argue that there are challenges and opportunities across the network, transport and application layers, and the application-transport interface of the network stack. In this work, we explore the trade-offs of these architectural choices in the context of web browsing and real-time video, and identify the constituting principles of a design that is general, performant, and deployable.
Datacenter Network Deserves Better Traffic Models

Traffic modeling of Datacenter Network (DCN) today is over-simplified, deviating from the ground truth. Adopted by numerous researchers, the common practice relies on the assumptions of traffic homogeneity and independence for ease of use. Based on our investigation of a real-world traffic dataset, we disprove these assumptions and point out the severe fidelity issue of the common practice that could invalidate many motivations and conclusions from influential research works. In this paper, we present Encore, a DCN traic modeling framework for ine-grained traic modeling and high-fidelity synthetic traffic generation. Leveraging machine learning techniques, Encore effectively extracts and preserves essential distribution and sequential features from raw traic. Preliminary experiments demonstrate that the traic generated by Encore not only restores the key features of real traffic but also achieves high consistency when used to evaluate network performance. We envision further expanding Encore to full-process traffic modeling and generation, and expect these critical improvements in traffic models can facilitate the DCN performance evaluation and optimization.
Green With Envy

Despite 40 years of active research on congestion control, there has been little or no consideration of how it impacts the energy usage of end-hosts or networking equipment. Particularly with the burgeoning energy consumption of data centers and wide-area networks, we argue that the time is ripe for the networking community to start thinking along these lines. To pave the way, we conduct lab experiments to measure the energy used by popular congestion control algorithms. We consider various aspects of congestion control and the rich research challenges that arise when we consider energy efficiency. Specifically, we find that fairness for the bandwidth allocated by congestion control can increase energy consumption by as high as 16%. We extrapolate these results to projected savings on the order of $10 million/year for large data centers.