Reviewerly/exHarmony huggingface dataset:
A Self-Supervised Learning Model for Unknown Internet Traffic Identification Based on Surge Period

The identification of Internet protocols provides a significant basis for keeping Internet security and improving Internet Quality of Service (QoS). However, the overwhelming developments and updating of Internet technologies and protocols have led to large volumes of unknown Internet traffic, which threaten the safety of the network environment a lot. Since most of the unknown Internet traffic does not have any labels, it is difficult to adopt deep learning directly. Additionally, the feature accuracy and identification model also impact the identification accuracy a lot. In this paper, we propose a surge period-based feature extraction method that helps remove the negative influence of background traffic in network sessions and acquire as many traffic flow features as possible. In addition, we also establish an identification model of unknown Internet traffic based on JigClu, the self-supervised learning approach to training unlabeled datasets. It finally combines with the clustering method and realizes the further identification of unknown Internet traffic. The model has been demonstrated with an accuracy of no less than 74% in identifying unknown Internet traffic with the public dataset ISCXVPN2016 under different scenarios. The work provides a novel solution for unknown Internet traffic identification, which is the most difficult task in identifying Internet traffic. We believe it is a great leap in Internet traffic identification and is of great significance to maintaining the security of the network environment.
Hyperspectral Reconnaissance: Joint Characterization of the Spectral Mixture Residual Delineates Geologic Unit Boundaries in the White Mountains, CA

We use a classic locale for geology education in the White Mountains, CA, to demonstrate a novel approach for using imaging spectroscopy (hyperspectral imaging) to generate base maps for the purpose of geologic mapping. The base maps produced in this fashion are complementary to, but distinct from, maps of mineral abundance. The approach synthesizes two concepts in imaging spectroscopy data analysis: the spectral mixture residual and joint characterization. First, the mixture residual uses a linear, generalizable, and physically based continuum removal model to mitigate the confounding effects of terrain and vegetation. Then, joint characterization distinguishes spectrally distinct geologic units by isolating residual, absorption-driven spectral features as nonlinear manifolds. Compared to most traditional classifiers, important strengths of this approach include physical basis, transparency, and near-uniqueness of result. Field validation confirms that this approach can identify regions of interest that contribute significant complementary information to PCA alone when attempting to accurately map spatial boundaries between lithologic units. For a geologist, this new type of base map can complement existing algorithms in exploiting the coming availability of global hyperspectral data for pre-field reconnaissance and geologic unit delineation.
High-Quality Video Watermarking Based on Deep Neural Networks for Video with HEVC Compression

This article presents a method for transparent watermarking of high-capacity watermarked video under H.265/HEVC (High-Efficiency Video Coding) compression conditions while maintaining high-quality encoded image. The aim of this paper is to present a method for watermark embedding using neural networks under conditions of subjecting video to lossy compression of the HEVC codec using the YUV420p color model chrominance channel for watermarking. This paper presents a method for training a deep neural network to embed a watermark when a compression channel is present. The discussed method is characterized by high accuracy of the video with an embedded watermark compared to the original. The PSNR (peak signal-to-noise ratio) values obtained are over 44 dB. The watermark capacity is 96 bits for an image with a resolution of 128 × 128. The method enables the complete recovery of a watermark from a single video frame compressed by the HEVC codec within the range of compression values defined by the CRF (constant rate factor) up to 22.
First Arrival Picking of Zero-Phase Seismic Data by Hilbert Envelope Empirical Half Window (HEEH) Method

First arrival travel time picking is an important step in many seismic data-processing applications. Most first arrival picking methods search for a sudden jump in seismic energy at trace onsets, which is clearly appropriate for minimum-phase data. This paper proposes a method for the first arrival picking of non-minimum phase data based on complex trace analysis. The Hilbert integral transform generates a complex seismic trace, followed by extraction of the envelope. The first arrival identification introduces an outlier detection method that uses the widely used three-sigma rule of thumb, which is commonly used in most software algorithms to identify outliers. The proposed method ultimately generates logical windows of ones (at the locations of outliers) and zeros (elsewhere). The first arrival is selected in the middle of the first outlier window. Testing the proposed method on zero-phase synthetic data with added 10% and 20% random noise, the method detected the true first arrivals accurately. Furthermore, tests on real Vibroseis data showed that the method recognizes the first arrivals with 67% accuracy within 20 milliseconds of their corresponding arrival times manually picked by an experienced geophysicist.
X-Education: Education of All Things with AI and Edge Computing—One Case Study for EFL Learning

Education usually only focuses on how to educate human beings with pedagogical or technical support. However, with artificial intelligence (AI) and edge computing, education can be extended and considered not only to educate human beings but also all things, such as physical or digital things. In this study, all things are given the opportunity to learn more about themselves and build their knowledge through interactions with other things, people, and AI agents. Thus, the X-Education framework is proposed in this study for educating all things, including human beings, physical, digital, and AI agents. One preliminary study for EFL writing was conducted to investigate not only whether all things can speed up their knowledge but also whether EFL learners as humans can also obtain the benefits of using X-Education. Further, the forwarding mechanisms of questioning and answering (Q&amp;A) were designed to speed up interactions among all things. In total, 22 learners were divided into two groups, the experimental group (EG) and the control group (CG), with/without the Q&amp;A forwarding mechanisms, respectively. A mixed-method approach with the two experimental phases was used in this study. The results showed that the knowledge of all things in the EG increased significantly more than the CG. Moreover, the EG received better EFL answers from the on-device AI with the forwarding mechanisms. They also felt that X-Education could help them to learn EFL writing better through Q&amp;A. Furthermore, it was demonstrated that X-Education can accommodate not only humans but also all things to improve their knowledge.
TopicVAE: Topic-aware Disentanglement Representation Learning for Enhanced Recommendation

Learning disentangled representations that reflect user preference based on user behavior (implicit feedback, such as click and purchase) and content information (e.g., plot description, poster) has become a hot research topic in modern recommender systems. However, most existing methods considering content information are not well-designed to disentangle user preference features due to neglecting the diversity of user preference on different semantic topics of items, resulting in sub-optimal performance and low interpretability. To address this problem, we propose a novelTopic-aware Disentangled Variational AutoEncoder (TopicVAE) to learn disentangled representations for enhanced recommendation. Specifically, we first utilize an attention-based topic extraction to extract the topic-level item representations and topic-item probability distribution from item content, and then introduce variational autoencoder to infer topic-level disentangled user representations. To guide the learning of topic-level disentanglement, we present a topic-guided self-supervised contrastive loss to promote the otherness of different topics by introducing a neighborhood-based user representation as guidance. Besides, a heuristic regularization is designed to force each dimension of the disentangled representations to independently reflect a fine-grained factor of a specific topic (e.g., red or blue for color) for feature-level disentanglement. Extensive experimental studies on three public datasets show that TopicVAE significantly outperforms several state-of-the-art baselines. Further empirical experiments also illustrate the interpretability of disentangled representations learned by TopicVAE.
Fine-tuning with Multi-modal Entity Prompts for News Image Captioning

News Image Captioning aims to generate descriptions for images embedded in news articles, including plentiful real-world concepts, especially about named entities. However, existing methods are limited in the entity-level template. Not only is it labor-intensive to craft the template, but it is error-prone due to local entity-aware, which solely constrains the prediction output at each language model decoding step with corrupted entity relationship. To overcome the problem, we investigate a concise and flexible paradigm to achieve global entity-aware by introducing a prompting mechanism with fine-tuning pre-trained models, named Fine-tuning with Multi-modal Entity Prompts for News Image Captioning (NewsMEP). Firstly, we incorporate two pre-trained models: (i) CLIP, translating the image with open-domain knowledge; (ii) BART, extended to encode article and image simultaneously. Moreover, leveraging the BART architecture, we can easily take the end-to-end fashion. Secondly, we prepend the target caption with two prompts to utilize entity-level lexical cohesion and inherent coherence in the pre-trained language model. Concretely, the visual prompts are obtained by mapping CLIP embeddings, and contextual vectors automatically construct the entity-oriented prompts. Thirdly, we provide an entity chain to control caption generation that focuses on entities of interest. Experiments results on two large-scale publicly available datasets, including detailed ablation studies, show that our NewsMEP not only outperforms state-of-the-art methods in general caption metrics but also achieves significant performance in precision and recall of various named entities.
Forcing the Whole Video as Background: An Adversarial Learning Strategy for Weakly Temporal Action Localization

With video-level labels, weakly supervised temporal action localization (WTAL) applies a localization-by-classification paradigm to detect and classify the action in untrimmed videos. Due to the characteristic of classification, class-specific background snippets are inevitably mis-activated to improve the discriminability of the classifier in WTAL. To alleviate the disturbance of background, existing methods try to enlarge the discrepancy between action and background through modeling background snippets with pseudo-snippet-level annotations, which largely rely on artificial hypotheticals. Distinct from the previous works, we present an adversarial learning strategy to break the limitation of mining pseudo background snippets. Concretely, the background classification loss forces the whole video to be regarded as the background by a background gradient reinforcement strategy, confusing the recognition model. Reversely, the foreground(action) loss guides the model to focus on action snippets under such conditions. As a result, competition between the two classification losses drives the model to boost its ability for action modeling. Simultaneously, a novel temporal enhancement network is designed to facilitate the model to construct temporal relation of affinity snippets based on the proposed strategy, for further improving the performance of action localization. Finally, extensive experiments conducted on THUMOS14 and ActivityNet1.2 demonstrate the effectiveness of the proposed method.
AGTGAN: Unpaired Image Translation for Photographic Ancient Character Generation

The study of ancient writings has great value for archaeology and philology. Essential forms of material are photographic characters, but manual photographic character recognition is extremely time-consuming and expertise-dependent. Automatic classification is therefore greatly desired. However, the current performance is limited due to the lack of annotated data. Data generation is an inexpensive but useful solution to data scarcity. Nevertheless, the diverse glyph shapes and complex background textures of photographic ancient characters make the generation task difficult, leading to unsatisfactory results of existing methods. To this end, we propose an unsupervised generative adversarial network called AGTGAN in this paper. By explicitly modeling global and local glyph shape styles, followed by a stroke-aware texture transfer and an associate adversarial learning mechanism, our method can generate characters with diverse glyphs and realistic textures. We evaluate our method on photographic ancient character datasets, e.g., OBC306 and CSDD. Our method outperforms other state-of-the-art methods in terms of various metrics and performs much better in terms of the diversity and authenticity of generated samples. With our generated images, experiments on the largest photographic oracle bone character dataset show that our method can achieve a significant increase in classification accuracy, up to 16.34%. The source code is available at https://github.com/Hellomystery/AGTGAN.
Imitated Detectors

Deep neural networks have shown great potential in many practical applications, yet their knowledge is at the risk of being stolen via exposed services (\eg APIs). In contrast to the commonly-studied classification model extraction, there exist no studies on the more challenging object detection task due to the sufficiency and efficiency of problem domain data collection. In this paper, we for the first time reveal that black-box victim object detectors can be easily replicated without knowing the model structure and training data. In particular, we treat it as black-box knowledge distillation and propose a teacher-student framework named Imitated Detector to transfer the knowledge of the victim model to the imitated model. To accelerate the problem domain data construction, we extend the problem domain dataset by generating synthetic images, where we apply the text-image generation process and provide short text inputs consisting of object categories and natural scenes; to promote the feedback information, we aim to fully mine the latent knowledge of the victim model by introducing an iterative adversarial attack strategy, where we feed victim models with transferable adversarial examples making victim provide diversified predictions with more information. Extensive experiments on multiple datasets in different settings demonstrate that our approach achieves the highest model extraction accuracy and outperforms other model stealing methods by large margins in the problem domain dataset. Our codes can be found at \urlhttps://github.com/LiangSiyuan21/Imitated-Detectors.
Adaptive Affine Transformation: A Simple and Effective Operation for Spatial Misaligned Image Generation

One challenging problem, named spatial misaligned image generation, describing a translation between two face/pose images with large spatial deformation, is widely faced in tasks of face/pose reenactment. Advanced researchers use the dense flow to solve this problem. However, under a complex spatial deformation, even using carefully designed networks, intrinsical complexities make it difficult to compute an accurate dense flow, leading to distorted results. Different from those dense flow based methods, we propose one simple but effective operator named AdaAT (Adaptive Affine Transformation) to realize misaligned image generation. AdaAT simulates spatial deformation by computing hundreds of affine transformations, resulting in less distortions. Without computing any dense flow, AdaAT directly carries out affine transformations in feature channel spaces. Furthermore, we package several AdaAT operators to one universal AdaAT module that is used for different face/pose generation tasks. To validate the effectiveness of our AdaAT, we conduct qualitative and quantitative experiments on four common datasets in the tasks of talking face generation, face reenactment, pose transfer and person image generation. We achieve state-of-the-art results on three of them.
Dream Painter: An Interactive Art Installation Bridging Audience Interaction, Robotics, and Creative AI

Dream Painter is an interactive robotic art installation that turns the audience's spoken dreams into a collective painting. By telling one's past dream, a participant guides the interactive robotic system in the latent space of the AI model that results in a multicolored line drawing. The artwork consists of several parts: an interaction station, a painting robot, a kinetic and animated mechanism that moves the paper roll when a drawing is finished, and the deep learning model that transforms a spoken word into a painting. All these interconnected components of hardware and software are arranged into an autonomous and interactive robotic art installation. The main aims of this project are to explore the interactive potential of AI technology and robotics, and trigger discussion over the deep learning applications in a wider sense. More precisely, this case study is primarily focused on the translation of different semiotic spaces as a trigger for creativity and audience interaction method.
Cartoon-Flow: A Flow-Based Generative Adversarial Network for Arbitrary-Style Photo Cartoonization

Photo cartoonization aims to convert photos of real-world scenes into cartoon-style images. Recently, generative adversarial network (GAN)-based methods for photo cartoonization have been proposed to generate pleasable cartoonized images. However, as these methods can transfer only learned cartoon styles to photos, they are limited in general-purpose applications where unlearned styles are often required. To address this limitation, an arbitrary style transfer (AST) method that transfers arbitrary artistic style into content images can be used. However, conventional AST methods do not perform satisfactorily in cartoonization for two reasons. First, they cannot capture the unique characteristics of cartoons that differ from common artistic styles. Second, they suffer from content leaks in which the semantic structure of the content is distorted. In this paper, to solve these problems, we propose a novel arbitrary-style photo cartoonization method, Cartoon-Flow. More specifically, we construct a new hybrid GAN with an invertible neural flow generator to effectively preserve content information. In addition, we introduce two new losses for cartoonization: (1) edge-promoting smooth loss to learn the unique characteristics of cartoons with smooth surfaces and clear edges, and (2) line loss to mimic the line drawing of cartoons. Extensive experiments demonstrate that the proposed method outperforms previous methods both quantitatively and qualitatively.
Invertible Rescaling Network and Its Extensions

Image rescaling is a commonly used bidirectional operation, which first downscales high-resolution images to fit various display screens or to be storage- and bandwidth-friendly, and afterward upscales the corresponding low-resolution images to recover the original resolution or the details in the zoom-in images. However, the non-injective downscaling mapping discards high-frequency contents, leading to the ill-posed problem for the inverse restoration task. This can be abstracted as a general image degradation–restoration problem with information loss. In this work, we propose a novel invertible framework to handle this general problem, which models the bidirectional degradation and restoration from a new perspective, i.e. invertible bijective transformation. The invertibility enables the framework to model the information loss of pre-degradation in the form of distribution, which could mitigate the ill-posed problem during post-restoration. To be specific, we develop invertible models to generate valid degraded images and meanwhile transform the distribution of lost contents to the fixed distribution of a latent variable during the forward degradation. Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable. We start from image rescaling and instantiate the model as Invertible Rescaling Network, which can be easily extended to the similar decolorization–colorization task. We further propose to combine the invertible framework with existing degradation methods such as image compression for wider applications. Experimental results demonstrate the significant improvement of our model over existing methods in terms of both quantitative and qualitative evaluations of upscaling and colorizing reconstruction from downscaled and decolorized images, and rate-distortion of image compression. Code is available at https://github.com/pkuxmq/Invertible-Image-Rescaling .
AggCast: Practical Cost-effective Scheduling for Large-scale Cloud-edge Crowdsourced Live Streaming

Conventional wisdom claims that in order to improve viewer engagement, the cloud-edge providers should serve the viewers with the nearest edge nodes, however, we show that doing this for crowdsourced live streaming (CLS) services can introduce significant costs inefficiency. We observe that the massive number of channels has greatly burdened the operating expenditure of the cloud-edge providers, and most importantly, unbalanced viewer distribution makes the edge nodes suffer significant costs inefficiency. To tackle the above concerns, we propose AggCast, a novel CLS scheduling framework to optimize the edge node utilization for the cloud-edge provider. The core idea of AggCast is to aggregate some viewers who are initially scattered on different regions, and assign them to fewer pre-selected nodes, thereby reducing bandwidth costs. In particular, by leveraging the insights obtained from our large-scale measurement, AggCast can not only ensure quality of experience (QoS), but also satisfy the systematic requirements of CLS services. AggCast has been A/B tested and fully deployed in a top cloud-edge provider in China for over eight months. The online and trace-driven experiments show that, compared to the common practice, AggCast can save over 15% back-to-source (BTS) bandwidth costs while having no negative impacts on QoS.