Reviewerly/exHarmony huggingface dataset:
A Neurally Guided Patch-Based Style Transfer for Mobile Devices

Style transfer is an application that has increased interest, primarily because of the impressive results obtained using neural networks. However, this application demands a lot of computational resources, thus preventing its use in low-end mobiles. The patch-based approach is an interesting alternative that consumes less memory. This work uses two methods to generate high-resolution stylized images. Gated convolution in the neural network and the Halide language in the patch-based implementation are used for optimization. As a result, we were able to apply the style transfer method on a mobile device with 4 GB of memory RAM running under 6 seconds for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1920\times 1080$</tex> image size preserving the high-frequency details.
Angular Penalty for Few-Shot Incremental 3D Object Learning

3D object recognition has garnered notable success in both academic and industrial contexts. However, the majority of existing 3D object recognition approaches are tailor-made for static scenarios that have ample availability of training instances. Therefore, we target a more arduous and pragmatic task: few-shot incremental 3D object learning (FSI3DL), which aims to learn the new 3D objects in an incremental manner, yet new classes only have a few training instances. However, this task presents two challenges: overfitting to few-shot, and catastrophic forgetting on previous classes due to the absence of previous training instances. To mitigate these challenges, we introduce an Angular Penalty method to consume irregular mesh directly and reserve embedding space for incoming new classes. Specifically, we design a graph convolution network that can take advantage of the better representation ability of mesh and overcome its irregularity. Moreover, we use an angular penalty loss to increase the similarity between inra-class instances and reduce the similarity between instances from different classes. This can leave space in embedding space for incoming new classes. Experiments on representative 3D object classification datasets demonstrate the better efficacy of our method.
C-SRCIL: Complex-valued Class-Incremental Learning for Signal Recognition

In military and civilian scenarios, many signal recognition tasks are often accompanied by the growth of signal classes due to signal camouflage or increased communication devices. However, conventional approaches based on deep learning cannot effectively cope with the growth of classes. To address this problem, in this paper, we propose a complex-valued class-incremental learning (CIL) framework for signal recognition (C-SRCIL), which can extract the latent complex-valued features of modulation signals and complete the CIL for complex-valued signal recognition. In our C-SRCIL, We first decouple the feature representation from the classifier to combat catastrophic forgetting, and introduce complex-valued neural networks and an integrated loss function in the feature representation. Then a complex-valued CIL adapter is designed based on the nodalization idea for updating the decoupled classifier in the incremental stage. Experiments on datasets RADIOML2016.10A and SIGNAL2022 show that C-SRCIL brings about a 1.32% performance improvement with a 39.96% reduction in training time compared to the state-of-the-art. It suggests that C-SRCIL extends the classification boundaries of existing models while preserving previous knowledge.
Improving Single-Step Adversarial Training By Local Smoothing

The excellent model obtained through natural data training in deep learning is easily tampered with by adversarial examples. After discovering that, adversarial training has become the best way to defend against adversarial attacks and improve the robustness of the model. Since it is expensive to frequently calculate adversarial examples in each epoch during the training process, most people prefer to choose a single-step adversarial training method. However, the single-step adversarial training method will cause catastrophic overfitting and make the model lose robustness forever. In this paper, we explain adversarial training from the perspective of data augmentation, using artificial binary data to explore the reason for the occurrence of this overfitting. We propose two methods, VFSAT(Various fixed-stepsize single-step adversarial training) and GradSum, to prevent the overfitting in term of local smoothing and improve the robustness of the model obtained by single-step adversarial training. Simultaneously, experiments on CIFAR-10 and Tiny ImageNet datasets were constructed and the proof that single-step adversarial training could also resist multi-step adversarial attacks was derived.
Filter-based Online Neuro-Fuzzy Model Learning using Noisy Measurements

Neuro-Fuzzy (NF) model is capable of learning the nonlinear mapping between inputs and outputs accurately from training data and is thus a powerful tool for identification of nonlinear dynamic systems. However, when deploying the trained model, the noisy measurement leads to bias model predictions. Besides, training data is insufficient to cover the whole operating space for nonlinear systems. To well capture the system response, this paper proposes a recursive least squares algorithm to enable the NF model self-adaptive to different operating conditions whilst being robustness against measurement noise. Building on the data filtering technique and the auxiliary model theory, the proposed algorithm achieves high model prediction accuracy for online implementations. Efficacy of the algorithm is demonstrated by two simulation cases.
Pre-training A Prompt Pool for Vision-Language Model

Pre-trained vision-language(VL) model improves the performance in vision-language tasks, but requires a large amount of labeled data to apply the model to downstream tasks. It is challenging to obtain good results with limited training data. Prompt Tuning(PT), which freezes pre-train language models(PLMs) and only tunes soft prompts, provides an effective solution for adapting PLMs to downstream tasks. However, PT performs comparably with Full-model Tuning(FT) when the data are sufficient and performs much worse in few-shot settings, primarily due to the initialization of soft prompts. In this paper, we propose a new training framework Pre-train a Prompt Pool for Vision-Language Models called “P3VLM”, to give better initialization to PT. The objective of P3VLM is to optimize prompts selected by the visual feature, allowing the prompt to learn relevant knowledge in different data domains and store it in the prompt pool. Then we use diverse prompts in the prompt pool as the initialization of PT instead of a single pre-trained prompt. In the pre-train stage, the training samples are used to train a prompt pool by selecting prompts that best match the visual features. In the downstream datasets, the model selects the best matching prompts as the initialization for PT, freezes the PLM, and only tunes the prompts. Extensive experiments show that our method obtains strong performance on two image caption datasets in both zero-shot and few-shot scenarios.
MVMAFOL: A Multi-Access Three-Layer Federated Online Learning Algorithm for Internet of Vehicles

With the development of intelligent transportation system, it is urgent to transmit and analyze traffic data based on Internet of Vehicles. Federated learning has the characteristics of privacy protection, distributed learning and making full use of the computing power of local devices, which is very suitable for IoV's streaming big data analysis. Classical federated learning has only one global model, thus it can't be applied directly to IoV, where different region may have different data distribution. It is preferable that model of different region can be trained separately. Besides, the high mobility of vehicle nodes allows them to enter other areas while collecting data. This paper tries to solve the above issues. Firstly, a new vehicle-edge-cloud three-layer federated learning architecture for Io V, the Mobile Vehicle Multiple Access Federated Online Learning (MVMAFOL) algorithm, is proposed, where each vehicle terminal can upload the local model to multiple edge servers to maximize the use of its local model and effectively improve the generalization ability of the global model. Besides, a Model Similarity Vehicle Multiple Access (MSVMA) algorithm is proposed to improve the performance of MVMAFOL in the regional model aggregation phase, and an Edge Cloud Parameter Amendment (ECPA) algorithm is proposed for the global model aggregation phase. Simulation results show that this scheme can effectively improve the accuracy of the trained model.
Efficient Temporal Action Localization with Temporal Attention and Gaussian Weight

The task of temporal action localization is to recognize the action categories and meanwhile detect the start and end time of each action instance. In this paper, we propose a temporal attention and gaussian weighted anchor-free method, named TG-TAL, for temporal action localization. Rather than using anchors, our method regresses action instances directly with video frames as samples. To better address the variable length of action instance, we introduce a multi-level prediction framework with temporal attention. An additional gaussian weight branch is also defined to enhance the classification performance on low-quality temporal segments. Extensive experiments demonstrate that our method is effective on various datasets. In particular, on THUMOS14, our method outperforms one-stage temporal action localization methods and establishes a new state-of-the-art performance with an mAP(%) of 41.9 at tIoU threshold 0.5. Our method also works with two-stages methods and proposal postprocessing methods. Combined with PGCN, our method surpasses the state-of-the-art methods at tIoU threshold 0.7 and achieves a new state-of-the-art performance of 24.1 in terms of mAP(%) on THUMOS14.
ED-T2V: An Efficient Training Framework for Diffusion-based Text-to-Video Generation

Diffusion models have achieved remarkable performance on image generation. However, It is difficult to reproduce this success on video generation because of expensive training cost. In fact, pretrained image generation models have already acquired visual generation capabilities and could be utilized for video generation. Thus, we propose an Efficient training framework for Diffusion-based Text-to-Video generation (ED-T2V), which is built on a pretrained text-to-image generation model. To model the temporal dynamic information, we propose temporal transformer blocks with novel identity attention and temporal cross-attention. ED-T2V has the following advantages: 1) most of the parameters of pretrained model are frozen to inherit the generation capabilities and reduce the training cost; 2) the identity attention requires the currently generated frame to attend to all positions of its previous frame, thus providing an efficient way to keep main content consistent across frames and enable movement generation; 3) the temporal cross-attention is proposed to construct associations between textual descriptions and multiple video tokens in the time dimension, which could better model video movement than traditional cross-attention methods. With the aforementioned benefits, ED-T2V not only significantly reduces the training cost of video diffusion models, but also has excellent generation fidelity and controllability.
Hidden Design Principles in Zero-Cost Performance Predictors for Neural Architecture Search

Neural Architecture Search is an active research field that aims to design neural networks automatically. Nevertheless, this is usually an expensive process since the search algorithm must evaluate the performance of many candidate solutions from a vast search space. Because of that, different strategies have been proposed to perform efficient Neural Architecture Search. The recent development of zero-cost performance predictors has shown a lot of promise due to the possibility of predicting a network's performance without training. On the other hand, a predictor's correlation with a model's performance may depend on the network search space and on the benchmark dataset. Each performance predictor might lead the search processes to favor very different network patterns. A design principle is defined as a restriction in a hyperparameter distribution that is expected to yield optimum network performance. In this work, we propose an automatic iterative approach to uncover the design principles of deep neural networks optimized by zero-cost performance predictors, and we discuss insightful information obtained by its application.
Few Shot Object Detection with Incompletely Annotated Samples

Few shot object detection aims to generalize the model to previously unseen classes with only a few training samples, which has been attached great attention due to its practicability in real scenes. Many existing methods hold the missed detection issue, mainly due to the problem of incompletely annotated samples. Specifically, only partial objects in a sample are labeled, resulting in unlabeled objects being used as the background for training. This problem is especially serious for few-shot learning. To solve this noisy label problem, we first propose a label calibration method based on confidence to correct potential incorrect labels, and then introduce the class center library to eliminate the negative impact of unlabeled objects. We conduct extensive experiments on PASCAL VOC and MS-COCO benchmarks, which proves the effectiveness of our approach and achieves the state-of-the-art results.
Improving Complex Knowledge Base Question Answering with Relation-Aware Subgraph Retrieval and Reasoning Network

Complex Knowledge Base Question Answering aims to answer a complex question over a Knowledge Base. A mainstream solution is based on information retrieval, which usually extracts a pivotal subgraph from entire Knowledge Base to locate candidate answers, and then determines the plausible answers with semantic matching between candidate answers and the question. However, such a paradigm can have two critical problems: 1) Complex Knowledge Base Question Answering can be sensitive to the subgraph, since a small subgraph may exclude the answers, while a large one may introduce a lot of noise; 2) directly deriving answers with semantic matching neglects the global topology in the Knowledge Base, which may limit the capability in answer reasoning. To tackle above challenges, we propose the Relation-Aware Subgraph Retrieval and Reasoning Network, where relations are emphasized to construct subgraphs and answer reasoning. Specifically, we present a Relation-Aware Subgraph Retrieval (RASR) method to initialize and prune subgraphs with the guidance of relation semantics. To compre-hensively understand the complex correlations between the question and candidate answers, we put forward a Relation-Aware Reasoning Network (RARN), which contains a text reasoning module focusing on the semantics understanding of the question and a graph reasoning module focusing on mining the topology between the topic entities and the answers. Experiments on two classical benchmark datasets show that our reasoning model outperforms the state-of-the-art results of Information Retrieval models. What's more, data statistical analysis on the subgraphs demonstrates the effectiveness of our proposed subgraph retrieval method.
Detection of Electric Component Based on Improved Faster-RCNN

In the era of smart grid development, it is an important task to improve the efficiency and intelligence of the grid control system. One of the problems that need to be solved is the tedious and error-prone manual drawing of the CAD design diagram, which requires maintainers to refer to the original electrical diagram for manual drawing. This is a huge workload and is prone to mistakes. To solve this problem, the automatic identification of electrical component in diagrams is particularly critical, and existing methods are inadequate in terms of algorithmic accuracy and robustness. To achieve better automatic detection, we propose an improved Faster-RCNN algorithm to detect electrical components in diagrams. In order to further improve the recognition efficiency for a multiple scales of electrical components, we replaced the original feature extractor VGG-16(Visual Geometry Group 16-layer model) with ResNet-50 as the feature extractor and introduce a feature fusion network based on attention block to strengthen the detection ability of multi-size electrical components. In the RPN network, the k-means++ algorithm is introduced to better generate anchor boxes. To overcome the problem of information loss caused by directly reducing the original grid drawing to the predetermined size, the overlapping sliding window is used to detect the high-resolution electrical diagram.
OFA<sup>2</sup>: A Multi-Objective Perspective for the Once-for-All Neural Architecture Search

Once-for-All (OFA) is a Neural Architecture Search (NAS) framework designed to address the problem of searching efficient architectures for devices with different resources constraints by decoupling the training and the searching stages. The computationally expensive process of training the OFA neural network is done only once, and then it is possible to perform multiple searches for subnetworks extracted from this trained network according to each deployment scenario. In this work we aim to give one step further in the search for efficiency by explicitly conceiving the search stage as a multi-objective optimization problem. A Pareto frontier is then populated with efficient, and already trained, neural architectures exhibiting distinct trade-offs among the conflicting objectives. This could be achieved by using any multi-objective evolutionary algorithm during the search stage, such as NSGA-II and SMS-EMOA. In other words, the neural network is trained once, the searching for subnetworks considering different hardware constraints is also done one single time, and then the user can choose a suitable neural network according to each deployment scenario. The conjugation of OFA and an explicit algorithm for multi-objective optimization opens the possibility of a posteriori decision-making in NAS, after sampling efficient subnetworks which are a very good approximation of the Pareto frontier, given that those subnetworks are already trained and ready to use. The source code is available at https://github.com/ito-rafael/once-for-all-2.
Handwritten Chinese Character Generation via Embedding, Decomposition and Discrimination

Handwritten Chinese characters generation (HCCG) aims to automatically generate handwritten Chinese character images with various styles, which is very important for training a high performance handwritten Chinese character recognition model. Most existing HCCG methods take Chinese characters as a whole and ignore their structures and details, resulting in the lack of fine-grained representations and monotonous styles, which further affects the generalization ability of handwritten Chinese character recognition model. In this paper, we propose a novel HCCG method, named Embedding, Decomposition and Discrimination Network (EDDNet), which decomposes a Chinese character into a component sequence to represent its internal structures and radicals, so as to achieve the diversity of writing styles through fine-grained representations of handwritten Chinese characters. First, we propose a local style embedding module (LSEM) to inject target styles into content features. Then we decompose characters into sequences of components and propose a fine-grained content discriminator (FCD) to maintain the content integrity of generated images. Extensive experiments demonstrate that EDDNet can generate handwritten Chinese character images with more perfect details and realistic writing styles. Moreover, it is easy to extend to unseen styles and unseen characters.
Learning What, Where and Which to Transfer

Deep learning models often require large datasets to perform well from scratch. Transfer learning methods solve this issue by using a pre-trained source network to improve a target network training. Recent approaches involve using feature maps from the source network to guide the target network training. The latest transfer learning methods use meta-networks to enhance the knowledge transfer process. These meta-networks bridge the source and target networks, deciding which pairs of feature map layers and channels should be matched for optimal knowledge transfer. This paper improves this approach by using pixel-level information, in addition to layers and channels, for better knowledge transfer. Our experiments on multiple datasets show that the proposed approach outperforms previous baselines in scenarios with limited labels per class. The source code is available at https://github.com/lucasdelimanogueira/L2T-www.
Interpretable Diversity Analysis: Visualizing Feature Representations in Low-Cost Ensembles

Diversity is an important consideration in the construction of robust neural network ensembles. A collection of well trained models will generalize better if they are diverse in the patterns they respond to and the predictions they make. Diversity is especially important for low-cost ensemble methods because members often share network structure in order to avoid training several independent models from scratch. Diversity is traditionally analyzed by measuring differences between the outputs of models. However, this gives little insight into how knowledge representations differ between ensemble members. This paper introduces several interpretability methods that can be used to qualitatively analyze diversity. We demonstrate these techniques by comparing the diversity of feature representations between child networks using two low-cost ensemble algorithms, Snapshot Ensembles and Prune and Tune Ensembles. We use the same pre-trained parent network as a starting point for both methods which allows us to explore how feature representations evolve over time. This approach to diversity analysis can lead to valuable insights and new perspectives for how we measure and promote diversity in ensemble methods.
Time-multiplexed Reservoir Computing with Percolating Networks of Nanoparticles

Percolating Networks of Nanoparticles (PNNs) are under investigation as candidates for physical implementations of reservoir computing (RC). Several promising features have been identified in PNNs, such as brain-like critical dynamics, long-range spatiotemporal correlations and nonlinear I-V characteristics. However, the information processing capability of PNNs remains to be demonstrated. Here we present detailed modelling of PNNs operating in the tunneling regime as delayed-dynamical reservoirs (DDRs). The computational capacity of these reservoirs is successfully demonstrated by their performance in two benchmark tasks: waveform discrimination and tenth-order nonlinear auto-regressive moving average (NARMA10) time series prediction. Furthermore, the interplay between the PNN response time and the delayed feedback is elucidated, providing valuable insight for future DDR design.