Reviewerly/exHarmony huggingface dataset:
FastSwap: A Lightweight One-Stage Framework for Real-Time Face Swapping

Recent face swapping frameworks have achieved high-fidelity results. However, the previous works suffer from high computation costs due to the deep structure and the use of off-the-shelf networks. To overcome such problems and achieve real-time face swapping, we propose a lightweight one-stage framework, FastSwap. We design a shallow network trained in a self-supervised manner without any manual annotations. The core of our framework is a novel decoder block, called Triple Adaptive Normalization (TAN) block, which effectively integrates the identity and pose information. Besides, we propose a novel data augmentation and switch-test strategy to extract the attributes from the target image, which further enables controllable attribute editing. Extensive experiments on VoxCeleb2 and wild faces demonstrate that our framework generates high-fidelity face swapping results in 123.22 FPS and better preserves the identity, pose, and attributes than other state-of-the-art methods. Furthermore, we conduct an in-depth study to demonstrate the effectiveness of our proposal.
Accelerating Falcon Post-Quantum Digital Signature Algorithm on Graphic Processing Units

Since 2016, the National Institute of Standards and Technology (NIST) has been performing a competition to standardize post-quantum cryptography (PQC).Although Falcon has been selected in the competition as one of the standard PQC algorithms because of its advantages in short key and signature sizes, its performance overhead is larger than that of other lattice-based cryptosystems.This study presents multiple methodologies to accelerate the performance of Falcon using graphics processing units (GPUs) for server-side use.Direct GPU porting significantly degrades performance because the Falcon reference codes require recursive functions in its sampling process.Thus, an iterative sampling approach for efficient parallel processing is presented.In this study, the Falcon software applied a fine-grained execution model and reported the optimal number of threads in a thread block.Moreover, the polynomial multiplication performance was optimized by parallelizing the number-theoretic transform (NTT)-based polynomial multiplication and the fast Fourier transform (FFT)-based multiplication.Furthermore, dummy-based parallel execution methods have been introduced to handle the thread divergence effects.The presented Falcon software on RTX 3090 NVIDA GPU based on the proposed methods with Falcon-512 and Falcon-1024 parameters outperform at 35.14, 28.84, and 34.64 times and 33.31, 27.45, and 34.40 times, respectively, better than the central processing unit (CPU) reference implementation using Advanced Vector Extensions 2 (AVX2) instructions on a Ryzen 9 5900X running at 3.7 GHz in key generation, signing, and verification, respectively.Therefore, the proposed Falcon software can be used in servers managing multiple concurrent clients for efficient certificate verification and be used as an outsourced key generation and signature generation server for Signature as a Service (SaS).
Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attacks

State-of-the-art generative model-based attacks against image classifiers overwhelmingly focus on single-object(i.e., single dominant object) images. Different from such settings, we tackle a more practical problem of generating adversarial perturbations using multi-object (i.e., multiple dominant objects) images as they are representative of most real-world scenes. Our goal is to design an attack strategy that can learn from such natural scenes by leveraging the local patch differences that occur inherently in such images (e.g. difference between the local patch on the object ‘person’ and the object ‘bike’ in a traffic scene). Our key idea is to misclassify an adversarial multi-object image by confusing the victim classifier for each local patch in the image. Based on this, we propose a novel generative attack (called Local Patch Difference or LPD-Attack) where a novel contrastive loss function uses the aforesaid local differences in feature space of multi-object scenes to optimize the perturbation generator. Through various experiments across diverse victim convolutional neural networks, we show that our approach outperforms baseline generative attacks with highly transferable perturbations when evaluated under different white-box and black-box settings.
Expert Recommendation in Community Question Answering via Heterogeneous Content Network Embedding

Expert Recommendation (ER) aims to identify domain experts with high expertise and willingness to provide answers to questions in Community Question Answering (CQA) web services.How to model questions and users in the heterogeneous content network is critical to this task.Most traditional methods focus on modeling questions and users based on the textual content left in the community while ignoring the structural properties of heterogeneous CQA networks and always suffering from textual data sparsity issues.Recent approaches take advantage of structural proximities between nodes and attempt to fuse the textual content of nodes for modeling.However, they often fail to distinguish the nodes' personalized preferences and only consider the textual content of a part of the nodes in network embedding learning, while ignoring the semantic relevance of nodes.In this paper, we propose a novel framework that jointly considers the structural proximity relations and textual semantic relevance to model users and questions more comprehensively.Specifically, we learn topology-based embeddings through a hierarchical attentive network learning strategy, in which the proximity information and the personalized preference of nodes are encoded and preserved.Meanwhile, we utilize the node's textual content and the text correlation between adjacent nodes to build the content-based embedding through a meta-context-aware skip-gram model.In addition, the user's relative answer quality is incorporated to promote the ranking performance.Experimental results show that our proposed framework consistently and significantly outperforms the state-of-the-art baselines on three real-world datasets by taking the deep semantic understanding and structural feature learning together.The performance of the proposed work is analyzed in terms of MRR, P@K, and MAP and is proven to be more advanced than the existing methodologies.
A Dynamic Multi-Attribute Resource Bidding Mechanism with Privacy Protection in Edge Computing

In edge computing, a reasonable edge resource bidding mechanism can enable edge providers and users to obtain benefits in a relatively fair fashion.To maximize such benefits, this paper proposes a dynamic multiattribute resource bidding mechanism (DMRBM).Most of the previous work mainly relies on a third-party agent to exchange information to gain optimal benefits.It is worth noting that when edge providers and users trade with thirdparty agents which are not entirely reliable and trustworthy, their sensitive information is prone to be leaked.Moreover, the privacy protection of edge providers and users must be considered in the dynamic pricing/transaction process, which is also very challenging.Therefore, this paper first adopts a privacy protection algorithm to prevent sensitive information from leakage.On the premise that the sensitive data of both edge providers and users are protected, the prices of providers fluctuate within a certain range.Then, users can choose appropriate edge providers by the price-performance ratio (PPR) standard and the reward of lower price (LPR) standard according to their demands.The two standards can be evolved by two evaluation functions.Furthermore, this paper employs an approximate computing method to get an approximate solution of DMRBM in polynomial time.Specifically, this paper models the bidding process as a non-cooperative game and obtains the approximate optimal solution based on two standards according to the game theory.Through the extensive experiments, this paper demonstrates that the DMRBM satisfies the individual rationality, budget balance, and privacy protection and it can also increase the task offloading rate and the system benefits.
Symbiotic Organisms Search with Deep Learning Driven Biomedical Osteosarcoma Detection and Classification

Osteosarcoma is one of the rare bone cancers that affect the individuals aged between 10 and 30 and it incurs high death rate.Early diagnosis of osteosarcoma is essential to improve the survivability rate and treatment protocols.Traditional physical examination procedure is not only a timeconsuming process, but it also primarily relies upon the expert's knowledge.In this background, the recently developed Deep Learning (DL) models can be applied to perform decision making.At the same time, hyperparameter optimization of DL models also plays an important role in influencing overall classification performance.The current study introduces a novel Symbiotic Organisms Search with Deep Learning-driven Osteosarcoma Detection and Classification (SOSDL-ODC) model.The presented SOSDL-ODC technique primarily focuses on recognition and classification of osteosarcoma using histopathological images.In order to achieve this, the presented SOSDL-ODC technique initially applies image pre-processing approach to enhance the quality of image.Also, MobileNetv2 model is applied to generate a suitable group of feature vectors whereas hyperparameter tuning of MobileNetv2 model is performed using SOS algorithm.At last, Gated Recurrent Unit (GRU) technique is applied as a classification model to determine proper class labels.In order to validate the enhanced osteosarcoma classification performance of the proposed SOSDL-ODC technique, a comprehensive comparative analysis was conducted.The obtained outcomes confirmed the betterment of SOSDL-ODC approach than the existing approaches as the former achieved a maximum accuracy of 97.73%.
HRNetO: Human Action Recognition Using Unified Deep Features Optimization Framework

Human action recognition (HAR) attempts to understand a subject's behavior and assign a label to each action performed.It is more appealing because it has a wide range of applications in computer vision, such as video surveillance and smart cities.Many attempts have been made in the literature to develop an effective and robust framework for HAR.Still, the process remains difficult and may result in reduced accuracy due to several challenges, such as similarity among actions, extraction of essential features, and reduction of irrelevant features.In this work, we proposed an end-toend framework using deep learning and an improved tree seed optimization algorithm for accurate HAR.The proposed design consists of a few significant steps.In the first step, frame preprocessing is performed.In the second step, two pre-trained deep learning models are fine-tuned and trained through deep transfer learning using preprocessed video frames.In the next step, deep learning features of both fine-tuned models are fused using a new Parallel Standard Deviation Padding Max Value approach.The fused features are further optimized using an improved tree seed algorithm, and select the best features are finally classified by using the machine learning classifiers.The experiment was carried out on five publicly available datasets, including UT-Interaction, Weizmann, KTH, Hollywood, and IXAMS, and achieved higher accuracy than previous techniques.
OCR-VQGAN: Taming Text-within-Image Generation

Synthetic image generation has recently experienced significant improvements in domains such as natural image or art generation. However, the problem of figure and diagram generation remains unexplored. A challenging aspect of generating figures and diagrams is effectively rendering readable texts within the images. To alleviate this problem, we present OCR-VQGAN, an image encoder, and decoder that leverages OCR pre-trained features to optimize a text perceptual loss, encouraging the architecture to preserve high-fidelity text and diagram structure. To explore our approach, we introduce the Paper2Fig100k dataset, with over 100k images of figures and texts from research papers. The figures show architecture diagrams and methodologies of articles available at arXiv.org from fields like artificial intelligence and computer vision. Figures usually include text and discrete objects, e.g., boxes in a diagram, with lines and arrows that connect them. We demonstrate the effectiveness of OCR-VQGAN by conducting several experiments on the task of figure reconstruction. Additionally, we explore the qualitative and quantitative impact of weighting different perceptual metrics in the overall loss function. We release code, models, and dataset at https://github.com/joanrod/ocr-vqgan.
Attribute-Based Authentication Scheme from Partial Encryption for Lattice with Short Key

Wireless network is the basis of the Internet of things and the intelligent vehicle Internet.Due to the complexity of the Internet of things and intelligent vehicle Internet environment, the nodes of the Internet of things and the intelligent vehicle Internet are more vulnerable to malicious destruction and attacks.Most of the proposed authentication and key agreement protocols for wireless networks are based on traditional cryptosystems such as large integer decomposition and elliptic curves.With the rapid development of quantum computing, these authentication protocols based on traditional cryptography will be more and more threatened, so it is necessary to design some authentication and key agreement protocols that can resist quantum attacks.In this paper, an anti-quantum authentication scheme for wireless networks based on lattice cryptosystem is constructed.In the attribute-based authentication scheme, the length of the authenticated public-private key pair depends on the maximum order and complexity of the formula in the algorithm.In the attribute-based authentication scheme, there is a certain correlation between the authenticated data and the attribute value of the user in the scheme.We show that the attribute-based authentication scheme gives an attribute-based with smaller public-private key pairs.The security of the attribute-based authentication scheme is based on the sub-exponential hard problem of the LWE (Learning With Errors).The Q-poly made by the adversary in the scheme, and our attribute-based authentication scheme guarantees that private data about user attributes and ciphertext cannot be obtained by malicious attackers.
Learning Latent Structural Relations with Message Passing Prior

Learning disentangled representations is an important topic in machine learning with a wide range of applications. Disentangled latent variables represent interpretable semantic information and reflect separate factors of variation in data. Although generative models can learn latent representations as well, most existing models ignore the structural information among latent variables. In this paper, we propose a novel approach to learn the disentangled latent structural representations from data using decomposable variational auto-encoders. We design a novel message passing prior for the latent representations to capture the interactions among different data components. Different from many previous methods that ignore data component or object interaction, our approach simultaneously learns component representation and encodes component relationships. We have applied our model to tasks of data segmentation and latent representation learning among different data components. Experiments on several benchmarks demonstrate the utility of the proposed method.
Avoiding Lingering in Learning Active Recognition by Adversarial Disturbance

This paper considers the active recognition scenario, where the agent is empowered to intelligently acquire observations for better recognition. The agents usually compose two modules, i.e., the policy and the recognizer, to select actions and predict the category. While using ground-truth class labels to supervise the recognizer, the policy is typically updated with rewards determined by the current in-training recognizer, like whether achieving correct predictions. However, this joint learning process could lead to unintended solutions, like a collapsed policy that only visits views that the recognizer is already sufficiently trained to obtain rewards, which harms the generalization ability. We call this phenomenon lingering to depict the agent being reluctant to explore challenging views during training. Existing approaches to tackle the exploration-exploitation trade-off could be ineffective as they usually assume reliable feedback during exploration to update the estimate of rarely-visited states. This assumption is invalid here as the reward from the recognizer could be insufficiently trained.To this end, our approach integrates another adversarial policy to constantly disturb the recognition agent during training, forming a competing game to promote active explorations and avoid lingering. The reinforced adversary, rewarded when the recognition fails, contests the recognition agent by turning the camera to challenging observations. Extensive experiments across two datasets validate the effectiveness of the proposed approach regarding its recognition performances, learning efficiencies, and especially robustness in managing environmental noises.
Scaling Neural Face Synthesis to High FPS and Low Latency by Neural Caching

Recent neural rendering approaches greatly improve image quality, reaching near photorealism. However, the underlying neural networks have high runtime, precluding telepresence and virtual reality applications that require high resolution at low latency. The sequential dependency of layers in deep networks makes their optimization difficult. We break this dependency by caching information from the previous frame to speed up the processing of the current one with an implicit warp. The warping with a shallow network reduces latency and the caching operations can further be parallelized to improve the frame rate. In contrast to existing temporal neural networks, ours is tailored for the task of rendering novel views of faces by conditioning on the change of the underlying surface mesh. We test the approach on view-dependent rendering of 3D portrait avatars, as needed for telepresence, on established benchmark sequences. Warping reduces latency by 70% (from 49.4ms to 14.9ms on commodity GPUs) and scales frame rates accordingly over multiple GPUs while reducing image quality by only 1%, making it suitable as part of end-to-end view-dependent 3D teleconferencing applications.
Style-Guided Inference of Transformer for High-resolution Image Synthesis

Transformer is eminently suitable for auto-regressive image synthesis which predicts discrete value from the past values recursively to make up full image. Especially, combined with vector quantised latent representation, the state-of-the-art auto-regressive transformer displays realistic high-resolution images. However, sampling the latent code from discrete probability distribution makes the output unpredictable. Therefore, it requires to generate lots of diverse samples to acquire desired outputs. To alleviate the process of generating lots of samples repetitively, in this article, we propose to take a desired output, a style image, as an additional condition without re-training the transformer. To this end, our method transfers the style to a probability constraint to re-balance the prior, thereby specifying the target distribution instead of the original prior. Thus, generated samples from the re-balanced prior have similar styles to reference style. In practice, we can choose either an image or a category of images as an additional condition. In our qualitative assessment, we show that styles of majority of outputs are similar to the input style.
Explainability-Aware One Point Attack for Point Cloud Neural Networks

Recent studies have shown an increased interest to investigate the reliability of point cloud networks by adversarial attacks. However, most of the existing studies aim to deceive humans, while few address the operation principles of the models themselves. In this work, we propose two adversarial methods: One Point Attack (OPA) and Critical Traversal Attack (CTA), which target the points crucial to predictions more precisely by incorporating explainability methods. Our results show that popular point cloud networks can be deceived with high success rate by shifting only one point from the input instance. We also show the interesting impact of different point attribution distributions on the adversarial robustness of point cloud networks. We discuss how our approaches facilitate the explainability study for point cloud networks. To the best of our knowledge, this is the first point-cloud-based adversarial approach concerning explainability. Our code is available at https://github.com/Explain3D/Exp-One-Point-Atk-PC.
D-Extract: Extracting Dimensional Attributes From Product Images

Product dimension is a crucial piece of information enabling customers make better buying decisions. E-commerce websites extract dimension attributes to enable customers filter the search results according to their requirements. The existing methods extract dimension attributes from textual data like title and product description. However, this textual information often exists in an ambiguous, disorganised structure. In comparison, images can be used to extract reliable and consistent dimensional information. With this motivation, we hereby propose two novel architecture to extract dimensional information from product images. The first namely Single-Box Classification Net-work is designed to classify each text token in the image, one at a time, whereas the second architecture namely Multi-Box Classification Network uses a transformer network to classify all the detected text tokens simultaneously. To attain better performance, the proposed architectures are also fused with statistical inferences derived from the product category which further increased the F1-score of the Single-Box Classification Network by 3.78% and Multi-Box Classification Network by ≈ 0.9%≈. We use distance super-vision technique to create a large scale automated dataset for pretraining purpose and notice considerable improvement when the models were pretrained on the large data before finetuning. The proposed model achieves a desirable precision of 91.54% at 89.75% recall and outperforms the other state of the art approaches by ≈ 4.76% in F1-score <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Semantic Segmentation of Degraded Images Using Layer-Wise Feature Adjustor

Semantic segmentation of degraded images is important for practical applications such as autonomous driving and surveillance systems. The degradation level, which represents the strength of degradation, is usually unknown in practice. Therefore, the semantic segmentation algorithm needs to take account of various levels of degradation. In this paper, we propose a convolutional neural network of semantic segmentation which can cope with various levels of degradation. The proposed network is based on the knowledge distillation from a source network trained with only clean images. More concretely, the proposed network is trained to acquire multi-layer features keeping consistency with the source network, while adjusting for various levels of degradation. The effectiveness of the proposed method is confirmed for different types of degradations: JPEG distortion, Gaussian blur and salt&pepper noise. The experimental comparisons validate that the proposed network outperforms existing networks for semantic segmentation of degraded images with various degradation levels.