Reviewerly/exHarmony huggingface dataset:
Pixel-by-Pixel Cross-Domain Alignment for Few-Shot Semantic Segmentation

In this paper we consider the task of semantic segmentation in autonomous driving applications. Specifically, we consider the cross-domain few-shot setting where training can use only few real-world annotated images and many annotated synthetic images. In this context, aligning the domains is made more challenging by the pixel-wise class imbalance that is intrinsic in the segmentation and that leads to ignoring the underrepresented classes and overfitting the well represented ones. We address this problem with a novel framework called Pixel-By-Pixel Cross-Domain Alignment (PixDA). We propose a novel pixel-by-pixel domain adversarial loss following three criteria: (i) align the source and the target domain for each pixel, (ii) avoid negative transfer on the correctly represented pixels, and (iii) regularize the training of infrequent classes to avoid overfitting. The pixel-wise adversarial training is assisted by a novel sample selection procedure, that handles the imbalance between source and target data, and a knowledge distillation strategy, that avoids overfitting towards the few target images. We demonstrate on standard synthetic-to-real benchmarks that PixDA outperforms previous state-of-the-art methods in (1-5)-shot settings.<sup>1</sup>
Unsupervised Adversarial Network Alignment with Reinforcement Learning

Network alignment, which aims at learning a matching between the same entities across multiple information networks, often suffers challenges from feature inconsistency, high-dimensional features, to unstable alignment results. This article presents a novel network alignment framework, Unsupervised Adversarial learning based Network Alignment(UANA), that combines generative adversarial network (GAN) and reinforcement learning (RL) techniques to tackle the above critical challenges. First, we propose a bidirectional adversarial network distribution matching model to perform the bidirectional cross-network alignment translations between two networks, such that the distributions of real and translated networks completely overlap together. In addition, two cross-network alignment translation cycles are constructed for training the unsupervised alignment without the need of prior alignment knowledge. Second, in order to address the feature inconsistency issue, we integrate a dual adversarial autoencoder module with an adversarial binary classification model together to project two copies of the same vertices with high-dimensional inconsistent features into the same low-dimensional embedding space. This facilitates the translations of the distributions of two networks in the adversarial network distribution matching model. Finally, we develop an RL based optimization approach to solve the vertex matching problem in the discrete space of the GAN model, i.e., directly select the vertices in target networks most relevant to the vertices in source networks, without unstable similarity computation that is sensitive to discriminative features and similarity metrics. Extensive evaluation on real-world graph datasets demonstrates the outstanding capability of UANA to address the unsupervised network alignment problem, in terms of both effectiveness and scalability.
Multi-modal Dictionary BERT for Cross-modal Video Search in Baidu Advertising

Due to their attractiveness, video advertisements are adored by advertisers. Baidu, as one of the leading search advertisement platforms in China, is putting more and more effort into video advertisements for its advertisement customers. Search-based video advertisement display is, in essence, a cross-modal retrieval problem, which is normally tackled through joint embedding methods. Nevertheless, due to the lack of interactions between text features and image features, joint embedding methods cannot achieve as high accuracy as its counterpart based on attention. Inspired by the great success achieved by BERT in NLP tasks, many cross-modal BERT models emerge and achieve excellent performance in cross-modal retrieval. Last year, Baidu also launched a cross-modal BERT, CAN, in video advertisement platform, and achieved considerably better performance than the previous joint-embedding model. In this paper, we present our recent work for video advertisement retrieval, Multi-modal Dictionary BERT (MDBERT) model. Compared with CAN and other cross-modal BERT models, MDBERT integrates a joint dictionary, which is shared among video features and word features. It maps the relevant word features and video features into the same codeword and thus fosters effective cross-modal attention. To support end-to-end training, we propose to soften the codeword assignment. Meanwhile, to enhance the inference efficiency, we adopt the product quantization to achieve fine-level feature space partition at a low cost. After launching MDBERT in Baidu video advertising platform, the conversion ratio (CVR) increases by 3.34%, bringing a considerable revenue boost for advertisers in Baidu.
Semi-Supervised Dual Relation Learning for Multi-Label Classification

In a real-world scenario, an object could contain multiple tags instead of a single categorical label. To this end, multi-label learning (MLL) emerged. In MLL, the feature distributions are long-tailed and the complex semantic label relation and the long-tailed training samples are the main challenges. Semi-supervised learning is a potential solution. While, existing methods are mainly designed for single class scenario while ignoring the latent label relations. In addition, they cannot well handle the distribution shift commonly existing across source and target domains. To this end, a Semi-supervised Dual Relation Learning (SDRL) framework for multi-label classification is proposed. SDRL utilizes a few labeled samples as well as large scale unlabeled samples in the training stage. It jointly explores the inter-instance feature-level relation and the intra-instance label-level relation even from the unlabeled samples. In our model, a dual-classifier structure is deployed to obtain domain invariant representations. The prediction results from the classifiers are further compared and the most confident predictions are extracted as pseudo labels. A trainable label relation tensor is designed to explicitly explore the pairwise latent label relations and refine the predicted labels. SDRL is able to effectively and efficiently explore the feature-label relation as well as the label-label relation knowledge without any extra semantic knowledge. We evaluated SDRL in general and zero-shot multi-label classification tasks and we concluded that SDRL is superior to other SOTA baselines. Furthermore, extensive ablation studies have been done which reveal the effectiveness of each component in our framework.
Optimized Security Algorithms for Intelligent and Autonomous Vehicular Transportation Systems

With the growth of the Internet of Vehicles (IoV) in Intelligent Autonomous Transport Systems (IATS), a huge volume of data is exchanged between vehicles in these newly developed infrastructures. As a result, the requirements of securing data exchange between vehicles, autonomous or otherwise, have also increased tremendously. Securing data transfer and keeping a record of each transaction becomes a necessity in IoV/IATS. In this paper, we propose some optimized security algorithms using symmetric encryption for secure multimedia data transfer between vehicles. The main feature of these optimized algorithms is that they use a lower amount of data to generate fingerprints. The algorithms convert approximately <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.7 \times 10^{5}$ </tex-math></inline-formula> samples of data into 3600 samples to generate the fingerprint. Fast Fourier Transform (FFT) is used to fetch the highest three peak values of the signal in the frequency domain. A centralized server authenticates the data transfer by comparing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$HASH$ </tex-math></inline-formula> of the fingerprints and also keeps the transaction record. Through experimental analysis, the performance of proposed algorithms is confirmed by achieving reduced size samples to generate fingerprints and their authentication at the server-side.
Ship target detection of unmanned surface vehicle base on EfficientDet

Formulae display:?Mathematical formulae have been encoded as MathML and are displayed in this HTML version using MathJax in order to improve their display. Uncheck the box to turn MathJax off. This feature requires Javascript. Click on a formula to zoom.
Evaluating the user Experience of a Photorealistic Social VR Movie

We all enjoy watching movies together. However, this is not always possible if we live apart. While we can remotely share our screens, the experience differs from being together. We present a social Virtual Reality (VR) system that captures, reconstructs, and transmits multiple users’ volumetric representations into a commercially produced 3D virtual movie, so they have the feeling of “being there” together. We conducted a 48-user experiment where we invited users to experience the virtual movie either using a Head Mounted Display (HMD) or using a 2D screen with a game controller. In addition, we invited 14 VR experts to experience both the HMD and the screen version of the movie and discussed their experiences in two focus groups. Our results showed that both end-users and VR experts found that the way they navigated and interacted inside a 3D virtual movie was novel. They also found that the photorealistic volumetric representations enhanced feelings of co-presence. Our study lays the groundwork for future interactive and immersive VR movie co-watching experiences.
Path Planning Based on Mixed Algorithm of RRT and Artificial Potential Field Method

Aiming at the problems of random sampling and low efficiency of the path planning algorithm of rapidly-exploring random trees (RRT), an improved algorithm combined with the artificial potential field method (APF) is proposed. This method first introduces a probability value in the expansion step of the random tree in the basic RRT algorithm to speed up the convergence of the random tree to the target node and adds a gravitational component to the random tree to guide the tree to grow towards the target point to speed up the search process. Establish a repulsion field around obstacles to limit the search area between obstacles and reduce the randomness of the path. In this research, in addition to the RRT algorithm, the RRT* algorithm will also be used for improvement. The simulation experiment results show that the proposed method is significantly optimized in time, path length and the number of iterations.
Popularity-Enhanced News Recommendation with Multi-View Interest Representation

News recommendation is of vital importance to alleviating in-formation overload. Recent research shows that precise modeling of news content and user interests become critical for news rec-ommendation. Existing methods usually utilize information such as news title, abstract, entities to predict Click Through Rate(CTR) or add some auxiliary tasks to a multi-task learning framework. However, none of them directly consider predicted news popularity and the degree of users' attention to popular news into the CTR prediction results. Meanwhile, multiple inter-ests may arise throughout users' browsing history. Thus it is hard to represent user interests via a single user vector. In this paper, we propose PENR, a Popularity-Enhanced News Recommenda-tion method, which integrates popularity prediction task to im-prove the performance of the news encoder. News popularity score is predicted and added to the final CTR, while news popu-larity is utilized to model the degree of users' tendency to follow hot news. Moreover, user interests are modeled from different perspectives via a subspace projection method that assembles the browsing history to multiple subspaces. In this way, we capture users' multi-view interest representations. Experiments on a real-world dataset validate the effectiveness of our PENR approach.
SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentation

Temporal action segmentation is a task to classify each frame in the video with an action label. However, it is quite expensive to annotate every frame in a large corpus of videos to construct a comprehensive supervised training dataset. Thus in this work we propose an unsupervised method, namely SSCAP, that operates on a corpus of unlabeled videos and predicts a likely set of temporal segments across the videos. SSCAP leverages Self-Supervised learning to extract distinguishable features and then applies a novel Co-occurrence Action Parsing algorithm to not only capture the correlation among sub-actions under-lying the structure of activities, but also estimate the temporal path of the sub-actions in an accurate and general way. We evaluate on both classic datasets (Breakfast, 50Sal-ads) and the emerging fine-grained action dataset (Fine-Gym) with more complex activity structures and similar sub-actions. Results show that SSCAP achieves state-of-the-art performance on all datasets and can even outperform some weakly-supervised approaches, demonstrating its effectiveness and generalizability.
A Sentiment and Style Controllable Approach for Chinese Poetry Generation

Sentiment and style control are two vital aspects in automatic poetry generation. Excellent Chinese classical poetry should express a certain emotion and embody a specific style at the same time. Existing work still has deficiencies in controlling sentiment and style simultaneously. To address above issues, in this paper, we propose a novel approach for Chinese classical poetry generation, which can generate sentiment-controllable and style-controllable poems. First, it classifies hundreds of thousands of poems by style, sentiment, format, and primary keyword. Then, it utilizes masking self-attention mechanism to associate multiple tags and verses. Besides, it can generate metrical rhyming verses with distinctive sentiment and style characteristics according to the tag-set and secondary keywords. Finally, this approach is applied in Chang Qing Yin, which can collaborate with users to polish generated poems, providing alternatives automatically. Experimental results show that our approach performs well in sentiment and style control, and quality of generated poems outperforms several strong baselines.
Missing Value Filling Based on the Collaboration of Cloud and Edge in Artificial Intelligence of Things

With the development of 5G technology and Internet of Things, all kinds of real life data are collected and recorded by a large number of sensors. It is of great significance to mine and analyze the hidden information in the data for applications like future prediction. However, due to interferences or instability of collection equipment, collected sensory data are often incomplete, and this incompleteness hinders the in-depth analysis of data in the cloud. Therefore, processing around missing values is significant. Relying on cloud machine learning methods is not enough to deal with the problem of missing data in the Artificial Intelligence of Things (AIoT) environment, however, edge computing provides a promising solution. In this article, gated recurrent units filling is employed at the edge nodes. A mobile edge node can not only find the historical information of the current missing data node but also acquire the data of the nodes adjacent to the missing data node. These ensure that the missing data are restored to the maximum extent at the source. The experimental results show that the missing value filling based on edge computing not only outperforms other filling methods in quality but also greatly reduces the energy consumption in AIoT.
SCARF: A Semantic Constrained Attention Refinement Network for Semantic Segmentation

Semantic segmentation has achieved great progress by exploiting the contextual dependencies. In this paper, we propose an end-to-end Semantic Constrained Attention ReFinement (SCARF) network, based on semantic constrained contextual dependencies, to fully utilize the semantic information across different layers. Our novelties lie in the following aspects: Firstly, we present a general framework for capturing the non-local contextual dependencies. Secondly, within the framework, we introduce an efficient Category Attention (CA) block to capture semantic-related context by using the category constraint from coarse segmentation, which reduces the computational complexity from O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) to O(n) for image with n pixels. Thirdly, we overcome the contextual information confusion problem by balancing the non-local contextual dependencies and the local consistency adaptively using a category-wise learning weight. Finally, we fully utilize the multi-scale semantic-related con-textual information by refining the segmentation iteratively across layers with semantic constraint. Extensive evaluations demonstrate that our SCARF network significantly improves the segmentation results and achieves superior performance 85.0% mIoU on PASCAL VOC 2012, 55.0% mIoU on PASCAL Context, and 82.1% mIoU on Cityscapes.
Performance Analysis of Zero-Trust multi-cloud

Zero Trust security model permits to secure cloud native applications while encrypting all network communication, authenticating, and authorizing every request. The service mesh can enable Zero Trust using a side-car proxy without changes to the application code. To the best of our knowledge, no previous work has provided a performance analysis of Zero Trust in a multi-cloud environment. This paper proposes a multi-cloud framework and a testing workflow to analyse performance of the data plane under load and the impact on the control plane, when Zero Trust is enabled. The results of preliminary tests show that Istio has reduced latency variability in responding to sequential HTTP requests. Results also reveal that the overall CPU and memory usage can increase based on service mesh configuration and the cloud environment.
An ensemble of random decision trees with local differential privacy in edge computing

Edge computing is an emerging computing paradigm, which offers a great opportunity to implement data mining-based services and applications for a large number of devices and sensors in Internet of Things. However, the new paradigm is faced with security and privacy challenges due to the diversity and the limited capability of edge components. In particular, data privacy is one of the most concerned problems for all the participants. In this paper, we propose a framework of privacy-preserving data mining based on private random decision trees in edge computing, which not only gives the strong privacy guarantee, but also provides a certain amount of data utility. Firstly, we design a preservation framework to implement private random decision trees satisfying local differential privacy. Secondly, we present the concrete implementations of algorithms and the corresponding task that each participant needs to undertake. Thirdly, we analyze the key factors to influence privacy and utility, including the allocation of data and privacy budget. Fourthly, we give the improved algorithms to further increase the utility with strong privacy preservation. Finally, extensive experiments demonstrate the good performance of our designed framework.
Illumination invariant face recognition using Fused Cross Lattice Pattern of Phase Congruency (FCLPPC)

This paper presents a new facial feature descriptor called Fused Cross Lattice Pattern of Phase Congruency (FCLPPC) for high accuracy, homogeneous and heterogeneous illumination invariant cross-modal face recognition. Using the dimensionless phase congruency features, an effective illumination-invariant local feature extractor has been devised. To this end, a novel multi-directional binary pattern named Cross Lattice Pattern (CLP) has been proposed. CLP is applied to the previously extracted invariant phase congruency feature maps to generate the CLPPC images. Finally, weighted alpha-blending has been performed on the CLPPC maps to generate the Fused CLPPC (FCLPPC) feature map. Recognition results on Extended Yale-B, TUFTS, CMU-PIE, and CASIA NIR-VIS datasets have been presented to depict the superiority of the proposed scheme over other state-of-the-art methods. Additionally, the proposed FCLPPC has been combined with a lightweight Convolutional Neural Network to further augment the recognition accuracy.
A conceptual new model for use of solar water heaters in hot and dry regions

Solar heater has been known as an application of the solar thermal energy that is commonly employed for water heating. Purpose of this research is to introduce a conceptual model for employing solar water heaters (SWHs) in Yazd, Iran. Moreover, structural equation modeling (SEM) is used for assessing effect(s) of the available parameters on the suggested model. Political-cultural-social issues, economic issues, financial support, geographic location, infrastructure, and interactions were assessed as the independent variables of the study. AMOS was used to implement the processes and design the research questionnaire. According to SEM, the dependent variable "interaction" via "economic issues" exerted a significant positive effect of 0.21. The "geographical location" (-0.15) had a significantly negative effect and "financial support" (0.15) had a significantly positive effect on the application of SWHs. In addition, "geographic location" via the "infrastructure" factor had a significantly positive effect of 0.21 on the implementation and success of the SWH projects in Yazd city. Regarding Root Mean Square Error (RMSE), values between 0.05 and 0.08 are generally acceptable and values closer to zero represent more desirable goodness-of-fit. Thus, the RMSE obtained for this model (0.000) verifies the goodness of fit achieved by the model. Based on the results, participants had a positive attitude towards the utilization of the SWH in Yazd. From the extensive study of the available literature reviews, novelty of this research work is related to applying a new conceptual model for SWH use in hot and dry regions for the first time. Also, applicability of solar water heater considering geographical location, infrastructure, interactions, financial support, economic issues, and cultural-social-political issues has not yet been investigated. It was concluded that for developing and promoting the application of the SWH systems in Yazd city, the government must especially address the economic concerns, financial supports as well as infrastructures related to these efforts. Moreover, enhancing the levels of local engineering awareness and technologies related to construction and maintenance of the SWH and upgrading the design and construction methods for the above efforts would lead to the use of the solar-based systems with the increased efficiency.