Reviewerly/exHarmony huggingface dataset:
PhotoWCT<sup>2</sup>: Compact Autoencoder for Photorealistic Style Transfer Resulting from Blockwise Training and Skip Connections of High-Frequency Residuals

Photorealistic style transfer is an image editing task with the goal to modify an image to match the style of another image while ensuring the result looks like a real photograph. A limitation of existing models is that they have many parameters, which in turn prevents their use for larger image resolutions and leads to slower run-times. We introduce two mechanisms that enable our design of a more compact model that we call PhotoWCT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , which preserves state-of-art stylization strength and photorealism. First, we introduce blockwise training to perform coarse-to-fine feature transformations that enable state-of-art stylization strength in a single autoencoder in place of the inefficient cascade of four autoencoders used in PhotoWCT. Second, we introduce skip connections of high-frequency residuals in order to preserve image quality when applying the sequential coarse-to-fine feature transformations. Our PhotoWCT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> model requires fewer parameters (e.g., 30.3% fewer) while supporting higher resolution images (e.g., 4K) and achieving faster stylization than existing models.
Expanding Relationship for Cross Domain Recommendation

Cross-domain recommendation technique is a promising way to alleviate data sparsity issues by transferring knowledge from an auxiliary domain to a target domain. However, most existing works focus on utilizing the same users among different domains, while ignoring domain-specific users which forms the majority in real-world circumstances. In this paper, we propose a novel cross-domain learning approach--Relation Expansion based Cross-Domain Recommendation (ReCDR) to improve recommendation accuracies on small-overlapped domains. ReCDR first models the interactions in each domain as a local graph. It then forms a shared network by expanding out relationships using pre-trained node similarities. On the enhanced graph, ReCDR adopts a hierarchical attention mechanism. The output embedding will finally be combined with the local feature to balance the result for dual-target task. The proposed model is thoroughly evaluated on three real-world datasets. Experiments demonstrate superior performance compared to state-of-the-art methods.
High performance and energy efficient sobel edge detection

Sobel edge detection is widely used in computer vision and image processing but its processing time becomes a serious problem in real-time environments, especially when an image is very large. Instead of utilizing a hardware-accelerated approach, we propose a purely software-based method which is simpler and cheaper. Our algorithm reduces the number of arithmetic operations and data loads, so that processing speed is increased and energy consumption reduced. The processing time is further reduced by the use of AVX intrinsics and OpenMP directives which distribute the workload among the AVX engines in a multi-core architecture. Our algorithm reduces the number of arithmetic operations by 22.73% compared to that of the state-of-the-art Sobel (SOAS) algorithm, while the number of data loads are reduced by 43.75% compared to SOAS. Performance and energy consumption comparisons between our algorithm and SOAS, as well as with the Sobel functions offered by the OpenCV and IPP libraries are investigated, and the results demonstrate that a multi-core version of our algorithm, implemented by AVX intrinsics, is on average 3.20, 9.34, and 13.99 times faster than IPP, SOAS, and OpenCV respectively. Also, it consumes an average of 2.91, 8.43, and 11.21 times less energy than IPP, SOAS, and OpenCV. Our algorithm, utilizing software modifications alone, benefits from both shorter development time and reduced cost compared to hardware approaches relying on an FPGA, ASIC, or GPU, making it more suitable for resource-constrained environments.
A graph convolutional topic model for short and noisy text streams

Learning hidden topics from data streams has become absolutely necessary but posed challenging problems such as concept drift as well as short and noisy data. Using prior knowledge to enrich a topic model is one of potential solutions to cope with these challenges. Prior knowledge that is derived from human knowledge (e.g. Wordnet) or a pre-trained model (e.g. Word2vec) is very valuable and useful to help topic models work better. However, in a streaming environment where data arrives continually and infinitely, existing studies are limited to exploiting these resources effectively. Especially, a knowledge graph, that contains meaningful word relations, is ignored. In this paper, to aim at exploiting a knowledge graph effectively, we propose a novel graph convolutional topic model (GCTM) which integrates graph convolutional networks (GCN) into a topic model and a learning method which learns the networks and the topic model simultaneously for data streams. In each minibatch, our method not only can exploit an external knowledge graph but also can balance the external and old knowledge to perform well on new data. We conduct extensive experiments to evaluate our method with both a human knowledge graph (Wordnet) and a graph built from pre-trained word embeddings (Word2vec). The experimental results show that our method achieves significantly better performances than state-of-the-art baselines in terms of probabilistic predictive measure and topic coherence. In particular, our method can work well when dealing with short texts as well as concept drift.
Real-time posture feedback for effective motor learning in table tennis in virtual reality

• Existing VR simulations of table tennis do not have posture learning as the focus of teaching the sport. • We have developed a VR table tennis system with real-time posture feedback to examine learning and training in VR. • To overcome the limitation of existing platforms, we simulated ball flight physics to provide a near-real-world experience. • The experimental group showed significant improvements in motor learning and ball returning techniques following VR training. • Through this system, we explore whether complex skills can be taught in VR and transferred to the real world. Table tennis is a demanding sport in terms of motor skills, requiring precise time-critical postures and techniques for the player to excel. Therefore, it is a compelling case study for motor learning in Virtual Reality (VR). Though VR table tennis training applications have been used to study the learning process in VR, each has not fully explored the role of real-time feedback on motor learning, which is a fundamental component of real-life table tennis training. To address this, we have developed a training system that provides multi-modal training instructions and real-time posture guidance feedback to assist beginner players in improving their playing outcomes. This system synthesizes a table tennis training environment by providing realistic visual and audio stimuli that includes a custom highly-accurate physics implementation of ball flight. An experimental group (n = 9) trained to learn correct posture and paddle handling for forehand and backhand drives. The results show that the participants improved significantly in terms of technique and ball return quality. This work is the first VR table tennis system that provides real-time posture feedback using a low-cost depth camera. Additionally, the participants’ significant improvement in playing posture over a short period of time shows the integral role of feedback in learning and further validates VR as an effective motor learning tool.
BEVDetNet: Bird's Eye View LiDAR Point Cloud based Real-time 3D Object Detection for Autonomous Driving

3D object detection based on LiDAR point clouds is a crucial module in autonomous driving particularly for long range sensing. Most of the research is focused on achieving higher accuracy and these models are not optimized for deployment on embedded systems from the perspective of latency and power efficiency. For high speed driving scenarios, latency is a crucial parameter as it provides more time to react to dangerous situations. Typically a voxel or point-cloud based 3D convolution approach is utilized for this module. Firstly, they are inefficient on embedded platforms as they are not suitable for efficient parallelization. Secondly, they have a variable runtime due to level of sparsity of the scene which is against the determinism needed in a safety system. In this work, we aim to develop a very low latency algorithm with fixed runtime. We propose a novel semantic segmentation architecture as a single unified model for object center detection using key points, box predictions and orientation prediction using binned classification in a simpler Bird's Eye View (BEV) 2D representation. The proposed architecture can be trivially extended to include semantic segmentation classes like road without any additional computation. The proposed model has a latency of 4 ms on the embedded Nvidia Xavier platform. The model is 5X faster than other top accuracy models with a minimal accuracy degradation of 2% in Average Precision at <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">${\mathbf{I}\mathbf{o}\mathbf{U}=\boldsymbol{0.5}}$</tex> on KITTI dataset.
Multi-Stage Fusion for Multi-Class 3D Lidar Detection

In autonomous driving, the robust and accurate perceptions of the environment is a fundamental and challenging task. Resorting to the advancing of different sensors such as LiDAR and Camera, the autonomous systems are able to capture and process complementary perceptual information for better detection and classifying objects. In this paper, we propose a LiDAR-Camera fusion method for multi-class 3D object detection. The proposed method makes the utmost use of data from the two sensors by multiple fusion stages, and can be learned in an end-to-end manner. First, we apply a multi-level gated adaptive fusion mechanism with the feature extraction backbone. This point-wise fusion stage assiduously exploits the image and point cloud inputs, and obtains joint semantic representations of the scene. Next, given the regions of interest (RoIs) proposed based on the LiDAR features, the corresponding Camera features are selected by RoI-based feature pooling. These features are used to enrich the LiDAR features in local regions and enhance the proposal refinement. Moreover, we introduce a multi-label classification task as an auxiliary regularization to the object detection network. Without relying on extra labels, it helps the model better mine the extracted features and discover hard object instances. The experiments conducted on the KITTI dataset have proved all our fusion strategies are effective.
An Enhanced SiamMask Network for Coastal Ship Tracking

Coastal ship tracking is significant for cargo transportation and route determination. However, there are few effective tracking methods for specialized ship tracking. Although Siamese networks have been commonly used for object tracking in the field of deep learning, the results for the ship are not accurate due to the lack of contour and edge information. In addition, scale variation and seawater cause unstable ship movements, which aggravates the reduction in tracking accuracy. Therefore, we propose an enhanced SiamMask network for coastal ship tracking. Compared to the previous Siamese network, our algorithm has the following three advantages. First, we apply the unity of visual object tracking and semisupervised object segmentation to the ship tracking task, which completes target tracking while outputting edge shape information. Second, we propose a refined feature pyramid network that utilizes a feature fusion module and enhanced residual module (ERM) to solve the problems of scale variation in datasets. Third, we propose an attentionwise cross correlation with a multidimension attention module (MDAM) to focus more on ship targets and suppress nontargets through autonomous learning at width, height, and channel levels, which creates a tradeoff between the accuracy and the robustness of tracking algorithms. The experimental results show that our method achieves leading performance in LMD-TShips, outperforming most of the state-of-the-art trackers. Code is available at <uri>https://gitee.com/EnhancedSiamShipTracking/code</uri>.
Mine Me but Don’t Single Me Out: Differentially Private Event Logs for Process Mining

The applicability of process mining techniques hinges on the availability of event logs capturing the execution of a business process. In some use cases, particularly those involving customer-facing processes, these event logs may contain private information. Data protection regulations restrict the use of such event logs for analysis purposes. One way of circumventing these restrictions is to anonymize the event log to the extent that no individual can be singled out using the anonymized log. This paper addresses the problem of anonymizing an event log in order to guarantee that, upon disclosure of the anonymized log, the probability that an attacker may single out any individual represented in the original log, does not increase by more than a threshold. The paper proposes a differentially private disclosure mechanism, which oversamples the cases in the log and adds noise to the timestamps to the extent required to achieve the above privacy guarantee. The paper reports on an empirical evaluation of the proposed approach using 14 real-life event logs in terms of data utility loss and computational efficiency.
SSSIC: Semantics-to-Signal Scalable Image Coding With Learned Structural Representations

We address the requirement of image coding for joint human-machine vision, i.e., the decoded image serves both human observation and machine analysis/understanding. Previously, human vision and machine vision have been extensively studied by image (signal) compression and (image) feature compression, respectively. Recently, for joint human-machine vision, several studies have been devoted to joint compression of images and features, but the correlation between images and features is still unclear. We identify the deep network as a powerful toolkit for generating structural image representations. From the perspective of information theory, the deep features of an image naturally form an entropy decreasing series: a scalable bitstream is achieved by compressing the features backward from a deeper layer to a shallower layer until culminating with the image signal. Moreover, we can obtain learned representations by training the deep network for a given semantic analysis task or multiple tasks and acquire deep features that are related to semantics. With the learned structural representations, we propose SSSIC, a framework to obtain an embedded bitstream that can be either partially decoded for semantic analysis or fully decoded for human vision. We implement an exemplar SSSIC scheme using coarse-to-fine image classification as the driven semantic analysis task. We also extend the scheme for object detection and instance segmentation tasks. The experimental results demonstrate the effectiveness of the proposed SSSIC framework and establish that the exemplar scheme achieves higher compression efficiency than separate compression of images and features.
Locally weighted regression with different kernel smoothers for software effort estimation

Estimating software effort has been a largely unsolved problem for decades. One of the main reasons that hinders building accurate estimation models is the often heterogeneous nature of software data with a complex structure. Typically, building effort estimation models from local data tend to be more accurate than using the entire data. Previous studies have focused on the use of clustering techniques and decision trees to generate local and coherent data that can help in building local prediction models. However, these approaches may fall short in some aspect due to limitations in finding optimal clusters and processing noisy data. In this paper we used a more sophisticated locality approach that can mitigate these shortcomings that is Locally Weighted Regression (LWR). This method provides an efficient solution to learn from local data by building an estimation model that combines multiple local regression models in k-nearest-neighbor based model. The main factor affecting the accuracy of this method is the choice of the kernel function used to derive the weights for local regression models. This paper investigates the effects of choosing different kernels on the performance of Locally Weighted Regression of a software effort estimation problem. After comprehensive experiments with 7 datasets, 10 kernels, 3 polynomial degrees and 4 bandwidth values with a total of 840 Locally Weighted Regression variants, we found that: 1) Uniform kernel functions cannot outperform non-uniform kernel functions, and 2) kernel type, polynomial degrees and bandwidth parameters have no specific effect on the estimation accuracy. In other words, no change in bandwidth or degree values occurred with a significant difference in kernel rankings. In short, Locally Weighted Regression methods with Triweight or Triangle kernel can perform better than more complex kernels. Hence, we encourage non-uniform kernel methods as smoother function with wide bandwidth and small polynomial degree.
Towards Demystifying Intra-Function Parallelism in Serverless Computing

Serverless computing offers a pay-per-use model with high elasticity and automatic scaling for a wide range of applications. Since cloud providers abstract most of the underlying infrastructure, these services work similarly to black-boxes. As a result, users can influence the resources allocated to their functions, but might not be aware that they have to parallelize them to profit from the additionally allocated virtual CPUs (vCPUs). In this paper, we analyze the impact of parallelization within a single function and container instance for AWS Lambda, Google Cloud Functions (GCF), and Google Cloud Run (GCR). We focus on compute-intensive workloads since they benefit greatly from parallelization. Furthermore, we investigate the correlation between the number of allocated CPU cores and vCPUs in serverless environments. Our results show that the number of available cores to a function/container instance does not always equal the number of allocated vCPUs. By parallelizing serverless workloads, we observed cost savings up to 81% for AWS Lambda, 49% for GCF, and 69.8% for GCR.
Dynamic Federated Learning-Based Economic Framework for Internet-of-Vehicles

Federated learning (FL) can empower Internet-of-Vehicles (IoV) networks by leveraging smart vehicles (SVs) to participate in the learning process with minimum data exchanges and privacy disclosure. The collected data and learned knowledge can help the vehicular service provider (VSP) improve the global model accuracy, e.g., for road safety as well as better profits for both VSP and participating SVs. Nonetheless, there exist major challenges when implementing the FL in IoV networks, such as dynamic activities and diverse quality-of-information (QoI) from a large number of SVs, VSP's limited payment budget, and profit competition among SVs. In this paper, we propose a novel dynamic FL-based economic framework for an IoV network to address these challenges. Specifically, the VSP first implements an SV selection method to determine a set of the best SVs for the FL process according to the significance of their current locations and information history at each learning round. Then, each selected SV can collect on-road information and propose a payment contract to the VSP based on its collected QoI. For that, we develop a multi-principal one-agent contract-based policy to maximize the profits of the VSP and learning SVs under the VSP's limited payment budget and asymmetric information between the VSP and SVs. Through experimental results using real-world on-road datasets, we show that our framework can converge 57% faster (even with only 10% of active SVs in the network) and obtain much higher social welfare of the network (up to 27.2 times) compared with those of other baseline FL methods.
OGSRN: Optical-guided super-resolution network for SAR image

Although Convolutional Neural Networks (CNNs) have significantly improved the development of image Super-Resolution (SR) technology in recent years, the existing SR methods for SAR image with large scale factors have rarely been studied due to technical difficulty. A more efficient method is to obtain comprehensive information to guide the SAR image reconstruction. Indeed, the co-registered High-Resolution (HR) optical image has been successfully applied to enhance the quality of SAR image due to its discriminative characteristics. Inspired by this, we propose a novel Optical-Guided Super-Resolution Network (OGSRN) for SAR image with large scale factors. Specifically, our proposed OGSRN consists of two sub-nets: a SAR image Super-Resolution U-Net (SRUN) and a SAR-to-Optical Residual Translation Network (SORTN). The whole process during training includes two stages. In stage-1, the SR SAR images are reconstructed by the SRUN. And an Enhanced Residual Attention Module (ERAM), which is comprised of the Channel Attention (CA) and Spatial Attention (SA) mechanisms, is constructed to boost the representation ability of the network. In stage-2, the output of the stage-1 and its corresponding HR SAR images are translated to optical images by the SORTN, respectively. And then the differences between SR images and HR images are computed in the optical space to obtain feedback information that can reduce the space of possible SR solution. After that, we can use the optimized SRUN to directly produce HR SAR image from Low-Resolution (LR) SAR image in the testing phase. The experimental results show that under the guidance of optical image, our OGSRN can achieve excellent performance in both quantitative assessment metrics and visual quality.
Improving displacement measurement accuracy by compensating for camera motion and thermal effect on camera sensor

Displacement measurement is a major concept in the daily maintenance and failure assessment of engineering structures. Vision-based systems offer a cost-effective, simple alternative for non-contact displacement measurement. However, its measurement is challenging because of the errors induced by camera instability (thermal effects on camera sensor and camera motion). Researchers have tried to compensate for the errors induced by camera instability in displacement calculation methods by placing fixed objects in the background. However, remotely-located objects in the background cannot fully allow for camera rotations. Supplementary sensors can be employed to measure the camera motion in six degrees of freedom (6-DOF), although this method is unable to compensate for thermal effects on camera sensor. This paper presents a dual-view system that can compensate for the camera instability by using a camera, a beam splitter, a convex lens, and a photomask. The beam splitter and convex lens enable the camera to capture dual-view data and measure the instability of the system by using the photomask. In laboratory tests, the Root Mean Squared Error (RMSE) of the displacements measured by the proposed system did not exceed 0.05 mm under the conditions of drastic temperature variations and strong winds. Outdoor tests showed that the reflection of sunlight on the objects will seriously affect the practical performance of the proposed system, and a robust localization method for the photomask is necessary. The measurement scheme design for the track structure vibration of high-speed railways proved the acceptable engineering practicability and installation flexibility of the dual-view system.