Reviewerly/exHarmony huggingface dataset:
Scene Re-ranking for Recommendation

Re-ranking is to refine the candidate ranking list of recommended items, such that the re-ranked list attracts users to purchase or click more items than the candidate one without re-ranking. Items in the candidate list are often ranked by their relevance to users' interests. It is thus important to exploit the mutual influence between items in the re-ranking process. Existing re-ranking models focus on only the pairwise influence between two items, and have limited capability to exploit the local mutual influence in a group of items. Users often show successive interests on a group of relevant items, e.g., mobile phone, phone covers, wireless headset, namely scene. We propose a novel re-ranking model that jointly exploits the local mutual influence in scenes and the global mutual influence between different scenes. Scene representations are learned by graph neural network and multi-head attention. In addition, matrix factorization is utilized to learn the interactive relationship between users and scenes. The final re-ranking list is generated by sorting the predicted scores of all scenes. The results on different datasets show that our method outperforms all other state-of-the-art algorithms significantly.
AC-PM: An Area-Efficient and Configurable Polynomial Multiplier for Lattice Based Cryptography

As the computation bottleneck in lattice-based cryptography (LBC), the polynomial multiplication based on number theoretic transform (NTT) has been continuously studied for flexible hardware implementations with high area-efficiency. This paper presents an area-efficient and configurable NTT-based polynomial multiplier (AC-PM) incorporating algorithmic and architectural level optimization techniques. For the core operation of polynomial multiplication, two low-complexity and fast modular multiplication algorithms are introduced with loose constraints of LBC-friendly primes. Based on the proposed algorithms, a reconfigurable processing element (RPE) is dedicatedly designed to execute all the operations in an NTT-based polynomial multiplication: NTT, inverse NTT (INTT), and coefficient-wise multiplication (CWM). The proposed AC-PM can be configured with different numbers of RPEs and supports various polynomial degrees without recompilation. Additionally, the dataflow complexity is greatly simplified. More importantly, to the best of our knowledge, the twiddle factors are reused, for the first time, to support both NTT and INTT with multiple polynomial degrees, which leads to increased flexibility of AC-PM with small overhead on hardware resource. FPGA implementation results demonstrate that the proposed AC-PM significantly outperforms the prior arts in both flexibility and area efficiency.
FLOAT: Factorized Learning of Object Attributes for Improved Multi-object Multi-part Scene Parsing

Multi-object multi-part scene parsing is a challenging task which requires detecting multiple object classes in a scene and segmenting the semantic parts within each object. In this paper, we propose FLOAT, a factorized label space framework for scalable multi-object multi-part parsing. Our framework involves independent dense prediction of object category and part attributes which increases scalability and reduces task complexity compared to the monolithic label space counterpart. In addition, we propose an inference-time ‘zoom’ refinement technique which significantly improves segmentation quality, especially for smaller objects/parts. Compared to state of the art, FLOAT obtains an absolute improvement of 2.0% for mean IOU (mIOU) and 4.8% for segmentation quality IOU (sqIOU) on the Pascal-Part-58 dataset. For the larger Pascal-Part-108 dataset, the improvements are 2.1% for mIOU and 3.9% for sqIOU. We incorporate previously excluded part attributes and other minor parts of the Pascal-Part dataset to create the most comprehensive and challenging version which we dub Pascal-Part-201. FLOAT obtains improvements of 8.6% for mIOU and 7.5% for sqIOU on the new dataset, demonstrating its parsing effectiveness across a challenging diversity of objects and parts. The code and datasets are available at floatseg.github.io.
Zoom-and-Reasoning: Joint Foreground Zoom and Visual-Semantic Reasoning Detection Network for Aerial Images

Aerial image object detection remains rather challenging, due to the small object gathering and confusion of inter-class similarities and intra-class diversity. Confronting such challenges, we propose a two-stage framework, ‘Zoom&Reasoning Det,’ which performs detection in a foreground highlight manner and leverages contextual relations to assist detection. In the coarse foreground zoom stage, different from earlier works that divide original images into patches and perform detection on each patch separately, we design Foreground Zoom Strategy (FZS), which zooms foreground dense regions from a coarse detector and packs them into one image. In the fine reasoning detect stage, motivated by a human visual mechanism that can achieve correct recognition by reasoning through context, we present Visual-Semantic Reasoning Network (VSRNet), consisting of Visual Reasoning Graph (VRG) and Semantic Reasoning Graph (SRG), simultaneously considering local visual and global semantic contextual relational information for each instance. Each instance feature representation is further refined by aggregating the outputs of VSRNet. Comprehensive experiments conducted on two challenging aerial image datasets, VisDrone and UAVDT, demonstrate the advantage of our method over the state-of-the-art.
Deep Attention-Guided Graph Clustering With Dual Self-Supervision

Existing deep embedding clustering methods fail to sufficiently utilize the available off-the-shelf information from feature embeddings and cluster assignments, limiting their performance. To this end, we propose a novel method, namely deep attention-guided graph clustering with dual self-supervision (DAGC). Specifically, DAGC first utilizes a heterogeneity-wise fusion module to adaptively integrate the features of the auto-encoder and the graph convolutional network in each layer and then uses a scale-wise fusion module to dynamically concatenate the multi-scale features in different layers. Such modules are capable of learning an informative feature embedding via an attention-based mechanism. In addition, we design a distribution-wise fusion module that leverages cluster assignments to acquire clustering results directly. To better explore the off-the-shelf information from the cluster assignments, we develop a dual self-supervision solution consisting of a soft self-supervision strategy with a Kullback-Leibler divergence loss and a hard self-supervision strategy with a pseudo supervision loss. Extensive experiments on nine benchmark datasets validate that our method consistently outperforms state-of-the-art methods. Especially, our method improves the ARI by more than 10.29% over the best baseline. The code will be publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZhihaoPENG-CityU/DAGC</uri> .
On the Instability of Relative Pose Estimation and RANSAC's Role

Relative pose estimation using the 5-point or 7-point Random Sample Consensus (RANSAC) algorithms can fail even when no outliers are present and there are enough inliers to support a hypothesis. These cases arise due to numerical instability of the 5- and 7-point minimal problems. This paper characterizes these instabilities, both in terms of minimal world scene configurations that lead to infinite condition number in epipolar estimation, and also in terms of the related minimal image feature pair correspondence configurations. The instability is studied in the context of a novel framework for analyzing the conditioning of minimal problems in multiview geometry, based on Riemannian manifolds. Experiments with synthetic and real-world data reveal that RANSAC does not only serve to filter out outliers, but RANSAC also selects for well-conditioned image data, sufficiently separated from the ill-posed locus that our theory predicts. These findings suggest that, in future work, one could try to accelerate and increase the success of RANSAC by testing only well-conditioned image data.
Traffic Analysis Through Deep-Learning-Based Image Segmentation From UAV Streaming

Alongside many traditional as well as novel applications, in the latest years, drones have been widely adopted as remote sensing platforms for road traffic monitoring in urban areas and on highways. The problem of traffic monitoring on Region of Interest (RoI) based on drone imagery is a challenging task, especially when the surveillance drone is constantly moving. In this work, two specific subtasks have been addressed. The goal of the first stage is to predict the RoI in drone imagery of traffic scenes using deep-learning (DL)-based approaches instead of traditional image processing; in this connection, the goal of the second task is to perform vehicle detection on the selected RoI. To ensure diversity and robustness, drone images with different altitudes, positions, and view points have been considered. To achieve these goals, two custom aerial data sets for RoI extraction and detection were built by collecting aerial sequences from flying unmanned aerial vehicles (UAVs) and by transmitting them to the base station leveraging 5G technology. Two different ad hoc DL-based architectures have been designed for the RoI extraction task to maximize the accuracy and inference speed, respectively, and have been evaluated on two different data sets: 1) a custom-built data set and 2) a Massachusetts roads data set. Finally, the models providing the best performance have been combined to further improve the overall results. Experimental tests show that the proposed framework represents a promising solution for drone-based road traffic monitoring in critical areas, exploiting imagery from a variety of viewing angles and altitudes.
Linear Frequency Residual Cepstral Features for Replay Spoof Detection on ASVSpoof 2019

Playing a pre-recorded speech to gain illegal access to Automatic Speaker Verification (ASV) system is one of the easiest attacks to execute but difficult to detect. Such attacks are called as replay attacks. Designing robust ASV systems from such attacks motivates to explore signal processing framework for Spoof Speech Detection (SSD). This paper exploits excitation source-based information in the form of Linear Frequency Residual Cepstral Coefficients (LFRCC) feature set for SSD task. In the source-filter model of speech production, the excitation source is also known to contain speaker-specific information. In this context, the residual obtained from Linear Prediction (LP) of speech is exploited in cepstral domain to detect replay attack. Improvements in results are obtained by choosing appropriate order of LP, as the order of LP controls the amount of information carried by the residual. Experiments performed on ASVSpoof 2019 Physical Access (PA) dataset using Gaussian Mixture Model (GMM) and Convolutional Neural Network (CNN) show that the optimal LP order is 8 which gives EER on the evaluation set as 17.30% and 15.21% using GMM and CNN classifiers, respectively.
Generation-Based Joint Luminance-Chrominance Learning for Underwater Image Quality Assessment

Underwater enhanced images (UEIs) are affected by not only the color cast and haze effect due to light attenuation and scattering, but also the over-enhancement and texture distortion caused by enhancement algorithms. However, existing underwater image quality assessment (UIQA) methods mainly focus on the inherent distortion caused by underwater optical imaging, and ignore the widespread artificial distortion, which leads to poor performance in evaluating UEIs. In this paper, a novel mapping-based underwater image quality representation is proposed. We divide underwater enhanced images into different domains and utilize a feature vector to measure the distance from the raw image domain to each enhanced image domain. The length and direction of the vector are defined as the enhancement degree and enhancement direction of the image. We construct a best enhancement direction and map other vectors to this direction to obtain the corresponding quality representation. Based on this, a novel network, called generation-based joint luminance-chrominance underwater image quality evaluation (GLCQE), is proposed, which is mainly divided into three parts: bi-directional reference generation module (BRGM), chromatic distortion evaluation network (CDEN), and sharpness distortion evaluation network (SDEN). BRGM is designed to generate two reference images about the unenhanced and the optimal enhanced versions of input UEI. In addition, the distortions in the luminance and chrominance domains of the UEI are analyzed. The luminance and chrominance channels of images are separated and input to SDEN and CDEN respectively to detect different distortions. A multi-scale feature mapping module is proposed in CDEN and SDEN to extract the feature representation of quality in chrominance and luminance of these images respectively. Moreover, a parallel spatial attention module is designed to focus on distortions in structural space by utilizing the different receptive fields of the convolution layer, due to the diverse manifestations of structural loss in the image. Finally, the mapped features extracted by two collaborative networks help the model evaluate the quality of underwater images more accurately. Extensive experiments demonstrate the superiority of our model against other representative state-of-the-art models.
Relationformer: A Unified Framework for Image-to-Graph Generation

A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blood-vessel network extraction, or scene graph generation. Traditionally, image-to-graph generation is addressed with a two-stage approach consisting of object detection followed by a separate relation prediction, which prevents simultaneous object-relation interaction. This work proposes a unified one-stage transformer-based framework, namely Relationformer that jointly predicts objects and their relations. We leverage direct set-based object prediction and incorporate the interaction among the objects to learn an object-relation representation jointly. In addition to existing [obj]-tokens, we propose a novel learnable token, namely [rln]-token. Together with [obj]-tokens, [rln]-token exploits local and global semantic reasoning in an image through a series of mutual associations. In combination with the pair-wise [obj]-token, the [rln]-token contributes to a computationally efficient relation prediction. We achieve state-of-the-art performance on multiple, diverse and multi-domain datasets that demonstrate our approach's effectiveness and generalizability. (Code is available at https://github.com/suprosanna/relationformer ).
Hybrid Quantum-Classical Recurrent Neural Networks for Time Series Prediction

This paper aims at solving time series prediction problems by means of a hybrid quantum-classical recurrent neural network. We propose a novel architecture based on stacked Long Short-Term Memory layers and a variational quantum layer. The latter employs a quantum feature map to embed input data into quantum states, which are then processed by a circuit ansatz. Finally, the expectation value of the circuit's outcome is taken over Pauli observables. Quantum properties such as superposition and entanglement are exploited to perform computations efficiently in a high-dimensional feature space. The proposed hybrid quantum-classical neural network is applied to a real-life challenging problem pertaining to the prediction of renewable energy time series. The comparison between the proposed approach and the classical counterpart shows that the former achieves better results in terms of prediction error, thus demonstrating better approximation of stochastic fluctuations and an overall effectiveness of the quantum variational approach also for prediction tasks.
Application of Federated Machine Learning in Manufacturing

A vast amount of data is created every minute, both in the private sector and industry. Whereas it is often easy to get hold of data in the private entertainment sector, in the industrial production environment it is much more difficult due to laws, preservation of intellectual property, and other factors. However, most machine learning methods require a data source that is sufficient in terms of quantity and quality. A suitable way to bring both requirements together is federated learning where learning progress is aggregated, but everyone remains the owner of their data. Federate learning was first proposed by Google researchers in 2016 and is used for example in the improvement of Google's keyboard Gboard. In contrast to billions of android users, comparable machinery is only used by few companies. This paper examines which other constraints prevail in production and which federated learning approaches can be considered as a result.
Effective Light Field De-Occlusion Network Based on Swin Transformer

Existing CNN-based light field de-occlusion (LF-DeOcc) methods suffer from occlusion removal performance degradation in the presence of large-size occlusions. In this paper, we infer that it is possibly caused by the limited receptive field of CNN and experimentally demonstrate that the de-occlusion performance is high-related to the receptive field. Therefore, a novel LF-DeOcc network based on Swin Transformer and CNN, which aims to exploit both global and local receptive fields, is firstly proposed for light field de-occlusion task. CNNs are employed at shallow layers to compensate for the deficiency of Transformers in extracting local features, while Transformers are employed at deep layers to capture the global patterns of large size occlusions. Hence, by integrating the global and local features, one could restore occlusion-free images effectively. For performance evaluation, a large dataset with mild-to-severe occlusions is developed and tested, with average occlusion rates of 19.82%, 32.13% and 40.56%, respectively. Experimental results show that the proposed network is superior to state-of-the-art methods, achieving 27.87 dB and 29.46 dB on the public dataset and our developed dataset, respectively. Finally, a new evaluation method has been presented in our work, i.e., by utilizing the real target detection task to evaluate the performance of LF de-occlusion algorithms. The practicability of our algorithm is validated using the new evaluation method.
Generating Robust DNN With Resistance to Bit-Flip Based Adversarial Weight Attack

Rowhammer Attack, a new DRAM-based attack, was developed exploiting weak cells to alter their content. Such attacks can be launched at the user level without requiring access permission to the victim memory cells. Leveraging such attacks, a new bit-flip-based adversarial weights attack (BFA) was developed targeting deep neural network models. When BFA attackers acquire a DNN model, they manipulate the existing DNN adversarial attack into locating vulnerable bits in the target DNN model. By flipping a subset of them using Rowhammer, they can crash that model within 30 trails. In this paper, we propose a lightweight and easy-to-deploy defense mechanism in the bit-level, Randomized Rotated and Nonlinear Encoding (RREC), which generates both robustness and fault-tolerant against BFA. Since flipping the most significant bit (MSB) in quantized data is too dangerous, we introduce randomized Rotation to obfuscate the bit order of model data and efficiently hide truly vulnerable bits with less vulnerable ones. Further, RREC reduces the average bit-flipped distance by more than 3x from the nonlinear encoding. It decreases the bit-flip distance among the majority of bits (including those vulnerable bits). Theoretically, RREC minimized the impact of a single bit BFA to 1/24 compared with baseline. Experimentally, RREC tolerates more than 17x flipped bits versus baseline model and 4.8x and 5.7x more bits compared with the existing BFA defenses (4B QAT and WR) with 0.01x to 0.08x of runtime latency. Moreover, we evaluate RREC against a newly emerged attack, Targeted-BFA, and it improves the defense rate from <inline-formula><tex-math notation="LaTeX">$5\%$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$95\%$</tex-math></inline-formula> .
Unsupervised Low-light Image Enhancement with Decoupled Networks

In this paper, we tackle the problem of enhancing real-world low-light images with significant noise in an unsupervised fashion. Conventional unsupervised approaches focus primarily on illumination or contrast enhancement but fail to suppress the noise in real-world low-light images. To address this issue, we decouple this task into two sub-tasks: illumination enhancement and noise suppression. We propose a two-stage, fully unsupervised model to handle these tasks separately. In the noise suppression stage, we propose an illumination-aware denoising model so that real noise at different locations is removed with the guidance of the illumination conditions. To facilitate the unsupervised training, we construct pseudo triplet samples and propose an adaptive content loss correspondingly to preserve contextual details. To thoroughly evaluate the performance of the enhancement models, we build a new unpaired real-world low-light enhancement dataset. Extensive experiments show that our proposed method outperforms the state-of-the-art unsupervised methods concerning both illumination enhancement and noise reduction.
SoCCMiner

Numerous tools exist for mining source code and software development process metrics. However, very few publicly available tools focus on source code comments, a crucial software artifact. This paper presents SoCCMiner (Source Code-Comments and Comment-Context Miner), a tool that offers multiple mining pipelines. It is the first readily available (plug-and-play) and customizable open-source tool for mining source code contextual information of comments at different granularities (Class comments, Method comments, Interface comments, and other granular comments). Mining comments at different source code granularities can aid researchers and practitioners working in a host of applications that focus on source code comments, such as Self-Admitted Technical Debt, Program Comprehension, and other applications. Furthermore, SoCCMiner is highly adaptable and extendable to include additional attributes and support other programming languages. This prototype supports the Java programming language.
A Survey on Table Question Answering: Recent Advances

Table Question Answering (Table QA) refers to providing precise answers from tables to answer a user's question. In recent years, there have been a lot of works on table QA, but there is a lack of comprehensive surveys on this research topic. Hence, we aim to provide an overview of available datasets and representative methods in table QA. We classify existing methods for table QA into five categories according to their techniques, which include semantic-parsing-based, generative, extractive, matching-based, and retriever-reader-based methods. Moreover, because table QA is still a challenging task for existing methods, we also identify and outline several key challenges and discuss the potential future directions of table QA.