Reviewerly/exHarmony huggingface dataset:
Random Survival Forests Incorporated by the Nadaraya-Watson Regression

An attention-based random survival forest (Att-RSF) is presented in the paper. The first main idea behind this model is to adapt the Nadaraya-Watson kernel regression to the random survival forest so that the regression weights or kernels can be regarded as trainable attention weights under important condition that predictions of the random survival forest are represented in the form of functions, for example, the survival function and the cumulative hazard function. Each trainable weight assigned to a tree and a training or testing example is defined by two factors: by the ability of corresponding tree to predict and by the peculiarity of an example which falls into a leaf of the tree. The second main idea behind Att-RSF is to apply the Huber's contamination model to represent the attention weights as the linear function of the trainable attention parameters. The Harrell's C-index (concordance index) measuring the prediction quality of the random survival forest is used to form the loss function for training the attention weights. The C-index jointly with the contamination model lead to the standard quadratic optimization problem for computing the weights, which has many simple algorithms for its solution. Numerical experiments with real datasets containing survival data illustrate Att-RSF.
Long-tail Detection with Effective Class-Margins

Large-scale object detection and instance segmentation face a severe data imbalance. The finer-grained object classes become, the less frequent they appear in our datasets. However, at test-time, we expect a detector that performs well for all classes and not just the most frequent ones. In this paper, we provide a theoretical understanding of the long-trail detection problem. We show how the commonly used mean average precision evaluation metric on an unknown test set is bound by a margin-based binary classification error on a long-tailed object detection training set. We optimize margin-based binary classification error with a novel surrogate objective called Effective Class-Margin Loss (ECM). The ECM loss is simple, theoretically well-motivated, and outperforms other heuristic counterparts on LVIS v1 benchmark over a wide range of architecture and detectors. Code is available at https://github.com/janghyuncho/ECM-Loss .
A CNN-Based Born-Again TSK Fuzzy Classifier Integrating Soft Label Information and Knowledge Distillation

This article proposes a CNN-based born-again Takagi–Sugeno–Kang (TSK) fuzzy classifier denoted as CNNBaTSK. CNNBaTSK achieves the following distinctive characteristics: 1) CNNBaTSK provides a new perspective of knowledge distillation with a noniterative learning method (least learning machine with knowledge distillation, LLM-KD) to solve the consequent parameters of fuzzy rule, where consequent parameters are trained jointly on the ground-truth label loss, knowledge distillation loss, and regularization term; 2) with the inherent advantage of the fuzzy rule, CNNBaTSK has the capability to express the dark knowledge acquired from the CNN in an interpretable manner. Specifically, the dark knowledge (soft label information) is partitioned into five fixed antecedent fuzzy spaces. The centers of each soft label information in different fuzzy rules are {0, 0.25, 0.5, 0.75, 1}, which may have corresponding linguistic explanations: { <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">very low, low, medium, high, very high</i> }. For the consequent part of the fuzzy rule, the original features are employed to train the consequent parameters that ensure the direct interpretability in the original feature space. The experimental results on the benchmark datasets and the CHB-MIT EEG dataset demonstrate that CNNBaTSK can simultaneously improve the classification performance and model interpretability.
End-to-End Weakly Supervised Object Detection with Sparse Proposal Evolution

Conventional methods for weakly supervised object detection (WSOD) typically enumerate dense proposals and select the discriminative proposals as objects. However, these two-stage “enumerate-and-select” methods suffer object feature ambiguity brought by dense proposals and low detection efficiency caused by the proposal enumeration procedure. In this study, we propose a sparse proposal evolution (SPE) approach, which advances WSOD from the two-stage pipeline with dense proposals to an end-to-end framework with sparse proposals. SPE is built upon a visual transformer equipped with a seed proposal generation (SPG) branch and a sparse proposal refinement (SPR) branch. SPG generates high-quality seed proposals by taking advantage of the cascaded self-attention mechanism of the visual transformer, and SPR trains the detector to predict sparse proposals which are supervised by the seed proposals in a one-to-one matching fashion. SPG and SPR are iteratively performed so that seed proposals update to accurate supervision signals and sparse proposals evolve to precise object regions. Experiments on VOC and COCO object detection datasets show that SPE outperforms the state-of-the-art end-to-end methods by 7.0% mAP and 8.1% AP50. It is an order of magnitude faster than the two-stage methods, setting the first solid baseline for end-to-end WSOD with sparse proposals. The code is available at https://github.com/MingXiangL/SPE .
Assessing Bias in Face Image Quality Assessment

Face image quality assessment (FIQA) attempts to improve face recognition (FR) performance by providing additional information about sample quality. Because FIQA methods attempt to estimate the utility of a sample for face recognition, it is reasonable to assume that these methods are heavily influenced by the underlying face recognition system. Although modern face recognition systems are known to perform well, several studies have found that such systems often exhibit problems with demographic bias. It is therefore likely that such problems are also present with FIQA techniques. To investigate the demographic biases associated with FIQA approaches, this paper presents a comprehensive study involving a variety of quality assessment methods (general-purpose image quality as-sessment, supervised face quality assessment, and unsupervised face quality assessment methods) and three diverse state-of-the-art FR models. Our analysis on the Balanced Faces in the Wild (BFW) dataset shows that all techniques considered are affected more by variations in race than sex. While the general-purpose image quality assessment methods appear to be less biased with respect to the two demographic factors considered, the supervised and unsupervised face image quality assessment methods both show strong bias with a tendency to favor white individuals (of either sex). In addition, we found that methods that are less racially biased perform worse overall. This suggests that the observed bias in FIQA methods is to a significant extent related to the underlying face recognition system.
AudioIMU: Enhancing Inertial Sensing-Based Activity Recognition with Acoustic Models

Modern commercial wearable devices are widely equipped with inertial measurement units (IMU) and microphones. The motion and audio signals captured by these sensors can be used for recognizing a variety of user physical activities. Compared to motion data, audio data contains rich contextual information of human activities, but continuous audio sensing also poses extra data sampling burdens and privacy issues. Given such challenges, this paper studies a novel approach to augment IMU models for human activity recognition (HAR) with the superior acoustic knowledge of activities. Specifically, we propose a teacher-student framework to derive an IMU-based HAR model. Instead of training with motion data alone, an advanced audio-based teacher model is incorporated to guide the student HAR model. Once trained, the HAR model only takes as inputs motion data for inference. Based on a semi-controlled study with 15 participants, we show that an IMU model augmented with the proposed framework outperforms the original baseline model without augmentation (74.4% versus 70.0% accuracy) for recognizing 23 activities of daily living. We further discuss a few insights regarding the difference of model performance with and without our framework and possible trade-offs for actual deployment.
Implementation Of A Hybrid Recommendation System For Movies

In today’s digitally growing world, recommendation system plays a very important role in the media and entertainment industry. Streaming service providers like Netflix, YouTube, Amazon Prime, etc. are making tremendous use of recommendation systems in order to endorse their content to the users. A recommendation system is a subclass of an Information Filtering System that seeks to predict the rating or the preference a user might give to an item. The selection of proper datasets and algorithms are the two most important pillars of a recommendation system. This prototype focuses more on content and a collaborative-based approach which later shifts its focus to a hybrid model. Mathematical Modelling like Cosine Similarity and Single Value Deposition (SVD) is used in this project. Designing a user interface is also a part of this project where any user can display the results.
Speaker's Emotions Recognition Module Based on the GoogleLeNet Neural Network

The article is devoted to the development of means for recognition of the emotions of the speaker, based on the neural network analysis of fixed fragments of the voice signal. The possibility of improving recognition tools through the use of a convolutional neural network of the GoogLeNet type is shown. The expediency of training the network on the examples of the TESS database is determined. A procedure for representing a voice signal in the form of a square grayscale image has been developed. Based on the use of the specified database, the modification of the GoogLeNet network was carried out. The architecture and software implementation of the recognition module have been developed. With the help of computer experiments, it has been established that after 150 epochs of training on a fairly limited training sample, the modified GoogLeNet makes it possible to achieve an accuracy of recognition of the speaker's emotions of approximately 0.97 on test examples. The obtained accuracy is comparable to the accuracy of the best modern systems of this purpose and confirms the promise of using neural network models like GoogLeNet for emotion recognition based on voice signal analysis. The need for further research aimed at reducing the resource intensity of the neural network model is shown.
Perspective Phase Angle Model for Polarimetric 3D Reconstruction

Current polarimetric 3D reconstruction methods, including those in the well-established shape from polarization literature, are all developed under the orthographic projection assumption. In the case of a large field of view, however, this assumption does not hold and may result in significant reconstruction errors in methods that make this assumption. To address this problem, we present the perspective phase angle (PPA) model that is applicable to perspective cameras. Compared with the orthographic model, the proposed PPA model accurately describes the relationship between polarization phase angle and surface normal under perspective projection. In addition, the PPA model makes it possible to estimate surface normals from only one single-view phase angle map and does not suffer from the so-called $$\pi $$ -ambiguity problem. Experiments on real data show that the PPA model is more accurate for surface normal estimation with a perspective camera than the orthographic model.
Generative Adversarial Network for SAR-to-Optical Image Translation with Feature Cross-Fusion Inference

The translation of synthetic aperture radar (SAR) to optical images provides a new solution for the interpretation of SAR images. Most of the existing translation networks are based on generative adversarial networks and use 9-residual blocks or U-Net structures in the feature inference phase. Both structures cause a large amount of information lost during the conversion of SAR image features to optical features, making the outline of the translated image blurred or semantic information lost. Aiming at this problem, this paper proposes a cross-fusion inference network structure, which preserves both high-resolution features and low-resolution features in the whole process of feature inference. Our proposed method broadens the network horizontally while deepening it vertically and improving the image translation performance. The experiments conducted on the public dataset sen1-2 show that the proposed method is superior to other networks.
An Unsupervised Domain Adaptive Approach for Multimodal 2D Object Detection in Adverse Weather Conditions

Integrating different representations from complementary sensing modalities is crucial for robust scene interpretation in autonomous driving. While deep learning architectures that fuse vision and range data for 2D object detection have thrived in recent years, the corresponding modalities can degrade in adverse weather or lighting conditions, ultimately leading to a drop in performance. Although domain adaptation methods attempt to bridge the domain gap between source and target domains, they do not readily extend to heterogeneous data distributions. In this work, we propose an unsupervised domain adaptation framework, which adapts a 2D object detector for RGB and LiDAR sensors to one or more target domains featuring adverse weather conditions. Our proposed approach consists of three components. First, a data augmentation scheme that simulates weather distortions is devised to add domain confusion and prevent overfitting on the source data. Second, to promote cross-domain foreground object alignment, we leverage the complementary features of multiple modalities through a multi-scale entropy-weighted domain discriminator. Finally, we use carefully designed pretext tasks to learn a more robust representation of the target domain data. Experiments performed on the DENSE dataset show that our method can substantially alleviate the domain gap under the single-target domain adaptation setting and the less explored yet more general multi-target domain adaptation setting.
CoGNet: Cooperative Graph Neural Networks

Graph representation learning has received increasing attention in recent years for many real-world applications. A major challenge in graph representation learning is the lack of labeled data. To address this challenge, Graph Neural Networks (GNNs) use message passing frameworks to combine information from unlabeled data with labeled data. However, the use of unlabeled data under the message passing framework is indirect in the training process where unlabeled data does not supervise the training process. To fully exploit the potential of unlabeled data, we propose a novel dual-view cooperative training framework for graph data where unlabeled data is involved in the training process for supervision. Specifically, we regard different views as the reasoning processes of two GNN models with which the models make predictions, integrating the understanding of different models on the underlying graph. To exchange information between models, we design a pseudo-label-based approach, where the two models mutually provide pseudo labels to each other iteratively. Moreover, to ensure the quality of pseudo labels, we propose an entropy-based pseudo-labels selection procedure and we adopt GNNExplainer to visualize different views in our framework. Our comprehensive experimental evaluation shows that our methods can boost the performance of state-of-the-art models.
Wnet: Audio-Guided Video Object Segmentation via Wavelet-Based Cross- Modal Denoising Networks

Audio-Guided video object segmentation is a challenging problem in visual analysis and editing, which automatically separates foreground objects from the background in a video sequence according to the referring audio expressions. However, existing referring video object segmentation works mainly focus on the guidance of text-based referring expressions, due to the lack of modeling the semantic representations of audio-video interaction contents. In this paper, we consider the problem of audio-guided video semantic segmentation from the viewpoint of end-to-end denoising encoder-decoder network learning. We propose the wavelet-based encoder network to learn the cross-modal representations of the video contents with audio-form queries. Specifically, we adopt the multi-head cross-modal attention layers to explore the potential relations of video and query contents. A 2-dimension discrete wavelet trans-form is merged into the transformer encoder to decompose the audio-video features. Next, we maximize mutual information between the encoded features and multi-modal features after cross-modal attention layers to enhance the au-dio guidance. Then, a self attention-free decoder network is developed to generate the target masks with frequency-domain transforms. In addition, we construct the first large-scale audio-guided video semantic segmentation dataset. The extensive experiments show the effectiveness of our method <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Code is available at: https://github.com/asudahkzj/Wnet.git.
A Text Classification Methodology to Assist a Large Technical Support System

Text-based tools for reporting technical issues and receiving support are widespread in commercial applications, such as customer services and internal corporate communication.Past issues recorded in such systems may provide valuable knowledge for better handling future interactions.Nevertheless, the predominance of short messages and the presence of specific domain subjects constitute additional challenges.In this work, we aim to build an assistant for a system operating in a large company that provides asynchronous services for technical support.It is known that some repetitive technical issues can be handled with simple standard messages, named templates.Thus, we propose a modular pipeline based on natural language processing and machine learning algorithms to enable raw text processing, feature extraction, and supervised learning to recommend suitable templates from a given textual description of the incoming issue.In a real-world scenario, the proposed pipeline achieved an average accuracy of 72.7%, a promising result for a setup with 9 classes and few labeled training instances.Moreover, a post hoc analysis shows how our methodology is able to correctly identify the words more closely related to the corresponding templates.
Refactoring debt

To meet project timelines or budget constraints, developers intentionally deviate from writing optimal code to feasible code in what is known as incurring Technical Debt (TD). Furthermore, as part of planning their correction, developers document these deficiencies as comments in the code (i.e., self-admitted technical debt or SATD). As a means of improving source code quality, developers often apply a series of refactoring operations to their codebase. In this study, we explore developers repaying this debt through refactoring operations by examining occurrences of SATD removal in the code of 76 open-source Java systems. Our findings show that TD payment usually occurs with refactoring activities and developers refactor their code to remove TD for specific reasons. We envision our findings supporting vendors in providing tools to better support developers in the automatic repayment of technical debt.
Verified Software Units for Simple DFA Modules and Objects in C

Finite state machines occur ubiquitously in modern software, often in the form of C code that is synthesized from higher-level descriptions. To explore how the resulting code bases can be integrated into foundational verification infrastructures, we present formal specifications and machine-checked proofs of DFA representations using VST, a higher-order separation logic for C implemented in the Coq proof assistant. Paying particular attention to modularity and API-level representation hiding, we consider statically linked modules as well as object-inspired programming styles. Exploiting the abstraction capabilities of a recent VST enhancement, Verified Software Units (VSU), we complement separate compilation by separate verification and obtain instances of behavioral subtyping as separation logic entailments between suitable object representation predicates.
A Unified Framework for Implicit Sinkhorn Differentiation

The Sinkhorn operator has recently experienced a surge of popularity in computer vision and related fields. One major reason is its ease of integration into deep learning frameworks. To allow for an efficient training of respective neural networks, we propose an algorithm that obtains analytical gradients of a Sinkhorn layer via implicit differentiation. In comparison to prior work, our framework is based on the most general formulation of the Sinkhorn operator. It allows for any type of loss function, while both the target capacities and cost matrices are differentiated jointly. We further construct error bounds of the resulting algorithm for approximate inputs. Finally, we demonstrate that for a number of applications, simply replacing automatic differentiation with our algorithm directly improves the stability and accuracy of the obtained gradients. Moreover, we show that it is computationally more efficient, particularly when resources like GPU memory are scarce. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Our implementation is available under the following link: https://github.com/marvin-eisenberger/implicit-sinkhorn