Miracle-dz/UniFAR-FTUs huggingface dataset:
Mesh based automatic image registration

This paper presents a novel method for automatic image registration. It represents image as triangular mesh and use triangle as feature primitive. First, it detects corner features and triangulates them into triangular mesh. Then, orrespondences of triangles from different images are established by evaluating the similarity of the triangular regions. Affine rectification is applied to establish pixel correspondences. Based on the triangle correspondences, the image transformation is estimated using RANSAC estimator. The proposed method is applied to various image pairs related by projective transformation, experimental results show that the method works successfully even under the case that there are large rotation or severe perspective deformation effect between the images.
X-ray sources in stellar winds

We consider a model of a binary X-ray source immersed in a stellar wind from its companion star. We discuss the geometry of the constant ionization and temperature contours in the model and describe the X-ray radiative transfer through the wind. The results are applied to make predictions about the presence and variability of iron K-shell fluorescence lines, to explain the light curve of Cen X-3 during a ''low'' to ''high'' state transition in 1972 July, and to predict variability with orbital phase of resonance ultraviolet lines such as N v lambdalambda1238, 1242 from the binary system.
Influence of Supernova SN Ia Rate and the Early Star Formation Rate on the Galactic Chemical Evolution

Based on the recently developed numerical approach to understand the formation and the chemical evolution of the milky-way galaxy in the solar neighborhood we study the influence of the supernova type SN Ia rates on the galactic chemical evolution. Supernova SN Ia plays an important role in producing the iron inventory of the galaxy. We also study the dependence of the chemical evolution on the star formation rate prevailing during the initial one billion years of the evolution of the galaxy. This era marks the formation of the galactic halo and the thick disk. A comparison of the elemental abundance distributions of the dwarf stars in the solar neighborhood is made among the various models simulated in the present work. In order to explain the majority of the observed elemental evolutionary trends, specifically those related with the galactic evolution of iron and oxygen, it would be essential to incorporate a major component of prompt SN Ia to the galactic evolution. The prompt SN Ia would produce significant fraction of SN Ia within the initial ~100 million years from the time of star formation. The essential requirement of prompt SN Ia would result in a significant enhancement of SN Ia rates during the earliest epoch of the galaxy. The elemental evolutionary trends also favor an enhancement in the star formation rate during the initial one billion years of the galaxy at least by a factor of three compared to the trend prevailing during the latter evolutionary time of the galaxy.
Thermo-Optic Dispersion Formula for RbTiOAsO4

The thermo-optic constants of the flux grown RbTiOAsO4 (RTA) have been measured from 0.4577 to 0.7931 µm. These results were fitted to the temperature phase-matching bandwidths (FWHM) for harmonic generation of the Nd:YAG laser and the Nd:YAG laser-pumped KTiOAsO4 (KTA) optical parametric oscillator (OPO), and were used to construct the thermo-optic dispersion formula that can be used for good reproduction of the temperature-dependent phase-matching conditions in the 0.459–3.129 µm range. Applications to the quasi phase-matching are presented.
Grammar Based Speaker Role Identification for Air Traffic Control Speech Recognition

—Automatic Speech Recognition (ASR) for air trafﬁc control is generally trained by pooling Air Trafﬁc Controller (ATCO) and pilot data into one set. This is motivated by the fact that pilot’s voice communications are more scarce than ATCOs. Due to this data imbalance and other reasons (e.g., varying acoustic conditions), the speech from ATCOs is usually recognized more accurately than from pilots. Automatically identifying the speaker roles is a challenging task, especially in the case of the noisy voice recordings collected using Very High Frequency (VHF) receivers or due to the unavailability of the push-to-talk (PTT) signal, i.e., both audio channels are mixed. In this work, we propose to (1) automatically segment the ATCO and pilot data based on an intuitive approach exploiting ASR transcripts and (2) subsequently consider an automatic recognition of ATCOs’ and pilots’ voice as two separate tasks. Our work is performed on VHF audio data with high noise levels, i.e., signal-to-noise (SNR) ratios below 15 dB, as this data is recognized to be helpful for various speech-based machine-learning tasks. Speciﬁcally, for the speaker role identiﬁcation task, the module is represented by a simple yet efﬁcient knowledge-based system exploiting a grammar deﬁned by the International Civil Aviation Organization (ICAO). The system accepts text as the input, either manually veriﬁed annotations or automatically generated transcripts. The developed approach provides an average accuracy in speaker role identiﬁcation of about 83%. Finally, we show that training an acoustic model for ASR tasks separately (i.e
Query Time Optimized Deep Learning Based Video Inference System

This is a project report about how we tune Focus[1], a video inference system that provides low cost and low latency, through two phases. In this report, we will decrease the query time by saving the middle layer output of the neural network. This is a trade-off strategy that involves using more space to save time. We show how this scheme works using prototype systems, and it saves 20% of the time. The code repository URL is here, https://github.com/iphyer/CS744 FocousIngestOpt.
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare

We introduce MegaPose, a method to estimate the 6D pose of novel objects, that is, objects unseen during training. At inference time, the method only assumes knowledge of (i) a region of interest displaying the object in the image and (ii) a CAD model of the observed object. The contributions of this work are threefold. First, we present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects. The shape and coordinate system of the novel object are provided as inputs to the network by rendering multiple synthetic views of the object's CAD model. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner. Third, we introduce a large-scale synthetic dataset of photorealistic images of thousands of objects with diverse visual and shape properties and show that this diversity is crucial to obtain good generalization performance on novel objects. We train our approach on this large synthetic dataset and apply it without retraining to hundreds of novel objects in real images from several pose estimation benchmarks. Our approach achieves state-of-the-art performance on the ModelNet and YCB-Video datasets. An extensive evaluation on the 7 core datasets of the BOP challenge demonstrates that our approach achieves performance competitive with existing approaches that require access to the target objects during training. Code, dataset and trained models are available on the project page: https://megapose6d.github. io/.
Deep Neural Networks integrating genomics and histopathological images for predicting stages and survival time-to-event in colon cancer

Motivation: There exist unexplained diverse variation within the predefined colon cancer stages using only features either from genomics or histopathological whole slide images as prognostic factors. Unraveling this variation will bring about improved in staging and treatment outcome, hence motivated by the advancement of Deep Neural Network (DNN) libraries and different structures and factors within some genomic dataset, we aggregate atypia patterns in histopathological images with diverse carcinogenic expression from mRNA, miRNA and DNA Methylation as an integrative input source into an ensemble deep neural network for colon cancer stages classification, and samples stratification into low or high risk survival groups. Results: The results of our Ensemble Deep Convolutional Neural Network (EDCNN) model show an improved performance in stages classification on the integrated dataset. The fused input features return Area under curve – Receiver Operating Characteristic curve (AUC-ROC) of 95.21% compared with AUC-ROC of 71.09% and 67.98% obtained when only genomics and images features are used for the stage’s classification, respectively. Also, the extracted features were used to split the patients into low or high-risk survival groups. Among the 2,548 fused features, 1,695 features showed a statistically
On proximal gradient mapping and its minimization in norm via potential function-based acceleration

The proximal gradient descent method, well-known for composite optimization, can be completely described by the concept of proximal gradient mapping. In this paper, we highlight our previous two discoveries of proximal gradient mapping--norm monotonicity and refined descent, with which we are able to extend the recently proposed potential function-based framework from gradient descent to proximal gradient descent.
Paraphrase Identification with Deep Learning: A Review of Datasets and Methods

The rapid advancement of AI technology has made text generation tools like GPT-3 and ChatGPT increasingly accessible, scalable, and effective. This can pose serious threat to the credibility of various forms of media if these technologies are used for plagiarism, including scientific literature and news sources. Despite the development of automated methods for paraphrase identification, detecting this type of plagiarism remains a challenge due to the disparate nature of the datasets on which these methods are trained. In this study, we review traditional and current approaches to paraphrase identification and propose a refined typology of paraphrases. We also investigate how this typology is represented in popular datasets and how under-representation of certain types of paraphrases impacts detection capabilities. Finally, we outline new directions for future research and datasets in the pursuit of more effective paraphrase detection using AI.
Unsupervised Time-Aware Sampling Network with Deep Reinforcement Learning for EEG-Based Emotion Recognition

Recognizing human emotions from complex, multivariate, and non-stationary electroencephalography (EEG) time series is essential in affective brain-computer interface. However, because continuous labeling of ever-changing emotional states is not feasible in practice, existing methods can only assign a fixed label to all EEG timepoints in a continuous emotion-evoking trial, which overlooks the highly dynamic emotional states and highly non-stationary EEG signals. To solve the problems of high reliance on fixed labels and ignorance of time-changing information, in this paper we propose a time-aware sampling network (TAS-Net) using deep reinforcement learning (DRL) for unsupervised emotion recognition, which is able to detect key emotion fragments and disregard irrelevant and misleading parts. Extensive experiments are conducted on three public datasets (SEED, DEAP, and MAHNOB-HCI) for emotion recognition using leave-one-subject-out cross-validation, and the results demonstrate the superiority of the proposed method against previous unsupervised emotion recognition methods.
Disentangling Prosody Representations With Unsupervised Speech Reconstruction

Human speech can be characterized by different components, including semantic content, speaker identity and prosodic information. Significant progress has been made in disentangling representations for semantic content and speaker identity in speech recognition and speaker verification tasks respectively. However, it is still an open challenging question to extract prosodic information because of the intrinsic association of different attributes, such as timbre and rhythm, and because of the need for supervised training schemes to achieve robust speech recognition. The aim of this article is to address the disentanglement of emotional prosody based on unsupervised reconstruction. Specifically, we identify, design, implement and integrate three crucial components in our proposed model Prosody2Vec: (1) a unit encoder that transforms speech signals into discrete units for semantic content, (2) a pretrained speaker verification model to generate speaker identity embeddings, and (3) a trainable prosody encoder to learn prosody representations. We first pretrain Prosody2Vec on unlabelled emotional speech corpora, then fine-tune the model on specific datasets to perform Speech Emotion Recognition (SER) and Emotional Voice Conversion (EVC) tasks. Both objective and subjective evaluations on the EVC task suggest that Prosody2Vec effectively captures general prosodic features that can be smoothly transferred to other emotional speech. In addition, our SER experiments on the IEMOCAP dataset reveal that the prosody features learned by Prosody2Vec are complementary and beneficial for the performance of widely used speech pretraining models and surpass the state-of-the-art methods when combining Prosody2Vec with HuBERT representations. Audio samples can be found on our demo website.
3DHumanGAN: Towards Photo-Realistic 3D-Aware Human Image Generation

We present 3DHumanGAN , a 3D-aware generative adversarial network (GAN) that synthesizes images of full-body humans with consistent appearances under different view-angles and body-poses. To tackle the representational and computational challenges in synthesizing the articulated structure of human bodies, we propose a novel generator architecture in which a 2D convolutional backbone is modulated by a 3D pose mapping network. The 3D pose mapping network is formulated as a renderable implicit function conditioned on a posed 3D human mesh. This design has several merits: i) it allows us to harness the power of 2D GANs to generate photo-realistic images; ii) it generates consistent images under varying view-angles and speciﬁable poses; iii) the model can beneﬁt from the 3D human prior. Our model is adversarially learned from a collection of web images needless of manual annotation. Code and models will be publicly available 12 .
Shared Coupling-bridge for Weakly Supervised Local Feature Learning

—Sparse local feature extraction is usually believed to be of important signiﬁcance in typical vision tasks such as simultaneous localization and mapping, image matching and 3D reconstruction. At present, it still has some deﬁciencies needing further improvement, mainly including the discrimination power of extracted local descriptors, the localization accuracy of detected keypoints, and the efﬁciency of local feature learning. This paper focuses on promoting the currently popular sparse local feature learning with camera pose supervision. Therefore, it pertinently proposes a Shared Coupling-bridge scheme with four light-weight yet effective improvements for weakly-supervised local feature (SCFeat) learning. It mainly contains: i) the Feature-Fusion-ResUNet Backbone (F2R-Backbone) for local descriptors learning, ii) a shared coupling-bridge normalization to improve the decoupling training of description network and detection network, iii) an improved detection network with peakiness measurement to detect keypoints and iv) the fundamental matrix error as a reward factor to further optimize feature detection training. Extensive experiments prove that our SCFeat improvement is effective. It could often obtain a state-of-the-art performance on classic image matching and visual localization. In terms of 3D reconstruction, it could still achieve competitive results. For sharing and communication, our source codes are available at https://github.com/sunjiayuanro/SCFeat.git.
Establishing a Stronger Baseline for Lightweight Contrastive Models

Recent research has reported a performance degradation in self-supervised contrastive learning for specially designed efficient networks, such as MobileNet and EfficientNet. A common practice to address this problem is to introduce a pretrained contrastive teacher model and train the lightweight networks with distillation signals generated by the teacher. However, it is time and resource consuming to pretrain a teacher model when it is not available. In this work, we aim to establish a stronger baseline for lightweight contrastive models without using a pretrained teacher model. Specifically, we show that the optimal recipe for efficient models is different from that of larger models, and using the same training settings as ResNet50, as previous research does, is inappropriate. Additionally, we observe a common issue in contrastive learning where either the positive or negative views can be noisy, and propose a smoothed version of InfoNCE loss to alleviate this problem. As a result, we successfully improve the linear evaluation results from 36.3% to 62.3% for MobileNet-V3-Large and from 42.2% to 65.8% for EfficientNet-B0 on ImageNet, closing the accuracy gap to ResNet50 with 5× fewer parameters. We hope our research will facilitate the usage of lightweight contrastive models.1
Autonomous Vehicle Navigation with LIDAR using Path Planning

​ : ​ In this paper, a complete framework for Autonomous Self Driving is implemented. LIDAR, Camera and IMU (Inertial Measurement Unit) sensors are used together. The entire data communication is managed using ROS (Robot Operating System) which provides a robust platform for implementation of Robotics Projects. Jetson Nano is used to provide powerful on-board processing capabilities. Sensor fusion is performed on the data received from the different sensors to improve the accuracy of the decision making and inferences that we derive from the data. This data is then used to create a localized map of the environment. In this step, the position of the vehicle is obtained with respect to the Mapping done using the sensor data. The different SLAM (Simultaneous Localization and Mapping) techniques used for this purpose are Hector Mapping and GMapping which are widely used mapping techniques in ROS. Apart from SLAM that primarily uses LIDAR data, Visual Odometry is implemented using a Monocular Camera. The sensor fused data is then used by Adaptive Monte Carlo Localization for car localization. Using the localized map developed, Path Planning techniques like "TEB planner" and “Dynamic Window Approach” are implemented for autonomous navigation of the vehicle. The last step in the Project is the implantation of Control which is the final decision making block in the pipeline that gives speed and steering data for the navigation that is compatible with Ackermann Kinematics. The implementation of such a control block under a ROS framework using the three sensors, viz, LIDAR, Camera and IMU is a novel approach that is undertaken in this project.
Mitigating Artifacts in Real-World Video Super-Resolution Models

The recurrent structure is a prevalent framework for the task of video super-resolution, which models the temporal dependency between frames via hidden states. When applied to real-world scenarios with unknown and complex degradations, hidden states tend to contain unpleasant artifacts and propagate them to restored frames. In this circumstance, our analyses show that such artifacts can be largely alleviated when the hidden state is replaced with a cleaner counterpart. Based on the observations, we propose a Hidden State Attention (HSA) module to mitigate artifacts in real-world video super-resolution. Specifically, we first adopt various cheap filters to produce a hidden state pool. For example, Gaussian blur filters are for smoothing artifacts while sharpening filters are for enhancing details. To aggregate a new hidden state that contains fewer artifacts from the hidden state pool, we devise a Selective Cross Attention (SCA) module, in which the attention between input features and each hidden state is calculated. Equipped with HSA, our proposed method, namely FastRealVSR, is able to achieve 2x speedup while obtaining better performance than Real-BasicVSR. Codes will be available at https://github.com/TencentARC/FastRealVSR.
Learning Invariant Subspaces of Koopman Operators-Part 2: Heterogeneous Dictionary Mixing to Approximate Subspace Invariance

This work builds on the models and concepts presented in part 1 to learn approximate dictionary representations of Koopman operators from data. Part I of this paper presented a methodology for arguing the subspace invariance of a Koopman dictionary. This methodology was demonstrated on the state-inclusive logistic lifting (SILL) basis. This is an affine basis augmented with conjunctive logistic functions. The SILL dictionary's nonlinear functions are homogeneous, a norm in data-driven dictionary learning of Koopman operators. In this paper, we discover that structured mixing of heterogeneous dictionary functions drawn from different classes of nonlinear functions achieve the same accuracy and dimensional scaling as the deep-learning-based deepDMD algorithm. We specifically show this by building a heterogeneous dictionary comprised of SILL functions and conjunctive radial basis functions (RBFs). This mixed dictionary achieves the same accuracy and dimensional scaling as deepDMD with an order of magnitude reduction in parameters, while maintaining geometric interpretability. These results strengthen the viability of dictionary-based Koopman models to solving high-dimensional nonlinear learning problems.
Ein Beitrag zum Entladungsverhalten langer Luftfunkenstrecken bei Blitzspannung

09107 Chemnitz www.tu-chemnitz.de Für die Menschen zählen Gewitter mit Donner und Blitz zu den ältesten bekannten Phänomenen der Natur. Die Tatsache, dass Blitze dennoch bis heute nicht abschließend erforscht sind, liegt in dem äußerst komplexen, schnellen und stochastischen Verhalten der Blitzentladung begründet. Der Autor befasste sich in Rahmen dieser Arbeit u.a. mit der Entwicklung eines modernen Messsystems, durch das die Erfassung von sowohl elektrischen als auch optischen Parametern der im Nanosekundenbereich ablaufenden Prozesse der Entladungen möglich wurde. Es handelt sich um technisch anspruchsvolle Messungen von hohen Stoßspannungen und schwachen Vorentladungsströmen in einem großen Frequenzbereich. Außerdem gelang es durch eine geschickte Synchronisation, die Entladungsprozesse präzise und erstmalig in einer Fotosequenz aufzunehmen. Auf Grundlage der Messergebnisse konnte der Entladungsprozess in mehrere Phasen der Vorentladung unterteilt und beschrieben werden. Vor allem beweisen die Messergebnisse die Existenz einer kanalartigen Vorentladung, die nicht zwingend zu einem Durchschlag führt. Dieser Mechanismus, der bei langsam ansteigenden Spannungen als stabile Leaderentladung benannt und mit Thermoionisation gekennzeichnet ist, war bei Blitzstoßspannungen bisher umstritten. Ali Shirvani Boroujeni Ein Beitrag zum Entladungsverhalten langer Luftfunkenstrecken bei Blitzspannung Fakultät für Elektrotechnik und Informationstechnik Professur für Energieund Hochspannungstechnik
Edge-Preserving Smoothers for Image Processing

Abstract Classical smoothers have limited usefulness in image processing, because sharp “edges” tend to be blurred. There is a literature on edge-preserving smoothers, but these all require moderately large “smooth stretches. ” Here we discuss an approach to this problem called “sigma filtering” and propose an improvement based on running M estimation. Both computational and theoretical aspects are developed. For image processing, the methods have a niche between standard filtering approaches and Bayes–Markov random-field analysis.