Miracle-dz/UniFAR-FTUs huggingface dataset:
Hyperspectral Unmixing Using Transformer Network

Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way into the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep neural network-based unmixing model with transformers. A transformer network captures nonlocal feature dependencies by interactions between image patches, which are not employed in convolutional neural network (CNN) models, and hereby has the ability to enhance the quality of the endmember spectra and the abundance maps. The proposed model is a combination of a convolutional autoencoder and a transformer. The hyperspectral data is encoded by the convolutional encoder. The transformer captures long-range dependencies between the representations derived from the encoder. The data are reconstructed using a convolutional decoder. We applied the proposed unmixing model to three widely used unmixing datasets, that is, Samson, Apex, and Washington DC Mall, and compared it with the state-of-the-art in terms of root mean squared error and spectral angle distance. The source code for the proposed model will be made publicly available at https://github.com/preetam22n/DeepTrans-HSU.
Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning

Any city-scale visual localization system has to overcome long-term appearance changes, such as varying illumination conditions or seasonal changes between query and database images. Since semantic content is more robust to such changes, we exploit semantic information to improve visual localization. In our scenario, the database consists of gnomonic views generated from panoramic images (e.g. Google Street View) and query images are collected with a standard field-of-view camera at a different time. To improve localization, we check the semantic similarity between query and database images, which is not trivial since the position and viewpoint of the cameras do not exactly match. To learn similarity, we propose training a CNN in a self-supervised fashion with contrastive learning on a dataset of semantically segmented images. With experiments we showed that this semantic similarity estimation approach works better than measuring the similarity at pixel-level. Finally, we used the semantic similarity scores to verify the retrievals obtained by a state-of-the-art visual localization method and observed that contrastive learning-based pose verification increases top-1 recall value to 0.90 which corresponds to a 2% improvement.
Automatic detection of low surface brightness galaxies from SDSS images

Low surface brightness (LSB) galaxies are galaxies with central surface brightness fainter than the night sky. Due to the faint nature of LSB galaxies and the comparable sky background, it is difficult to search LSB galaxies automatically and efficiently from large sky survey. In this study, we established the Low Surface Brightness Galaxies Auto Detect model (LSBG-AD), which is a data-driven model for end-to-end detection of LSB galaxies from Sloan Digital Sky Survey (SDSS) images. Object detection techniques based on deep learning are applied to the SDSS field images to identify LSB galaxies and estimate their coordinates at the same time. Applying LSBG-AD to 1120 SDSS images, we detected 1197 LSB galaxy candidates, of which 1081 samples are already known and 116 samples are newly found candidates. The B-band central surface brightness of the candidates searched by the model ranges from 22 mag arcsec $^ {- 2} $ to 24 mag arcsec $^ {- 2} $, quite consistent with the surface brightness distribution of the standard sample. 96.46\% of LSB galaxy candidates have an axis ratio ($b/a$) greater than 0.3, and 92.04\% of them have $fracDev\_r$\textless 0.4, which is also consistent with the standard sample. The results show that the LSBG-AD model learns the features of LSB galaxies of the training samples well, and can be used to search LSB galaxies without using photometric parameters. Next, this method will be used to develop efficient algorithms to detect LSB galaxies from massive images of the next generation observatories.
Random quantum circuits are approximate unitary t-designs in depth O(nt5+o(1))

The applications of random quantum circuits range from quantum computing and quantum many-body systems to the physics of black holes. Many of these applications are related to the generation of quantum pseudorandomness: Random quantum circuits are known to approximate unitary t-designs. Unitary t-designs are probability distributions that mimic Haar randomness up to tth moments. In a seminal paper, Brandão, Harrow and Horodecki prove that random quantum circuits on qubits in a brickwork architecture of depth O(nt10.5) are approximate unitary t-designs. In this work, we revisit this argument, which lower bounds the spectral gap of moment operators for local random quantum circuits by Ω(n−1t−9.5). We improve this lower bound to Ω(n−1t−4−o(1)), where the o(1) term goes to 0 as t→∞. A direct consequence of this scaling is that random quantum circuits generate approximate unitary t-designs in depth O(nt5+o(1)). Our techniques involve Gao's quantum union bound and the unreasonable effectiveness of the Clifford group. As an auxiliary result, we prove fast convergence to the Haar measure for random Clifford unitaries interleaved with Haar random single qubit unitaries.
Probing highly collimated photon-jets with deep learning

Many extensions of the standard model (SM) predict the existence of axion-like particles and/or dark Higgs in the sub-GeV scale. Two new sub-GeV particles, a scalar and a pseudoscalar, produced through the Higgs boson exotic decays, are investigated. The decay signatures of these two new particles with highly collimated photons in the final states are discriminated from the ones of SM backgrounds using the Convolutional Neural Networks and Boosted Decision Trees techniques. The sensitivities of searching for such new physics signatures at the Large Hadron Collider are obtained.
Wind farm layout optimisation using set based multi-objective bayesian optimisation

Wind energy is one of the cleanest renewable electricity sources and can help in addressing the challenge of climate change. One of the drawbacks of wind-generated energy is the large space necessary to install a wind farm; this arises from the fact that placing wind turbines in a limited area would hinder their productivity and therefore not be economically convenient. This naturally leads to an optimisation problem, which has three specific challenges: (1) multiple conflicting objectives (2) computationally expensive simulation models and (3) optimisation over design sets instead of design vectors. The first and second challenges can be addressed by using multi-objective Bayesian optimisation (BO). However, the traditional BO cannot be applied as the optimisation function in the problem relies on design sets instead of design vectors. This paper extends the applicability of multi-objective BO to set based optimisation for solving the wind farm layout problem. We use a set-based kernel in Gaussian process to quantify the correlation between wind farms (with a different number of turbines). The results on the given data set of wind energy and direction clearly show the potential of using set-based multi-objective BO.
Improved Relation Networks for End-to-End Speaker Verification and Identification

Speaker identification systems in a real-world scenario are tasked to identify a speaker amongst a set of enrolled speakers given just a few samples for each enrolled speaker. This paper demonstrates the effectiveness of meta-learning and relation networks for this use case. We propose improved relation networks for speaker verification and few-shot (unseen) speaker identification. The use of relation networks facilitates joint training of the frontend speaker encoder and the backend model. Inspired by the use of prototypical networks in speaker verification and to increase the discriminability of the speaker embeddings, we train the model to classify samples in the current episode amongst all speakers present in the training set. Furthermore, we propose a new training regime for faster model convergence by extracting more information from a given meta-learning episode with negligible extra computation. We evaluate the proposed techniques on VoxCeleb, SITW and VCTK datasets on the tasks of speaker verification and unseen speaker identification. The proposed approach outperforms the existing approaches consistently on both tasks.
Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging: Theory, algorithms, and applications

Plug-and-play (PnP) priors constitute one of the most widely used frameworks for solving computational imaging problems through the integration of physical models and learned models. PnP leverages high-fidelity physical sensor models and powerful machine learning methods for prior modeling of data to provide state-of-the-art reconstruction algorithms. PnP algorithms alternate between minimizing a data fidelity term to promote data consistency and imposing a learned regularizer in the form of an image denoiser. Recent highly successful applications of PnP algorithms include biomicroscopy, computerized tomography (CT), magnetic resonance imaging (MRI), and joint ptychotomography. This article presents a unified and principled review of PnP by tracing its roots, describing its major variations, summarizing main results, and discussing applications in computational imaging. We also point the way toward further developments by discussing recent results on equilibrium equations that formulate the problem associated with PnP algorithms.
Error Identification Strategies for Python Jupyter Notebooks

Computational notebooks-such as Jupyter or Colab-combine text and data analysis code. They have become ubiquitous in the world of data science and exploratory data analysis. Since these notebooks present a different programming paradigm than conventional IDE-driven programming, it is plausible that debugging in computational notebooks might also be different. More specifically, since creating notebooks blends domain knowledge, statistical analysis, and programming, the ways in which notebook users find and fix errors in these different forms might be different. In this paper, we present an exploratory, observational study on how Python Jupyter notebook users find and understand potential errors in notebooks. Through a conceptual replication of study design investigating the error identification strategies of R notebook users, we presented users with Python Jupyter notebooks pre-populated with common notebook errors-errors rooted in either the statistical data analysis, the knowledge of domain concepts, or in the programming. We then analyzed the strategies our study participants used to find these errors and determined how successful each strategy was at identifying errors. Our findings indicate that while the notebook programming environment is different from the environments used for traditional programming, debugging strategies remain quite similar. It is our hope that the insights presented in this paper will help both notebook tool designers and educators make changes to improve how data scientists discover errors more easily in the notebooks they write.
Investigating Modality Bias in Audio Visual Video Parsing

We focus on the audio-visual video parsing (AVVP) problem that involves detecting audio and visual event labels with temporal boundaries. The task is especially challenging since it is weakly supervised with only event labels available as a bag of labels for each video. An existing state-of-the-art model for AVVP uses a hybrid attention network (HAN) to generate cross-modal features for both audio and visual modalities, and an attentive pooling module that aggregates predicted audio and visual segment-level event probabilities to yield video-level event probabilities. We provide a detailed analysis of modality bias in the existing HAN architecture, where a modality is completely ignored during prediction. We also propose a variant of feature aggregation in HAN that leads to an absolute gain in F-scores of about 2% and 1.6% for visual and audio-visual events at both segment-level and event-level, in comparison to the existing HAN model.
Diﬀusion of Ruthenium in Graphite: The Role of Clustering & Interlayer Stacking.

The diﬀusion, penetration and intercalation of metallic atomic dopants is an important question for various graphite applications in engineering and nanotechnology. We have performed systematic ﬁrst-principles calculations of the behaviour of ruthenium nanoclusters on a graphene monolayer and intercalated into a bilayer. Our computational results show that at a suﬃciently high density of single Ru atom interstitials, intercalated atoms can shear the surrounding lattice to an AA stacking conﬁguration, an eﬀect which weakens with increasing cluster size. Moreover, the interlayer stacking conﬁguration, in turn, has a signiﬁcant eﬀect on cluster diﬀusion. We therefore ﬁnd diﬀerent trends in diﬀusivity as a function of cluster size and interlayer stacking. For monolayer graphene and an AA graphene bilayer, the formation of small clusters generally lowers diﬀusion barriers, while the opposite behaviour is found for the preferred AB stacking conﬁguration. These results demonstrate that conditions of local impurity concentration and interlayer disregistry are able to regulate the diﬀusivity of metallic impurities in graphite.
Perfectly Perform Machine Learning Task with Imperfect Optical Hardware Accelerator

Optical architectures have been emerging as an energy-efficient and high-throughput hardware platform to accelerate computationally intensive general matrix-matrix multiplications (GEMMs) in modern machine learning (ML) algorithms. However, the inevitable imperfection and nonuniformity in large-scale optoelectronic devices prevent the scalable deployment of optical architectures, particularly those with innovative nano-devices. Here, we report an optical ML hardware to accelerate GEMM operations based on cascaded spatial light modulators and present a calibration procedure that enables accurate calculations despite the non-uniformity and imperfection in devices and system. We further characterize the hardware calculation accuracy under different configurations of electrical-optical interfaces. Finally, we deploy the developed optical hardware and calibration procedure to perform a ML task of predicting the intersubband plasmon frequency in single-wall carbon nanotubes. The obtained prediction accuracy from the optical hardware agrees well with that obtained using a general purpose electronic graphic process unit.
Comparison of single carrier 200G 4QAM, 8QAM and 16QAM in a WDM field trial demonstration over 612 km SSMF

We present results of a field trial carried on Telecom Italia EDFA-only legacy link with 0.3 dB/km average fiber attenuation. Single carrier 200G WDM DP-4QAM, DP-8QAM and DP-16QAM were successfully transmitted with system margin of 4.1 dB, 4.3 dB and 2.6 dB respectively.
Going Beyond RF: How AI-enabled Multimodal Beamforming will Shape the NextG Standard

Incorporating artificial intelligence and machine learning (AI/ML) methods within the 5G wireless standard promises autonomous network behavior and ultra-low-latency reconfiguration. However, the effort so far has purely focused on learning from radio frequency (RF) signals. Future standards and next-generation (nextG) networks beyond 5G will have two significant evolutions over the state-of-the-art 5G implementations: (i) massive number of antenna elements, scaling up to hundreds-to-thousands in number, and (ii) inclusion of AI/ML in the critical path of the network reconfiguration process that can access sensor feeds from a variety of RF and non-RF sources. While the former allows unprecedented flexibility in 'beamforming', where signals combine constructively at a target receiver, the latter enables the network with enhanced situation awareness not captured by a single and isolated data modality. This survey presents a thorough analysis of the different approaches used for beamforming today, focusing on mmWave bands, and then proceeds to make a compelling case for considering non-RF sensor data from multiple modalities, such as LiDAR, Radar, GPS for increasing beamforming directional accuracy and reducing processing time. This so called idea of multimodal beamforming will require deep learning based fusion techniques, which will serve to augment the current RF-only and classical signal processing methods that do not scale well for massive antenna arrays. The survey describes relevant deep learning architectures for multimodal beamforming, identifies computational challenges and the role of edge computing in this process, dataset generation tools, and finally, lists open challenges that the community should tackle to realize this transformative vision of the future of beamforming.
indic-punct: An automatic punctuation restoration and inverse text normalization framework for Indic languages

Automatic Speech Recognition (ASR) generates text which is most of the times devoid of any punctuation. Absence of punctuation is text can affect readability. Also, down stream NLP tasks such as sentiment analysis, machine translation, greatly benefit by having punctuation and sentence boundary information. We present an approach for automatic punctuation of text using a pretrained IndicBERT model. Inverse text normalization is done by hand writing weighted finite state transducer (WFST) grammars. We have developed this tool for 11 Indic languages namely Hindi, Tamil, Telugu, Kannada, Gujarati, Marathi, Odia, Bengali, Assamese, Malayalam and Punjabi. All code and data is publicly. available
General Lossless Polarization and Phase Transformation Using Bilayer Metasurfaces

Metasurface optical elements enable wavefront control and polarization manipulation with subwavelength resolution. Metasurfaces made of linearly birefringent meta‐atoms such as rectangular nanoposts are commonly used to control phase and polarization, but a single layer of these meta‐atoms cannot implement transformations with elliptically polarized or circularly polarized eigenstates (i.e., chiral transformations). Here, it is shown that two cascaded metasurface layers comprising linearly birefringent meta‐atoms provide sufficiently many degrees of freedom to implement arbitrary unitary phase and polarization transformations. A systematic design method for such metastructures is described and used to design a bifocal metalens that cannot be implemented by a single layer of birefringent meta‐atoms. The presented design and implementation techniques introduce a systematic approach for realizing the most general form of lossless polarization and phase transformations using metasurfaces with high efficiency.
Quantum Mechanical Description of Thermo-Optic Interaction

Thermo-optic interaction signiﬁcantly diﬀers from the usual particle-particle interac-tions in physics, as it is retarded in time. A prominent platform for realising this kind of interaction are photon Bose-Einstein condensates, which are created in dye-ﬁlled mi-crocavities. The dye solution continually absorbs and re-emits these photons, causing the photon gas to thermalise and to form a Bose-Einstein condensate. Because of a non-ideal quantum eﬃciency, these cycles heat the dye solution, creating a medium that provides an eﬀective thermo-optic photon-photon interaction. So far, only a mean-ﬁeld description of this process exists. This paper goes beyond by working out a quantum mechanical description of the eﬀective thermo-optic photon-photon interaction. To this end, the self-consistent modelling of the temperature diﬀusion builds the backbone of the modelling. Furthermore, the manyfold experimental timescales allow for deriving an approximate Hamiltonian. The resulting quantum theory is applied in the perturbative regime to both a harmonic and a box potential for investigating its prospect for precise measurements of the eﬀective photon-photon interaction strength.
The CO Emission in the Taffy Galaxies (UGC 12914/15) at 60 pc Resolution. I. The Battle for Star Formation in the Turbulent Taffy Bridge

We present Atacama Large Millimeter/submillimeter Array observations at a spatial resolution of 0. ″2 (60 pc) of CO emission from the Taffy galaxies (UGC 12914/5). The observations are compared with narrowband Paα, mid-IR, radio continuum and X-ray imaging, plus optical spectroscopy. The galaxies have undergone a recent head-on collision, creating a massive gaseous bridge that is known to be highly turbulent. The bridge contains a complex web of narrow molecular filaments and clumps. The majority of the filaments are devoid of star formation, and fall significantly below the Kennicutt–Schmidt relationship for normal galaxies, especially for the numerous regions undetected in Paα emission. Within the loosely connected filaments and clumps of gas we find regions of high velocity dispersion that appear gravitationally unbound for a wide range of likely values of X CO. Like the “Firecracker” region in the Antennae system, they would require extremely high external dynamical or thermal pressure to stop them dissipating rapidly on short crossing timescales of 2–5 Myr. We suggest that the clouds may be transient structures within a highly turbulent multiphase medium that is strongly suppressing star formation. Despite the overall turbulence in the system, stars seem to have formed in compact hotspots within a kiloparsec-sized extragalactic H ii region, where the molecular gas has a lower velocity dispersion than elsewhere, and shows evidence for a collision with an ionized gas cloud. Like the shocked gas in the Stephan’s Quintet group, the conditions in the Taffy bridge shows how difficult it is to form stars within a turbulent, multiphase, gas.
PIE-Net: Photometric Invariant Edge Guided Network for Intrinsic Image Decomposition

Intrinsic image decomposition is the process of recovering the image formation components (reflectance and shading) from an image. Previous methods employ either explicit priors to constrain the problem or implicit constraints as formulated by their losses (deep learning). These methods can be negatively influenced by strong illumination conditions causing shading-reflectance leakages. Therefore, in this paper, an end-to-end edge-driven hybrid CNN approach is proposed for intrinsic image decomposition. Edges correspond to illumination invariant gradients. To handle hard negative illumination transitions, a hierarchical approach is taken including global and local refinement layers. We make use of attention layers to further strengthen the learning process. An extensive ablation study and large scale experiments are conducted showing that it is beneficial for edge-driven hybrid IID networks to make use of illumination invariant descriptors and that separating global and local cues helps in improving the performance of the network. Finally, it is shown that the proposed method obtains state of the art performance and is able to generalise well to real world images. The project page with pretrained models, finetuned models and network code can be found at https://ivi.fnwi. uva.nl/cv/pienet/.