Miracle-dz/UniFAR-FTUs huggingface dataset:
Temporal-wise Attention Spiking Neural Networks for Event Streams Classification

How to effectively and efficiently deal with spatio-temporal event streams, where the events are generally sparse and non-uniform and have the μs temporal resolution, is of great value and has various real-life applications. Spiking neural network (SNN), as one of the brain-inspired event-triggered computing models, has the potential to extract effective spatio-temporal features from the event streams. However, when aggregating individual events into frames with a new higher temporal resolution, existing SNN models do not attach importance to that the serial frames have different signal-to-noise ratios since event streams are sparse and non-uniform. This situation interferes with the performance of existing SNNs. In this work, we propose a temporal-wise attention SNN (TA-SNN) model to learn frame-based representation for processing event streams. Concretely, we extend the attention concept to temporal-wise input to judge the significance of frames for the final decision at the training stage, and discard the irrelevant frames at the inference stage. We demonstrate that TA-SNN models improve the accuracy of event streams classification tasks. We also study the impact of multiple-scale temporal resolutions for frame-based representation. Our approach is tested on three different classification tasks: gesture recognition, image classification, and spoken digit recognition. We report the state-of-the-art results on these tasks, and get the essential improvement of accuracy (almost 19%) for gesture recognition with only 60 ms.
Retrieving Linear and Nonlinear Optical Dispersions of Matter: Combined Experiment-Numerical Ellipsometry in Silicon, Gold and Indium Tin Oxide

The predominant methods currently used to determine nonlinear optical constants like the nonlinear refractive index n2 or the third order susceptibility χ(3) rely mostly on experimental, open and closed z-scan techniques and beam deflection methods. While these methods work well when the linear absorption is relatively small or negligible, the retrieval process is more complicated for a strongly scattering, dispersive or absorbing medium. The study of optics at the nanoscale in the picosecond or femtosecond laser pulsed regimes demands the development of new theoretical tools, and diverse experimental approaches, to extract and verify both linear and nonlinear optical dispersions exhibited by matter, especially when material constituents are fashioned into nanostructures of arbitrary shape. We present a practical, combined experimental and theoretical approach based on the hydrodynamic model that uses experimental results of harmonic generation conversion efficiencies to retrieve complex, nonlinear dispersion curves, not necessarily only for third order processes. We provide examples for materials that are of special interest to nanophotonics, for example, silicon, gold, and indium tin oxide (ITO), which displays nonlocal effects and a zero-crossing of the real part of the dielectric constant. The results for silicon and gold compare well with analytical predictions of nonlinear dispersion based on the nonlinear oscillator model. Based on our assessment of third harmonic generation conversion efficiencies in silicon, we predict χ ω ( 3 ) and χ 3 ω ( 3 ) are of order 10−17 (m/V)2 in the visible and near IR ranges, with respective peaks of 10−14 (m/V)2 and 10−16 (m/V)2 in the UV range. Similarly, gold’s χ ω ( 3 ) and χ 3 ω ( 3 ) are of order 10−17–10−16 (m/V)2, and predict χ ω ( 3 ) ∼ 10 − 17 ( m / V ) 2 and χ 3 ω ( 3 ) ∼ 10 − 18 ( m / V ) 2 for ITO. These results clearly suggest that judicious exploitation of the nonlinear dispersion of ordinary semiconductors has the potential to transform device physics in spectral regions that extend well into the UV range.
ReDAL: Region-based and Diversity-aware Active Learning for Point Cloud Semantic Segmentation

Despite the success of deep learning on supervised point cloud semantic segmentation, obtaining large-scale point-by-point manual annotations is still a significant challenge. To reduce the huge annotation burden, we propose a Region-based and Diversity-aware Active Learning (ReDAL), a general framework for many deep learning approaches, aiming to automatically select only informative and diverse sub-scene regions for label acquisition. Observing that only a small portion of annotated regions are sufficient for 3D scene understanding with deep learning, we use softmax entropy, color discontinuity, and structural complexity to measure the information of sub-scene regions. A diversity-aware selection algorithm is also developed to avoid redundant annotations resulting from selecting informative but similar regions in a querying batch. Extensive experiments show that our method highly outperforms previous active learning strategies, and we achieve the performance of 90% fully supervised learning, while less than 15% and 5% annotations are required on S3DIS and SemanticKITTI datasets, respectively.
SN 2019hcc: a Type II supernova displaying early O ii lines

We present optical spectroscopy together with ultraviolet, optical, and near-infrared photometry of SN 2019hcc, which resides in a host galaxy at redshift 0.044, displaying a sub-solar metallicity. The supernova spectrum near peak epoch shows a ‘w’ shape at around 4000 Å which is usually associated with O ii lines and is typical of Type I superluminous supernovae. SN 2019hcc post-peak spectra show a well-developed H α P-Cygni profile from 19 d past maximum and its light curve, in terms of its absolute peak luminosity and evolution, resembles that of a fast-declining Hydrogen-rich supernova (SN IIL). The object does not show any unambiguous sign of interaction as there is no evidence of narrow lines in the spectra or undulations in the light curve. Our tardis spectral modelling of the first spectrum shows that carbon, nitrogen, and oxygen (CNO) at 19 000 K reproduce the ‘w’ shape and suggests that a combination of non-thermally excited CNO and metal lines at 8000 K could reproduce the feature seen at 4000 Å. The Bolometric light-curve modelling reveals that SN 2019hcc could be fit with a magnetar model, showing a relatively strong magnetic field (B > 3 × 1014 G), which matches the peak luminosity and rise time without powering up the light curve to superluminous luminosities. The high-energy photons produced by the magnetar would then be responsible for the detected O ii lines. As a consequence, SN 2019hcc shows that a ‘w’ shape profile at around 4000 Å, usually attributed to O ii, is not only shown in superluminous supernovae and hence it should not be treated as the sole evidence of the belonging to such a supernova type.
4D effective action from the non-Abelian DBI action with a magnetic flux background

We study a systematic derivation of four dimensional N = 1 supersymmetric effective theory from ten dimensional non-Abelian Dirac-Born-Infeld action compactified on a six dimensional torus with magnetic fluxes on the D-branes. We find a new type of matter Kähler metric while gauge kinetic function and superpotential are consistent with previous studies. For the ten dimensional action, we use a symmetrized trace prescription and focus on the bosonic part up to O(F 4). In the presence of the supersymmetry, four dimensional chiral fermions can be obtained via index theorem. The new matter Kähler metric is independent of flavor but depends on the fluxes, 4D dilaton, Kähler moduli and complex structure moduli, and will be always positive definite if an induced Ramond-Ramond charge of the D-branes on which matters are living are positive. We read the superpotential from an F-term scalar quartic interaction derived from the ten dimensional action and the contribution of the new matter Kähler metric to the scalar potential which we derive turns out to be consistent with the supergravity formulation. ∗y.abe@gauge.scphys. kyoto-u.ac. jp †thigaki@rk.phys.keio.ac. jp ‡kobayashi@particle.sci.hokudai.ac. jp §s-takada@particle.sci.hokudai.ac. jp ar X iv :2 10 7. 11 96 1v 2 [ he pth ] 1 6 D ec 2 02 1
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition

Graph convolutional networks (GCNs) have been widely used and achieved remarkable results in skeleton-based action recognition. In GCNs, graph topology dominates feature aggregation and therefore is the key to extracting representative features. In this work, we propose a novel Channel-wise Topology Refinement Graph Convolution (CTR-GC) to dynamically learn different topologies and effectively aggregate joint features in different channels for skeleton-based action recognition. The proposed CTR-GC models channel-wise topologies through learning a shared topology as a generic prior for all channels and refining it with channel-specific correlations for each channel. Our refinement method introduces few extra parameters and significantly reduces the difficulty of modeling channel-wise topologies. Furthermore, via reformulating graph convolutions into a unified form, we find that CTR-GC relaxes strict constraints of graph convolutions, leading to stronger representation capability. Combining CTR-GC with temporal modeling modules, we develop a powerful graph convolutional network named CTR-GCN which notably outperforms state-of-the-art methods on the NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.1
Learning from 2D: Pixel-to-Point Knowledge Transfer for 3D Pretraining

Most 3D neural networks are trained from scratch owing to the lack of large-scale labeled 3D datasets. In this paper, we present a novel 3D pretraining method by leveraging 2D networks learned from rich 2D datasets. We propose the contrastive pixel-to-point knowledge transfer to effectively utilize the 2D information by mapping the pixel-level and point-level features into the same embedding space. Due to the heterogeneous nature between 2D and 3D networks, we introduce the back-projection function to align the features between 2D and 3D to make the transfer possible. Additionally, we devise an upsampling feature projection layer to increase the spatial resolution of high-level 2D feature maps, which enables learning fine-grained 3D representations. With a pretrained 2D network, the proposed pretraining process requires no additional 2D or 3D labeled data, further alleviating the expensive 3D data annotation cost. To the best of our knowledge, we are the first to exploit existing 2D trained weights to pretrain 3D deep neural networks. Our intensive experiments show that the 3D models pretrained with 2D knowledge boost the performances of 3D networks across various real-world 3D downstream tasks.
Computational Graphs for Matrix Functions

Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve them, and eventually derive new ones. The accuracy of these matrix techniques can be characterized by the accuracy of their scalar counterparts, thus designing algorithms for matrix functions can be regarded as a scalar-valued optimization problem. The derivatives needed during the optimization can be calculated automatically by exploiting the structure of the DAG in a fashion analogous to backpropagation. This article describes GraphMatFun.jl, a Julia package that offers the means to generate and manipulate computational graphs, optimize their coefficients, and generate Julia, MATLAB, and C code to evaluate them efficiently at a matrix argument. The software also provides tools to estimate the accuracy of a graph-based algorithm and thus obtain numerically reliable methods. For the exponential, for example, using a particular form (degree-optimal) of polynomials produces implementations that in many cases are cheaper, in terms of computational cost, than the Padé-based techniques typically used in mathematical software. The optimized graphs and the corresponding generated code are available online.
Investigations into the characteristics and influences of nonequilibrium evolution

In order to estimate qualitatively the inﬂuence of nonequilibrium evolution in relativistic heavy ion collisions, we use the three dimensional Ising model with Metropolis algorithm to study the evolution from nonequilibrium to equilibrium on the phase boundary. The evolution of order parameter approaches its equilibrium value exponentially, the same as that given by Langevin equation. The average relaxation time is deﬁned which is demonstrated to well represent the relaxation time in dynamical equations. It is shown that the average relaxation time at critical temperature diverges as the z th power of system size. The third and the fourth cumulants of order parameter during the nonequilibrium evolution could be either positive or negative, depending on the observation time, consistent with dynamical models at T > T c . It is found that the nonequilibrium evolution at T > T c lasts very short, and the inﬂuence is weaker than that at T < T c . Those qualitative features are instructive to determine experimentally the critical point and the phase boundary of QCD.
Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings

This paper presents a PINN training framework that employs (1) pre-training steps that accelerates and improve the robustness of the training of physics-informed neural network with auxiliary data stored in point clouds, (2) a net-to-net knowledge transfer algorithm that improves the weight initialization of the neural network and (3) a multi-objective optimization algorithm that may improve the performance of a physical-informed neural network with competing constraints. We consider the training and transfer and multi-task learning of physics-informed neural network (PINN) as multi-objective problems where the physics constraints such as the governing equation, boundary conditions, thermodynamic inequality, symmetry, and invariant properties, as well as point cloud used for pre-training can sometimes lead to conflicts and necessitating the seek of the Pareto optimal solution. In these situations, weighted norms commonly used to handle multiple constraints may lead to poor performance, while other multi-objective algorithms may scale poorly with increasing dimensionality. To overcome this technical barrier, we adopt the concept of vectorized objective function and modify a gradient descent approach to handle the issue of conflicting gradients. Numerical experiments are compared the benchmark boundary value problems solved via PINN. The performance of the proposed paradigm is compared against the classical equal-weighted norm approach. Our numerical experiments indicate that the brittleness and lack of robustness demonstrated in some PINN implementations can be overcome with the proposed strategy.
Extending Challenge Sets to Uncover Gender Bias in Machine Translation: Impact of Stereotypical Verbs and Adjectives

Human gender bias is reflected in language and text production. Because state-of-the-art machine translation (MT) systems are trained on large corpora of text, mostly generated by humans, gender bias can also be found in MT. For instance when occupations are translated from a language like English, which mostly uses gender neutral words, to a language like German, which mostly uses a feminine and a masculine version for an occupation, a decision must be made by the MT System. Recent research showed that MT systems are biased towards stereotypical translation of occupations. In 2019 the first, and so far only, challenge set, explicitly designed to measure the extent of gender bias in MT systems has been published. In this set measurement of gender bias is solely based on the translation of occupations. With our paper we present an extension of this challenge set, called WiBeMT, which adds gender-biased adjectives and sentences with gender-biased verbs. The resulting challenge set consists of over 70, 000 sentences and has been translated with three commercial MT systems: DeepL Translator, Microsoft Translator, and Google Translate. Results show a gender bias for all three MT systems. This gender bias is to a great extent significantly influenced by adjectives and to a lesser extent by verbs.
Optimizing topological switching in confined 2D-Xene nanoribbons via finite-size effects

Optimizing Topological Switching in Confined 2D-Xene Nanoribbons via Finite-Size Effects Muhammad Nadeem,1, a) Chao Zhang,2 Dimitrie Culcer,3, b) Alex R. Hamilton,3, b) Michael S. Fuhrer,4, c) and Xiaolin Wang1, a) 1)Institute for Superconducting and Electronic Materials (ISEM), Australian Institute for Innovative Materials (AIIM), University of Wollongong, Wollongong, New South Wales 2525, Australia. 2)School of Physics, University of Wollongong, Wollongong, NSW 2522, Australia 3)School of Physics, University of New South Wales, Sydney 2052, Australia. 4)School of Physics and Astronomy, Monash University, Clayton, Victoria 3800, Australia.
Helicity-selective Raman scattering from in-plane anisotropic α-MoO3

Hyperbolic crystals like α-MoO3 can support large wavevectors and photon density as compared to the commonly used dielectric crystals, which makes them a highly desirable platform for compact photonic devices. The extreme anisotropy of the dielectric constant in these crystals is intricately linked with the anisotropic character of the phonons, which along with photon confinement leads to the rich physics of phonon polaritons. However, the chiral nature of phonons in these hyperbolic crystals have not been studied in detail. In this study, we report our observations of helicity selective Raman scattering from flakes of α-MoO3. Both helicity-preserving and helicity-reversing Raman scattering are observed. Our studies reveal that helical selectivity is largely governed by the underlying crystal symmetry. This study shed light on the chiral character of the high symmetry phonons in these hyperbolic crystals. It paves the way for exploiting proposed schemes of coupling chiral phonon modes into propagating surface plasmon polaritons and realizing compact photonic circuits based on helical polarized light.
An interview method for engagement with personal sensor data

Whether investigating research questions or designing systems, many researchers and designers need to engage users with their personal data. However, it is difﬁcult to successfully design user-facing tools for interacting with personal data without ﬁrst understanding what users want to do with their data. Techniques for raw data exploration, sketching, or physicalization can avoid the perils of tool development, but prevent direct analytical access to users’ rich personal data streams. We present a new method that directly tackles this challenge: the data engagement interview. This interview method incorporates an analyst to provide real-time analysis of personal data streams, granting interview participants opportunities to directly engage their data, and interviewers to observe and ask questions throughout this engagement. We describe the method’s development through a case study with asthmatic participants, share insights and guidance from our experience, and report a broad set of insights from these interviews.
Tunable coupling scheme for implementing two-qubit gates on fluxonium qubits

I. N. Moskalenko, 2 I. S. Besedin, 2, ∗ I. A. Simakov, 2, 3, 4 and A. V. Ustinov 2, 5 National University of Science and Technology ”MISIS”, 119049 Moscow, Russia Russian Quantum Center, 143025 Skolkovo, Moscow, Russia Skolkovo Institute of Science and Technology, 143026 Moscow, Russia Moscow Institute of Physics and Technology, 141701 Dolgoprundy, Russia Physikalisches Institut, Karlsruhe Institute of Technology, Karlsruhe, Germany (Dated: September 28, 2021)
Facetron: A Multi-Speaker Face-to-Speech Model Based on Cross-Modal Latent Representations

In this paper, we propose a multi-speaker face-to-speech waveform generation model that also works for unseen speaker conditions. Using a generative adversarial network (GAN) with linguistic and speaker characteristic features as auxiliary conditions, our method directly converts face images into speech waveforms under an end-to-end training framework. The linguistic features are extracted from lip movements using a lip-reading model, and the speaker characteristic features are predicted from face images using a face encoder trained through cross-modal learning with a pre-trained acoustic model. Since these two features are uncorrelated and controlled independently, we can flexibly synthesize speech waveforms whose speaker characteristics vary depending on the input face images. We show the superiority of our proposed model over conventional methods in terms of objective and subjective evaluation results.
Federated causal inference in heterogeneous observational data

We are interested in estimating the effect of a treatment applied to individuals at multiple sites, where data is stored locally for each site. Due to privacy constraints, individual‐level data cannot be shared across sites; the sites may also have heterogeneous populations and treatment assignment mechanisms. Motivated by these considerations, we develop federated methods to draw inferences on the average treatment effects of combined data across sites. Our methods first compute summary statistics locally using propensity scores and then aggregate these statistics across sites to obtain point and variance estimators of average treatment effects. We show that these estimators are consistent and asymptotically normal. To achieve these asymptotic properties, we find that the aggregation schemes need to account for the heterogeneity in treatment assignments and in outcomes across sites. We demonstrate the validity of our federated methods through a comparative study of two large medical claims databases.
Boosting Entity-Aware Image Captioning With Multi-Modal Knowledge Graph

Entity-aware image captioning aims to describe named entities and events related to the image by utilizing the background knowledge in the associated article. This task remains challenging as it is difficult to learn the association between named entities and visual cues due to the long-tail distribution of named entities. Furthermore, the complexity of the article brings difficulty in extracting fine-grained relationships between entities to generate informative event descriptions about the image. To tackle these challenges, we propose a novel approach that constructs a multi-modal knowledge graph (MMKG) to associate the visual objects with named entities and capture the relationship between entities simultaneously with the help of external knowledge collected from the web. Specifically, we build a text sub-graph by extracting named entities and their relationships from the article, and build an image sub-graph by detecting the objects in the image. To connect these two sub-graphs, we propose a cross-modal entity matching module trained using a knowledge base that contains Wikipedia entries and the corresponding images. Finally, the MMKG is integrated into the captioning model via a graph attention mechanism. Extensive experiments on both GoodNews and NYTimes800 k datasets demonstrate the effectiveness of our method.