Miracle-dz/UniFAR-FTUs huggingface dataset:
Assisting human experts in the interpretation of their visual process: A case study on assessing copper surface adhesive potency

Deep Neural Networks are often though to lack interpretability due to the distributed nature of their internal representations. In contrast, humans can generally justify, in natural language, for their answer to a visual question with simple common sense reasoning. However, human introspection abilities have their own limits as one often struggles to justify for the recognition process behind our lowest level feature recognition ability: for instance, it is difficult to precisely explain why a given texture seems more characteristic of the surface of a finger nail rather than a plastic bottle. In this paper, we showcase an application in which deep learning models can actually help human experts justify for their own low-level visual recognition process: We study the problem of assessing the adhesive potency of copper sheets from microscopic pictures of their surface. Although highly trained material experts are able to qualitatively assess the surface adhesive potency, they are often unable to precisely justify for their decision process. We present a model that, under careful design considerations, is able to provide visual clues for human experts to understand and justify for their own recognition process. Not only can our model assist human experts in their interpretation of the surface characteristics, we show how this model can be used to test different hypothesis of the copper surface response to different manufacturing processes
Multiwavelength observations of GRB 140629A

Aims.We investigate the long gamma-ray burst (GRB) 140629A through multiwavelength observations to derive the properties of the dominant jet and its host galaxy.Methods. The afterglow and host galaxy observations were taken in the optical (Swift/UVOT and various facilities worldwide), infrared (Spitzer), and X-rays (Swift/XRT) between 40 s and 3 yr after the burst trigger.Results. Polarisation observations by the MASTER telescope indicate that this burst is weakly polarised. The optical spectrum contains absorption features, from which we confirm the redshift of the GRB as originating atz = 2.276 ± 0.001. We performed spectral fitting of the X-rays to optical afterglow data and find there is no strong spectral evolution. We determine the hydrogen column densityNHto be 7.2 × 1021cm−2along the line of sight. The afterglow in this burst can be explained by a blast wave jet with a long-lasting central engine expanding into a uniform medium in the slow cooling regime. At the end of energy injection, a normal decay phase is observed in both the optical and X-ray bands. An achromatic jet break is also found in the afterglow light curves ∼0.4 d after trigger. We fit the multiwavelength data simultaneously with a model based on a numerical simulation and find that the observations can be explained by a narrow uniform jet in a dense environment with an opening angle of 6.7° viewed 3.8° off-axis, which released a total energy of 1.4 × 1054erg. Using the redshift and opening angle, we find GRB 140629A follows both the Ghirlanda and Amati relations. From the peak time of the light curve, identified as the onset of the forward shock (181s after trigger), the initial Lorentz factor (Γ0) is constrained in the range 82–118. Fitting the host galaxy photometry, we find the host to be a low mass, star-forming galaxy with a star formation rate of log (SFR) 1.1+0.9−0.4M⊙yr−1. We obtain a value of the neutral hydrogen density by fitting the optical spectrum, logNHI = 21.0 ± 0.3, classifying this host as a damped Lyman-alpha. High ionisation lines (N V, Si IV) are also detected in the spectrum.
LPAT: Learning to Predict Adaptive Threshold for Weakly-supervised Temporal Action Localization

Recently, Weakly-supervised Temporal Action Localization (WTAL) has been densely studied because it can free us from costly annotating temporal boundaries of actions. One prevalent strategy is obtaining action score sequences over time and then truncating segments of scores higher than a fixed threshold at every kept snippet. However, the threshold is not modeled in the training process and manually setting the threshold introduces expert knowledge, which damages the coherence of systems and makes it unfair for comparisons. In this paper, we propose to adaptively set the threshold at each snippet to be its background score, which can be learned to predict (LPAT). In both training and testing time, the predicted threshold is leveraged to localize action segments and the scores of these segments are allocated for video classification. We also identify an important constraint to improve the confidence of generated proposals, and model it as a novel loss term, which facilitates the video classification loss to improve models' localization ability. As such, our LPAT model is able to generate accurate action proposals with only video-level supervision. Extensive experiments on two standard yet challenging datasets, i.e., THUMOS'14 and ActivityNet1.2, show significant improvement over state-of-the-art methods.
Hybrid sideways/longitudinal swimming in the monoflagellate Shewanella oneidensis: from aerotactic band to biofilm

Shewanella oneidensis MR-1 are facultative aerobic electroactive bacteria with an appealing potential for sustainable energy production and bioremediation. They gather around air sources, forming aerotactic bands and biofilms. Here, we experimentally follow the evolution of the band around an air bubble, and we find good agreement with the numerical solutions of the pertinent transport equations. Video microscopy reveals a transition between motile and non-motile MR-1 upon oxygen depletion, preventing further development of the biofilm. We discover that MR-1 can alternate between longitudinal fast and sideways slow swimming. The resulting bimodal velocity distributions change in response to different oxygen concentrations and gradients, supporting the biological functions of aerotaxis and confinement.
Self-enhancement of automatic tunnel accident detection (TAD) on CCTV by AI deep-learning

The deep-learning-based tunnel accident detection (TAD) system (Lee 2019) has installed a system capable of monitoring 9 CCTVs at XX site in November, 2018. The initial deep-learning training was started by studying 70,914 labeled images and label data. However, sunlight, the tail light of a vehicle, and the warning light of the working vehicle were recognized as a fire, and many pedestrians were detected in the lane of the tunnel or a black elongated black object. To solve these problems, as shown in Fig. 1, the false detection data detected in the field were trained with labeled data and reapplied in the field. As a result, false detection of pedestrians and fire could be significantly reduced.
Exploiting video sequences for unsupervised disentangling in generative adversarial networks

In this work we present an adversarial training algorithm that exploits correlations in video to learn --without supervision-- an image generator model with a disentangled latent space. The proposed methodology requires only a few modifications to the standard algorithm of Generative Adversarial Networks (GAN) and involves training with sets of frames taken from short videos. We train our model over two datasets of face-centered videos which present different people speaking or moving the head: VidTIMIT and YouTube Faces datasets. We found that our proposal allows us to split the generator latent space into two subspaces. One of them controls content attributes, those that do not change along short video sequences. For the considered datasets, this is the identity of the generated face. The other subspace controls motion attributes, those attributes that are observed to change along short videos. We observed that these motion attributes are face expressions, head orientation, lips and eyes movement. The presented experiments provide quantitative and qualitative evidence supporting that the proposed methodology induces a disentangling of this two kinds of attributes in the latent space.
POSE.R

The article presents a distributed algorithm, called Prediction-based Opportunistic Sensing for Resilient and Efficient Sensor Networks (POSE.R), where the sensor nodes utilize predictions of the targets' positions to probabilistically control their multi-modal operating states to track the targets. There are two desired features of the algorithm: energy efficiency and resilience. If the target is traveling through a high-node-density area, then an optimal sensor selection approach is employed that maximizes a joint cost function of remaining energy and geometric diversity around the target’s position. This provides energy efficiency and increases the network lifetime while preventing redundant nodes from tracking the target. However, if the target is traveling through a low-node-density area or in a coverage gap (e.g., formed by node failures or non-uniform node deployment), then a potential game is played amongst the surrounding nodes to optimally expand their sensing ranges via minimizing energy consumption and maximizing target coverage. This provides resilience, that is, the self-healing capability to track the target in the presence of low node densities and coverage gaps. The algorithm is comparatively evaluated against existing approaches through Monte Carlo simulations that demonstrate its superiority in terms of tracking performance, network-resilience, and network-lifetime.
Vision-Infused Deep Audio Inpainting

Multi-modality perception is essential to develop interactive intelligence. In this work, we consider a new task of visual information-infused audio inpainting, i.e., synthesizing missing audio segments that correspond to their accompanying videos. We identify two key aspects for a successful inpainter: (1) It is desirable to operate on spectrograms instead of raw audios. Recent advances in deep semantic image inpainting could be leveraged to go beyond the limitations of traditional audio inpainting. (2) To synthesize visually indicated audio, a visual-audio joint feature space needs to be learned with synchronization of audio and video. To facilitate a large-scale study, we collect a new multi-modality instrument-playing dataset called MUSIC-Extra-Solo (MUSICES) by enriching MUSIC dataset. Extensive experiments demonstrate that our framework is capable of inpainting realistic and varying audio segments with or without visual contexts. More importantly, our synthesized audio segments are coherent with their video counterparts, showing the effectiveness of our proposed Vision-Infused Audio Inpainter (VIAI).
Lozenge tilings of hexagons with removed core and satellites

We consider regions obtained from 120 degree rotationally invariant hexagons by removing a core and three equal satellites (all equilateral triangles) so that the resulting region is both vertically symmetric and 120 degree rotationally invariant, and give simple product formulas for the number of their lozenge tilings. We describe a new method of approach for proving these formulas, and give the full details for an illustrative special case. As a byproduct, we are also able to generalize this special case in a different direction, by finding a natural counterpart of a twenty year old formula due to Ciucu, Eisenk\"olbl, Krattenthaler and Zare, which went unnoticed until now. The general case of the original problem will be treated in a subsequent paper. We then work out consequences for the correlation of holes, which were the original motivation for this study.
Ultrashort waveguide tapers based on Luneburg lens

In integrated photonic circuits, silicon-on-insulator waveguides with different geometries have been employed to realize a variety of components. Therefore, efficient coupling of two different waveguides is crucial. In this paper, focusing property of the Luneburg lens is exploited to design waveguide tapers. The Luneburg lens, truncated in a shape of a parabolic taper with reduced footprint, is utilized to connect a 10 μm wide waveguide to a 0.5 μm one with the same thickness with an average coupling loss of 0.35 dB in the entire O, E, S, C, L, and U bands of optical communications. The proposed compact taper with the length of 11 μm is implemented by varying the thickness of the guiding layer and compared with three conventional tapers with the same length. However, designing a coupler to connect waveguides with different thicknesses and widths is more challenging. By applying quasi-conformal transformation optics, we flatten the Luneburg lens and consequently increase the refractive index on the flattened side. As a result, we are able to couple two waveguides with different thicknesses and widths. The numerical simulations are used to evaluate the theoretically designed tapers. To our knowledge, this is the first study presenting ultrashort tapers based on truncated Luneburg lens.
Resilience for Landslide Geohazards and Promoting Strategies in the Three Gorges Reservoir Area

Recently, resilience is increasingly used as a concept for understanding natural disaster systems. Landslide is one of the most frequent geohazards in the Three Gorges Reservoir Area (TGRA). However, it is difficult to measure local disaster resilience, because of special geographical location in the TGRA and the special disaster landslide. Current approaches to disaster resilience evaluation are usually limited either by the qualitative method or properties of different disaster. Therefore, practical evaluating methods for the disaster resilience are needed. In this study, we developed an indicator system to evaluate landslides disaster resilience in the TGRE at the county level. It includes two properties of inherent geological stress and external social response, which are summarized into physical stress and social forces. The evaluated disaster resilience can be simulated for promoting strategies with fuzzy cognitive map (FCM). The results show that, (1) The strong resilience counties are: Fuling and Yunyang. The weak resilience counties are Wulong, Xingshan, Zhong, Zigui and Dudukou. (2) The resilience of the TGRA has spatial auto-correlation, that is, the areas with similar resilience are gathered in geographical location. (3) Proper policy guideline is essential to promote the disaster resilience. Policy promotes the system from all aspects in the TGRA.
Towards Fine-Grained Prosody Control for Voice Conversion

In a typical voice conversion system, previous works utilized various acoustic features (such as the pitch, voiced/unvoiced flag and aperiodicity) of the source speech to control the prosody of converted speech. However, prosody is related with many factors, such as the intonation, stress and rhythm. It is a challenging task to perfectly describe prosody through hand-crafted acoustic features. To address these difficulties, we propose to use prosody embeddings to describe prosody. These embeddings are learned from the source speech in an unsupervised manner. To verify the effectiveness of our proposed method, we conduct experiments on our Mandarin corpus. Experimental results show that our proposed method can improve the speech quality and speaker similarity of the converted speech. What’s more, we observe that our method can even achieve promising results in singing conditions.
On the frequency correlations of low-frequency QPOs with kilohertz QPOs in accreting millisecond X-ray pulsars

We investigate frequency correlations of low frequency (LF, <80 Hz) and kHz quasi-periodic oscillations (QPOs) using the complete RXTE data sets on six accreting millisecond X-ray pulsars (AMXPs) and compare them to those of non-pulsating neutron star (NS) low-mass X-ray binaries with known spin. For the AMXPs SAX J1808.4−3658 and XTE J1807−294, we find frequency-correlation power-law indices that, surprisingly, are significantly lower than in the non-pulsars, and consistent with the relativistic precession model (RPM) prediction of 2.0 appropriate to test-particle orbital and Lense–Thirring precession frequencies. As previously reported, power-law normalizations are significantly higher in these AMXPs than in the non-pulsating sources, leading to requirements on the NS specific moment of inertia in this model that cannot be satisfied with realistic equations of state. At least two other AMXPs show frequency correlations inconsistent with those of SAX J1808.4−3658 and XTE J1807−294, and possibly similar to those of the non-pulsating sources; for two AMXPs no conclusions could be drawn. We discuss these results in the context of a model that has had success in black hole (BH) systems involving a torus-like hot inner flow precessing due to (prograde) frame dragging, and a scenario in which additional (retrograde) magnetic and classical precession torques not present in BH systems are also considered. We show that a combination of these interpretations may accommodate our results.
Bootstrapping deep music separation from primitive auditory grouping principles

Separating an audio scene such as a cocktail party into constituent, meaningful components is a core task in computer audition. Deep networks are the state-of-the-art approach. They are trained on synthetic mixtures of audio made from isolated sound source recordings so that ground truth for the separation is known. However, the vast majority of available audio is not isolated. The brain uses primitive cues that are independent of the characteristics of any particular sound source to perform an initial segmentation of the audio scene. We present a method for bootstrapping a deep model for music source separation without ground truth by using multiple primitive cues. We apply our method to train a network on a large set of unlabeled music recordings from YouTube to separate vocals from accompaniment without the need for ground truth isolated sources or artificial training mixtures.
Face Detection on Surveillance Images

In last few decades, a lot of progress has been made in the field of face detection. Various face detection methods have been proposed by numerous researchers working in this area. The two well-known benchmarking platform: the FDDB and WIDER face detection provide quite challenging scenarios to assess the efficacy of the detection methods. These benchmarking data sets are mostly created using images from the public network ie. the Internet. A recent, face detection and open-set recognition challenge has shown that those same face detection algorithms produce high false alarms for images taken in surveillance scenario. This shows the difficult nature of the surveillance environment. Our proposed body pose based face detection method was one of the top performers in this competition. In this paper, we perform a comparative performance analysis of some of the well known face detection methods including the few used in that competition, and, compare them to our proposed body pose based face detection method. Experiment results show that, our proposed method that leverages body information to detect faces, is the most realistic approach in terms of accuracy, false alarms and average detection time, when surveillance scenario is in consideration.
Scale-Equivariant Steerable Networks

The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various tasks due to the changing distances between the objects and the camera. First, we introduce the general theory for building scale-equivariant convolutional networks with steerable filters. We develop scale-convolution and generalize other common blocks to be scale-equivariant. We demonstrate the computational efficiency and numerical stability of the proposed method. We compare the proposed models to the previously developed methods for scale equivariance and local scale invariance. We demonstrate state-of-the-art results on MNIST-scale dataset and on STL-10 dataset in the supervised learning setting.
Preventing Adversarial Use of Datasets through Fair Core-Set Construction

We propose improving the privacy properties of a dataset by publishing only a strategically chosen "core-set" of the data containing a subset of the instances. The core-set allows strong performance on primary tasks, but forces poor performance on unwanted tasks. We give methods for both linear models and neural networks and demonstrate their efficacy on data.
Optimal fermion-to-qubit mapping via ternary trees with applications to reduced quantum states learning

We introduce a fermion-to-qubit mapping defined on ternary trees, where any single Majorana operator on an $n$-mode fermionic system is mapped to a multi-qubit Pauli operator acting nontrivially on $\lceil \log_3(2n+1)\rceil$ qubits. The mapping has a simple structure and is optimal in the sense that it is impossible to construct Pauli operators in any fermion-to-qubit mapping acting nontrivially on less than $\log_3(2n)$ qubits on average. We apply it to the problem of learning $k$-fermion reduced density matrix (RDM), a problem relevant in various quantum simulation applications. We show that using the ternary-tree mapping one can determine the elements of all $k$-fermion RDMs, to precision $\epsilon$, by repeating a single quantum circuit for $\lesssim (2n+1)^k \epsilon^{-2}$ times. This result is based on a method we develop here that allows one to determine the elements of all $k$-qubit RDMs, to precision $\epsilon$, by repeating a single quantum circuit for $\lesssim 3^k \epsilon^{-2}$ times, independent of the system size. This improves over existing schemes for determining qubit RDMs.
A Graph-Based Framework to Bridge Movies and Synopses

Inspired by the remarkable advances in video analytics, research teams are stepping towards a greater ambition – movie understanding. However, compared to those activity videos in conventional datasets, movies are significantly different. Generally, movies are much longer and consist of much richer temporal structures. More importantly, the interactions among characters play a central role in expressing the underlying story. To facilitate the efforts along this direction, we construct a dataset called Movie Synopses Associations (MSA) over 327 movies, which provides a synopsis for each movie, together with annotated associations between synopsis paragraphs and movie segments. On top of this dataset, we develop a framework to perform matching between movie segments and synopsis paragraphs. This framework integrates different aspects of a movie, including event dynamics and character interactions, and allows them to be matched with parsed paragraphs, based on a graph-based formulation. Our study shows that the proposed framework remarkably improves the matching accuracy over conventional feature-based methods. It also reveals the importance of narrative structures and character interactions in movie understanding. Dataset and code are available at: https://ycxioooong.github. io/projects/moviesyn
Deep Learning for Whole Slide Image Analysis: An Overview

The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artifacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies.