Miracle-dz/UniFAR-FTUs huggingface dataset:
FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Semi-supervised Optical Flow Estimation

Optical flow estimation is an important yet challenging problem in the field of video analytics. The features of different semantics levels/layers of a convolutional neural network can provide information of different granularity. To exploit such flexible and comprehensive information, we propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. The pyramid correlation mapping module takes advantage of the multi-scale correlations of global/local patches by aggregating features of different scales to form a multi-level cost volume. The residual reconstruction module aims to reconstruct the sub-band high-frequency residuals of finer optical flow in each stage. Based on the pyramid correlation mapping, we further propose a correlation-warping-normalization (CWN) module to efficiently exploit the correlation dependency. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods - FlowNet2, LiteFlowNet and PWC-Net on the Final pass of Sintel dataset, respectively.
Increased pentraxin-3 level in patients with slow coronary flow

Amac: Koroner yavas akim (YKA), anjiyografisinde normal koroner arterlere sahip olup opak maddenin koronerlerin distaline gec ulasmasi ile karakterizedir. YKA'in inflamasyon ve yuksek duyarlikli C reaktif protein (hs-CRP) gibi inflamatuvar belirtecler ile iliskisi bilinmektedir. Pentraksin-3 (PTX- 3), yeni bir akut faz reaktani olup CRP gibi pentraksin ailesinin bir uyesidir. Biz bu calismada YKA hastalarinda PTX- 3 duzeyini arastirdik. Yontem: Calismaya YKA saptanan 25 hasta ve koroner arter hastaligi (KAH) olan 26 hasta alindi. Yavas koroner akim ve KAH tanisi koroner anjiyografi ile konuldu. Kardiyoloji poliklinigine basvurmus iskemik bulgularin gozlenmedigi 24 saglikli birey kontrol grubu olarak alindi. Tum grubun PTX-3 ve hs-CRP calisildi. Bulgular: KYA grubundaki hastalarin PTX-3 ve hs-CRP seviyesi kontrol grubuna gore daha yuksekti(sirasiyla 0.52 ± 0.2 ng/ml ve 0.20 ±0.08 ng/ml, p< 0.001; 1.1±0.4 mg/dl ve 0.6±0.5 mg/dl, p< 0.001). Ancak KYA grubu ile KAH grubu arasinda serum PTX-3 ile hs-CRP seviyesinde fark bulunmadi (sirasiyla 0.52 ± 0.2 ng/ml ve 0.58 ± 0.18 ng/ml, p: 0.24; 1.1 ± 0.4 mg/dl ve 1.1 ± 0.6 mg/dl; p: 0.32). Korelasyon analizi sonucu serum PTX-3 ve hs-CRP seviyeleri birbirleriyle iliskili bulundu. (Rho=0.34, p: 0.003). Sonuc: PTX-3, yeni bir inflamatuvar marker olup YKA olan hastalarda yukselmistir ve bu hastalarda inflamatuvar durumu yansitmada bir belirtectir. Objective: Slow coronary flow (SCF) is defined as late opacification in the epicardial coronary artery without significant stenosis on the coronary angiographic images. The association between SCF and inflammation and inflammatory markers such as high sensitivity-C reactive protein (hs- CRP) is well known. Pentraxin-3 (PTX-3), a new acute phase reactant, is a member of pentraxine family like hs-CRP. We investigated the association between PTX-3 and hs-CRP in patients with SCF. Method: The study included 25 patients with SCF and 26 patients with coronary artery disease (CAD) whose diagnoses were made by coronary angiography. The control group consisted of 24 healthy subjects admitted cardiology outpatient clinic without any sign of ischemia. From the all study population PTX-3 and hs-CRP levels were measured. Results: The SCF group had significantly increased PTX-3 and hs- CRP levels than the control group (0.52 ± 0.2 ng/ml vs 0.20 ±0.08 ng/ml, p< 0.001; 1.1±0.4 mg/dl vs 0.6±0.5 mg/dl, p< 0.001, respectively). However there were no differences in levels of PTX-3 and hs-CRP between the SCF and the CAD groups (0.52 ± 0.2 ng/ml vs 0.58 ± 0.18 ng/ml, p: 0.24; 1.1 ± 0.4 mg/dl vs 1.1 ± 0.6 mg/dl; p: 0.32, respectively). Correlation analysis revealed a positive correlation between serum PTX-3 levels and hs-CRP levels (r=0.34, p: 0.003). Conclusion: PTX-3, a novel inflammatory marker, is elevated in patients with SCF and may be reflecting the inflammatory status in patients with SCF.
Low Latency Volumetric Video Edge Cloud Streaming

Volumetric video is an emerging key technology for immersive representation of 3D spaces and objects. The enhanced immersion of volumetric videos leads to new use cases such as streaming of six-degrees-of-freedom (6DoF) videos in which the user can freely change his position and orientation. However, rendering volumetric videos as 3D representations (mesh or point cloud) requires lots of computational power and transmission of such volumetric data requires lots of bandwidth. To mitigate this issue, rendering a 2D view from the volumetric data at a cloud/edge server and streaming that as a 2D video is a feasible solution. However, network-based processing brings additional network and processing latency. In order to reduce the motion-to-photon latency, prediction of the future user pose is necessary. We developed a 6DoF user movement prediction model for very low latency streaming services and investigated its potential to further reduce the motion-to-photon latency for different prediction windows. Our results show that the developed prediction model on average reduces the positional rendering errors caused by the motion-to-photon latency compared to a baseline system in which no prediction is performed.
TailorGAN: Making User-Defined Fashion Designs

Attribute editing has become an important and emerging topic of computer vision. In this paper, we consider a task: given a reference garment image A and another image B with target attribute (collar/sleeve), generate a photorealistic image which combines the texture from reference A and the new attribute from reference B. The highly convoluted attributes and the lack of paired data are the main challenges to the task. To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e.g., collar and sleeves) without paired data. Our method consists of a reconstruction learning step and an adversarial learning step. The model learns texture and location information through reconstruction learning. And, the model’s capability is generalized to achieve single-attribute manipulation by adversarial learning. Meanwhile, we compose a new dataset, named GarmentSet, with annotation of landmarks of collars and sleeves on clean garment images. Extensive experiments on this dataset and real-world samples demonstrate that our method can synthesize much better results than the state-of-the-art methods in both quantitative and qualitative comparisons.
Dynamics of the mean-field-interacting quantum kicked rotor

We study the dynamics of the many-body atomic kicked rotor with interactions at the mean-field level, governed by the Gross-Pitaevskii equation. We show that dynamical localization is destroyed by the interaction, and replaced by a subdiffusive behavior. In contrast to results previously obtained from a simplified version of the Gross-Pitaevskii equation, the subdiffusive exponent does not appear to be universal. By studying the phase of the mean-field wave function, we propose a new approximation that describes correctly the dynamics at experimentally relevant times close to the start of subdiffusion, while preserving the reduced computational cost of the former approximation.
Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition

Affective computing and cognitive theory are widely used in modern human-computer interaction scenarios. Human faces, as the most prominent and easily accessible features, have attracted great attention from researchers. Since humans have rich emotions and developed musculature, there exist a lot of fine-grained expressions in real-world applications. However, it is extremely time-consuming to collect and annotate a large number of facial images, of which may even require psychologists to correctly categorize them. To the best of our knowledge, the existing expression datasets are only limited to several basic facial expressions, which are not sufficient to support our ambitions in developing successful human-computer interaction systems. To this end, a novel Fine-grained Facial Expression Database - F2ED is contributed in this paper, and it includes more than 200k images with 54 facial expressions from 119 persons. Considering the phenomenon of uneven data distribution and lack of samples is common in real-world scenarios, we further evaluate several tasks of few-shot expression learning by virtue of our F2ED, which are to recognize the facial expressions given only few training instances. These tasks mimic human performance to learn robust and general representation from few examples. To address such few-shot tasks, we propose a unified task-driven framework - Compositional Generative Adversarial Network (Comp-GAN) learning to synthesize facial images and thus augmenting the instances of few-shot expression classes. Extensive experiments are conducted on F2ED and existing facial expression datasets, i.e., JAFFE and FER2013, to validate the efficacy of our F2ED in pre-training facial expression recognition network and the effectiveness of our proposed approach Comp-GAN to improve the performance of few-shot recognition tasks.
A Study of Mental Maps in Immersive Network Visualization

The visualization of a network influences the quality of the mental map that the viewer develops to understand the network. In this study, we investigate the effects of a 3D immersive visualization environment compared to a traditional 2D desktop environment on the comprehension of a network’s structure. We compare the two visualization environments using three tasks—interpreting network structure, memorizing a set of nodes, and identifying the structural changes—commonly used for evaluating the quality of a mental map in network visualization. The results show that participants were able to interpret network structure more accurately when viewing the network in an immersive environment, particularly for larger networks. However, we found that 2D visualizations performed better than immersive visualization for tasks that required spatial memory.
SeismiQB - a novel framework for deep learning with seismic data

In recent years, Deep Neural Networks were successfully adopted in numerous domains to solve various image-related tasks, ranging from simple classification to fine borders annotation. Naturally, many researches proposed to use it to solve geological problems. Unfortunately, many of the seismic processing tools were developed years before the era of machine learning, including the most popular SEG-Y data format for storing seismic cubes. Its slow loading speed heavily hampers experimentation speed, which is essential for getting acceptable results. Worse yet, there is no widely-used format for storing surfaces inside the volume (for example, seismic horizons). To address these problems, we've developed an open-sourced Python framework with emphasis on working with neural networks, that provides convenient tools for (i) fast loading seismic cubes in multiple data formats and converting between them, (ii) generating crops of desired shape and augmenting them with various transformations, and (iii) pairing cube data with labeled horizons or other types of geobodies.
Reionization with galaxies and active galactic nuclei

In this work we investigate the properties of the sources that reionized the intergalactic medium (IGM) in the high-redshift Universe. Using a semi-analytical model aimed at reproducing galaxies and black holes in the first ∼1.5 Gyr of the Universe, we revisit the relative role of star formation and black hole accretion in producing ionizing photons that can escape into the IGM. Both star formation and black hole accretion are regulated by supernova feedback, resulting in black hole accretion being stunted in low-mass haloes. We explore a wide range of combinations for the escape fraction of ionizing photons (redshift-dependent, constant, and scaling with stellar mass) from both star formation ($\langle f_{\rm esc}^{\rm sf} \rangle$) and AGN ($f_{\rm esc}^{\rm bh}$) to find: (i) the ionizing budget is dominated by stellar radiation from low stellar mass ($M_*\lt 10^9 \, {\rm \rm M_\odot }$) galaxies at z > 6 with the AGN contribution (driven by $M_{bh}\gt 10^6 \, {\rm \rm M_\odot }$ black holes in $M_* \gtrsim 10^9\, {\rm \rm M_\odot }$ galaxies) dominating at lower redshifts; (ii) AGN only contribute $10-25{{\ \rm per\ cent}}$ to the cumulative ionizing emissivity by z = 4 for the models that match the observed reionization constraints; (iii) if the stellar mass dependence of $\langle f_{\rm esc}^{\rm sf} \rangle$ is shallower than $f_{\rm esc}^{\rm bh}$, at z < 7 a transition stellar mass exists above which AGN dominate the escaping ionizing photon production rate; (iv) the transition stellar mass decreases with decreasing redshift. While AGN dominate the escaping emissivity above the knee of the stellar mass function at z ∼ 6.8, they take-over at stellar masses that are a tenth of the knee mass by z = 4.
Two-Phase Object-Based Deep Learning for Multi-temporal SAR Image Change Detection

Change detection is one of the fundamental applications of synthetic aperture radar (SAR) images. However, speckle noise presented in SAR images has a negative effect on change detection, leading to frequent false alarms in the mapping products. In this research, a novel two-phase object-based deep learning approach is proposed for multi-temporal SAR image change detection. Compared with traditional methods, the proposed approach brings two main innovations. One is to classify all pixels into three categories rather than two categories: unchanged pixels, changed pixels caused by strong speckle (false changes), and changed pixels formed by real terrain variation (real changes). The other is to group neighbouring pixels into superpixel objects such as to exploit local spatial context. Two phases are designed in the methodology: (1) Generate objects based on the simple linear iterative clustering (SLIC) algorithm, and discriminate these objects into changed and unchanged classes using fuzzy c-means (FCM) clustering and a deep PCANet. The prediction of this Phase is the set of changed and unchanged superpixels. (2) Deep learning on the pixel sets over the changed superpixels only, obtained in the first phase, to discriminate real changes from false changes. SLIC is employed again to achieve new superpixels in the second phase. Low rank and sparse decomposition are applied to these new superpixels to suppress speckle noise significantly. A further clustering step is applied to these new superpixels via FCM. A new PCANet is then trained to classify two kinds of changed superpixels to achieve the final change maps. Numerical experiments demonstrate that, compared with benchmark methods, the proposed approach can distinguish real changes from false changes effectively with significantly reduced false alarm rates, and achieve up to 99.71% change detection accuracy using multi-temporal SAR imagery.
Spatiotemporal Camera-LiDAR Calibration: A Targetless and Structureless Approach

The demand for multimodal sensing systems for robotics is growing due to the increase in robustness, reliability and accuracy offered by these systems. These systems also need to be spatially and temporally co-registered to be effective. In this letter, we propose a targetless and structureless spatiotemporal camera-LiDAR calibration method. Our method combines a closed-form solution with a modified structureless bundle adjustment where the coarse-to-fine approach does not require an initial guess on the spatiotemporal parameters. Also, as 3D features (structure) are calculated from triangulation only, there is no need to have a calibration target or to match 2D features with the 3D point cloud which provides flexibility in the calibration process and sensor configuration. We demonstrate the accuracy and robustness of the proposed method through both simulation and real data experiments using multiple sensor payload configurations mounted to hand-held, aerial and legged robot systems. Also, qualitative results are given in the form of a colorized point cloud visualization.
Overly Optimistic Prediction Results on Imbalanced Data: Flaws and Benefits of Applying Over-sampling

Information extracted from electrohysterography recordings could potentially prove to be an interesting additional source of information to estimate the risk on preterm birth. Recently, a large number of studies have reported near-perfect results to distinguish between recordings of patients that will deliver term or preterm using a public resource, called the Term/Preterm Electrohysterogram database. However, we argue that these results are overly optimistic due to a methodological flaw being made. In this work, we focus on one specific type of methodological flaw: applying over-sampling before partitioning the data into mutually exclusive training and testing sets. We show how this causes the results to be biased using two artificial datasets and reproduce results of studies in which this flaw was identified. Moreover, we evaluate the actual impact of over-sampling on predictive performance, when applied prior to data partitioning, using the same methodologies of related studies, to provide a realistic view of these methodologies' generalization capabilities. We make our research reproducible by providing all the code under an open license.
Online Convex Optimization for Caching Networks

We study the problem of wireless edge caching when file popularity is unknown and possibly non-stationary. A bank of <inline-formula> <tex-math notation="LaTeX">$J$ </tex-math></inline-formula> caches receives file requests and a utility is accrued for each request depending on the serving cache. The network decides dynamically which files to store at each cache and how to route them, in order to maximize total utility. The request sequence is assumed to be drawn from an arbitrary distribution, capturing time-variance, temporal and spatial locality of requests. For this challenging setting, we propose the <italic>Bipartite Supergradient Caching Algorithm</italic> (BSCA) which provably exhibits no regret (<inline-formula> <tex-math notation="LaTeX">$R_{T}/T \to 0$ </tex-math></inline-formula>). That is, as the time horizon <inline-formula> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> increases, BSCA achieves (at least) the same utility with the cache configuration that we would have chosen knowing all future requests. The learning rate of the algorithm is characterized by its regret expression <inline-formula> <tex-math notation="LaTeX">$R_{T}\! =\! O(\sqrt {JT})$ </tex-math></inline-formula>, which is independent of the file library size. For the single-cache case, we prove that this is the lowest attainable bound. BSCA requires at each step <inline-formula> <tex-math notation="LaTeX">$J$ </tex-math></inline-formula> projections on intersections of boxes and simplices, for which we propose a tailored algorithm. Our model is the first that draws a connection between the network caching problem and <italic>Online Convex Optimization</italic>, and we demonstrate its generality by discussing various practical extensions and presenting a trace-driven comparison with state-of-the-art competitors.
On-Device Information Extraction from Screenshots in form of tags

We propose a method to make mobile Screenshots easily searchable. In this paper, we present the workflow in which we: 1) pre-processed a collection of screenshots, 2) identified script present in image, 3) extracted unstructured text from images, 4) identified language of the extracted text, 5) extracted keywords from the text, 6) identified tags based on image features, 7) expanded tag set by identifying related keywords, 8) inserted image tags with relevant images after ranking and indexed them to make it searchable on device. We made the pipeline which supports multiple languages and executed it on-device, which addressed privacy concerns. We developed novel architectures for components in the pipeline, optimized performance and memory for on-device computation. We observed from experimentation that the solution developed can reduce overall user effort and improve end user experience while searching, whose results are published.
Sinogram super-resolution and denoising convolutional neural network (SRCN) for limited data photoacoustic tomography

The quality of the reconstructed photoacoustic image largely depends on the amount of photoacoustic (PA) boundary data available, which in turn is proportional to the number of detectors employed. In case of limited data (owing to less number of detectors due to cost/instrumentation constraints), the reconstructed PA images suffer from artifacts and are often noisy. In this work, for the first time, a deep learning based model was developed to super resolve and denoise the photoacoustic sinogram data. The proposed method was compared with existing nearest neighbor interpolation and wavelet based denoising techniques and was shown to outperform them both in numerical and in-vivo cases. The improvement obtained in Root Mean Square Error (RMSE) and Peak Signal to Noise Ratio (PSNR) for the reconstructed PA image using the sinogram data that was super-resolved and denoised using proposed neural network based method was as high as 41.70 % and 6.93 dB respectively compared to utilizing limited sinogram data.
Note on all-order Landau-level structures of the Heisenberg-Euler effective actions for QED and QCD

We investigate the Landau-level structures encoded in the famous Heisenberg-Euler (HE) effective action in constant electromagnetic fields. We first discuss the HE effective actions for scalar and spinor QED, and then extend it to the QCD analogue in the covariantly constant chromo-electromagnetic fields. We identify all the Landau levels and the Zeeman energies starting out from the proper-time representations at the one-loop order, and derive the vacuum persistence probability for the Schwinger mechanism in the summation form over independent contributions of the all-order Landau levels. We find an enhancement of the Schwinger mechanism catalyzed by a magnetic field for spinor QED and, in contrast, a stronger exponential suppression for scalar QED due to the "zero-point energy" of the Landau quantization. For QCD, we identify the discretized energy levels of the transverse and longitudinal gluon modes on the basis of their distinct Zeeman energies, and explicitly confirm the cancellation between the longitudinal-gluon and ghost contributions in the Schwinger mechanism. We also discuss the unstable ground state of the perturbative gluon excitations known as the Nielsen-Olesen instability.