Miracle-dz/UniFAR-FTUs huggingface dataset:
SVS-net: A Novel Semantic Segmentation Network in Optical Coherence Tomography Angiography Images

Automated vascular segmentation on optical coherence tomography angiography (OCTA) is important for the quantitative analyses of retinal microvasculature in neuroretinal and systemic diseases. Despite recent improvements, artifacts continue to pose challenges in segmentation. Our study focused on removing the speckle noise artifact from OCTA images when performing segmentation. Speckle noise is common in OCTA and is particularly prominent over large non-perfusion areas. It may interfere with the proper assessment of retinal vasculature. In this study, we proposed a novel Supervision Vessel Segmentation network (SVS-net) to detect vessels of different sizes. The SVS-net includes a new attention-based module to describe vessel positions and facilitate the understanding of the network learning process. The model is efficient and explainable and could be utilized to reduce the need for manual labeling. Our SVS-net had better performance in accuracy, recall, F1 score, and Kappa score when compared to other well recognized models.
Sentence-Permuted Paragraph Generation

Generating paragraphs of diverse contents is important in many applications. Existing generation models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. Our idea is permuting the sentence orders to improve the content diversity of multi-sentence paragraph. We propose a novel framework PermGen whose objective is to maximize the expected log-likelihood of output paragraph distributions with respect to all possible sentence orders. PermGen uses hierarchical positional embedding and designs new procedures for training, and decoding in the sentence-permuted generation. Experiments on three paragraph generation benchmarks demonstrate PermGen generates more diverse outputs with a higher quality than existing models.
Predicting Discourse Trees from Transformer-based Neural Summarizers

Previous work indicates that discourse information benefits summarization. In this paper, we explore whether this synergy between discourse and summarization is bidirectional, by inferring document-level discourse trees from pre-trained neural summarizers. In particular, we generate unlabeled RST-style discourse trees from the self-attention matrices of the transformer model. Experiments across models and datasets reveal that the summarizer learns both, dependency- and constituency-style discourse information, which is typically encoded in a single head, covering long- and short-distance discourse dependencies. Overall, the experimental results suggest that the learned discourse information is general and transferable inter-domain.
Exporters' reaction to positive foreign demand shocks

I use the quasi-natural experiment of the 2018 African swine fever (ASF) outbreak in China to analyze swine exporters' reaction to a foreign market's positive demand shock. I use the universe of Spanish firms' export transactions to China and other countries, and compare the performance of swine and other exporters before and after the ASF. The ASF almost tripled Spanish swine exporters' sales to China. Swine exporters did not increase exported product portfolio or export revenue concentration in their best-performing products in China after the ASF. The increase in exports to China positively impacted export revenue and survival in third markets. This positive impact was especially intense for small swine exporters. Domestic sales also increased for swine exporters with liquidity constraints before the ASF.
Attentive Max Feature Map for Acoustic Scene Classification with Joint Learning considering the Abstraction of Classes

The attention mechanism has been widely adopted in acoustic scene classification. However, we find that during the process of attention exclusively emphasizing information, it tends to excessively discard information although improving the performance. We propose a mechanism referred to as the attentive max feature map which combines two effective techniques, attention and max feature map, to further elaborate the attention mechanism and mitigate the abovementioned phenomenon. Furthermore, we explore various joint learning methods that utilize additional labels originally generated for subtask B (3-classes) on top of existing labels for subtask A (10-classes) of the DCASE2020 challenge. We expect that using two kinds of labels simultaneously would be helpful because the labels of the two subtasks differ in their degree of abstraction. Applying two proposed techniques, our proposed system achieves state-of-the-art performance among single systems on subtask A. In addition, because the model has a complexity comparable to subtask B's requirement, it shows the possibility of developing a system that fulfills the requirements of both subtasks; generalization on multiple devices and low-complexity.
Spatial Knowledge Acquisition in Virtual and Physical Reality: A Comparative Evaluation

Virtual Reality (VR) head-mounted displays (HMDs) have been studied widely as tools for the most diverse kinds of training activities. One particular kind that is the basis for many real-world applications is spatial knowledge acquisition and navigation. For example, knowing well by heart escape routes can be an important factor for firefighters and soldiers. Prior research on how well knowledge acquired in virtual worlds translates to the real, physical one has had mixed results, with some suggesting spatial learning in VR is akin to using a regular 2D display. However, VR HMDs have evolved drastically in the last decade, and little is known about how spatial training skills in a simulated environment using up-to-date VR HMDs compare to training in the real world. In this paper, we investigate how people trained in a VR maze compare against those trained in a physical maze in terms of recall of the position of items inside the environment. While our results did not find significant differences in time performance for people who experienced the physical and those who trained in VR, other behavioral factors were different.
A comparative study of Different Machine Learning Regressors For Stock Market Prediction

For the development of successful share trading strategies, forecasting the course of action of the stock market index is important. Effective prediction of closing stock prices could guarantee investors attractive benefits. Machine learning algorithms have the ability to process and forecast almost reliable closing prices for historical stock patterns. In this article, we intensively studied NASDAQ stock market and targeted to choose the portfolio of ten different companies belongs to different sectors. The objective is to compute opening price of next day stock using historical data. To fulfill this task nine different Machine Learning regressor applied on this data and evaluated using MSE and R2 as performance metric.
Are Multilingual BERT models robust? A Case Study on Adversarial Attacks for Multilingual Question Answering

Recent approaches have exploited weaknesses in monolingual question answering (QA) models by adding adversarial statements to the passage. These attacks caused a reduction in state-of-the-art performance by almost 50%. In this paper, we are the first to explore and successfully attack a multilingual QA (MLQA) system pre-trained on multilingual BERT using several attack strategies for the adversarial statement reducing performance by as much as 85%. We show that the model gives priority to English and the language of the question regardless of the other languages in the QA pair. Further, we also show that adding our attack strategies during training helps alleviate the attacks.
Mass and Environment as Drivers of Galaxy Evolution. IV. On the Quenching of Massive Central Disk Galaxies in the Local Universe

The phenomenological study of evolving galaxy populations in Peng et al. has shown that star forming galaxies can be quenched by two distinct processes: mass quenching and environment quenching. To explore the mass quenching process in local galaxies, we study the massive central disk galaxies with stellar mass above the Schechter characteristic mass. In Zhang et al., we showed that during the quenching of the massive central disk galaxies as their star formation rate decreases, their molecular gas mass and star formation efficiency drop rapidly but their H i gas mass remains surprisingly constant. To identify the underlying physical mechanisms, in this work we analyze the change during quenching of various structure parameters, bar frequency, and active galactic nucleus (AGN) activity. We find three closely related facts. On average, as star formation rate decreases in these galaxies: (1) they become progressively more compact, indicated by their significantly increasing concentration index, bulge-to-total mass ratio, and central velocity dispersion, which are mainly driven by the growth and compaction of their bulge component; (2) the frequency of barred galaxies increases dramatically, and at a given concentration index the barred galaxies have a significantly higher quiescent fraction than unbarred galaxies, implying that the galactic bar may play an important role in mass quenching; and (3) the “AGN” frequency increases dramatically from 10% on the main sequence to almost 100% for the most quiescent galaxies, which is mainly driven by the sharp increase of LINERs. These observational results lead to a self-consistent picture of how mass quenching operates.
Identification of unknown parameters and prediction with hierarchical matrices

Statistical analysis of massive datasets very often implies expensive linear algebra operations with large dense matrices. Typical tasks are an estimation of unknown parameters of the underlying statistical model and prediction of missing values. We developed the H-MLE procedure, which solves these tasks. The unknown parameters can be estimated by maximizing the joint Gaussian log-likelihood function, which depends on a covariance matrix. To decrease high computational cost, we approximate the covariance matrix in the hierarchical (H-) matrix format. The H-matrix technique allows us to work with inhomogeneous covariance matrices and almost arbitrary locations. Especially, H-matrices can be applied in cases when the matrices under consideration are dense and unstructured. For validation purposes, we implemented three machine learning methods: the k-nearest neighbors (kNN), random forest, and deep neural network. The best results (for the given datasets) were obtained by the kNN method with three or seven neighbors depending on the dataset. The results computed with the H-MLE method were compared with the results obtained by the kNN method. The developed H-matrix code and all datasets are freely available online.
Continual Learning From Unlabeled Data Via Deep Clustering

Continual learning, a promising future learning strategy, aims to learn new tasks incrementally using less computation and memory resources instead of retraining the model from scratch whenever new task arrives. However, existing approaches are designed in supervised fashion assuming all data from new tasks have been annotated, which are not practical for many real-life applications. In this work, we introduce a new framework to make continual learning feasible in unsupervised mode by using pseudo label obtained from cluster assignments to update model. We focus on image classiﬁcation task under class-incremental setting and assume no class label is provided for training in each incremental learning step. For illustration purpose, we apply k-means clustering, knowledge distillation loss and exemplar set as our baseline solution, which achieves competitive re-sults even compared with supervised approaches on both challenging CIFAR-100 and ImageNet (ILSVRC) datasets. We also demonstrate that the performance of our baseline solution can be further improved by incorporating recently developed supervised continual learning techniques, showing great potential for our framework to minimize the gap between supervised and unsupervised continual learning.
NT5?! Training T5 to Perform Numerical Reasoning

Numerical reasoning over text (NRoT) presents unique challenges that are not well addressed by existing pre-training objectives. We explore five sequential training schedules that adapt a pre-trained T5 model for NRoT. Our final model is adapted from T5, but further pre-trained on three datasets designed to strengthen skills necessary for NRoT and general reading comprehension before being fine-tuned on the Discrete Reasoning over Text (DROP) dataset. The training improves DROP's adjusted F1 performance (a numeracy-focused score) from 45.90 to 70.83. Our model closes in on GenBERT (72.4), a custom BERT-Base model using the same datasets with significantly more parameters. We show that training the T5 multitasking framework with multiple numerical reasoning datasets of increasing difficulty, good performance on DROP can be achieved without manually engineering partitioned functionality between distributed and symbol modules.
The Wishart-Sachdev-Ye-Kitaev model: Exact spectral density, Q-Laguerre polynomials, and quantum chaos

We study the Wishart-Sachdev-Ye-Kitaev (WSYK) model consisting of two q̂-body Sachdev-Ye-Kitaev (SYK) models with general complex couplings, one the complex conjugate of the other, living in off-diagonal blocks of the larger WSYK Hamiltonian. In the limit of large number N of Majoranas and q̂ ∝ √ N , we employ diagrammatic techniques, and a generalized Riordan-Touchard formula, to show that the spectral density of the model is exactly given by the weight function of the Al-Salam-Chihara Q-Laguerre polynomials. For a fixed q̂, the analytical spectral density, with Q = Q(q̂, N) computed analytically, is still in excellent agreement with numerical results obtained by exact diagonalization. The spectrum is positive with a hard edge at zero energy and, for odd q̂, low-energy excitations grow as a stretched exponential, though the functional form is different from that of the supersymmetric SYK model. The microscopic spectral density and level statistics close to the hard edge are very well approximated by that of an ensemble of random matrices belonging to non-standard universality classes which indicates quantum chaotic dynamics. The observed features of the model are consistent with the existence of a gravity dual in the low-energy limit.
Faster Algorithms for Rooted Connectivity in Directed Graphs

We consider the fundamental problems of determining the rooted and global edge and vertex connectivities (and computing the corresponding cuts) in directed graphs. For rooted (and hence also global) edge connectivity with small integer capacities we give a new randomized Monte Carlo algorithm that runs in time $\tilde{O}(n^2)$. For rooted edge connectivity this is the first algorithm to improve on the $\Omega(n^3)$ time bound in the dense-graph high-connectivity regime. Our result relies on a simple combination of sampling coupled with sparsification that appears new, and could lead to further tradeoffs for directed graph connectivity problems. We extend the edge connectivity ideas to rooted and global vertex connectivity in directed graphs. We obtain a $(1 + \epsilon)$-approximation for rooted vertex connectivity in $\tilde{O}(nW/\epsilon)$ time where $W$ is the total vertex weight (assuming integral vertex weights); in particular this yields an $\tilde{O}(n^2/\epsilon)$ time randomized algorithm for unweighted graphs. This translates to a $\tilde{O}(\kappa nW)$ time exact algorithm where $\kappa$ is the rooted connectivity. We build on this to obtain similar bounds for global vertex connectivity. Our results complement the known results for these problems in the low connectivity regime due to work of Gabow [9] for edge connectivity from 1991, and the very recent work of Nanongkai et al. [24] and Forster et al. [7] for vertex connectivity.
Probing a Bose metal via electrons: inescapable non-Fermi liquid scattering and pseudogap physics

Non-Fermi liquid behavior and pseudogap formation are among the most well-known examples of exotic spectral features observed in several strongly correlated materials such as the hole-doped cuprates, nickelates, iridates, ruthenates, ferropnictides, doped Mott organics, transition metal dichalcogenides, heavy fermions, d- and f-electron metals, etc. We demonstrate that these features are inevitable consequences when fermions couple to an unconventional Bose metal (Hegg et al 2021 Proc. Natl. Acad. Sci. 118) mean field consisting of lower-dimensional coherence. Not only do we find both exotic phenomena, but also a host of other features that have been observed e.g. in the cuprates including nodal anti-nodal dichotomy and pseudogap asymmetry (symmetry) in momentum (real) space. Obtaining these exotic and heretofore mysterious phenomena via a mean field offers a simple, universal, and therefore widely applicable explanation for their ubiquitous empirical appearance.
Continual Learning for Fake Audio Detection

Fake audio attack becomes a major threat to the speaker verification system. Although current detection approaches have achieved promising results on dataset-specific scenarios, they encounter difficulties on unseen spoofing data. Fine-tuning and retraining from scratch have been applied to incorporate new data. However, fine-tuning leads to performance degradation on previous data. Retraining takes a lot of time and computation resources. Besides, previous data are unavailable due to privacy in some situations. To solve the above problems, this paper proposes detecting fake without forgetting, a continual-learning-based method, to make the model learn new spoofing attacks incrementally. A knowledge distillation loss is introduced to loss function to preserve the memory of original model. Supposing the distribution of genuine voice is consistent among different scenarios, an extra embedding similarity loss is used as another constraint to further do a positive sample alignment. Experiments are conducted on the ASVspoof2019 dataset. The results show that our proposed method outperforms fine-tuning by the relative reduction of average equal error rate up to 81.62%.
On Energy-Based Models with Overparametrized Shallow Neural Networks

Energy-based models (EBMs) are a simple yet powerful framework for generative modeling. They are based on a trainable energy function which defines an associated Gibbs measure, and they can be trained and sampled from via well-established statistical tools, such as MCMC. Neural networks may be used as energy function approximators, providing both a rich class of expressive models as well as a flexible device to incorporate data structure. In this work we focus on shallow neural networks. Building from the incipient theory of overparametrized neural networks, we show that models trained in the so-called"active"regime provide a statistical advantage over their associated"lazy"or kernel regime, leading to improved adaptivity to hidden low-dimensional structure in the data distribution, as already observed in supervised learning. Our study covers both maximum likelihood and Stein Discrepancy estimators, and we validate our theoretical results with numerical experiments on synthetic data.
Floquet higher-order Weyl and nexus semimetals

This work reports the general design and characterization of two exotic, anomalous nonequilibrium topological phases. In equilibrium systems, the Weyl nodes or the crossing points of nodal lines may become the transition points between higher-order and first-order topological phases defined on two-dimensional slices, thus featuring both hinge Fermi arc and surface Fermi arc. We advance this concept by presenting a strategy to obtain, using time-sequenced normal insulator phases only, Floquet higher-order Weyl semimetals and Floquet higher-order nexus semimetals, where the concerned topological singularities in the three-dimensional Brillouin zone border anomalous two-dimensional higher-order Floquet phases. The fascinating topological phases we obtain are previously unknown and can be experimentally studied using, for example, a three-dimensional lattice of coupled ring resonators.
Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks

In this paper, we propose a novel layer based on fast Walsh-Hadamard transform (WHT) and smooth-thresholding to replace 1 × 1 convolution layers in deep neural networks. In the WHT domain, we denoise the transform domain coefficients using the new smooth-thresholding non-linearity, a smoothed version of the well-known soft-thresholding operator. We also introduce a family of multiplication-free operators from the basic 2×2 Hadamard transform to implement 3 × 3 depthwise separable convolution layers. Using these two types of layers, we replace the bottleneck layers in MobileNet-V2 to reduce the network’s number of parameters with a slight loss in accuracy. For example, by replacing the final third bottleneck layers, we reduce the number of parameters from 2.270M to 947K. This reduces the accuracy from 95.21% to 92.88% on the CIFAR-10 dataset. Our approach significantly improves the speed of data processing. The fast Walsh-Hadamard transform has a computational complexity of O(m log2 m). As a result, it is computationally more efficient than the 1 × 1 convolution layer. The fast Walsh-Hadamard layer processes a tensor in ℝ10×32×32×1024 about 2 times faster than 1 × 1 convolution layer on NVIDIA Jetson Nano computer board.
Planning with Entity Chains for Abstractive Summarization

Pre-trained transformer-based sequence-to-sequence models have become the go-to solution for many text generation tasks, including summarization. However, the results produced by these models tend to contain signiﬁcant issues such as hallucinations and irrelevant passages. One solution to mitigate these problems is to incorporate better content planning in neural summarization. We propose to use entity chains (i.e., chains of entities mentioned in the summary) to better plan and ground the generation of abstractive summaries. In particular, we augment the target by prepending it with its entity chain. We experimented with both pre-training and ﬁnetuning with this content planning objec-tive. When evaluated on CNN/DailyMail, SAMSum and XSum, models trained with this objective improved on entity correctness and summary conciseness, and achieved state-of-the-art performance on ROUGE for SAMSum and XSum.
Reward Optimization for Neural Machine Translation with Learned Metrics

Neural machine translation (NMT) models are conventionally trained with token-level negative log-likelihood (NLL), which does not guarantee that the generated translations will be optimized for a selected sequence-level evaluation metric. Multiple approaches are proposed to train NMT with BLEU as the reward, in order to directly improve the metric. However, it was reported that the gain in BLEU does not translate to real quality improvement, limiting the application in industry. Recently, it became clear to the community that BLEU has a low correlation with human judgment when dealing with state-of-the-art models. This leads to the emerging of model-based evaluation metrics. These new metrics are shown to have a much higher human correlation. In this paper, we investigate whether it is beneficial to optimize NMT models with the state-of-the-art model-based metric, BLEURT. We propose a contrastive-margin loss for fast and stable reward optimization suitable for large NMT models. In experiments, we perform automatic and human evaluations to compare models trained with smoothed BLEU and BLEURT to the baseline models. Results show that the reward optimization with BLEURT is able to increase the metric scores by a large margin, in contrast to limited gain when training with smoothed BLEU. The human evaluation shows that models trained with BLEURT improve adequacy and coverage of translations. Code is available via this https URL.