Miracle-dz/UniFAR-FTUs huggingface dataset:
Toposes of Topological Monoid Actions

We demonstrate that categories of continuous actions of topological monoids on discrete spaces are Grothendieck toposes. We exhibit properties of these toposes, giving a solution to the corresponding Morita-equivalence problem. We characterize these toposes in terms of their canonical points. We identify natural classes of representatives with good topological properties, 'powder monoids' and then 'complete monoids', for the Morita-equivalence classes of topological monoids. Finally, we show that the construction of these toposes can be made (2-)functorial by considering geometric morphisms induced by continuous semigroup homomorphisms.
COVID-Net CXR-S: Deep Convolutional Neural Network for Severity Assessment of COVID-19 Cases from Chest X-ray Images

The world is still struggling in controlling and containing the spread of the COVID-19 pandemic caused by the SARS-CoV-2 virus. The medical conditions associated with SARS-CoV-2 infections have resulted in a surge in the number of patients at clinics and hospitals, leading to a significantly increased strain on healthcare resources. As such, an important part of managing and handling patients with SARS-CoV-2 infections within the clinical workflow is severity assessment, which is often conducted with the use of chest X-ray (CXR) images. In this work, we introduce COVID-Net CXR-S, a convolutional neural network for predicting the airspace severity of a SARS-CoV-2 positive patient based on a CXR image of the patient’s chest. More specifically, we leveraged transfer learning to transfer representational knowledge gained from over 16,000 CXR images from a multinational cohort of over 15,000 SARS-CoV-2 positive and negative patient cases into a custom network architecture for severity assessment. Experimental results using the RSNA RICORD dataset showed that the proposed COVID-Net CXR-S has potential to be a powerful tool for computer-aided severity assessment of CXR images of COVID-19 positive patients. Furthermore, radiologist validation on select cases by two board-certified radiologists with over 10 and 19 years of experience, respectively, showed consistency between radiologist interpretation and critical factors leveraged by COVID-Net CXR-S for severity assessment. While not a production-ready solution, the ultimate goal for the open source release of COVID-Net CXR-S is to act as a catalyst for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative new clinical decision support solutions for helping clinicians around the world manage the continuing pandemic.
GODSAC*: Graph Optimized DSAC* for Robot Relocalization

Deep learning based camera pose estimation from monocular camera images has seen a recent uptake in Visual SLAM research. Even though such pose estimation approaches have excellent results in small confined areas like offices and apartment buildings, they tend to do poorly when applied to larger areas like outdoor settings, mainly because of the scarcity of distinctive features. We propose GODSAC* as a camera pose estimation approach that augments pose predictions from a trained neural network with noisy odometry data through the optimization of a pose graph. GODSAC* outperforms the state-of-the-art approaches in pose estimation accuracy, as we demonstrate in our experiments.
PREDICT: Persian Reverse Dictionary

Finding the appropriate words to convey concepts (i.e., lexical access) is essential for effective communication. Reverse dictionaries fulfill this need by helping individuals to find the word(s) which could relate to a specific concept or idea. To the best of our knowledge, this resource has not been available for the Persian language. In this paper, we compare four different architectures for implementing a Persian reverse dictionary (PREDICT). We evaluate our models using (phrase,word) tuples extracted from the only Persian dictionaries available online, namely Amid, Moein, and Dehkhoda where the phrase describes the word. Given the phrase, a model suggests the most relevant word(s) in terms of the ability to convey the concept. The model is considered to perform well if the correct word is one of its top suggestions. Our experiments show that a model consisting of Long Short-Term Memory (LSTM) units enhanced by an additive attention mechanism is enough to produce suggestions comparable to (or in some cases better than) the word in the original dictionary. The study also reveals that the model sometimes produces the synonyms of the word as its output which led us to introduce a new metric for the evaluation of reverse dictionaries called Synonym Accuracy accounting for the percentage of times the event of producing the word or a synonym of it occurs. The assessment of the best model using this new metric also indicates that at least 62% of the times, it produces an accurate result within the top 100 suggestions.
Curious Representation Learning for Embodied Intelligence

Self-supervised representation learning has achieved remarkable success in recent years. By subverting the need for supervised labels, such approaches are able to utilize the numerous unlabeled images that exist on the Internet and in photographic datasets. Yet to build truly intelligent agents, we must construct representation learning algorithms that can learn not only from datasets but also learn from environments. An agent in a natural environment will not typically be fed curated data. Instead, it must explore its environment to acquire the data it will learn from. We propose a framework, curious representation learning (CRL), which jointly learns a reinforcement learning policy and a visual representation model. The policy is trained to maximize the error of the representation learner, and in doing so is incentivized to explore its environment. At the same time, the learned representation becomes stronger and stronger as the policy feeds it ever harder data to learn from. Our learned representations enable promising transfer to downstream navigation tasks, performing better than or comparably to ImageNet pretraining without using any supervision at all. In addition, despite being trained in simulation, our learned representations can obtain interpretable results on real images. Code is available at https://yilundu.github. io/crl/.
Strong Fulde-Ferrell Larkin-Ovchinnikov pairing fluctuations in polarized Fermi systems

Michele Pini, Pierbiagio Pieri, 3 and G. Calvanese Strinati 4, 5 School of Science and Technology, Physics Division, Università di Camerino, 62032 Camerino (MC), Italy Dipartimento di Fisica e Astronomia, Università di Bologna, 40127 Bologna (BO), Italy INFN, Sezione di Bologna, 40127 Bologna (BO), Italy INFN, Sezione di Perugia, 06123 Perugia (PG), Italy CNR-INO, Istituto Nazionale di Ottica, Sede di Firenze, 50125 Firenze (FI), Italy (Dated: July 15, 2021)
Kinetic modeling of pyrolysis of three Iranian waste oils in a micro-fluidized bed

ABSTRACT Different approaches have been used to convert the waste materials into a clean syngas or other chemicals such as methanol. Among them, pyrolysis is a good candidate to produce the synthesis gas and volatile matters for industrial and refinery applications. In this work, we studied the kinetic and chemical behavior of three Iranian waste oils through a kinetic model and an experimental study. The experiments carried out in a micro-FB reactor, which is a good option for low emissions. Results showed that the reaction temperature and reaction rate are two of the most important factors for maximum conversion level of fuel. Results also showed an optimum value for reaction rate. The modeling results validated against the experimental measurements and found to be in good agreements.
RotLSTM: Rotating Memories in Recurrent Neural Networks

Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.
Towards Kitaev spin liquid in 3d transition metal compounds

This paper reviews the current progress on searching the Kitaev spin liquid state in 3[Formula: see text] electron systems. Honeycomb cobaltates were recently proposed as promising candidates to realize the Kitaev spin liquid state, due to the more localized wavefunctions of [Formula: see text] ions compared with that of [Formula: see text] and [Formula: see text] ions, and also the easy tunability of the exchange Hamiltonian in favor of Kitaev interaction. Several key parameters that have large impacts on the exchange constants, such as the charge-transfer gap and the trigonal crystal field, are identified and discussed. Specifically, tuning crystal field effect by means of strain or pressure is emphasized as an efficient phase control method driving the magnetically ordered cobaltates into the spin liquid state. Experimental results suggesting the existence of strong Kitaev interactions in layered honeycomb cobaltates are discussed. Finally, the future research directions are briefly outlined.
Sciatic nerve regeneration induced by glycosaminoglycan and laminin mimetic peptide nanofiber gels

In the USA, 20 million patients suffer from neuropathy caused by peripheral nerve injuries, which costs approximately $150 billion annually. For longer nerve gaps and multiple injury sites, it is essential to use nerve guidance conduits for healthy pathfinding of regenerating axons. Here, extracellular matrix mimetic peptide nanofiber hydrogels were used for functionalizing guidance conduits to enhance neuronal regeneration in the distal stump of full transaction sciatic nerve injury in rats with functional repair. Conduits filled with heparan sulfate and laminin mimetic peptide nanofibers significantly improved electromyography response and promoted neuronal regeneration in a rat model of sciatic nerve defect. In addition, Schwann cells cultured on these nanofibers showed increased viability and significantly enhanced nerve growth factor (NGF) release. Overall, these results suggest that extracellular matrix mimetic peptide nanofibers present a promising treatment option for peripheral nerve injuries.
Generalized Spatially Coupled Parallel Concatenated Convolutional Codes With Partial Repetition

We introduce generalized spatially coupled parallel concatenated codes (GSC-PCCs), a class of spatially coupled turbo-like codes obtained by coupling parallel concatenated codes (PCCs) with a fraction of information bits repeated before the PCC encoding. GSC-PCCs can be seen as a generalization of the original spatially coupled parallel concatenated convolutional codes (SC-PCCs) proposed by Moloudi et al. [1]. To characterize the asymptotic performance of GSC-PCCs, we derive the corresponding density evolution equations and compute their decoding thresholds. We show that the proposed codes have some nice properties such as threshold saturation and that their decoding thresholds improve with the repetition factor $q$. Most notably, our analysis suggests that the proposed codes asymptotically approach the capacity as $q$ tends to infinity with any given constituent convolutional code.
Positnn: Training Deep Neural Networks with Mixed Low-Precision Posit

Low-precision formats have proven to be an efficient way to reduce not only the memory footprint but also the hardware resources and power consumption of deep learning computations. Under this premise, the posit numerical format appears to be a highly viable substitute for the IEEE floating-point, but its application to neural networks training still requires further research. Some preliminary results have shown that 8-bit (and even smaller) posits may be used for inference and 16-bit for training, while maintaining the model accuracy. The presented research aims to evaluate the feasibility to train deep convolutional neural networks using posits. For such purpose, a software framework was developed to use simulated posits and quires in end-to-end training and inference. This implementation allows using any bit size, configuration, and even mixed precision, suitable for different precision requirements in various stages. The obtained results suggest that 8-bit posits can substitute 32-bit floats during training with no negative impact on the resulting loss and accuracy.
Frequency and Time Dependence of Linear Polarization in Turbulent Jets of Blazars

Time-variable polarization is an extremely valuable observational tool to probe the dynamical physical conditions of blazar jets. Since 2008, we have been monitoring the flux and linear polarization of a sample of gamma-ray bright blazars at optical frequencies. Some of the observations were performed on nightly or intra-night time-scales in four optical bands, providing information on the frequency and time dependence of the polarization. The observed behavior is similar to that found in simulations of turbulent plasma in a relativistic jet that contains a standing shock and/or a helical background magnetic field. Similar simulations predict the characteristics of X-ray synchrotron polarization of blazars that will be measured in the future by the Imaging X-ray Polarimetry Explorer (IXPE).
LFI-CAM: Learning Feature Importance for Better Visual Explanation

Class Activation Mapping (CAM) is a powerful technique used to understand the decision making of Convolutional Neural Network (CNN) in computer vision. Recently, there have been attempts not only to generate better visual explanations, but also to improve classification performance using visual explanations. However, previous works still have their own drawbacks. In this paper, we propose a novel architecture, LFI-CAM*** (Learning Feature Importance Class Activation Mapping), which is trainable for image classification and visual explanation in an end-to-end manner. LFI-CAM generates attention map for visual explanation during forward propagation, and simultaneously uses attention map to improve classification performance through the attention mechanism. Feature Importance Network (FIN) focuses on learning the feature importance instead of directly learning the attention map to obtain a more reliable and consistent attention map. We confirmed that LFI-CAM is optimized not only by learning the feature importance but also by enhancing the backbone feature representation to focus more on important features of the input image. Experiments show that LFI-CAM outperforms baseline models’ accuracy on classification tasks as well as significantly improves on previous works in terms of attention map quality and stability over different hyper-parameters.
Initialization and Regularization of Factorized Neural Layers

Factorized layers--operations parameterized by products of two or more matrices--occur in a variety of deep learning contexts, including compressed model training, certain types of knowledge distillation, and multi-head self-attention architectures. We study how to initialize and regularize deep nets containing such layers, examining two simple, understudied schemes, spectral initialization and Frobenius decay, for improving their performance. The guiding insight is to design optimization routines for these networks that are as close as possible to that of their well-tuned, non-decomposed counterparts; we back this intuition with an analysis of how the initialization and regularization schemes impact training with gradient descent, drawing on modern attempts to understand the interplay of weight-decay and batch-normalization. Empirically, we highlight the benefits of spectral initialization and Frobenius decay across a variety of settings. In model compression, we show that they enable low-rank methods to significantly outperform both unstructured sparsity and tensor methods on the task of training low-memory residual networks; analogs of the schemes also improve the performance of tensor decomposition techniques. For knowledge distillation, Frobenius decay enables a simple, overcomplete baseline that yields a compact model from over-parameterized training without requiring retraining with or pruning a teacher network. Finally, we show how both schemes applied to multi-head attention lead to improved performance on both translation and unsupervised pre-training.
A Perceptual Distortion Reduction Framework for Adversarial Perturbation Generation

Most of the adversarial attack methods suffer from large perceptual distortions such as visible artifacts, when the attack strength is relatively high. These perceptual distortions contain a certain portion which contributes less to the attack success rate. This portion of distortions, which is induced by unnecessary modiﬁcations and lack of proper perceptual distortion constraint, is the target of the proposed framework. In this paper, we propose a perceptual distortion reduction framework to tackle this problem from two perspectives. We guide the perturbation addition process to reduce unnecessary modiﬁcations by proposing an activated region transfer attention mask, which intends to transfer the activated regions of the target model from the correct prediction to incorrect ones. Note that an ensemble model is adopted to predict the activated regions of the unseen models in the black-box setting of our framework. Besides, we propose a perceptual distortion constraint and add it into the ob-jective function of adversarial attack to jointly optimize the perceptual distortions and attack success rate. Extensive experiments have veriﬁed the effectiveness of our framework on several baseline methods.
Model Counting meets F0 Estimation

Constraint satisfaction problems (CSP's) and data stream models are two powerful abstractions to capture a wide variety of problems arising in different domains of computer science. Developments in the two communities have mostly occurred independently and with little interaction between them. In this work, we seek to investigate whether bridging the seeming communication gap between the two communities may pave the way to richer fundamental insights. To this end, we focus on two foundational problems: model counting for CSP's and computation of zeroth frequency moments F0 for data streams. Our investigations lead us to observe striking similarity in the core techniques employed in the algorithmic frameworks that have evolved separately for model counting and F0 computation. We design a recipe for translation of algorithms developed for F0 estimation to that of model counting, resulting in new algorithms for model counting. We then observe that algorithms in the context of distributed streaming can be transformed to distributed algorithms for model counting. We next turn our attention to viewing streaming from the lens of counting and show that framing F0 estimation as a special case of #DNF counting allows us to obtain a general recipe for a rich class of streaming problems, which had been subjected to case-specific analysis in prior works. In particular, our view yields a state-of-the art algorithm for multidimensional range efficient F0 estimation with a simpler analysis.
Modelling of Ocean Waves with the Alber Equation: Application to Non-Parametric Spectra and Generalisation to Crossing Seas

The Alber equation is a phase-averaged second-moment model used to study the statistics of a sea state, which has recently been attracting renewed attention. We extend it in two ways: firstly, we derive a generalized Alber system starting from a system of nonlinear Schrödinger equations, which contains the classical Alber equation as a special case but can also describe crossing seas, i.e., two wavesystems with different wavenumbers crossing. (These can be two completely independent wavenumbers, i.e., in general different directions and different moduli.) We also derive the associated two-dimensional scalar instability condition. This is the first time that a modulation instability condition applicable to crossing seas has been systematically derived for general spectra. Secondly, we use the classical Alber equation and its associated instability condition to quantify how close a given nonparametric spectrum is to being modulationally unstable. We apply this to a dataset of 100 nonparametric spectra provided by the Norwegian Meteorological Institute and find that the vast majority of realistic spectra turn out to be stable, but three extreme sea states are found to be unstable (out of 20 sea states chosen for their severity). Moreover, we introduce a novel “proximity to instability” (PTI) metric, inspired by the stability analysis. This is seen to correlate strongly with the steepness and Benjamin–Feir Index (BFI) for the sea states in our dataset (>85% Spearman rank correlation). Furthermore, upon comparing with phase-resolved broadband Monte Carlo simulations, the kurtosis and probability of rogue waves for each sea state are also seen to correlate well with the PTI (>85% Spearman rank correlation).
Leveraging Deep Representations of Radiology Reports in Survival Analysis for Predicting Heart Failure Patient Mortality

Utilizing clinical texts in survival analysis is difficult because they are largely unstructured. Current automatic extraction models fail to capture textual information comprehensively since their labels are limited in scope. Furthermore, they typically require a large amount of data and high-quality expert annotations for training. In this work, we present a novel method of using BERT-based hidden layer representations of clinical texts as covariates for proportional hazards models to predict patient survival outcomes. We show that hidden layers yield notably more accurate predictions than predefined features, outperforming the previous baseline model by 5.7% on average across C-index and time-dependent AUC. We make our work publicly available at https://github.com/bionlplab/heart_failure_mortality.
Distributionally Robust Risk Map for Learning-Based Motion Planning and Control: A Semidefinite Programming Approach

In this article, we propose a novel safety specification tool, called the distributionally robust risk map (DR-risk map), for a mobile robot operating in a learning-enabled environment. Given the robot's position, the map aims to reliably assess the conditional value-at-risk (CVaR) of collision with obstacles whose movements are inferred by Gaussian process regression (GPR). Unfortunately, the inferred distribution is subject to errors, making it difficult to accurately evaluate the CVaR of collision. To overcome this challenge, our tool measures the risk under the worst-case distribution in a so-called ambiguity set that characterizes allowable distribution errors. To resolve the infinite-dimensionality issue inherent in the construction of the DR-risk map, we derive a tractable semidefinite programming formulation that provides an upper bound of the risk, exploiting techniques from modern distributionally robust optimization. As a concrete application for motion planning, a distributionally robust RRT* algorithm is considered using the risk map that addresses distribution errors caused by GPR. Furthermore, a motion control method is devised using the DR-risk map in a learning-based model predictive control (MPC) formulation. In particular, a neural network approximation of the risk map is proposed to reduce the computational cost in solving the MPC problem. The performance and utility of the proposed risk map are demonstrated through simulation studies that show its ability to ensure the safety of mobile robots despite learning errors.