Miracle-dz/UniFAR-FTUs huggingface dataset:
NITI: Training Integer Neural Networks Using Integer-Only Arithmetic

Low bitwidth integer arithmetic has been widely adopted in hardware implementations of deep neural network inference applications. However, despite the promised energy-efficiency improvements demanding edge applications, the use of low bitwidth integer arithmetic for neural network training remains limited. Unlike inference, training demands high dynamic range and numerical accuracy for high quality results, making the use of low-bitwidth integer arithmetic particularly challenging. To address this challenge, we present a novel neural network training framework called NITI that exclusively utilizes low bitwidth integer arithmetic. NITI stores all parameters and accumulates intermediate values as 8-bit integers while using no more than 5 bits for gradients. To provide the necessary dynamic range during the training process, a per-layer block scaling exponentiation scheme is utilized. By deeply integrating with the rounding procedures and integer entropy loss calculation, the proposed scaling scheme incurs only minimal overhead in terms of storage and additional computation. Furthermore, a hardware-efficient pseudo-stochastic rounding scheme that eliminates the need for external random number generation is proposed to facilitate conversion from wider intermediate arithmetic results to lower precision for storage. Since NITI operates only with standard 8-bit integer arithmetic and storage, it is possible to accelerate it using existing low bitwidth operators originally developed for inference in commodity accelerators. To demonstrate this, an open-source software implementation of end-to-end training, using native 8-bit integer operations in modern GPUs is presented. In addition, experiments have been conducted on an FPGA-based training accelerator to evaluate the hardware advantage of NITI. When compared with an equivalent training setup implemented with floating point storage and arithmetic, NITI has no accuracy degradation on the MNIST and CIFAR10 datasets. On ImageNet, NITI achieves similar accuracy as state-of-the-art integer training frameworks without relying on full-precision floating-point first and last layers.
Likelihood Landscape and Local Minima Structures of Gaussian Mixture Models

In this paper, we study the landscape of the population negative log-likelihood function of Gaussian Mixture Models with a general number of components. Due to nonconvexity, there exist multiple local minima that are not globally optimal, even when the mixture is well-separated. We show that all local minima share the same form of structure that partially identifies the component centers of the true mixture, in the sense that each local minimum involves a non-overlapping combination of fitting multiple Gaussians to a single true component and fitting a single Gaussian to multiple true components. Our results apply to the setting where the true mixture components satisfy a certain separation condition, and are valid even when the number of components is over-or under-specified. For Gaussian mixtures with three components, we obtain sharper results in terms of the scaling with the separation between the components.
Angled crested type water waves with surface tension II: Zero surface tension limit

This is the second paper in a series of papers analyzing angled crested type water waves with surface tension. We consider the 2D capillary gravity water wave equation and assume that the fluid is inviscid, incompressible, irrotational and the air density is zero. In the first paper \cite{Ag19} we constructed a weighted energy which generalizes the energy of Kinsey and Wu \cite{KiWu18} to the case of non-zero surface tension, and proved a local wellposedness result. In this paper we prove that under a suitable scaling regime, the zero surface tension limit of these solutions with surface tension are solutions to the gravity water wave equation which includes waves with angled crests.
Study of prosthetic heart valve thrombosis and outcomes after thrombolysis

Background: The study aimed to evaluate the clinical presentation, diagnostic features, treatment strategies, and complications of prosthetic heart valve thrombosis (PHVT) and to determine efficacy, outcomes and complications of thrombolytic therapy during hospital stay.Methods: This was a prospective, observational, single-centre study carried out between March, 2016 and December, 2017 at a tertiary care centre in India. Total 110 patients with history of prosthetic heart valve replacement and symptoms related to PHVT were included in the study. Patients underwent thrombolysis, surgery or conservative management for treatment of PHVT, as per their individual clinical presentation. Clinical profile and treatment outcomes were assessed using a pre-tested, semi-structured questionnaire and clinical assessment.Results: Mean age of the patients was 39.4±12.5 years. Most of the patients presented with NYHA class II and III (85.4%) symptoms. Total 20.9% of patients were poorly compliant with anticoagulants. Thrombolysis was initial treatment in 105 (95.5%) patients. Five (4.6%) patients were treated with heparin. Two patients underwent surgery after failed thrombolysis. Mortality in the thrombolysis group was 6.6%. Embolism occurred in 8.6% of the group and major bleeding in 1.9%. One patient who underwent surgery died.Conclusions: In light of these results, it can be concluded that most cases of PHVT are due to inadequate anticoagulation and poor monitoring mainly in patients belonging to the lower socioeconomic group. Thrombolysis can be considered as first line therapy where the immediate surgical options are remote.
Replica Analysis of the Linear Model With Markov or Hidden Markov Signal Priors

This paper estimates free energy, average mutual information, and minimum mean square error (MMSE) of a linear model under two assumptions: 1) the source is generated by a Markov chain; 2) the source is generated via a hidden Markov model. Our estimates are based on the replica method in statistical physics. We show that under the posterior mean estimator, the linear model with Markov sources or hidden Markov sources is decoupled into single-input AWGN channels with state information available at both encoder and decoder where the state distribution follows the left Perron-Frobenius eigenvector with unit Manhattan norm of the stochastic matrix of Markov chains. Numerical results show that the free energies and MSEs obtained via the replica method are closely approximate to their counterparts achieved by the Metropolis-Hastings algorithm or some well-known approximate message passing algorithms in the research literature.
Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect

As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on re-weighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the "bad" while keep the "good". We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.
Multiple-Association Supporting HTC/MTC in Limited-Backhaul Capacity Ultra-Dense Networks

Coexistence of Human-Type Communications (HTCs) and Machine-Type Communications (MTCs) is inevitable. Ultra-Dense Networks (UDNs) will be efficacious in supporting both types of communications. In a UDN, a massive number of low-power and low-cost Small Cells (SCs) are deployed with density higher than that of the HTC users. In such a scenario, the backhaul capacities constitute an intrinsic bottleneck for the system. Hence, we propose a multiple association scheme where each HTC user associates to and activates multiple SCs to overcome the backhaul capacity constraints mainly encountered in the downlink. In addition, having more active cells allows for more MTC devices to be supported by the network. Using tools from stochastic geometry, we formulate a novel mathematical framework investigating the performance of HTC in both downlink and uplink as well as the uplink MTC. Stretched Exponential Path Loss (SEPL) model is considered to practically reflect the UDN environment. Extensive simulations were conducted to verify the accuracy of the mathematical analysis under different system parameters. Results show the existence of an optimum number of SCs to which an HTC user may connect under backhaul capacity constraints. Besides, the proposed multiple-association scheme improves the performance of MTC in terms of both ASE and density of supported devices.
The Leading Intraseasonal Variability Mode of Wintertime Surface Air Temperature over the North American Sector

In this study, detailed characteristics of the leading intraseasonal variability mode of boreal winter surface air temperature (SAT) over the North American (NA) sector are investigated. This intraseasonal SAT mode, characterized by two anomalous centers with an opposite sign—one over central NA and another over east Siberia (ES)/Alaska—bears a great resemblance to the “warm Arctic–cold continent” pattern of the interannual SAT variability over NA. This intraseasonal SAT mode and associated circulation exert a pronounced influence on regional weather extremes, including precipitation over the northwest coast of NA, sea ice concentration over the Chukchi and Bering Seas, and extreme warm and cold events over the NA continent and Arctic region. Surface warming and cooling signals of the intraseasonal SAT mode are connected to temperature anomalies in a deep-tropospheric layer up to 300 hPa with a decreasing amplitude with altitude. Particularly, a coupling between the troposphere and stratosphere is found during evolution of the intraseasonal SAT variability, although whether the stratospheric processes are essential in sustaining the leading intraseasonal SAT mode is difficult to determine based on observations alone. Two origins of wave sources are identified in contributing to vertically propagating planetary waves near Alaska: one over ES/Alaska associated with local intraseasonal variability and another from the subtropical North Pacific via Rossby wave trains induced by tropical convective activity over the western Pacific, possibly associated with the Madden–Julian oscillation.
Differentially Private Adversarial Robustness Through Randomized Perturbations

Deep Neural Networks, despite their great success in diverse domains, are provably sensitive to small perturbations on correctly classified examples and lead to erroneous predictions. Recently, it was proposed that this behavior can be combatted by optimizing the worst case loss function over all possible substitutions of training examples. However, this can be prone to weighing unlikely substitutions higher, limiting the accuracy gain. In this paper, we study adversarial robustness through randomized perturbations, which has two immediate advantages: (1) by ensuring that substitution likelihood is weighted by the proximity to the original word, we circumvent optimizing the worst case guarantees and achieve performance gains; and (2) the calibrated randomness imparts differentially-private model training, which additionally improves robustness against adversarial attacks on the model outputs. Our approach uses a novel density-based mechanism based on truncated Gumbel noise, which ensures training on substitutions of both rare and dense words in the vocabulary while maintaining semantic similarity for model robustness.
Rotated Binary Neural Network

Binary Neural Network (BNN) shows its predominance in reducing the complexity of deep neural networks. However, it suffers severe performance degradation. One of the major impediments is the large quantization error between the full-precision weight vector and its binary vector. Previous works focus on compensating for the norm gap while leaving the angular bias hardly touched. In this paper, for the first time, we explore the influence of angular bias on the quantization error and then introduce a Rotated Binary Neural Network (RBNN), which considers the angle alignment between the full-precision weight vector and its binarized version. At the beginning of each training epoch, we propose to rotate the full-precision weight vector to its binary vector to reduce the angular bias. To avoid the high complexity of learning a large rotation matrix, we further introduce a bi-rotation formulation that learns two smaller rotation matrices. In the training stage, we devise an adjustable rotated weight vector for binarization to escape the potential local optimum. Our rotation leads to around 50% weight flips which maximize the information gain. Finally, we propose a training-aware approximation of the sign function for the gradient backward. Experiments on CIFAR-10 and ImageNet demonstrate the superiorities of RBNN over many state-of-the-arts. Our source code, experimental settings, training logs and binary models are available at this https URL.
Quantitative and Qualitative Evaluation of Explainable Deep Learning Methods for Ophthalmic Diagnosis

Background: The lack of explanations for the decisions made by algorithms such as deep learning has hampered their acceptance by the clinical community despite highly accurate results on multiple problems. Recently, attribution methods have emerged for explaining deep learning models, and they have been tested on medical imaging problems. The performance of attribution methods is compared on standard machine learning datasets and not on medical images. In this study, we perform a comparative analysis to determine the most suitable explainability method for retinal OCT diagnosis. Methods: A commonly used deep learning model known as Inception v3 was trained to diagnose 3 retinal diseases - choroidal neovascularization (CNV), diabetic macular edema (DME), and drusen. The explanations from 13 different attribution methods were rated by a panel of 14 clinicians for clinical significance. Feedback was obtained from the clinicians regarding the current and future scope of such methods. Results: An attribution method based on a Taylor series expansion, called Deep Taylor was rated the highest by clinicians with a median rating of 3.85/5. It was followed by two other attribution methods, Guided backpropagation and SHAP (SHapley Additive exPlanations). Conclusion: Explanations of deep learning models can make them more transparent for clinical diagnosis. This study compared different explanations methods in the context of retinal OCT diagnosis and found that the best performing method may not be the one considered best for other deep learning tasks. Overall, there was a high degree of acceptance from the clinicians surveyed in the study. Keywords: explainable AI, deep learning, machine learning, image processing, Optical coherence tomography, retina, Diabetic macular edema, Choroidal Neovascularization, Drusen
Modelling neutron star mountains

As the era of gravitational-wave astronomy has well and truly begun, gravitational radiation from rotating neutron stars remains elusive. Rapidly spinning neutron stars are the main targets for continuous-wave searches since, according to general relativity, provided they are asymmetrically deformed, they will emit gravitational waves. It is believed that detecting such radiation will unlock the answer to why no pulsars have been observed to spin close to the break-up frequency. We review existing studies on the maximum mountain that a neutron star crust can support, critique the key assumptions and identify issues relating to boundary conditions that need to be resolved. In light of this discussion, we present a new scheme for modelling neutron star mountains. The crucial ingredient for this scheme is a description of the fiducial force which takes the star away from sphericity. We consider three examples: a source potential which is a solution to Laplace’s equation, another solution which does not act in the core of the star and a thermal pressure perturbation. For all the cases, we find that the largest quadrupoles are between a factor of a few to two orders of magnitude below previous estimates of the maximum-mountain size.
TernaryBERT: Distillation-aware Ultra-low Bit BERT

Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks.However, these models are both computation and memory expensive, hindering their deployment to resource-constrained devices. In this work, we propose TernaryBERT, which ternarizes the weights in a fine-tuned BERT model. Specifically, we use both approximation-based and loss-aware ternarization methods and empirically investigate the ternarization granularity of different parts of BERT. Moreover, to reduce the accuracy degradation caused by the lower capacity of low bits, we leverage the knowledge distillation technique in the training process. Experiments on the GLUE benchmark and SQuAD show that our proposed TernaryBERT outperforms the other BERT quantization methods, and even achieves comparable performance as the full-precision model while being 14.9x smaller.
Targeted VAE: Structured Inference and Targeted Learning for Causal Parameter Estimation

Undertaking causal inference with observational data is extremely useful across a wide range of domains including the development of medical treatments, advertisements and marketing, and policy making. There are two main challenges associated with undertaking causal inference using observational data: treatment assignment heterogeneity (i.e., differences between the treated and untreated groups), and an absence of counterfactual data (i.e. not knowing what would have happened if an individual who did get treatment, were instead to have not been treated). We address these two challenges by combining structured inference and targeted learning. To our knowledge, Targeted Variational AutoEncoder (TVAE) is the first method to incorporate targeted learning into deep latent variable models. Results demonstrate competitive and state of the art performance.
Masses of the 
QQQ¯Q¯
 tetraquarks in the relativistic diquark–antidiquark picture

Masses of the ground-state tetraquarks composed from heavy $c$ and $b$ quarks and antiquarks are calculated in the diquark-antidiquark picture in the framework of the relativistic quark model based on the quasipotential approach. The quasipotentials of the quark-quark and diquark-antidiquark interactions are constructed similarly to the previous consideration of mesons and baryons. It is assumed that the diquark and antidiquark interact in the tetraquark as a whole, and the internal structure of the diquarks is taken into account. All such tetraquarks are found above the thresholds of decays to two heavy quarkonia. This is a result of the consideration of the diquark not to be a pointlike object. Therefore, such tetraquarks can be observed as broad structures decaying dominantly to quarkonia. The broad structure next to the di-$J/\ensuremath{\psi}$ mass threshold, recently observed by the LHCb Collaboration, can correspond to the ground ${2}^{++}$-state tetraquark consisting of four charm quarks.
Few Distance Sets in $\ell_p$ Spaces and $\ell_p$ Product Spaces.

Kusner asked if $n+1$ points is the maximum number of points in $\mathbb{R}^n$ such that the $\ell_p$ distance between any two points is $1$. We present an improvement to the best known upper bound when $p$ is large in terms of $n$, as well as a generalization of the bound to $s$-distance sets. We also study equilateral sets in the $\ell_p$ sums of Euclidean spaces, deriving upper bounds on the size of an equilateral set for when $p=\infty$, $p$ is even, and for any $1\le p<\infty$.
Acoustic scattering by impedance screens/cracks with fractal boundary: well-posedness analysis and boundary element approximation

We study time-harmonic scattering in $\mathbb{R}^n$ ($n=2,3$) by a planar screen (a "crack" in the context of linear elasticity), assumed to be a non-empty bounded relatively open subset $\Gamma$ of the hyperplane $\mathbb{R}^{n-1}\times \{0\}$, on which impedance (Robin) boundary conditions are imposed. In contrast to previous studies, $\Gamma$ can have arbitrarily rough (possibly fractal) boundary. To obtain well-posedness for such $\Gamma$ we show how the standard impedance boundary value problem and its associated system of boundary integral equations must be supplemented with additional solution regularity conditions, which hold automatically when $\partial\Gamma$ is smooth. We show that the associated system of boundary integral operators is compactly perturbed coercive in an appropriate function space setting, strengthening previous results. This permits the use of Mosco convergence to prove convergence of boundary element approximations on smoother "prefractal" screens to the limiting solution on a fractal screen. We present accompanying numerical results, validating our theoretical convergence results, for three-dimensional scattering by a Koch snowflake and a square snowflake.
Effect of oyster shell,limestone and calcium premix on egg production and egg shell quality

Starcross brown 579 laying pullets (81), 22-week-old, were fed on isonitrogenous and isoenergetic diets with same level of dietary calcium from different sources, e.g. oyster shell, limestone and calcium premix to evaluate the effect of different dietary calcium sources on egg production and egg shell quality up to 33 week of age. No significant difference (P&gt0.05) in feed intake, hen day egg production, egg weight, shell weight, shell per cent, shell thickness and survivability was noted among pullets on different dietary calcium sources. Egg weight, shell weight, shell per cent and shell thickness was increased with the advancement of age of pullets on calcium premix. Although there was no statistically signiticant difference found in terms of egg production and egg quality on 3 different dietary calcium sources but calcium premix seemed to increase egg production, egg weight, and shell weight than did oyster shell and limestone. The study therefore, suggested that, any of these calcium sources can be considered for laying chicken, but calcium premix would be better at the first phase of production.