Miracle-dz/UniFAR-FTUs huggingface dataset:
Evaluating Generative Adversarial Networks on Explicitly Parameterized Distributions

The true distribution parameterizations of commonly used image datasets are inaccessible. Rather than designing metrics for feature spaces with unknown characteristics, we propose to measure GAN performance by evaluating on explicitly parameterized, synthetic data distributions. As a case study, we examine the performance of 16 GAN variants on six multivariate distributions of varying dimensionalities and training set sizes. In this learning environment, we observe that: GANs exhibit similar performance trends across dimensionalities; learning depends on the underlying distribution and its complexity; the number of training samples can have a large impact on performance; evaluation and relative comparisons are metric-dependent; diverse sets of hyperparameters can produce a "best" result; and some GANs are more robust to hyperparameter changes than others. These observations both corroborate findings of previous GAN evaluation studies and make novel contributions regarding the relationship between size, complexity, and GAN performance.
Multimodal deep learning for short-term stock volatility prediction

Stock market volatility forecasting is a task relevant to assessing market risk. We investigate the interaction between news and prices for the one-day-ahead volatility prediction using state-of-the-art deep learning approaches. The proposed models are trained either end-to-end or using sentence encoders transfered from other tasks. We evaluate a broad range of stock market sectors, namely Consumer Staples, Energy, Utilities, Heathcare, and Financials. Our experimental results show that adding news improves the volatility forecasting as compared to the mainstream models that rely only on price data. In particular, our model outperforms the widely-recognized GARCH(1,1) model for all sectors in terms of coefficient of determination $R^2$, $MSE$ and $MAE$, achieving the best performance when training from both news and price data.
Reconciling modern machine learning and the bias-variance trade-off

The question of generalization in machine learning---how algorithms are able to learn predictors from a training sample to make accurate predictions out-of-sample---is revisited in light of the recent breakthroughs in modern machine learning technology. The classical approach to understanding generalization is based on bias-variance trade-offs, where model complexity is carefully calibrated so that the fit on the training sample reflects performance out-of-sample. However, it is now common practice to fit highly complex models like deep neural networks to data with (nearly) zero training error, and yet these interpolating predictors are observed to have good out-of-sample accuracy even for noisy data. How can the classical understanding of generalization be reconciled with these observations from modern machine learning practice? In this paper, we bridge the two regimes by exhibiting a new "double descent" risk curve that extends the traditional U-shaped bias-variance curve beyond the point of interpolation. Specifically, the curve shows that as soon as the model complexity is high enough to achieve interpolation on the training sample---a point that we call the "interpolation threshold"---the risk of suitably chosen interpolating predictors from these models can, in fact, be decreasing as the model complexity increases, often below the risk achieved using non-interpolating models. The double descent risk curve is demonstrated for a broad range of models, including neural networks and random forests, and a mechanism for producing this behavior is posited.
Hierarchical LSTMs with Adaptive Attention for Visual Captioning

Recent progress has been made in using attention based encoder-decoder framework for image and video captioning. Most existing decoders apply the attention mechanism to every generated word including both visual words (e.g., “gun” and “shooting”) and non-visual words (e.g., “the”, “a”). However, these non-visual words can be easily predicted using natural language model without considering visual signals or attention. Imposing attention mechanism on non-visual words could mislead and decrease the overall performance of visual captioning. Furthermore, the hierarchy of LSTMs enables more complex representation of visual data, capturing information at different scales. Considering these issues, we propose a hierarchical LSTM with adaptive attention (hLSTMat) approach for image and video captioning. Specifically, the proposed framework utilizes the spatial or temporal attention for selecting specific regions or frames to predict the related words, while the adaptive attention is for deciding whether to depend on the visual information or the language context information. Also, a hierarchical LSTMs is designed to simultaneously consider both low-level visual information and high-level language context information to support the caption generation. We design the hLSTMat model as a general framework, and we first instantiate it for the task of video captioning. Then, we further instantiate our hLSTMarefine it and apply it to the imioning task. To demonstrate the effectiveness of our proposed framework, we test our method on both video and image captioning tasks. Experimental results show that our approach achieves the state-of-the-art performance for most of the evaluation metrics on both tasks. The effect of important components is also well exploited in the ablation study.
Finger-GAN: Generating Realistic Fingerprint Images Using Connectivity Imposed GAN

Generating realistic biometric images has been an interesting and, at the same time, challenging problem. Classical statistical models fail to generate realistic-looking fingerprint images, as they are not powerful enough to capture the complicated texture representation in fingerprint images. In this work, we present a machine learning framework based on generative adversarial networks (GAN), which is able to generate fingerprint images sampled from a prior distribution (learned from a set of training images). We also add a suitable regularization term to the loss function, to impose the connectivity of generated fingerprint images. This is highly desirable for fingerprints, as the lines in each finger are usually connected. We apply this framework to two popular fingerprint databases, and generate images which look very realistic, and similar to the samples in those databases. Through experimental results, we show that the generated fingerprint images have a good diversity, and are able to capture different parts of the prior distribution. We also evaluate the Frechet Inception distance (FID) of our proposed model, and show that our model is able to achieve good quantitative performance in terms of this score.
Capacity Results for the K-User Broadcast Channel with Two Nested Multicast Messages

The K-user discrete memoryless (DM) broadcast channel (BC) with two nested multicast messages is studied in which one common message is to be multicast to all receivers and the second private message to a subset of receivers. The receivers that must decode both messages are referred to as private receivers and the others that must decode only the common message as common receivers. For two nested multicast messages, we establish the capacity region for several classes of DM BCs characterized by the respective associated sets of pair-wise relationships between and among the common and private receivers, each described by the well-known more capable or less noisy conditions. For three classes of DM BCs, the capacity region is simply achieved by superposition coding and the proofs of the converses rely on a recently found information inequality. The achievable rate region is then enhanced through the addition of a splitting of the private message into as many parts as there are common receivers and indirect decoding. A closed-form two-dimensional polyhedral description is obtained for it for a given coding distribution. Through a converse result that relies on the well-known Csiszar sum lemma and the information inequality, a specialization of this region that involves splitting the private message into just two sub-messages is proved to be the capacity region for several classes of DM BCs, beyond those for which superposition coding alone is capacity optimal, thereby underscoring the benefit of rate-splitting. All previously known capacity results for DM BCs with two nested multicast messages for the two and three-receiver DM BCs as well as DM BCs with one private or one common receiver are included in the general framework presented in this work.
A Continuous-Time Analysis of Distributed Stochastic Gradient

We analyze the effect of synchronization on distributed stochastic gradient algorithms. By exploiting an analogy with dynamical models of biological quorum sensing, where synchronization between agents is induced through communication with a common signal, we quantify how synchronization can significantly reduce the magnitude of the noise felt by the individual distributed agents and their spatial mean. This noise reduction is in turn associated with a reduction in the smoothing of the loss function imposed by the stochastic gradient approximation. Through simulations on model nonconvex objectives, we demonstrate that coupling can stabilize higher noise levels and improve convergence. We provide a convergence analysis for strongly convex functions by deriving a bound on the expected deviation of the spatial mean of the agents from the global minimizer for an algorithm based on quorum sensing, the same algorithm with momentum, and the elastic averaging SGD (EASGD) algorithm. We discuss extensions to new algorithms that allow each agent to broadcast its current measure of success and shape the collective computation accordingly. We supplement our theoretical analysis with numerical experiments on convolutional neural networks trained on the CIFAR-10 data set, where we note a surprising regularizing property of EASGD even when applied to the non-distributed case. This observation suggests alternative second-order in time algorithms for nondistributed optimization that are competitive with momentum methods.
Secure Modulo Sum via Multiple Access Channel

We discuss secure computation of modular sum when multiple access channel from distinct players $A_{1}, \ldots, A_{c}$ to a third party (Receiver) is given. Then, we define the secure modulo sum capacity as the supremum of the transmission rate of modulo sum without information leakage of other information. We derive its useful lower bound, which is numerically calculated under a realistic model that can be realizable as a Gaussian multiple access channel (MAC).
Maximum Likelihood Estimation and Graph Matching in Errorfully Observed Networks

Abstract Given a pair of graphs with the same number of vertices, the inexact graph matching problem consists in finding a correspondence between the vertices of these graphs that minimizes the total number of induced edge disagreements. We study this problem from a statistical framework in which one of the graphs is an errorfully observed copy of the other. We introduce a corrupting channel model, and show that in this model framework, the solution to the graph matching problem is a maximum likelihood estimator (MLE). Necessary and sufficient conditions for consistency of this MLE are presented, as well as a relaxed notion of consistency in which a negligible fraction of the vertices need not be matched correctly. The results are used to study matchability in several families of random graphs, including edge independent models, random regular graphs, and small-world networks. We also use these results to introduce measures of matching feasibility, and experimentally validate the results on simulated and real-world networks. Supplemental files for this article are available online.
MAP moving horizon estimation for threshold measurements with application to field monitoring

This paper deals with state estimation of a spatially distributed system given noisy measurements from pointwise‐in‐time‐and‐space threshold sensors spread over the spatial domain of interest. A maximum a posteriori probability (MAP) approach is undertaken and a moving horizon (MH) approximation of the MAP cost function is adopted. It is proved that, under system linearity and log‐concavity of the noise probability density functions, the proposed MH‐MAP state estimator amounts to the solution, at each sampling interval, of a convex optimization problem. Moreover, a suitable centralized solution for large‐scale systems is proposed with a substantial decrease of the computational complexity. The latter algorithm is shown to be feasible for the state estimation of spatially dependent dynamic fields described by partial differential equations via the use of the finite element spatial discretization method. A simulation case study concerning estimation of a diffusion field is presented in order to demonstrate the effectiveness of the proposed approach. Quite remarkably, the numerical tests exhibit a noise‐assisted behavior of the proposed approach in that the estimation accuracy results optimal in the presence of measurement noise with non‐null variance.
Deep Learning based Early Detection and Grading of Diabetic Retinopathy Using Retinal Fundus Images

Diabetic Retinopathy (DR) is a constantly deteriorating disease, being one of the leading causes of vision impairment and blindness. Subtle distinction among different grades and existence of many significant small features make the task of recognition very challenging. In addition, the present approach of retinopathy detection is a very laborious and time-intensive task, which heavily relies on the skill of a physician. Automated detection of diabetic retinopathy is essential to tackle these problems. Early-stage detection of diabetic retinopathy is also very important for diagnosis, which can prevent blindness with proper treatment. In this paper, we developed a novel deep convolutional neural network, which performs the early-stage detection by identifying all microaneurysms (MAs), the first signs of DR, along with correctly assigning labels to retinal fundus images which are graded into five categories. We have tested our network on the largest publicly available Kaggle diabetic retinopathy dataset, and achieved 0.851 quadratic weighted kappa score and 0.844 AUC score, which achieves the state-of-the-art performance on severity grading. In the early-stage detection, we have achieved a sensitivity of 98% and specificity of above 94%, which demonstrates the effectiveness of our proposed method. Our proposed architecture is at the same time very simple and efficient with respect to computational time and space are concerned.
Over-Parameterized Deep Neural Networks Have No Strict Local Minima For Any Continuous Activations

In this paper, we study the loss surface of the over-parameterized fully connected deep neural networks. We prove that for any continuous activation functions, the loss function has no bad strict local minimum, both in the regular sense and in the sense of sets. This result holds for any convex and continuous loss function, and the data samples are only required to be distinct in at least one dimension. Furthermore, we show that bad local minima do exist for a class of activation functions
3D Point Capsule Networks

In this paper, we propose 3D point-capsule networks, an auto-encoder designed to process sparse 3D point clouds while preserving spatial arrangements of the input data. 3D capsule networks arise as a direct consequence of our unified formulation of the common 3D auto-encoders. The dynamic routing scheme and the peculiar 2D latent space deployed by our capsule networks bring in improvements for several common point cloud-related tasks, such as object classification, object reconstruction and part segmentation as substantiated by our extensive evaluations. Moreover, it enables new applications such as part interpolation and replacement.
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling

Work on the problem of contextualized word representation---the development of reusable neural network components for sentence understanding---has recently seen a surge of progress centered on the unsupervised pretraining task of language modeling with methods like ELMo. This paper contributes the first large-scale systematic study comparing different pretraining tasks in this context, both as complements to language modeling and as potential alternatives. The primary results of the study support the use of language modeling as a pretraining task and set a new state of the art among comparable models using multitask learning with language models. However, a closer look at these results reveals worryingly strong baselines and strikingly varied results across target tasks, suggesting that the widely-used paradigm of pretraining and freezing sentence encoders may not be an ideal platform for further work.
Early Stone Manipulation in Urinary Tract Infection Associated with Obstructing Nephrolithiasis

A urinary tract infection (UTI) and sepsis secondary to an obstructing stone are one of the few true urological emergencies. The accepted management of infected ureteral stones includes emergent decompression of the collecting system as well as antibiotic therapy. Despite this, no consensus guidelines clarify the optimal time to undergo definitive stone management following decompression. Historically, our institution has performed ureteroscopy with laser lithotripsy (URS-LL) treatment at least 1 to 2 weeks after decompression to allow for clinical improvement and completion of an antibiotic course. In this case series, we retrospectively review four cases in which patients had a documented UTI secondary to an obstructive ureteral stone. The patients underwent urgent decompression and, based on labs and clinical improvement, were subsequently treated with URS-LL. The presented patients received URS-LL within 5 days of decompression and antibiotics. The patients had no sepsis related postoperative complications from the accelerated course of treatment, resulting in discharge within 2 days following URS-LL. We provide a detailed examination of each patient presentation to describe our institution's experience with treating infected kidney stones within days of urgent decompression in order to question the previous standard of treating an infected kidney stone with a more delayed intervention.
Predictors of intentional intoxication using decision tree modeling analysis: a retrospective study

Objective The suicide rate in South Korea is very high and is expected to increase in coming years. Intoxication is the most common suicide attempt method as well as one of the common reason for presenting to an emergency medical center. We used decision tree modeling analysis to identify predictors of risk for suicide by intentional intoxication. Methods A single-center, retrospective study was conducted at our hospital using a 4-year registry of the institute from January 1, 2013 to December 31, 2016. Demographic factors, such as sex, age, intentionality, therapeutic adherence, alcohol consumption, smoking status, physical disease, cancer, psychiatric disease, and toxicological factors, such as type of intoxicant and poisoning severity score were collected. Candidate risk factors based on the decision tree were used to select variables for multiple logistic regression analysis. Results In total, 4,023 patients with intoxication were enrolled as study participants, with 2,247 (55.9%) identified as cases of intentional intoxication. Reported annual percentages of intentional intoxication among patients were 628/937 (67.0%), 608/1,082 (56.2%), 536/1,017 (52.7), 475/987 (48.1%) from 2013 to 2016. Significant predictors identified based on decision tree analysis were alcohol consumption, old age, psychiatric disease, smoking, and male sex; those identified based on multiple regression analysis were alcohol consumption, smoking, male sex, psychiatric disease, old age, poor therapeutic adherence, and physical disease. Conclusion We identified important predictors of suicide risk by intentional intoxication. A specific and realistic approach to analysis using the decision tree modeling technique is an effective method to determine those groups at risk of suicide by intentional intoxication.
Inositols in Insulin Signaling and Glucose Metabolism

In the past decades, both the importance of inositol for human health and the complex interaction between glucose and inositol have been the subject of increasing consideration. Glucose has been shown to interfere with cellular transmembrane transport of inositol, inhibiting, among others, its intestinal absorption. Moreover, intracellular glucose is required for de novo biosynthesis of inositol through the inositol-3-phosphate synthase 1 pathway, while a few glucose-related metabolites, like sorbitol, reduce intracellular levels of inositol. Furthermore, inositol, via its major isomers myo-inositol and D-chiro-inositol, and probably some of its phosphate intermediate metabolites and correlated enzymes (like inositol hexakisphosphate kinase) participate in both insulin signaling and glucose metabolism by influencing distinct pathways. Indeed, clinical data support the beneficial effects exerted by inositol by reducing glycaemia levels and hyperinsulinemia and buffering negative effects of sustained insulin stimulation upon the adipose tissue and the endocrine system. Due to these multiple effects, myoIns has become a reliable treatment option, as opposed to hormonal stimulation, for insulin-resistant PCOS patients.
RIPK3-Mediated Necroptosis and Neutrophil Infiltration Are Associated with Poor Prognosis in Patients with Alcoholic Cirrhosis

Alcoholic cirrhosis is an end-stage liver disease with impaired survival and often requires liver transplantation. Recent data suggests that receptor-interacting protein kinase-3- (RIPK3-) mediated necroptosis plays an important role in alcoholic cirrhosis. Additionally, neutrophil infiltration is the most characteristic pathologic hallmark of alcoholic hepatitis. Whether RIPK3 level is correlated with neutrophil infiltration or poor prognosis in alcoholic cirrhotic patients is still unknown. We aimed to determine the correlation of RIPK3 and neutrophil infiltration with the prognosis in the end-stage alcoholic cirrhotic patients. A total of 20 alcoholic cirrhotic patients subjected to liver transplantation and 5 normal liver samples from control patients were retrospectively enrolled in this study. Neutrophil infiltration and necroptosis were assessed by immunohistochemical staining for myeloperoxidase (MPO) and RIPK3, respectively. The noninvasive score system (model for end-stage liver disease (MELD)) and histological score systems (Ishak, Knodell, and ALD grading and ALD stage) were used to evaluate the prognosis. Neutrophil infiltration was aggravated in patients with a high MELD score (≥32) in the liver. The MPO and RIPK3 levels in the liver were positively related to the Ishak score. The RIPK3 was also significantly and positively related to the Knodell score. In conclusion, RIPK3-mediated necroptosis and neutrophil-mediated alcoholic liver inflammatory response are highly correlated with poor prognosis in patients with end-stage alcoholic cirrhosis. RIPK3 and MPO might serve as potential predictors for poor prognosis in alcoholic cirrhotic patients.