Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimating Potential Outcome Distributions with Collaborating Causal Networks

Traditional causal inference approaches leverage observational study data to estimate the difference in observed (factual) and unobserved (counterfactual) outcomes for a potential treatment, known as the Conditional Average Treatment Effect (CATE). However, CATE corresponds to the comparison on the first moment alone, and as such may be insufficient in reflecting the full picture of treatment effects. As an alternative, estimating the full potential outcome distributions could provide greater insights. However, existing methods for estimating treatment effect potential outcome distributions often impose restrictive or overly-simplistic assumptions about these distributions. Here, we propose Collaborating Causal Networks (CCN), a novel methodology which goes beyond the estimation of CATE alone by learning the full potential outcome distributions. Estimation of outcome distributions via the CCN framework does not require restrictive assumptions of the underlying data generating process (e.g. Gaussian errors). Additionally, our proposed method facilitates estimation of the utility of each possible treatment and permits individual-specific variation through utility functions (e.g. risk tolerance variability). CCN not only extends outcome estimation beyond traditional risk difference, but also enables a more comprehensive decision making process through definition of flexible comparisons. Under assumptions commonly made in the causal inference literature, we show that CCN learns distributions that asymptotically capture the correct potential outcome distributions. Furthermore, we propose an adjustment approach that is empirically effective in alleviating sample imbalance between treatment groups in observational studies. Finally, we evaluate the performance of CCN in multiple experiments on both synthetic and semi-synthetic data. We demonstrate that CCN learns improved distribution estimates compared to existing Bayesian and deep generative methods as well as improved decisions with respects to a variety of utility functions.
Deep Neural Networks and Tabular Data: A Survey

Heterogeneous tabular data are the most commonly used form of data and are essential for numerous critical and computationally demanding applications. On homogeneous datasets, deep neural networks have repeatedly shown excellent performance and have therefore been widely adopted. However, their adaptation to tabular data for inference or data generation tasks remains highly challenging. To facilitate further progress in the field, this work provides an overview of state-of-the-art deep learning methods for tabular data. We categorize these methods into three groups: data transformations, specialized architectures, and regularization models. For each of these groups, our work offers a comprehensive overview of the main approaches. Moreover, we discuss deep learning approaches for generating tabular data and also provide an overview over strategies for explaining deep models on tabular data. Thus, our first contribution is to address the main research streams and existing methodologies in the mentioned areas while highlighting relevant challenges and open research questions. Our second contribution is to provide an empirical comparison of traditional machine learning methods with 11 deep learning approaches across five popular real-world tabular datasets of different sizes and with different learning objectives. Our results, which we have made publicly available as competitive benchmarks, indicate that algorithms based on gradient-boosted tree ensembles still mostly outperform deep learning models on supervised learning tasks, suggesting that the research progress on competitive deep learning models for tabular data is stagnating. To the best of our knowledge, this is the first in-depth overview of deep learning approaches for tabular data; as such, this work can serve as a valuable starting point to guide researchers and practitioners interested in deep learning with tabular data.
The Lyman Alpha Reference Sample XII. Morphology of extended Lyman alpha emission in star-forming galaxies

Aims. We use Hubble space telescope data of 45 nearby star-forming galaxies to investigate properties of Lyman-alpha (Ly α ) halos, Ly α morphology, and the star-forming characteristics of galaxies. We study how the morphology of Ly α emission is related to other Ly α observables. Furthermore, we study the interdependencies of Ly α morphological quantities. Methods. We studied the spatial extent of Ly α using surface brightness proﬁles in the following two ways i) using circular apertures and ii) within faint Ly α isophotes. We also measured the average intensity and the size of the regions with a high star formation rate density. The morphology of the galaxies was quantiﬁed by computing centroid position, axis ratio, and position angle in the Ly α , ultraviolet continuum, and I band maps. Results. We found that galaxies with more extended star-forming regions possess larger Ly α halos. Furthermore, galaxies with more elongated Ly α morphology are also more extended in Ly α . Our data suggest that Ly α bright galaxies appear rounder in their Ly α morphology, and there is less of a contribution from their Ly α halo to their overall luminosity. We compared our results with studies at high redshift and found that whilst the Ly α extent in the inner regions of the galaxies in our sample are similar to the high − z Ly α emitters (LAEs), Ly α halos are more extended in high − z LAEs. Conclusions. Our analysis suggests that the Ly α morphology a ﬀ ects the measurement of other observable quantities concerning Ly α emission, and some of the conclusions drawn from high redshift LAEs might be biased towards galaxies with speciﬁc Ly α shapes. In particular, faint Ly α emitters have larger Ly α scale lengths and halo fractions. This implies that faint Ly α emitters are harder to detect at high redshift than previously believed.
KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Survey

This is a tutorial and survey paper on Karush-Kuhn-Tucker (KKT) conditions, first-order and second-order numerical optimization, and distributed optimization. After a brief review of history of optimization, we start with some preliminaries on properties of sets, norms, functions, and concepts of optimization. Then, we introduce the optimization problem, standard optimization problems (including linear programming, quadratic programming, and semidefinite programming), and convex problems. We also introduce some techniques such as eliminating inequality, equality, and set constraints, adding slack variables, and epigraph form. We introduce Lagrangian function, dual variables, KKT conditions (including primal feasibility, dual feasibility, weak and strong duality, complementary slackness, and stationarity condition), and solving optimization by method of Lagrange multipliers. Then, we cover first-order optimization including gradient descent, line-search, convergence of gradient methods, momentum, steepest descent, and backpropagation. Other first-order methods are explained, such as accelerated gradient method, stochastic gradient descent, mini-batch gradient descent, stochastic average gradient, stochastic variance reduced gradient, AdaGrad, RMSProp, and Adam optimizer, proximal methods (including proximal mapping, proximal point algorithm, and proximal gradient method), and constrained gradient methods (including projected gradient method, projection onto convex sets, and Frank-Wolfe method). We also cover non-smooth and $\ell_1$ optimization methods including lasso regularization, convex conjugate, Huber function, soft-thresholding, coordinate descent, and subgradient methods. Then, we explain second-order methods including Newton's method for unconstrained, equality constrained, and inequality constrained problems.
Deep Instance Segmentation with High-Resolution Automotive Radar

—Automotive radar provides reliable environmental perception in all-weather conditions with affordable cost, but it hardly supplies semantic and geometry information due to the sparsity of radar detection points. With the development of high-resolution automotive radar in recent years, instance segmentation becomes possible by using automotive radar. Its data contain rich contexts such as Radar Cross Section and micro-Doppler effects, and sometimes can provide detection when the ﬁeld of view is obscured. The outcome from instance segmentation could be potentially used as the input of trackers for tracking targets. In this paper, we propose two efﬁcient methods for instance segmentation with radar detection points, one is implemented in an end-to-end deep learning driven fashion using PointNet++ framework, and the other is based on clustering of the radar detection points with semantic information. Both approaches can be further improved by implementing visual multi-layer perceptron. The effectiveness of the proposed methods is veriﬁed using experimental results on the recent RadarScenes dataset.
Investigating Fairness of Ocular Biometrics Among Young, Middle-Aged, and Older Adults

A number of studies suggest bias of the face biometrics, i.e., face recognition and soft-biometric estimation methods, across gender, race, and age-groups. There is a recent urge to investigate the bias of different biometric modalities toward the deployment of fair and trustworthy biometric solutions. Ocular biometrics has obtained increased attention from academia and industry due to its high accuracy, security, privacy, and ease of use in mobile devices. A recent study in 2020 also suggested the fairness of ocular-based user recognition across males and females. This paper aims to evaluate the fairness of ocular biometrics in the visible spectrum among age-groups; young, middle, and older adults. Thanks to the availability of the latest large-scale 2020 UFPR ocular biometric dataset, with subjects acquired in the age range 18–79 years, to facilitate this study. Experimental results suggest the overall equivalent performance of ocular biometrics across gender and age-groups in user verification and gender-classification. Performance difference for older adults at lower false match rate and young adults was noted at user verification and age-classification, respectively. This could be attributed to inherent characteristics of the biometric data from these age-groups impacting specific applications, which suggest a need for advancement in sensor technology and software solutions.
DeepEdge: A Deep Reinforcement Learning Based Task Orchestrator for Edge Computing

The improvements in the edge computing technology pave the road for diversified applications that demand real-time interaction. However, due to the mobility of the end-users and the dynamic edge environment, it becomes challenging to handle the task offloading with high performance. Moreover, since each application in mobile devices has different characteristics, a task orchestrator must be adaptive and have the ability to learn the dynamics of the environment. For this purpose, we develop a deep reinforcement learning based task orchestrator, DeepEdge, which learns to meet different task requirements without needing human interaction even under the heavily-loaded stochastic network conditions in terms of mobile users and applications. Given the dynamic offloading requests and time-varying communication conditions, we successfully model the problem as a Markov process and then apply the Double Deep Q-Network (DDQN) algorithm to implement DeepEdge. To evaluate the robustness of DeepEdge, we experiment with four different applications including image rendering, infotainment, pervasive health, and augmented reality in the network under various loads. Furthermore, we compare the performance of our agent with the four different task offloading approaches in the literature. Our results show that DeepEdge outperforms its competitors in terms of the percentage of satisfactorily completed tasks.
A General Strategy for In Situ Growth of All‐Inorganic CsPbX3 (X = Br, I, and Cl) Perovskite Nanocrystals in Polymer Fibers toward Significantly Enhanced Water/Thermal Stabilities

Currently, all‐inorganic CsPbX3 (X = Br, I, and Cl) perovskites (IPs) are emerging as excellent candidate materials for exploring optoelectronic devices, due to their superior optical/electronic performances. However, their intrinsic poor stability greatly limits their practical applications. Here, a general strategy is reported for in situ growth of all‐inorganic perovskite nanocrystals (IPNCs) in polymer fibers with highly uniform size and spatial distribution, which is based on one‐step electrospinning of solutions containing IPs precursors and polymers. It is verified that the IPNCs of CsPbX3 can be uniformly encapsulated within the polymer fibers with finely tuned compositions, by rationally adjusting the ratios of PbX2 and CsX salts in the raw solutions. Consequently, the photoluminescence (PL) emissions of CsPbX3@polymer fibers can be readily tuned to cover the whole visible range. The obtained CsPbBr3@polymer fibers exhibit fundamentally improved water/thermal stabilities with a PL quantum yield (QY) of 48%. Their PL QY retains beyond 70% of its original value after being immersed in water for 192 h and maintains over 50% after being heated at 80 °C for 120 min. Furthermore, the light emitting diodes with high brightness based on CsPbBr3@polymer fibers are constructed, suggesting their promising applications.
Global Convergence and Stability of Stochastic Gradient Descent

In machine learning, stochastic gradient descent (SGD) is widely deployed to train models using highly non-convex objectives with equally complex noise models. Unfortunately, SGD theory often makes restrictive assumptions that fail to capture the non-convexity of real problems, and almost entirely ignore the complex noise models that exist in practice. In this work, we make substantial progress on this shortcoming. First, we establish that SGD's iterates will either globally converge to a stationary point or diverge under nearly arbitrary nonconvexity and noise models. Under a slightly more restrictive assumption on the joint behavior of the non-convexity and noise model that generalizes current assumptions in the literature, we show that the objective function cannot diverge, even if the iterates diverge. As a consequence of our results, SGD can be applied to a greater range of stochastic optimization problems with confidence about its global convergence behavior and stability.
Which will be your firm’s next technology? Comparison between machine learning and network-based algorithms

We reconstruct the innovation dynamics of about two hundred thousand companies by following their patenting activity for about ten years. We define the technology portfolios of these companies as the set of the technological sectors present in the patents they submit. By assuming that companies move more frequently towards related sectors, we leverage their past activity to build network-based and machine learning algorithms to forecast the future submissions of patents in new sectors. We compare different prediction methodologies using suitable evaluation metrics, showing that tree-based machine learning algorithms outperform the standard methods based on networks of co-occurrences. This methodology can be applied by firms and policymakers to disentangle, given the present innovation activity, the feasible technological sectors from those that are out of reach.
Autoregressive Diffusion Models

We introduce Autoregressive Diffusion Models (ARDMs), a model class encompassing and generalizing order-agnostic autoregressive models (Uria et al., 2014) and absorbing discrete diffusion (Austin et al., 2021), which we show are special cases of ARDMs under mild assumptions. ARDMs are simple to implement and easy to train. Unlike standard ARMs, they do not require causal masking of model representations, and can be trained using an efficient objective similar to modern probabilistic diffusion models that scales favourably to highly-dimensional data. At test time, ARDMs support parallel generation which can be adapted to fit any given generation budget. We find that ARDMs require significantly fewer steps than discrete diffusion models to attain the same performance. Finally, we apply ARDMs to lossless compression, and show that they are uniquely suited to this task. Contrary to existing approaches based on bits-back coding, ARDMs obtain compelling results not only on complete datasets, but also on compressing single data points. Moreover, this can be done using a modest number of network calls for (de)compression due to the model's adaptable parallel generation.
In Vivo assessment of hypertensive nephrosclerosis using ultrasound localization microscopy.

PURPOSE As a typical chronic kidney disease (CKD), hypertensive nephrosclerosis (HN) is a common syndrome of hypertension, characterized by chronic kidney microvascular damage. Early diagnosis of microvascular damage using conventional ultrasound imaging encounters challenges in sensitivity and specificity owing to the inherent diffraction limit. Ultrasound localization microscopy (ULM) has been developed to obtain microvasculature and microvascular hemodynamics within the kidney, and would be a promising tool for early diagnosis of CKD. METHODS In this study, the advantage of quantitative indexes obtained by using ULM (mean arterial blood flow speeds of different segments of interlobular arteries) over indexes obtained using conventional clinical serum (β2-microglobulin, serum urea nitrogen and creatinine) and urine (24-hour urine volume and urine protein) tests and ultrasound Doppler imaging [peak systolic velocity (PSV), end-diastolic velocity (EDV) and resistance index (RI)] and contrast-enhanced ultrasound imaging [CEUS; rise time (RT), peak intensity (IMAX), mean transit time (mTT) and area under the time-intensity curve (AUC)] for early diagnosis of HN was investigated. Examinations were carried out on 6 spontaneously hypertensive rats (SHR) and 5 normal Wistar-Kyoto (WKY) rats at the age of 10 weeks. RESULTS The experimental results showed that the indicators derived from conventional clinical inspections (serum and urine tests) and ultrasound imaging (PSV, EDV, RI, RT, IMAX, mTT and AUC) did not show significant difference between hypertensive and healthy rats (p > 0.05), while the TTP of the SHR group (28.52 ± 5.52 s) derived from CEUS is significantly higher than that of the WKY group (18.68 ± 7.32 s; p < 0.05). The mean blood flow speed in interlobular artery of SHR (12.47 ± 1.06 mm/s) derived from ULM is significantly higher than that of WKY rats (10.13 ± 1.17 mm/s; p < 0.01). CONCLUSION The advantages of ULM over conventional clinical inspections and ultrasound imaging methods for early diagnosis of HN were validated. The quantitative results showed that ULM can effectively diagnose HN at the early stage by detecting the blood flow speed changes of interlobular arteries. ULM may promise a reliable technique for early diagnosis of HN in the future. This article is protected by copyright. All rights reserved.
NoiER: An Approach for Training More Reliable Fine-Tuned Downstream Task Models

The recent development in pretrained language models that are trained in a self-supervised fashion, such as BERT, is driving rapid progress in natural language processing. However, their brilliant performance is based on leveraging syntactic artefacts of the training data rather than fully understanding the intrinsic meaning of language. The excessive exploitation of spurious artefacts is a problematic issue: the distribution collapse problem, which is the phenomenon that the model fine-tuned on downstream tasks is unable to distinguish out-of-distribution sentences while producing a high-confidence score. In this paper, we argue that the distribution collapse is a prevalent issue in pretrained language models and propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional out-of-distribution detection evaluation metrics by 55% on average compared to the original fine-tuned models.
Breast Cancer Diagnosis in Two-View Mammography Using End-to-End Trained EfficientNet-Based Convolutional Network

Some recent studies have described deep convolutional neural networks to diagnose breast cancer in mammograms with similar or even superior performance to that of human experts. One of the best techniques does two transfer learnings: the first uses a model trained on natural images to create a “patch classifier” that categorizes small subimages; the second uses the patch classifier to scan the whole mammogram and create the “single-view whole-image classifier”. We propose to make a third transfer learning to obtain a “two-view classifier” to use the two mammographic views: bilateral craniocaudal and mediolateral oblique. We use EfficientNet as the basis of our model. We “end-to-end” train the entire system using CBIS-DDSM dataset. To ensure statistical robustness, we test our system twice using: (a) 5-fold cross validation; and (b) the original training/test division of the dataset. Our technique reached an AUC of 0.9344 using 5-fold cross validation (accuracy, sensitivity and specificity are 85.13% at the equal error rate point of ROC). Using the original dataset division, our technique achieved an AUC of 0.8483, as far as we know the highest reported AUC for this problem, although the subtle differences in the testing conditions of each work do not allow for an accurate comparison. The inference code and model are available at https://github.com/dpetrini/two-views-classifier
Multimodal datasets: misogyny, pornography, and malignant stereotypes

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects.