Miracle-dz/UniFAR-FTUs huggingface dataset:
MixDehazeNet : Mix Structure Block For Image Dehazing Network

Image dehazing is a typical task in the low-level vision field. Previous studies verified the effectiveness of the large convolutional kernel and attention mechanism in dehazing. However, there are two drawbacks: the multi-scale properties of an image are readily ignored when a large convolutional kernel is introduced, and the standard series connection of an attention module does not sufficiently consider an uneven hazy distribution. In this paper, we propose a novel framework named Mix Structure Image Dehazing Network (MixDehazeNet), which solves two issues mentioned above. Specifically, it mainly consists of two parts: the multi-scale parallel large convolution kernel module and the enhanced parallel attention module. Compared with a single large kernel, parallel large kernels with multi-scale are more capable of taking partial texture into account during the dehazing phase. In addition, an enhanced parallel attention module is developed, in which parallel connections of attention perform better at dehazing uneven hazy distribution. Extensive experiments on three benchmarks demonstrate the effectiveness of our proposed methods. For example, compared with the previous state-of-the-art methods, MixDehazeNet achieves a significant improvement (42.62dB PSNR) on the SOTS indoor dataset. The code is released in https://github.com/AmeryXiong/MixDehazeNet.
On the Minimal Knowledge Required for Solving Stellar Consensus

Byzantine Consensus is fundamental for building consistent and fault-tolerant distributed systems. In traditional quorum-based consensus protocols, quorums are defined using globally known assumptions shared among all participants. Motivated by decentralized applications on open networks, the Stellar blockchain relaxes these global assumptions by allowing each participant to define its quorums using local information. A similar model called Consensus with Unknown Participants (CUP) studies the minimal knowledge required to solve consensus in ad-hoc networks where each participant knows only a subset of other participants of the system. We prove that Stellar cannot solve consensus using the initial knowledge provided to participants in the CUP model, even though CUP can. We propose an oracle called sink detector that augments this knowledge, enabling Stellar participants to solve consensus.
FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsing

Textual scene graph parsing has become increasingly important in various vision-language applications, including image caption evaluation and image retrieval. However, existing scene graph parsers that convert image captions into scene graphs often suffer from two types of errors. First, the generated scene graphs fail to capture the true semantics of the captions or the corresponding images, resulting in a lack of faithfulness. Second, the generated scene graphs have high inconsistency, with the same semantics represented by different annotations. To address these challenges, we propose a novel dataset, which involves re-annotating the captions in Visual Genome (VG) using a new intermediate representation called FACTUAL-MR. FACTUAL-MR can be directly converted into faithful and consistent scene graph annotations. Our experimental results clearly demonstrate that the parser trained on our dataset outperforms existing approaches in terms of faithfulness and consistency. This improvement leads to a significant performance boost in both image caption evaluation and zero-shot image retrieval tasks. Furthermore, we introduce a novel metric for measuring scene graph similarity, which, when combined with the improved scene graph parser, achieves state-of-the-art (SOTA) results on multiple benchmark datasets for the aforementioned tasks. The code and dataset are available at https://github.com/zhuang-li/FACTUAL .
Design of Rim-Located Reconfigurable Reflectarrays for Interference Mitigation in Reflector Antennas

Radio telescopes are susceptible to interference arriving through its sidelobes. If a reflector antenna could be retrofitted with an adaptive null steering system, it could potentially mitigate this interference. The design of a reflectarray which can be used to reconfigure a radio telescopes radiation pattern by driving a null to the angle of incoming interference is presented. The reflectarray occupies only a portion of the rim of the original reflector and lays conformal to the paraboloid within this region. The conformal reflectarray contains unit cells with 1-bit reconfigurability stemming from two symmetrically placed PIN diodes. It is found that the dielectric and switch losses introduced by the reflectarray do not significantly affect the radio telescopes efficiency since the reflectarray is placed only along the outer rim of the reflector which is weakly illuminated. Simulation results of an L-band reconfigurable reflectarray for an 18m prime focus fed parabola are presented.
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

In recent years, language models have drastically grown in size, and the abilities of these models have been shown to improve with scale. The majority of recent scaling laws studies focused on high-compute high-parameter count settings, leaving the question of when these abilities begin to emerge largely unanswered. In this paper, we investigate whether the effects of pre-training can be observed when the problem size is reduced, modeling a smaller, reduced-vocabulary language. We show the benefits of pre-training with masked language modeling (MLM) objective in models as small as 1.25M parameters, and establish a strong correlation between pre-training perplexity and downstream performance (GLUE benchmark). We examine downscaling effects, extending scaling laws to models as small as ~1M parameters. At this scale, we observe a break of the power law for compute-optimal models and show that the MLM loss does not scale smoothly with compute-cost (FLOPs) below $2.2 \times 10^{15}$ FLOPs. We also find that adding layers does not always benefit downstream performance.
On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models

Deep neural networks (DNNs) lack the precise semantics and definitive probabilistic interpretation of probabilistic graphical models (PGMs). In this paper, we propose an innovative solution by constructing infinite tree-structured PGMs that correspond exactly to neural networks. Our research reveals that DNNs, during forward propagation, indeed perform approximations of PGM inference that are precise in this alternative PGM structure. Not only does our research complement existing studies that describe neural networks as kernel machines or infinite-sized Gaussian processes, it also elucidates a more direct approximation that DNNs make to exact inference in PGMs. Potential benefits include improved pedagogy and interpretation of DNNs, and algorithms that can merge the strengths of PGMs and DNNs.
Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making

Pre-trained transformers are often fine-tuned to aid clinical decision-making using limited clinical notes. Model interpretability is crucial, especially in high-stakes domains like medicine, to establish trust and ensure safety, which requires human engagement. We introduce SUFO, a systematic framework that enhances interpretability of fine-tuned transformer feature spaces. SUFO utilizes a range of analytic and visualization techniques, including Supervised probing, Unsupervised similarity analysis, Feature dynamics, and Outlier analysis to address key questions about model trust and interpretability. We conduct a case study investigating the impact of pre-training data where we focus on real-world pathology classification tasks, and validate our findings on MedNLI. We evaluate five 110M-sized pre-trained transformer models, categorized into general-domain (BERT, TNLR), mixed-domain (BioBERT, Clinical BioBERT), and domain-specific (PubMedBERT) groups. Our SUFO analyses reveal that: (1) while PubMedBERT, the domain-specific model, contains valuable information for fine-tuning, it can overfit to minority classes when class imbalances exist. In contrast, mixed-domain models exhibit greater resistance to overfitting, suggesting potential improvements in domain-specific model robustness; (2) in-domain pre-training accelerates feature disambiguation during fine-tuning; and (3) feature spaces undergo significant sparsification during this process, enabling clinicians to identify common outlier modes among fine-tuned models as demonstrated in this paper. These findings showcase the utility of SUFO in enhancing trust and safety when using transformers in medicine, and we believe SUFO can aid practitioners in evaluating fine-tuned language models for other applications in medicine and in more critical domains.
Assess and Summarize: Improve Outage Understanding with Large Language Models

Cloud systems have become increasingly popular in recent years due to their flexibility and scalability. Each time cloud computing applications and services hosted on the cloud are affected by a cloud outage, users can experience slow response times, connection issues or total service disruption, resulting in a significant negative business impact. Outages are usually comprised of several concurring events/source causes, and therefore understanding the context of outages is a very challenging yet crucial first step toward mitigating and resolving outages. In current practice, on-call engineers with in-depth domain knowledge, have to manually assess and summarize outages when they happen, which is time-consuming and labor-intensive. In this paper, we first present a large-scale empirical study investigating the way on-call engineers currently deal with cloud outages at Microsoft, and then present and empirically validate a novel approach (dubbed Oasis) to help the engineers in this task. Oasis is able to automatically assess the impact scope of outages as well as to produce human-readable summarization. Specifically, Oasis first assesses the impact scope of an outage by aggregating relevant incidents via multiple techniques. Then, it generates a human-readable summary by leveraging fine-tuned large language models like GPT-3.x. The impact assessment component of Oasis was introduced in Microsoft over three years ago, and it is now widely adopted, while the outage summarization component has been recently introduced, and in this article we present the results of an empirical evaluation we carried out on 18 real-world cloud systems as well as a human-based evaluation with outage owners. The results obtained show that Oasis can effectively and efficiently summarize outages, and lead Microsoft to deploy its first prototype which is currently under experimental adoption by some of the incident teams.
Extended states in one-dimensional aperiodic lattices with linearly varying patches

We introduce a family of 1D aperiodic tight-binding models with linearly varying patches of A-type sites with on-site energies $\epsilon_A=0$ connected by single B-type sites with $\epsilon_B=W$. We analytically show such structures have strong spatial correlations. We theoretically find states are extended at resonance levels in the vicinity of $E^\kappa_M=-2\cos\frac{\kappa\pi}{M}$ if they are allowed energies, where $M=md$ are the size differences of patches, $d$ is the variation rate of patch sizes, $m\in \mathcal{N_{+}}$ and $\kappa=1,2,\cdots,M-1$. Related delocalization-localization transitions are explored. Numerical evidences are in excellent quantitative agreement with theoretical predictions.
Improving the Generalizability of Trajectory Prediction Models with Frenét-Based Domain Normalization

Predicting the future trajectories of robots' nearby objects plays a pivotal role in applications such as autonomous driving. While learning-based trajectory prediction methods have achieved remarkable performance on public benchmarks, the generalization ability of these approaches remains questionable. The poor generalizability on unseen domains, a well-recognized defect of data-driven approaches, can potentially harm the real-world performance of trajectory prediction models. We are thus motivated to improve models' generalization ability instead of merely pursuing high accuracy on average. Due to the lack of benchmarks for quantifying the generalization ability of trajectory predictors, we first construct a new benchmark called argoverse-shift, where the data distributions of domains are significantly different. Using this benchmark for evaluation, we identify that the domain shift problem seriously hinders the generalization of trajectory predictors since state-of-the-art approaches suffer from severe performance degradation when facing those out-of-distribution scenes. To enhance the robustness of models against domain shift problem, we propose a plug-and-play strategy for domain normalization in trajectory prediction. Our strategy utilizes the Frenét coordinate frame for modeling and can effectively narrow the domain gap of different scenes caused by the variety of road geometry and topology. Experiments show that our strategy noticeably boosts the prediction performance of the state-of-the-art in domains that were previously unseen to the models, thereby improving the generalization ability of data-driven trajectory prediction methods.
Isolating the Temperature Feedback Loop and Its Effects on Surface Temperature

AbstractClimate feedback processes are known to substantially amplify the surface warming response to an increase of greenhouse gases. When the forcing and feedbacks modify the temperature response they trigger temperature feedback loops that amplify the direct temperature changes due to the forcing and nontemperature feedbacks through the thermal–radiative coupling between the atmosphere and surface. This study introduces a new feedback-response analysis method that can isolate and quantify the effects of the temperature feedback loops of individual processes on surface temperature from their corresponding direct surface temperature responses.The authors analyze a 1% yr−1 increase of CO2 simulation of the NCAR CCSM4 at the time of CO2 doubling to illustrate the new method. The Planck sensitivity parameter, which indicates colder regions experience stronger surface temperature responses given the same change in surface energy flux, is the inherent factor that leads to polar warming amplification (PWA). Th.
The acquisition of English modal constructions: a corpus-based analysis.

The English modal system is complex, exhibiting many-to-one, and one-to-many, form-function mappings. Usage-based approaches emphasise the role of the input in acquisition but rarely address the impact of form-function mappings on acquisition. To test whether consistent form-function mappings facilitate acquisition, we analysed two dense mother-child corpora at age 3 and 4. We examined the influence on acquisition of input features including form-function mapping frequency and the number of functions a modal signifies, using innovative methodological controls for other aspects of the input (e.g., form frequency) and child characteristics (e.g., age as a proxy for socio-cognitive development). The children were more likely to produce the frequent modals and form-function mappings of their input but modals with fewer functions in caregiver speech did not promote acquisition of these forms. Our findings support usage-based approaches to language acquisition and demonstrate the importance of applying appropriate controls when investigating relationships between input and development.
Is the association of chronotype with adolescent behavior problems mediated through social jetlag?

ABSTRACT We examined the associations of chronotype with behavior problems in a cross-sectional study of 957 Colombian adolescents (mean age, 14.6 years; 56% female), in addition to the mediating role of social jetlag. The midpoint of bedtime and waketime on free days, corrected for sleep debt accumulated during school week (MSFsc), was estimated from parent reports and used to assess chronotype. Behavior problems were evaluated through the Youth Self-Report (YSR) and the parent-completed Child Behavior Checklist (CBCL) questionnaires. We estimated adjusted mean differences with 95% CI in externalizing, internalizing, attention, social, and thought problem scores per one hour difference in chronotype using linear regression. Later chronotype was related to internalizing and externalizing behavior problems. Eveningness was associated with higher adjusted mean YSR scores (unit difference per hour) in externalizing behavior (1.0; 95% CI: 0.6, 1.5), internalizing behavior (0.6; 95% CI: 0.2, 1.1), attention problems (0.2; 95% CI: 0.0, 0.3), social problems (0.4; 95% CI: 0.1, 0.8), and thought problems (0.3; 95% CI: 0.1, 0.6). Similar patterns were observed with the CBCL. The associations of chronotype with somatic complaints and social problems were stronger in boys than they were in girls. Later chronotype was related to social jetlag but social jetlag was only associated with somatic complaints and attention problems, and mediated 16% and 26% of their corresponding associations with chronotype. In conclusion, later chronotype is associated with behavior problems in adolescence. Social jetlag does not substantially mediate these associations.
Hydroclimatic streamflow prediction using Least Square-Support Vector Regression

In last two decades, many Artificial Intelligence-based models have gained recognition in the field of hydrologic forecasting. In this study, potential of least square support vector regression model is explored in the context of streamflow prediction using hydroclimatic inputs. This study is conducted in Narmada river basin up to Sandia gauging station and Mahanadi river basin up to Basantpur gauging station. Four different hydroclimatic variables, namely rainfall, maximum temperature, minimum temperature and streamflow values of previous day are used as input variables. Prediction performances are assessed in terms of different statistical measures, namely – correlation coefficient, root mean square error and Nash–Sutcliffe efficiency. Whereas these statistics indicate an overall impressive result, in particular, low and medium ranges of streamflows are found to have better correspondence between observed and predicted values.
A phase 2 multicenter study of docetaxel‐PM and trastuzumab‐pkrb combination therapy in recurrent or metastatic salivary gland carcinomas

Salivary duct carcinoma (SDC) is uncommon but is the most aggressive subtype of salivary gland carcinomas. The high positivity rate for human epidermal growth factor receptor 2 (HER2) led to an investigation of the efficacy of HER2‐targeted agents. Docetaxel‐PM (polymeric micelle) is a low‐molecular‐weight, nontoxic, biodegradable, and docetaxel‐loaded micellar formulation. Trastuzumab‐pkrb is a biosimilar to trastuzumab.
Systemic inflammation factors as survival prognosis markers in ovarian neoplasm and the relationship with cancer-associated inflammatory mediators—a review

At the level of the genital system, ovarian neoplasm is the most frequent cause of morbidity and mortality. In the specialized literature, the coexistence of an inflammatory process is admitted from the early stages of the evolution of this pathology. Starting from the importance of this process, both in determinism and in the evolution of carcinogenesis and summarizing the field of knowledge, for this study we considered two objectives: the first was the presentation of the pathogenic mechanism, through which chronic +ovarian inflammation is involved in the process of carcinogenesis, and the second is the justification of the clinical utility of the three parameters, accepted as biomarkers of systemic inflammation: neutrophil-lymphocyte ratio, platelet lymphocyte ratio, and lymphocyte-monocyte ratio in the assessment of prognosis. The study highlights the acceptance of these hematological parameters, with practical utility, as prognostic biomarkers in ovarian cancer, based on the intrinsic link with cancer-associated inflammatory mediators. Based on the data from the specialized literature, the conclusion is that in ovarian cancer, the inflammatory process induced by the presence of the tumor, induces changes in the types of circulating leukocytes, with immediate effects on the markers of systemic inflammation.
A nonparametric regression alternative to empirical Bayes approaches to simultaneous estimation

The simultaneous estimation of multiple unknown parameters lies at heart of a broad class of important problems across science and technology. Currently, the state-of-the-art performance in the such problems is achieved by nonparametric empirical Bayes methods. However, these approaches still suffer from two major issues. First, they solve a frequentist problem but do so by following Bayesian reasoning, posing a philosophical dilemma that has contributed to somewhat uneasy attitudes toward empirical Bayes methodology. Second, their computation relies on certain density estimates that become extremely unreliable in some complex simultaneous estimation problems. In this paper, we study these issues in the context of the canonical Gaussian sequence problem. We propose an entirely frequentist alternative to nonparametric empirical Bayes methods by establishing a connection between simultaneous estimation and penalized nonparametric regression. We use flexible regularization strategies, such as shape constraints, to derive accurate estimators without appealing to Bayesian arguments. We prove that our estimators achieve asymptotically optimal regret and show that they are competitive with or can outperform nonparametric empirical Bayes methods in simulations and an analysis of spatially resolved gene expression data.
Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity

We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification that are formulated from distributionally robust optimization (DRO) perspective, where the uncertainty in the given label information are modeled and captured by taking the worse case of distributional weights. The benefits of this perspective are several fold: (i) it provides a unified framework to explain the classical cross-entropy (CE) loss and SVM loss and their variants, (ii) it includes a special family corresponding to the temperature-scaled CE loss, which is widely adopted but poorly understood; (iii) it allows us to achieve adaptivity to the uncertainty degree of label information at an instance level. Our contributions include: (1) we study both consistency and robustness by establishing top-$k$ ($\forall k\geq 1$) consistency of LDR losses for multi-class classification, and a negative result that a top-$1$ consistent and symmetric robust loss cannot achieve top-$k$ consistency simultaneously for all $k\geq 2$; (2) we propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance; (3) we demonstrate stable and competitive performance for the proposed adaptive LDR loss on 7 benchmark datasets under 6 noisy label and 1 clean settings against 13 loss functions, and on one real-world noisy dataset. The code is open-sourced at \url{https://github.com/Optimization-AI/ICML2023_LDR}.