pwc-archive/papers-with-abstracts huggingface dataset:
General-Purpose vs. Domain-Adapted Large Language Models for Extraction of Structured Data from Chest Radiology Reports
Authors: Ali H. Dhanaliwala, Rikhiya Ghosh, Sanjeev Kumar Karn, Poikavila Ullaskrishnan, Oladimeji Farri, Dorin Comaniciu, Charles E. Kahn
Date: 2023-11-28 00:00:00
Tasks: Language Modelling, Large Language Model

Radiologists produce unstructured data that can be valuable for clinical care when consumed by information systems. However, variability in style limits usage. Study compares system using domain-adapted language model (RadLing) and general-purpose LLM (GPT-4) in extracting relevant features from chest radiology reports and standardizing them to common data elements (CDEs). Three radiologists annotated a retrospective dataset of 1399 chest XR reports (900 training, 499 test) and mapped to 44 pre-selected relevant CDEs. GPT-4 system was prompted with report, feature set, value set, and dynamic few-shots to extract values and map to CDEs. Output key:value pairs were compared to reference standard at both stages and an identical match was considered TP. F1 score for extraction was 97% for RadLing-based system and 78% for GPT-4 system. F1 score for mapping was 98% for RadLing and 94% for GPT-4; difference was statistically significant (P<.001). RadLing's domain-adapted embeddings were better in feature extraction and its light-weight mapper had better f1 score in CDE assignment. RadLing system also demonstrated higher capabilities in differentiating between absent (99% vs 64%) and unspecified (99% vs 89%). RadLing system's domain-adapted embeddings helped improve performance of GPT-4 system to 92% by giving more relevant few-shot prompts. RadLing system offers operational advantages including local deployment and reduced runtime costs.
Robust Correlated Equilibrium: Definition and Computation
Authors: Rahul Misra, Rafał Wisniewski, Carsten Skovmose Kallesøe, Manuela L. Bujorianu
Date: 2023-11-29 00:00:00

We study N-player finite games with costs perturbed due to time-varying disturbances in the underlying system and to that end we propose the concept of Robust Correlated Equilibrium that generalizes the definition of Correlated Equilibrium. Conditions under which the Robust Correlated Equilibrium exists are specified and a decentralized algorithm for learning strategies that are optimal in the sense of Robust Correlated Equilibrium is proposed. The primary contribution of the paper is the convergence analysis of the algorithm and to that end, we propose an extension of the celebrated Blackwell's Approachability theorem to games with costs that are not just time-average as in the original Blackwell's Approachability Theorem but also include time-average of previous algorithm iterates. The designed algorithm is applied to a practical water distribution network with pumps being the controllers and their costs being perturbed by uncertain consumption by consumers. Simulation results show that each controller achieves no regret and empirical distributions converge to the Robust Correlated Equilibrium.
Efficient Computation of Sparse and Robust Maximum Association Estimators
Authors: Pia Pfeiffer, Andreas Alfons, Peter Filzmoser
Date: 2023-11-29 00:00:00

Robust statistical estimators offer resilience against outliers but are often computationally challenging, particularly in high-dimensional sparse settings. Modern optimization techniques are utilized for robust sparse association estimators without imposing constraints on the covariance structure. The approach splits the problem into a robust estimation phase, followed by optimization of a decoupled, biconvex problem to derive the sparse canonical vectors. An augmented Lagrangian algorithm, combined with a modified adaptive gradient descent method, induces sparsity through simultaneous updates of both canonical vectors. Results demonstrate improved precision over existing methods, with high-dimensional empirical examples illustrating the effectiveness of this approach. The methodology can also be extended to other robust sparse estimators.
Calabi-Yau Four/Five/Six-folds as $\mathbb{P}^n_\textbf{w}$ Hypersurfaces: Machine Learning, Approximation, and Generation
Authors: Edward Hirst, Tancredi Schettini Gherardini
Date: 2023-11-28 00:00:00
Tasks: Symbolic Regression

Calabi-Yau four-folds may be constructed as hypersurfaces in weighted projective spaces of complex dimension 5 defined via weight systems of 6 weights. In this work, neural networks were implemented to learn the Calabi-Yau Hodge numbers from the weight systems, where gradient saliency and symbolic regression then inspired a truncation of the Landau-Ginzburg model formula for the Hodge numbers of any dimensional Calabi-Yau constructed in this way. The approximation always provides a tight lower bound, is shown to be dramatically quicker to compute (with compute times reduced by up to four orders of magnitude), and gives remarkably accurate results for systems with large weights. Additionally, complementary datasets of weight systems satisfying the necessary but insufficient conditions for transversality were constructed, including considerations of the interior point, reflexivity, and intradivisibility properties. Overall producing a classification of this weight system landscape, further confirmed with machine learning methods. Using the knowledge of this classification, and the properties of the presented approximation, a novel dataset of transverse weight systems consisting of 7 weights was generated for a sum of weights $\leq 200$; producing a new database of Calabi-Yau five-folds, with their respective topological properties computed. Further to this an equivalent database of candidate Calabi-Yau six-folds was generated with approximated Hodge numbers.
$Q_{bias}$ -- A Dataset on Media Bias in Search Queries and Query Suggestions
Authors: Fabian Haak, Philipp Schaer
Date: 2023-11-29 00:00:00
Tasks: Articles

This publication describes the motivation and generation of $Q_{bias}$, a large dataset of Google and Bing search queries, a scraping tool and dataset for biased news articles, as well as language models for the investigation of bias in online search. Web search engines are a major factor and trusted source in information search, especially in the political domain. However, biased information can influence opinion formation and lead to biased opinions. To interact with search engines, users formulate search queries and interact with search query suggestions provided by the search engines. A lack of datasets on search queries inhibits research on the subject. We use $Q_{bias}$ to evaluate different approaches to fine-tuning transformer-based language models with the goal of producing models capable of biasing text with left and right political stance. Additionally to this work we provided datasets and language models for biasing texts that allow further research on bias in online information search.
Creator Context for Tweet Recommendation
Authors: Spurthi Amba Hombaiah, Tao Chen, Mingyang Zhang, Michael Bendersky, Marc Najork, Matt Colen, Sergey Levi, Vladimir Ofitserov, Tanvir Amin
Date: 2023-11-29 00:00:00
Tasks: Articles

When discussing a tweet, people usually not only refer to the content it delivers, but also to the person behind the tweet. In other words, grounding the interpretation of the tweet in the context of its creator plays an important role in deciphering the true intent and the importance of the tweet. In this paper, we attempt to answer the question of how creator context should be used to advance tweet understanding. Specifically, we investigate the usefulness of different types of creator context, and examine different model structures for incorporating creator context in tweet modeling. We evaluate our tweet understanding models on a practical use case -- recommending relevant tweets to news articles. This use case already exists in popular news apps, and can also serve as a useful assistive tool for journalists. We discover that creator context is essential for tweet understanding, and can improve application metrics by a large margin. However, we also observe that not all creator contexts are equal. Creator context can be time sensitive and noisy. Careful creator context selection and deliberate model structure design play an important role in creator context effectiveness.
Attribute Simulation for Item Embedding Enhancement in Multi-interest Recommendation
Authors: Yaokun Liu, Xiaowang Zhang, Minghui Zou, Zhiyong Feng
Date: 2023-11-29 00:00:00
Tasks: Attribute

Although multi-interest recommenders have achieved significant progress in the matching stage, our research reveals that existing models tend to exhibit an under-clustered item embedding space, which leads to a low discernibility between items and hampers item retrieval. This highlights the necessity for item embedding enhancement. However, item attributes, which serve as effective and straightforward side information for enhancement, are either unavailable or incomplete in many public datasets due to the labor-intensive nature of manual annotation tasks. This dilemma raises two meaningful questions: 1. Can we bypass manual annotation and directly simulate complete attribute information from the interaction data? And 2. If feasible, how to simulate attributes with high accuracy and low complexity in the matching stage? In this paper, we first establish an inspiring theoretical feasibility that the item-attribute correlation matrix can be approximated through elementary transformations on the item co-occurrence matrix. Then based on formula derivation, we propose a simple yet effective module, SimEmb (Item Embedding Enhancement via Simulated Attribute), in the multi-interest recommendation of the matching stage to implement our findings. By simulating attributes with the co-occurrence matrix, SimEmb discards the item ID-based embedding and employs the attribute-weighted summation for item embedding enhancement. Comprehensive experiments on four benchmark datasets demonstrate that our approach notably enhances the clustering of item embedding and significantly outperforms SOTA models with an average improvement of 25.59% on Recall@20.
DepthFL: Depthwise Federated Learning for Heterogeneous Clients
Authors: Minjae Kim, Sangyoon Yu, Suhyun Kim, Soo-Mook Moon
Date: 2023-05-01 00:00:00
Proceeding: ICLR 2023 5
Tasks: Federated Learning

Federated learning is for training a global model without collecting private local data from clients. As they repeatedly need to upload locally-updated weights or gradients instead, clients require both computation and communication resources enough to participate in learning, but in reality their resources are heterogeneous. To enable resource-constrained clients to train smaller local models, width scaling techniques have been used, which reduces the channels of a global model. Unfortunately, width scaling suffers from heterogeneity of local models when averaging them, leading to a lower accuracy than when simply excluding resource-constrained clients from training. This paper proposes a new approach based on depth scaling called DepthFL. DepthFL defines local models of different depths by pruning the deepest layers off the global model, and allocates them to clients depending on their available resources. Since many clients do not have enough resources to train deep local models, this would make deep layers partially-trained with insufficient data, unlike shallow layers that are fully trained. DepthFL alleviates this problem by mutual self-distillation of knowledge among the classifiers of various depths within a local model. Our experiments show that depth-scaled local models build a global model better than width-scaled ones, and that self-distillation is highly effective in training data-insufficient deep layers.
Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm
Authors: Fardin ahmadizar, Khabat Soltanian, Fardin akhlaghiantab, Ioannis tsoulos
Date: 2015-03-01 00:00:00
Proceeding: 2015 2015 3

The most important problems with exploiting artificial neural networks (ANNs) are to design the network topology, which usually requires an excessive amount of expert’s effort, and to train it. In this paper, a new evolutionary-based algorithm is developed to simultaneously evolve the topology and the connection weights of ANNs by means of a new combination of grammatical evolution (GE) and genetic algorithm (GA). GE is adopted to design the network topology while GA is incorporated for better weight adaptation. The proposed algorithm needs to invest a minimal expert’s effort for customization and is capable of generating any feedforward ANN with one hidden layer. Moreover, due to the fact that the generalization ability of an ANN may decrease because of overfitting problems, the algorithm utilizes a novel adaptive penalty approach to simplify ANNs generated through the evolution process. As a result, it produces much simpler ANNs that have better generalization ability and are easy to implement. The proposed method is tested on some real world classification datasets, and the results are statistically compared against existing methods in the literature. The results indicate that our algorithm outperforms the other methods and provides the best overall performance in terms of the classification accuracy and the number of hidden neurons. The results also present the contribution of the proposed penalty approach in the simplicity and generalization ability of the generated networks
Survival Analysis Algorithms based on Decision Trees with Weighted Log-rank Criteria
Authors: Iulii Vasilev, Mikhail Petrovskiy, Igor Mashechkin
Date: 2022-02-03 00:00:00
Proceeding: Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods 2022 2
Tasks: Survival Analysis

Survival Analysis is an important tool to predict time-to-event in many applications, including but not limited to medicine, insurance, manufacturing and others. The state-of-the-art statistical approach is based on Cox proportional hazards. Though, from a practical point of view, it has several important disadvantages, such as strong assumptions on proportional over time hazard functions and linear relationship between time independent covariates and the log hazard. Another technical issue is an inability to deal with missing data directly. To overcome these disadvantages machine learning survival models based on recursive partitioning approach have been developed recently. In this paper, we propose a new survival decision tree model that uses weighted log-rank split criteria. Unlike traditional log-rank criteria the weighted ones allow to give different priority to events with different time stamps. It works with missing data directly while searching the best splitting point, its s ize is controlled by p-value threshold with Bonferroni adjustment and quantile based discretization is used to decrease the number of potential candidates for splitting points. Also, we investigate how to improve the accuracy of the model with bagging ensemble of the proposed decision tree models. We introduce an experimental comparison of the proposed methods against Cox proportional risk regression and existing tree-based survival models and their ensembles. According to the obtained experimental results, the proposed methods show better performance on several benchmark public medical datasets in terms of Concordance index and Integrated Brier Score metrics.
Guarantees for Self-Play in Multiplayer Games via Polymatrix Decomposability
Authors: Revan MacQueen, James R. Wright
Date: 2023-10-17 00:00:00
Proceeding: NeurIPS 2023 11

Self-play is a technique for machine learning in multi-agent systems where a learning algorithm learns by interacting with copies of itself. Self-play is useful for generating large quantities of data for learning, but has the drawback that the agents the learner will face post-training may have dramatically different behavior than the learner came to expect by interacting with itself. For the special case of two-player constant-sum games, self-play that reaches Nash equilibrium is guaranteed to produce strategies that perform well against any post-training opponent; however, no such guarantee exists for multiplayer games. We show that in games that approximately decompose into a set of two-player constant-sum games (called constant-sum polymatrix games) where global $\epsilon$-Nash equilibria are boundedly far from Nash equilibria in each subgame (called subgame stability), any no-external-regret algorithm that learns by self-play will produce a strategy with bounded vulnerability. For the first time, our results identify a structural property of multiplayer games that enable performance guarantees for the strategies produced by a broad class of self-play algorithms. We demonstrate our findings through experiments on Leduc poker.
Sensitivity of Survival Analysis Metrics
Authors: Iulii Vasilev, Mikhail Petrovskiy, Igor Mashechkin
Date: 2023-10-11 00:00:00
Proceeding: Mathematics 2023 10
Tasks: Sensitivity, Specificity, Survival Analysis

Survival analysis models allow for predicting the probability of an event over time. The specificity of the survival analysis data includes the distribution of events over time and the proportion of classes. Late events are often rare and do not correspond to the main distribution and strongly affect the quality of the models and quality assessment. In this paper, we identify four cases of excessive sensitivity of survival analysis metrics and propose methods to overcome them. To set the equality of observation impacts, we adjust the weights of events based on target time and censoring indicator. According to the sensitivity of metrics, 𝐴𝑈𝑃𝑅𝐶 (area under Precision-Recall curve) is best suited for assessing the quality of survival models, and other metrics are used as loss functions. To evaluate the influence of the loss function, the 𝐵𝑎𝑔𝑔𝑖𝑛𝑔 model uses ones to select the size and hyperparameters of the ensemble. The experimental study included eight real medical datasets. The proposed modifications of 𝐼𝐵𝑆 (Integrated Brier Score) improved the quality of 𝐵𝑎𝑔𝑔𝑖𝑛𝑔 compared to the classical loss functions. In addition, in seven out of eight datasets, the 𝐵𝑎𝑔𝑔𝑖𝑛𝑔 with new loss functions outperforms the existing models of the scikit-survival library.
Adaptive Sampling for Weighted Log-Rank Survival Trees Boosting
Authors: Iulii Vasilev, Mikhail Petrovskiy, Igor Mashechkin
Date: 2023-01-27 00:00:00
Proceeding: Lecture Notes in Computer Science 2023 1
Tasks: Survival Analysis

The field of survival analysis is devoted to predicting the probability and time of the occurrence of an event. The global problem is to predict the event probability over time. It has applications in healthcare, credit scoring, etc. The most widely used method for assessing the covariate impacts on survival is the Cox proportional hazards approach. However, the assumption of non-overlapping survival functions usually does not hold on real data, and the linear dependence on features limits the quality of the method. There are tree-based machine learning methods to solve these problems. Usually, to evaluate the difference between the samples, it used the log-rank test. Obtained survival decision tree models also have strong interpretability, they can evaluate the importance of predictors, but they demonstrate inferior performance in comparison to Cox proportional hazards models.

To overcome these issues, this paper proposes a new boosting of the survival decision tree model that uses adaptive sampling and weighted log-rank split criteria. The model iteratively corrects an error in the ensemble. Each decision tree is trained on a sample, taking into account the weights of observations and subsequently adjusting the probabilities of getting into the next sample. We introduce an experimental comparison of the proposed adaptive boosting method against Cox proportional hazard and widely used survival trees and their ensembles: random forest and gradient boosting. Experiments on healthcare datasets show that our model outperforms the state-of-the-art survival models in terms of the following metrics: the concordance index, the integrated Brier score, and the integrated AUC.
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models
Authors: Jean Kaddour, Oscar Key, Piotr Nawrot, Pasquale Minervini, Matt J. Kusner
Date: 2023-07-12 00:00:00
Proceeding: NeurIPS 2023 11

The computation necessary for training Transformer-based language models has skyrocketed in recent years. This trend has motivated research on efficient training algorithms designed to improve training, validation, and downstream performance faster than standard training. In this work, we revisit three categories of such algorithms: dynamic architectures (layer stacking, layer dropping), batch selection (selective backprop, RHO loss), and efficient optimizers (Lion, Sophia). When pre-training BERT and T5 with a fixed computation budget using such methods, we find that their training, validation, and downstream gains vanish compared to a baseline with a fully-decayed learning rate. We define an evaluation protocol that enables computation to be done on arbitrary machines by mapping all computation time to a reference machine which we call reference system time. We discuss the limitations of our proposed protocol and release our code to encourage rigorous research in efficient training procedures: https://github.com/JeanKaddour/NoTrainNoGain.
Efficient Activation Function Optimization through Surrogate Modeling
Authors: Garrett Bingham, Risto Miikkulainen
Date: 2023-01-13 00:00:00
Proceeding: NeurIPS 2023 11

Carefully designed activation functions can improve the performance of neural networks in many machine learning tasks. However, it is difficult for humans to construct optimal activation functions, and current activation function search algorithms are prohibitively expensive. This paper aims to improve the state of the art through three steps: First, the benchmark datasets Act-Bench-CNN, Act-Bench-ResNet, and Act-Bench-ViT were created by training convolutional, residual, and vision transformer architectures from scratch with 2,913 systematically generated activation functions. Second, a characterization of the benchmark space was developed, leading to a new surrogate-based method for optimization. More specifically, the spectrum of the Fisher information matrix associated with the model's predictive distribution at initialization and the activation function's output distribution were found to be highly predictive of performance. Third, the surrogate was used to discover improved activation functions in several real-world tasks, with a surprising finding: a sigmoidal design that outperformed all other activation functions was discovered, challenging the status quo of always using rectifier nonlinearities in deep learning. Each of these steps is a contribution in its own right; together they serve as a practical and theoretical foundation for further research on activation function optimization.