amonfortc/FibroPred · Hugging Face

README: FibroPred Predictive System

Description

FibroPred is a predictive system designed to analyze clinical data and provide predictions for specific medical outcomes related to fibrosis. The tool employs machine learning models, primarily Random Forest classifiers, to forecast the likelihood of various events or conditions, such as mortality, necessity for transplantation, and progressive disease.

The system provides an easy-to-use interface built with Gradio, allowing users to input feature values and obtain predictions. Models and their configurations are stored and loaded dynamically, ensuring modularity and adaptability.

Features

Prediction Targets:
- Death: Likelihood of patient mortality based on clinical and diagnostic features.
- Binary Diagnosis: Classification of patients into specific diagnostic categories.
- Necessity of Transplantation: Assessment of whether a patient is likely to need a transplant.
- Progressive Disease: Prediction of disease progression based on longitudinal data.
Dynamic Model Loading: Models and feature sets are loaded dynamically from a pre-configured directory (models).
Gradio Interface: A tabbed interface for each prediction target, where users can input values interactively and receive predictions in real-time.

Predictive Features

Below are the features used for prediction across all targets:

Pedigree: Represents the familial history related to fibrotic conditions.
Age at diagnosis: Age of the patient at the time of diagnosis.
FVC (L) at diagnosis: Forced vital capacity in liters at the time of diagnosis, reflecting lung function.
FVC (%) at diagnosis: Forced vital capacity as a percentage of the expected value for the patient’s age and sex.
DLCO (%) at diagnosis: Diffusion capacity for carbon monoxide as a percentage, measuring gas exchange efficiency in the lungs.
RadioWorsening2y: Radiological assessment of lung deterioration over two years.
Severity of telomere shortening - Transform 4: Indicates the degree of telomere shortening.
Progressive disease: Binary variable indicating whether the disease is progressive or stable.
Biopsy: Binary variable indicating whether a biopsy was performed.
Genetic mutation studied in patient: Binary variable indicating the presence of specific genetic mutations.
Comorbidities: Binary variable representing the presence of relevant comorbidities.
Tobacco use: Binary variable reflecting whether the patient has a history of tobacco use.

Setup Instructions

Clone or download the repository.
Ensure Python 3.8+ is installed.
Install dependencies using the command:
```
pip install -r requirements.txt
```
Place trained models in the models directory. Models should be .pkl files containing:
- model: Trained Random Forest model.
- features: Feature list used during model training.

Usage

Run the application:
```
python app.py
```
Access the Gradio interface through the displayed local or public URL.
Select a prediction tab, input feature values, and click "Submit" to get predictions.

Key Scripts

app.py:
- Implements the Gradio interface.
- Maps user-friendly model names to actual models and their features.
fibropred_model.py:
- Contains the preprocessing pipeline, including imputation and feature selection.
- Includes functions for training, evaluation, and visualization.
model_utils.py:
- Functions to load models and their features dynamically.
- Handles predictions using preloaded models.
requirements.txt:
- Lists the Python dependencies required for the system.

Technical Highlights

Machine Learning Framework: Models are built using scikit-learn, leveraging Random Forest classifiers for robust predictions.
Visualization: The script includes utilities for plotting feature importance, ROC-AUC curves, and overfitting diagnostics.
Dynamic Handling: Feature lists and models are dynamically linked, ensuring flexibility when adding new prediction targets.

Results

The performance and feature importance for the prediction targets are detailed below:

Prediction Target: Death

Cross-validation Accuracy: The cross-validation results for "Death" show some variability across folds, but overall, the model achieves consistently high accuracy.
Train vs Test Accuracy:

Feature Importance: Features such as "Progressive disease", "DLCO (%) at diagnosis", and "FVC (%) at diagnosis" are the most influential.

ROC-AUC Curve: The ROC-AUC curve illustrates strong model performance, with an AUC of 0.92.

Prediction Target: Binary Diagnosis

Cross-validation Accuracy: Variability in cross-validation accuracy is observed, but the model maintains high performance across most folds.

Feature Importance: Key predictors include "Age at diagnosis", "Pedigree", and "Tobacco use".

ROC-AUC Curve: The high AUC value of 0.95 indicates excellent discrimination ability.

Prediction Target: Progressive Disease

Cross-validation Accuracy: Accuracy scores across folds highlight variability, but peaks show strong model performance.
Train vs Test Accuracy:

Feature Importance: "DLCO (%) at diagnosis", "Age at diagnosis", and "Pedigree" emerge as the dominant predictors.

ROC-AUC Curve: With an AUC of 0.98, the model demonstrates exceptional predictive power.

Prediction Target: Necessity of Transplantation

Cross-validation Accuracy: Cross-validation reveals excellent model accuracy.

Feature Importance: "RadioWorsening2y", "FVC (%) 1 year after diagnosis", and "Comorbidities" are critical.

ROC-AUC Curve: The model achieves an AUC of 1.00.

Future Improvements

Optimizing Variable Names: Review and refine variable naming conventions for clarity.
Improving Model Precision: Retrain the model with a larger dataset.
Identifying Optimal Medical Variables: Simplify the model by removing less relevant variables.
Testing Model Performance with Reduced Variables: Assess predictive performance with a reduced set of variables.
Expanding Dataset Diversity: Incorporate data from diverse demographics.
Adding Longitudinal Data Analysis: Integrate temporal patterns in disease progression.
Real-time Model Retraining: Develop mechanisms for real-time updates.

Associated Space

Check out the interactive demo of this model on Hugging Face Spaces:

This README provides a comprehensive guide to understanding and using the FibroPred predictive system effectively.