Spaces:
Runtime error
Runtime error
summarization
T5 Summarisation Using Pytorch Lightning
Instructions
- Clone the repo.
- Run
make dirs
to create the missing parts of the directory structure described below. - Optional: Run
make virtualenv
to create a python virtual environment. Skip if using conda or some other env manager.- Run
source env/bin/activate
to activate the virtualenv.
- Run
- Run
make requirements
to install required python packages. - Put the raw data in
data/raw
. - To save the raw data to the DVC cache, run
dvc commit raw_data.dvc
- Edit the code files to your heart's desire.
- Process your data, train and evaluate your model using
dvc repro eval.dvc
ormake reproduce
- When you're happy with the result, commit files (including .dvc files) to git.
Project Organization
βββ LICENSE
βββ Makefile <- Makefile with commands like `make dirs` or `make clean`
βββ README.md <- The top-level README for developers using this project.
βββ data
β βββ processed <- The final, canonical data sets for modeling.
β βββ raw <- The original, immutable data dump.
β
βββ eval.dvc <- The end of the data pipeline - evaluates the trained model on the test dataset.
β
βββ models <- Trained and serialized models, model predictions, or model summaries
β
βββ notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
β the creator's initials, and a short `-` delimited description, e.g.
β `1.0-jqp-initial-data-exploration`.
β
βββ process_data.dvc <- Process the raw data and prepare it for training.
βββ raw_data.dvc <- Keeps the raw data versioned.
β
βββ references <- Data dictionaries, manuals, and all other explanatory materials.
β
βββ reports <- Generated analysis as HTML, PDF, LaTeX, etc.
β βββ figures <- Generated graphics and figures to be used in reporting
β βββ metrics.txt <- Relevant metrics after evaluating the model.
β βββ training_metrics.txt <- Relevant metrics from training the model.
β
βββ requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
β generated with `pip freeze > requirements.txt`
β
βββ setup.py <- makes project pip installable (pip install -e .) so src can be imported
βββ src <- Source code for use in this project.
β βββ __init__.py <- Makes src a Python module
β β
β βββ data <- Scripts to download or generate data
β β βββ make_dataset.py
β β
β βββ models <- Scripts to train models and then use trained models to make
β β β predictions
β β βββ predict_model.py
β β βββ train_model.py
β β
β βββ visualization <- Scripts to create exploratory and results oriented visualizations
β βββ visualize.py
β
βββ tox.ini <- tox file with settings for running tox; see tox.testrun.org
βββ train.dvc <- Traing a model on the processed data.
Project based on the cookiecutter data science project template. #cookiecutterdatascience