Added some description to the README.md

README.md

@@ -5,4 +5,48 @@ sdk_version: 4.25.0
 app_file: change_visualizer.py
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+# Description
+
+This project is a main artifact of the "Research on evaluation for AI Commit Message Generation" research.
+
+# Structure (important components)
+
+- ### Configuration: [config.py](config.py)
+    - Grazie API JWT token and Hugging Face token must be stored as environment variables.
+- ### Visualization app -- a Gradio application that is currently deployed
+  at https://huggingface.co/spaces/JetBrains-Research/commit-rewriting-visualization.
+    - Shows
+        - The "golden" dataset of manually collected samples; the dataset is downloaded on startup
+          from https://huggingface.co/datasets/JetBrains-Research/commit-msg-rewriting
+        - The entire dataset that includes the synthetic samples; the dataset is downloaded on startup
+          from https://huggingface.co/datasets/JetBrains-Research/synthetic-commit-msg-rewriting
+        - Some statistics collected for the dataset (and its parts); computed on startup
+
+      _Note: datasets updated => need to restart the app to see the changes._
+    - Files
+        - [change_visualizer.py](change_visualizer.py)
+- ### Data processing pipeline (_note: datasets and files names can be changed in the configuration file_)
+    - Run the whole pipeline by running [run_pipeline.py](run_pipeline.py)
+        - All intermediate results are stored as files defined in config
+    - Intermediate steps (can run them separately by running the corresponding files
+      from [generation_steps](generation_steps)). The input is then taken from the previous step's artifact.
+    - Generate the synthetic samples
+        - Files [generation_steps/synthetic_end_to_start.py](generation_steps/synthetic_end_to_start.py)
+          and [generation_steps/synthetic_start_to_end.py](generation_steps/synthetic_start_to_end.py)
+        - The first generation step (end to start) downloads the `JetBrains-Research/commit-msg-rewriting`
+          and `JetBrains-Research/lca-commit-message-generation` datasets from
+          Hugging Face datasets.
+    - Compute metrics
+        - File [generation_steps/metrics_analysis.py](generation_steps/metrics_analysis.py)
+        - Includes the functions for all metrics
+        - Downloads `JetBrains-Research/lca-commit-message-generation` Hugging Face dataset.
+    - The resulting artifact (dataset with golden and synthetic samples, attached reference messages and computed
+      metrics) is saved to the file [output/synthetic.csv](output/synthetic.csv). It should be uploaded
+      to https://huggingface.co/datasets/JetBrains-Research/synthetic-commit-msg-rewriting **manually**.
+- ### Data analysis
+    - [analysis_util.py](analysis_util.py) -- some functions used for data analysis, e.g., correlations computation.
+    - [analysis.ipynb](analysis.ipynb) -- compute the correlations, the resulting tables.
+    - [chart_processing.ipynb](chart_processing.ipynb) -- Jupyter Notebook that draws the charts that were used in the
+      presentation/thesis.