Added model evaluation results

README.md

@@ -67,54 +67,62 @@ print(prediction)
 ### Training Data
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+The model was fine-tuned on the hot paths dataset: [zhaojer/compiler_hot_paths](https://huggingface.co/datasets/zhaojer/compiler_hot_paths)
 
-[More Information Needed]
+The dataset is already split into train, validation, test sets with necessary columns/data needed for training/fine-tuning. No further preprocessing was performed for the data.
+
+The data (in the `path` column) were tokenized using the standard `BertTokenizer` for the `bert-base-uncased` model.
 
 ### Training Procedure
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
-
-[More Information Needed]
-
+We defined accuracy and AUROC as evaluation metrics for the model.
 
-#### Training Hyperparameters
+The model was fine-tuned for 3 epochs with standard hyperparameters, which took about 10 minutes to complete using NVIDIA T4 GPU.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Detailed Training Hyperparameters
 
-#### Speeds, Sizes, Times [optional]
+- `evaluation_strategy="epoch"`
+- `logging_strategy="epoch"`
+- `save_strategy="epoch"`
+- `num_train_epochs=3`
+- `per_device_train_batch_size=16`
+- `per_device_eval_batch_size=16`
+- `learning_rate=5e-5`
+- `load_best_model_at_end=True`
+- `metric_for_best_model="accuracy"`
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+Note: Anything not explicitly stated used default value.
 
 ## Evaluation
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
-
-#### Testing Data
+### Testing Data
 
 <!-- This should link to a Dataset Card if possible. -->
+The testing data consist of 68 hot paths and 92 cold paths generated from 4 distinct C programs.
+They are also from [zhaojer/compiler_hot_paths](https://huggingface.co/datasets/zhaojer/compiler_hot_paths); please see its dataset card for how the testing data were created.
+The model had never seen these testing data previously.
 
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
+### Metrics
 
 <!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+We evaluated the model on the testing data using the following metrics:
+- Loss (available by default)
+- Accuracy
+- AUROC
+- Precision, Recall, F1 score
+- Confusion matrix
 
 ### Results
 
-[More Information Needed]
+| Loss | Accuracy | AUROC | Precision | Recall |  F1  |
+| ---- | -------- | ----- | --------- | ------ | ---- |
+| 0.0620 | 0.9875 | 0.9952| 1.0000    | 0.9706 | 0.99 |
 
-#### Summary
+|               | Actually Hot | Actually Cold |
+| ------------- | -----------  | ------------  |
+| Predicted Hot | 66           |    0          |
+| Predicted Cold| 2            |    92         |