wasertech commited on
Commit
651bbfe
1 Parent(s): 55b1e66

update custom message

Browse files
Files changed (1) hide show
  1. constants.py +5 -5
constants.py CHANGED
@@ -120,19 +120,19 @@ Custom splits and potential data leakage during training can indeed lead to misl
120
 
121
  To address these concerns and ensure the reliability of metrics on the leaderboard:
122
 
123
- #### 1. Transparency in Training Data
124
  Model submissions should come with **detailed information about the training data** used, including whether they have seen the specific test sets used for evaluation. This **transparency enables the community to assess the validity of the results**.
125
 
126
- #### 2. Standardized Evaluation
127
  Promote the use of **standardized evaluation datasets and testing procedures across models**. This helps prevent data leakage and **ensures fair comparisons**.
128
 
129
- #### 3. Verification and Validation
130
  Implement verification processes to **check the integrity of submitted models**. This could include cross-validation checks to **identify any potential issues with custom splits or data leakage**.
131
 
132
- #### 4. Community Engagement
133
  Encourage **active participation and feedback from the ASR community**. Regular discussions and collaborations can help **identify and address issues related to data integrity and model evaluations**.
134
 
135
- #### 5. Documentation
136
  Models added to the leaderboard should **provide comprehensive documentation**, including information on **dataset usage, preprocessing steps, and any custom splits employed** during training.
137
 
138
  By focusing on these aspects, we can enhance trust in the metrics and evaluations within the ASR community and ensure that the models added to the leaderboard are reliable and accurately represent their performance. It's essential for the community to work together to maintain transparency and data integrity.
 
120
 
121
  To address these concerns and ensure the reliability of metrics on the leaderboard:
122
 
123
+ ##### 1. Transparency in Training Data
124
  Model submissions should come with **detailed information about the training data** used, including whether they have seen the specific test sets used for evaluation. This **transparency enables the community to assess the validity of the results**.
125
 
126
+ ##### 2. Standardized Evaluation
127
  Promote the use of **standardized evaluation datasets and testing procedures across models**. This helps prevent data leakage and **ensures fair comparisons**.
128
 
129
+ ##### 3. Verification and Validation
130
  Implement verification processes to **check the integrity of submitted models**. This could include cross-validation checks to **identify any potential issues with custom splits or data leakage**.
131
 
132
+ ##### 4. Community Engagement
133
  Encourage **active participation and feedback from the ASR community**. Regular discussions and collaborations can help **identify and address issues related to data integrity and model evaluations**.
134
 
135
+ ##### 5. Documentation
136
  Models added to the leaderboard should **provide comprehensive documentation**, including information on **dataset usage, preprocessing steps, and any custom splits employed** during training.
137
 
138
  By focusing on these aspects, we can enhance trust in the metrics and evaluations within the ASR community and ensure that the models added to the leaderboard are reliable and accurately represent their performance. It's essential for the community to work together to maintain transparency and data integrity.