Category: competition-setup
Question: How to creating a new scoring metric for Kaggle competition?
Answer: You can implement a new metric in a Python notebook at this [link](http://www.kaggle.com/code/metrics/new) or from the Host > Evaluation Metric tab on a competition. Metric notebooks can be published and shared, but currently only Kaggle staff can add metrics to the public metric listing. If you think your metric is a good candidate for general use, please make the notebook public and post in the [competition hosting forum](https://www.kaggle.com/discussions/competition-hosting).

Before your metric executes, Kaggle automatically reads the solution and submission file into Pandas dataframes, aligns the solution and submission rows based on a provided id column, and calls a `score()` function. Your metric code needs to define this `score()` function and it must return a single float. Almost all solution files are split into a `Public` and `Private` set by way of a `Usage` column in the file. The `score()` function is called separately for each of these respective sets.

Your `score()` function must satisfy the following constraints:

- Accept the arguments `solution: pd.DataFrame, submission: pd.DataFrame, row_id_column_name: str`, in that order. You can add any other keyword arguments that you need after those three. Any additional keyword arguments are configured on a per-competition basis on the Evaluation Metric page.
- All arguments and the return value of `score` must have type annotations.
- Default argument values are encouraged but not required.
- `score()` must return a finite float.
- `score()` must have a docstring. The docstring will be shown to competition hosts on the evaluation tab after they have selected a metric. We encourage you to include at least the same sections covered in our [example metric's docstring](https://www.kaggle.com/metric/example-metric-code): a general description of the metric, explanations of each of the `score` arguments, references for the metric math, and examples of valid use.
- In order to prevent data leaks from the solution file, errors must specify who will see the details. Only errors raised as `ParticipantVisibleError` will be visible to all participants.
- Error messages will be truncated to 280 characters.
- The scoring runtime is limited to 30 minutes total for the `Public` and `Private` splits combined.
- Metric notebooks do not have internet access and cannot use accelerators, so your `score()` function must not rely on these notebook features.

Once your code is ready, you will also need to define some metadata in the `Metric` section of the notebook sidebar. You must save this metadata separately from the rest of the notebook.

- **Name**: your metric will use the metric notebook's name. Save the metadata to update the name.
- **Description**: a short (less than 255 characters) description of the metric.
- **Category**: the main use of the metric, such as clustering or regression.
- **Leaderboard sort order**: toggle this to indicate if a higher score is better or worse.
- **Pass complete submission**: Advanced use only. You almost certainly only want to use this if your submission can have a different number of rows than the solution file. When enabled, your metric will receive the entire submission file for both the public and private scoring rounds. Your metric will need to manage matching the solution and submission rows using the `row_id_column_name`.

You will need to use the dedicated `Save` button in the Metric section of the notebook sidebar for this metadata, in addition to the `Save & Validate` button used to save the notebook's source code. When you save your metric, your notebook will first be committed like any other notebook, followed by a series of metric-specific validation checks. This validation step will also re-run any unit test functions and doctests that are discoverable with Pytest. We strongly encourage you to include test cases, but they are not mandatory.

If the validation step fails, your notebook code will still save, but no new metric version will be created. We recommend reviewing this [example metric](https://www.kaggle.com/metric/example-metric-code) or [metric template](https://www.kaggle.com/code/metric/metric-template/) before you begin coding.