Why does dataset merge fail when tools have different parameters?

Hi, I have a question about SFT (Supervised Fine-tuning) for an agent model.

Suppose I want to fine-tune an agent model that may receive two different tools: tool1 and tool2. These tools have different parameters and types in their schema definitions.

When I try to merge datasets containing different tool definitions, I get the following error:

TypeError: Couldn't cast array of type
struct<refundFee: struct<description: string, type: string>, ... , servicerId: struct<description: string, type: string>>
to
{
  'refundFee': {'description': Value(dtype='string'), 'type': Value(dtype='string')},
  ...
  'templateId': {'description': Value(dtype='string'), 'type': Value(dtype='string')}
}
From my understanding, the merge fails because the tools column's nested structure is different across datasets — e.g., one struct contains an extra field servicerId while the other does not. This causes HuggingFace Datasets (and its underlying Apache Arrow schema) to reject the merge.

My question is: why is it designed this way?

Is this strict schema matching a hard requirement of the library?
Is there a recommended way to merge datasets with different tool schemas (different parameters and types)?
For an agent model supporting multiple tools, what's the best practice for preparing/merging training data without losing flexibility?
Any guidance or design rationale would be greatly appreciated. Thanks!


Hi @hitszxs,
 This is indeed by design,

The `datasets` library is built on top of [Apache Arrow](https://arrow.apache.org/), which uses a **columnar storage format** with strict schema requirements. When you try to concatenate/merge datasets, the library checks if features can be aligned using the [`_check_if_features_can_be_aligned`](https://github.com/huggingface/datasets/blob/main/src/datasets/features/features.py#L2297-L2316) function.

Two datasets can be merged if:
1. Columns with the same name have the **same type**, OR
2. One of them has `Value("null")` (representing missing data)

For struct types (nested dictionaries like your tool schemas), **all fields must match exactly**. This ensures type safety and efficient columnar storage.

## Workarounds for Your Use Case
 Store tools as JSON strings

Instead of using nested struct types, store the tool definitions as JSON strings




Hi ! with #8027 you will be able to use the `Json()` type for tool fields where you need to store dictionaries with arbitrary keys and values. For example you can define the features of a tool calling dataset like this:

```python
from datasets import Features, Json, List, Value

features = Features({
    'messages': List({
        'role': Value('string'),
        'content': Value('string'),
        'tool_calls': List(Json()),
    })
})
```

and then before concatenating or interleaving datasets you can make sure they all have the same types:

```python
from datasets import concatenate_datasets

datasets = [ds.cast(features) for ds in datasets]
merged = concatenate_datasets(datasets)
```

Small tip: to avoid the `.cast()` step you could try to load the dataset with the specified features ahead of time:

```python
from datasets import load_dataset

ds = load_dataset(..., features=features)
```