Realloc error
I wonder if you faced an error "realloc of size ... failed" while working with it and if you did could you please share how you dealed with it? I got it while preparing dataset. I tried to change batch_size but when I run command batched=True it got me an error saying that "string indices must be integers"
Here is how whole section looks like:
def prepare_dataset(batch):
audio = batch["audio"]
# batched output is "un-batched"
batch["input_values"] = processor(audio["array"], sampling_rate=audio["sampling_rate"]).input_values[0]
batch["input_length"] = len(batch["input_values"])
with processor.as_target_processor():
batch["labels"] = processor(batch["sentence"]).input_ids
return batch
train =train.map(prepare_dataset, remove_columns=train.column_names)
test = test.map(prepare_dataset, remove_columns=test.column_names)
Hey @Bauyrjan ,
I am not sure if batched=True was the reason of the error, but it is probably some dimension error that you facing there. Please try to debug the code inside prepare_dataset function.
I also have to note that this code is pretty much following the standard code written from @huggingface-co team. here is a link to the code I followed.
Please let me know this helped!