Issue 1:
Feature request

Thank you for the awesome framework!

For my work I wanted to use transformers.pipelines.token_classification.TokenClassificationPipeline in batch mode, since it is much faster on GPU, but I wanted to keep all the functionality for grouping entities.

So I want to suggest something like this:

nlp = pipeline("ner", model=model,
tokenizer=tokenizer,
device = 0 if torch.cuda.is_available() else -1,
aggregation_strategy="average", batch_size=16)

Motivation

I implemented it for myself and think it would be cool to have this functionality "out-of-the-box" for community to enjoy the speed up. (And it really gives a huge speed up)

Your contribution

I am willing to contribute and implement this change for TokenClassification task (also for TextClassification, FeatureExtraction should be pretty much same). Have not worked with other pipelines, so not sure how batching is implemented there, but I am willing to try and contribute.

Issue 2:
System Info

python: 3.8

transformers: 4.23.1

Who can help?

@patrickvonplaten

Information

The official example scripts

My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)

My own task or dataset (give details below)

Reproduction

from transformers import Wav2Vec2Tokenizer, Wav2Vec2CTCTokenizer

def main() -> None:
# [NOTE]: plsace insert save_dir path
output_dir = r"./output_dir"

encoder1_tokenizer = Wav2Vec2CTCTokenizer.from_pretrained("facebook/wav2vec2-base-960h")
encoder2_tokenizer = Wav2Vec2Tokenizer.from_pretrained("facebook/wav2vec2-base-960h")

encoder1_tokenizer.vocab_files_name = {
"vocab_file": "encoder1_vocab.json",
"tokenizer_config_file": "encoder1_tokenizer_config.json",
}
encoder2_tokenizer.vocab_files_name = {
"vocab_file": "encoder2_vocab.json",
"tokenizer_config_file": "encoder2_tokenizer_config.json",
}

encoder1_tokenizer.save_vocabulary(save_diretory=output_dir)
encoder2_tokenizer.save_vocabulary(save_diretory=output_dir)

if "__main__" in __name__:
main()

Expected behavior

This is the problem that occurred when the bi-encoder had to be placed on Wav2Vec2.

When i saving the model

To prevent overwriting due to duplication when saving in tokenizer, i may have named files that are saved differently.

I overridden vocab_files_name as shown in Reproduction.

But later I found out that the file hasn't been renamed

So I looked up the problem and found the following problem

tokenization_wav2vec2.py > L127-161

class Wav2Vec2CTCTokenizer(PreTrainedTokenizer):

"""
Constructs a Wav2Vec2CTC tokenizer.
This tokenizer inherits from [`PreTrainedTokenizer`] which contains some of the main methods. Users should refer to
the superclass for more information regarding such methods.
Args:
vocab_file (`str`):
File containing the vocabulary.
bos_token (`str`, *optional*, defaults to `"
"`):
The beginning of sentence token.
eos_token (`str`, *optional*, defaults to `"
"`):
The end of sentence token.
unk_token (`str`, *optional*, defaults to `"<unk>"`):
The unknown token. A token that is not in the vocabulary cannot be converted to an ID and is set to be this
token instead.
pad_token (`str`, *optional*, defaults to `"<pad>"`):
The token used for padding, for example when batching sequences of different lengths.
word_delimiter_token (`str`, *optional*, defaults to `"|"`):
The token used for defining the end of a word.
do_lower_case (`bool`, *optional*, defaults to `False`):
Whether or not to accept lowercase input and lowercase the output when decoding.
**kwargs
Additional keyword arguments passed along to [`PreTrainedTokenizer`]
"""

vocab_files_names = VOCAB_FILES_NAMES # [NOTE]: here
pretrained_vocab_files_map = PRETRAINED_VOCAB_FILES_MAP
max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES
model_input_names = ["input_ids", "attention_mask"]

def __init__(

VOCAB_FILES_NAMES sets the default value for vocab_files_names

tokenization_wav2vec2.py > L595-606

def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None) -> Tuple[str]:
if not os.path.isdir(save_directory):
logger.error(f"Vocabulary path ({save_directory}) should be a directory")
return
vocab_file = os.path.join(
save_directory, (filename_prefix + "-" if filename_prefix else "") + VOCAB_FILES_NAMES["vocab_file"] # [NOTE]: here
)

with open(vocab_file, "w", encoding="utf-8") as f:
f.write(json.dumps(self.encoder, indent=2, sort_keys=True, ensure_ascii=False) + "\n")

return (vocab_file,)

However, if you look at the save_vocabulary function, "vocab_files_names" is not included in the checked part, but "VOCAB_FILES_NAMES" is included, so the file name does not change even if you override "vocab_files_names"

So I want to change that "VOCAB_FILES_NAMES" part to "vocab_files_names"

And this was on Wav2Vec2CTCTokenizer and Wav2Vec2Tokenizer