text: 
  
  TITLE
  Batch and smart batch support for pipelines.
  
  COMMENTS
  2
  
  REACTIONS
  +1: 9
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🚀 Feature request
  
  ## Motivation
  I want to use `TextClassificationPipeline` to classify a large number of texts.
  The naive approach is:
  
  ```python
  model = AutoModelForSequenceClassification.from_pretrained(model_dir)
  tokenizer = AutoTokenizer.from_pretrained(model_dir)
  pipeline = TextClassificationPipeline(
      model=model,
      tokenizer=tokenizer,
      framework="pt",
      device=0,
  )
  results = pipeline(unlabeled_text_list)
  ```
  
  But this gives me CUDA OOM error when `unlabeled_text_list` is long.
  
  What about adding batch support that lets you specify the batch size and maybe also support for multiprocessing tokenization?
  
  When possible smart batching would be nice. See this: https://github.com/UKPLab/sentence-transformers/issues/454#issuecomment-699496454
  
  What do you think?