text: 
  
  TITLE
  Encapsulate all forward passes of integration tests with "with torch.no_grad()"
  
  COMMENTS
  7
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  Each model in the Transformers library has one or more corresponding integration tests, which make sure the implementation in Transformers returns the same exact logits/hidden states as the original implementation on the same input data.
  
  e.g. the integration test for the Vision Transformer (ViT) can be found [here](https://github.com/huggingface/transformers/blob/df085d8ea8b823b85ecfe2e1968b5741983f98f3/tests/test_modeling_vit.py#L336). It verifies the shape of the logits as well as the first values on a cats image.
  
  However, a lot of these integration tests don't leverage `with torch.no_grad()`, which make sure no gradients are computed (which isn't necessary when doing inference).
  
  Hence, it would be great if someone could encapsulate the forward passes of the integration tests with `with torch.no_grad()`.