PyTorch's int8 & int4 WoQ GEMMs expect contiguous activations
Author: sanchitintel
Pull request: yes

# What does this PR do?

PyTorch's [int8](https://pytorch.org/cppdocs/api/function_namespaceat_1aea35d581753365d7db9494cc34de9f0e.html) & [int4](https://pytorch.org/cppdocs/api/function_namespaceat_1adeda9630914278ac02d7fd758da19e3d.html) weight-only-quantization GEMMs (used to perform computation corresponding to linear when models are quantized with torchao's int4 or int8 weight-only quantizations) [expect contiguous activations (at least for CPU)](https://github.com/pytorch/pytorch/blob/bc40ccf6aab3093efb1f765f9c0599384d2a0fb6/aten/src/ATen/native/LinearAlgebra.cpp#L3476).
But `hidden_states[:, slice_indices, :]` is being used to get logits in the following line, 
```
logits = self.lm_head(hidden_states[:, slice_indices, :])
```

Added a call to `contiguous()` after `hidden_states[:, slice_indices, :]`. When batch-size is 1, `logits` is contiguous anyway, so this change affects batch-size > 1.

Alternatively those PyTorch int4 & int8 weight-only quantization GEMM kernels would have to be modified to support non-contiguous activations.

Existing UTs should suffice to test changes.

Thanks!


cc @ArthurZucker


Comments:
Thanks a lot for reviewing, @RocketKnight1!

`make fix-copies` made changes to derivative models, but at my end, I see an error after `python utils/check_docstrings.py --fix_and_overwrite`, so `make-fix-copies` couldn't complete -

<details>

```
python utils/check_table.py --fix_and_overwrite
python utils/check_dummies.py --fix_and_overwrite
python utils/check_doctest_list.py --fix_and_overwrite
python utils/check_docstrings.py --fix_and_overwrite

Traceback (most recent call last):
  File "/env/transformers/utils/check_docstrings.py", line 1063, in <module>
    check_docstrings(overwrite=args.fix_and_overwrite, check_all=args.check_all)
  File "/env/transformers/utils/check_docstrings.py", line 981, in check_docstrings
    for modified_file_diff in repo.index.diff(repo.refs.main.commit):
  File "/env/lib/python3.10/site-packages/git/util.py", line 1200, in __getattr__
    return list.__getattribute__(self, attr)
AttributeError: 'IterableList' object has no attribute 'main'
```

</details>

I'll try fixing it, so that the `repo-consistency` check may pass. Thanks!

UPDATE: This error occurred when git origin was my transformers fork, but didn't occur when git origin was `https://github.com/huggingface/transformers`. After undoing 2 lint changes made by `make-fix-copies`, the CI check passed





Looks good to me now, but since a lot of models are affected after all I should double-check with a core maintainer. cc @arthurzucker for final review!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_36078). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for reviewing,@ArthurZucker!

I agree that it may degrade performance for some use-cases, as it'd create a copy after slicing.

It can alternatively be fixed in PyTorch by special-casing for non-contiguous activations (for `torch._weight_int8pack_mm` & `torch._weight_int4pack_mm`) on CPU by one of the two ways:

1. Creating a contiguous copy of the activation tensor in quantized GEMM kernels (which would have the same effect as this change), or
2. Supporting non-contiguous activation tensors in GEMM computation (the leading dimension of the activation tensor would be quite large in this case).

I'll check the performance impact of each for input shapes commonly encountered for popular LLMs, and will fix it in PyTorch instead.

Thanks again!