Adding call for contribution for the S4 model
Author: Rocketknight1
Pull request: yes

Looking for community contributors to add the S4 model! ([repo](https://github.com/HazyResearch/state-spaces), [paper](https://arxiv.org/abs/2111.00396))

Comments:
Hi @Rocketknight1,
I would like to work on this S4 model


Hi @kamalkraj, sure thing! I thought this was quite an unusual and challenging model, but you've done some really good work in other PRs, so I think you should be able for it. Note that since the original repository is PyTorch-only, we'll probably want a PyTorch version of the model only to start, and we can leave a TF port for a future PR (which you don't have to do, of course!). Is that okay with you?

Yes, @Rocketknight. 


Sure! In that case I'll leave this PR un-merged for now. You can see most of the details in the .md file. The best place to get started is probably to get their [standalone example](https://github.com/HazyResearch/state-spaces/blob/main/example.py) working with `pykeops`. If you encounter any difficulties, or you're not sure how to proceed, you can let me know here or on Slack. We have direct communication with the original paper authors too, so we should be able to ask them if we encounter any issues with the underlying theory. Thank you for this!

Okay

@Rocketknight1 
Got the standalone version working with `pykeops`
https://github.com/kamalkraj/S4-Standalone

@kamalkraj Nice!

Hi @Rocketknight1,
I just skimmed through the paper and code, If I understood correctly, I only need to port the autoregressive Language Model to HF, right?



Hi @kamalkraj - for now, that should be fine, yes. I think it's best to start with porting a single model correctly and then we can decide what to do afterwards!

Okay


I have started working on the model.
make fixup is throwing an error ```  S4Config in TYPE_HINT but not in _import_structure.```

```python
from typing import TYPE_CHECKING

# rely on isort to merge the imports
from ...file_utils import _LazyModule, is_torch_available


_import_structure = {"configuration_s4": ["S4Config"]}


if is_torch_available():
    _import_structure["modeling_s4"] = ["S4ForCausalLM", "S4Layer", "S4Model", "S4PreTrainedModel"]


if TYPE_CHECKING:
    from .configuration_s4 import S4Config

    if is_torch_available():
        from .modeling_s4 import S4ForCausalLM, S4Layer, S4Model, S4PreTrainedModel

else:
    import sys

    sys.modules[__name__] = _LazyModule(__name__, globals()["__file__"], _import_structure)
```

![Screenshot 2021-12-14 at 8 54 31 PM](https://user-images.githubusercontent.com/17096858/146027775-341ade3b-9024-439e-92b7-8fe663e3d0d5.png)

`parse_init` failed to parse the file properly. Is there anything wrong with this `__init__.py` file

@Rocketknight1 @sgugger 

The script is not used to have something taht fits in one line for the declaration of `_import_structure`. Will fix it one day but in the meantime, you should replace it by 
```
# fmt: off
_import_structure = {
    "configuration_s4": ["S4Config"]
}
# fmt: on
```
The flags fmt will tell black to be a good boy and leave the style as is, and the script will parse the init properly.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Closing this call for contributions PR since @kamalkraj is already working on the actual model PR.