[WIP][Research projects] Add folder for Music AI / Music Transformers
Author: asigalov61
Pull request: yes
Labels: name: wontfix

A new dir specifically for Music AI/Music Transformers.

Created as suggested by Patrick von Platen. 

I am still figuring out PRs so please correct this PR if I've done something wrong.

Thank you for your help/guidance and for the welcome to the Huggingface community :)

Looking forward to contributing what I can :)


 GPT2: @LysandreJik, @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 


Comments:
Hey @asigalov61, 

The folder should be under `examples/research_projects` => so at `examples/research_projects/music_transformers`. There you are very free to add the files you want. If you want people to use your code, we suggest to add nice & readable code with a well-thought API, and a nice README.md that explains how to use your code and that also shows a nice use case.

Got it!

Thank you for the advice 🙂

Do I have to use Hugginface transformers to post in that dir? Or I can use my implementations also?

I also mostly work in Google Colabs, so is it ok to post colabs?

Is the API a requirement to post? I do not always convert colabs to Python so I need to know this to figure out what to give priority to.

Thank you.
________________________________
From: Patrick von Platen <notifications@github.com>
Sent: Tuesday, December 29, 2020 4:44 AM
To: huggingface/transformers <transformers@noreply.github.com>
Cc: Alex <asigalov61@hotmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [huggingface/transformers] [WIP][Research projects] Add folder for Music AI / Music Transformers (#9337)


Hey @asigalov61<https://github.com/asigalov61>,

The folder should be under examples/research_projects => so at examples/research_projects/music_transformers. There you are very free to add the files you want. If you want people to use your code, we suggest to add nice & readable code with a well-thought API, and a nice README.md that explains how to use your code and that also shows a nice use case.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/huggingface/transformers/pull/9337#issuecomment-752062712>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANNXLI6A3S2M2GFCU2LLHXTSXHFMLANCNFSM4VNBN4YQ>.


> Got it! Thank you for the advice  Do I have to use Hugginface transformers to post in that dir? Or I can use my implementations also? I also mostly work in Google Colabs, so is it ok to post colabs? Is the API a requirement to post? I do not always convert colabs to Python so I need to know this to figure out what to give priority to. Thank you.
> […](#)
> ________________________________ From: Patrick von Platen <notifications@github.com> Sent: Tuesday, December 29, 2020 4:44 AM To: huggingface/transformers <transformers@noreply.github.com> Cc: Alex <asigalov61@hotmail.com>; Mention <mention@noreply.github.com> Subject: Re: [huggingface/transformers] [WIP][Research projects] Add folder for Music AI / Music Transformers (#9337) Hey @asigalov61<https://github.com/asigalov61>, The folder should be under examples/research_projects => so at examples/research_projects/music_transformers. There you are very free to add the files you want. If you want people to use your code, we suggest to add nice & readable code with a well-thought API, and a nice README.md that explains how to use your code and that also shows a nice use case. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<[#9337 (comment)](https://github.com/huggingface/transformers/pull/9337#issuecomment-752062712)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANNXLI6A3S2M2GFCU2LLHXTSXHFMLANCNFSM4VNBN4YQ>.

It should be based on hugging face transformers. If it's just a notebook - it might make more sense to just add it here: https://github.com/huggingface/transformers/tree/master/notebooks#community-notebooks

I was unable to translate my implementation to Huggingface transformers code because docs and examples were unclear so if someone can help me do that, I would really appreciate it.

Thank you so much.

Just in case here is the direct link to Google Colab because I am not sure how ot PR it (lol)...

https://github.com/asigalov61/transformers/blob/master/examples/music_transformers/Music_Reformer_TPU_Edition.ipynb

Hi! Could you let us know which docs/instructions were unclear? What were you trying to do, how can we help out? Thanks.

Certainly.

First of all, I could not find any clear and exact info on how to work with custom text datasets. You mostly provide examples for datasets from your library but not much else. For example, I could not find info on how to do basic things like loading a custom txt file or how to easily tokenize it to be compatible with huggingface implementations.

Another example would be the lack of complete notebooks/code. Like in the Reformer notebook by Peter, this one:
notebooks/PyTorch_Reformer.ipynb at master · patrickvonplaten/notebooks (github.com)<https://github.com/patrickvonplaten/notebooks/blob/master/PyTorch_Reformer.ipynb>
there is not a single word about SentencePiece which was used to create Crime and Punishment tokenizer model.

Also, I could not find a single example for GPT2 models + text, also in a basic and easy to use/try format.

Your docs/examples are probably ok if one invests enough time and effort (or if one has pre-existing knowledge and experience with your implementations) but w/o doing so, the examples/docs are not sufficient for beginners/newcomers due to a rather steep learning curve and effort/time required. Friendly IMHO please as I do appreciate your work and your efforts regardless.

To be very specific and relevant to the subject at hand, I need to know how to process and tokenize a simple line-by-line txt file that would work with Peter's Reformer's example? And also, I wanted to ask if you guys support Google Colab TPUs in any way as Reformer would train slower on GPUs?

I hope this makes sense.

Thank you for your help/time and understanding.

Alex






________________________________
From: Lysandre Debut <notifications@github.com>
Sent: Friday, January 22, 2021 7:02 AM
To: huggingface/transformers <transformers@noreply.github.com>
Cc: Alex <asigalov61@hotmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [huggingface/transformers] [WIP][Research projects] Add folder for Music AI / Music Transformers (#9337)


Hi! Could you let us know which docs/instructions were unclear? What were you trying to do, how can we help out? Thanks.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/huggingface/transformers/pull/9337#issuecomment-765464464>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANNXLI3YECRJSE7MSZGRUA3S3GHO3ANCNFSM4VNBN4YQ>.


Thank you for your feedback, there are indeed some aspects of the documentation which are lacking on which we are actively working.

Regarding what you mention, maybe these links can help you:

> how to work with custom text datasets.

We actually have an entire page dedicated to that aspect: [custom datasets](https://huggingface.co/transformers/custom_datasets.html?highlight=custom%20text)

> Another example would be the lack of complete notebooks/code. Like in the Reformer notebook [...] there is not a single word about SentencePiece which was used to create Crime and Punishment tokenizer model.

Indeed! If I may point you to other notebooks, the first of our official notebooks (hosted on this repository, see the `notebooks` folder at the root) is on training a tokenizer: [01-training-tokenizers](https://github.com/huggingface/transformers/blob/master/notebooks/01-training-tokenizers.ipynb). It is also visible from our [notebook documentation page](https://huggingface.co/transformers/notebooks.html).

Training a tokenizer isn't done by the Transformers library in itself, but by the [Tokenizers library](https://github.com/huggingface/tokenizers). I invite you to check their [documentation](https://huggingface.co/docs/tokenizers/python/latest/quicktour.html) which contains a lot of information regarding training tokenizers.

> Also, I could not find a single example for GPT2 models + text, also in a basic and easy to use/try format.

If I may point you to the following parts of the documentation:
- The [GPT-2 reference](https://huggingface.co/transformers/model_doc/gpt2.html) contains several snippets on how to to use GPT-2 models with text
- The [generation utils](https://huggingface.co/transformers/internal/generation_utils.html#utilities-for-generation) showcase how to leverage GPT-2 to generate text. 
- Checking the documentation regarding the [generate](https://huggingface.co/transformers/main_classes/model.html?highlight=generate#generation) method would probably be of interest as it has an identical API on all models - even if the code examples do not showcase GPT-2 directly.
- Several [GPT-2-based notebooks](https://huggingface.co/transformers/notebooks.html) are available on the notebooks page in the documentation
- Finally, we focus on having an identical API between all models. Checking a guide on how to generate text has very good chances of working for any other models. The quickstart on generating text [available here](https://huggingface.co/transformers/task_summary.html#text-generation) could be of use to you, as you simply need to replace the identifier of the model checkpoint by the GPT-2 checkpoint you're interested in.

> To be very specific and relevant to the subject at hand, I need to know how to process and tokenize a simple line-by-line txt file that would work with Peter's Reformer's example? And also, I wanted to ask if you guys support Google Colab TPUs in any way as Reformer would train slower on GPUs?

See below for some pointers on how you can achieve this:
- If you need to train a tokenizer, I invite you to check out the first notebook I mention: [01-training-tokenizers](https://github.com/huggingface/transformers/blob/master/notebooks/01-training-tokenizers.ipynb)
- Once you have your tokenizer you can train your model by either loading your dataset in `datasets` as it is shown in Patrick's notebook (and is simpler! in that case you may be interested in [loading a dataset from a local file](https://huggingface.co/docs/datasets/loading_datasets.html#loading-from-local-files)), or you can load a text file as it is shown in the [custom datasets](https://huggingface.co/transformers/custom_datasets.html?highlight=custom%20text) documentation.

Regarding TPU training, @patrickvonplaten can chime in about the Reformer especially.


Thanks once again for your feedback.

@LysandreJik Thank you very much for the most detailed and helpful info. Much appreciated and I will definitely check it all out in a little bit as you have suggested quite a lot. This will be very useful to me and I am really looking forward to contributing to the huggingface community however I can :)

Most sincerely,

Alex

Great! We're looking forward to your contributions :) Let us know if we can help down the road.

@LysandreJik 

Thank you for the welcome and offer to help. Much appreciated.

You can indeed help as I have run into problems pretty quickly...

So I have spent a few hours trying to make Peter's Reformer colab work with my dataset but to no avail, unfortunately...

Here is the colab: https://colab.research.google.com/drive/1R8jkADMi0vRDwaNTEz_XGQkZUsg_tm_p?usp=sharing

No matter what I do or try, I get errors on training execution... I think I have loaded the dataset correctly but I most certainly can be mistaken...I know that Peter's colab works with default CP setup but I can't make it work just yet....

I saved the output/errors in the colab so that you (or anyone else can take a look) + I am attaching my dataset for you to check out...

Now, I know that my dataset may not be perfect/compatible with Peter's implementation due to encoding and because it is a music dataset...so I am aware that it may not be that all straightforward in this particular case...

@LysandreJik if you can help/suggest something here, I will really appreciate it as I really want to make it work for you guys...

[Efficient-Virtuoso-Music-TXT-Dataset.zip](https://github.com/huggingface/transformers/files/5882917/Efficient-Virtuoso-Music-TXT-Dataset.zip)


P.S. Two questions: 

1) Can you enable Discussions on your repo...its a new GitHub feature and I think it would be a much better place for this kind of discussion/help/support questions??? Or if there is a place already that you prefer, we can move there with this...

2) Any news on the Performer implementations??? It is the latest and the greatest from Google and I already tried it with other's people implementations because it may be more suitable for music than Reformer + its brand new (like 6 month old)...

Thank you for your time and responses.

Alex.





This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.