GPT-2 trained on 24 yrs of US patent grants
Author: goji-patai
Pull request: no

### Model description

Hi,
    I have a GPT2 model available that was made from scratch and has been trained on 1976-2000 US patent grants ~>1M docs.  I think it is a cool/useful  example of Huggingface's gpt2 implementation.  I continue to update this model as I get new data.  I have a local streamlit implementation with greedy and beam searches.  Top-k, top_p and temperature variables are randomized within optimized ranges .  Should you accept to implement it, I will give you the model, and a streamlit implementation to be open-source.
    What this model can and cannot do:
Can:
    -Given an invention idea e.g. (totally made up)"an ultrasonic toothbrush with sensors to detect dental caries..." it gives intriguing results
    - it is not limited to one tech.  The training data was randomly sampled so as to cover a large tech space.
    -It gives even more intriguing results when prompted by a preamble (first ~ 7-10 words) of an existing granted patent.  In a non-scientific or exhaustive testing of this model, I have gotten generated text similar to inventions that were applied for and granted several years later than 2000.  This observation may or may not be generalizable to other fields of tech.
Cannot:
    -Prompts not grounded in the laws of physics as we currently understand them will generate unsatisfying results.  So, no perpetual machines, flying carpets, or unicorns.  Additionally, there are certain subject matters that cannot be patented by law in the US.  Prompts about these subject matters will not give meaningful responses.
    -it is limited by its training to those ideas that have been patented until 2000. 


Best regards,
Gojeb Frehywot
gfrehywot@gmail.com

P.S.  Should you implement this model as you have done for other gpt2 models, I have no interest in any possible patentable invention a user might generate/come up with

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

My name is Gojeb Frehywot.  I have a streamlit implementation of this text generating model on my local machine.  I do not use a repository on GITHUB.  
Can upload to GITHUB do so if you are interested to look into this request further.
Thank you.
Gojeb Frehywot
gfrehyowt@gmail.com

Comments:
Hi @goji-patai - cool model! 

This kind of post is best shared in places like our forum's [Show and Tell](https://discuss.huggingface.co/c/show-and-tell/65) section, or the `i-made-this` channel in our [discord](https://t.co/1n75wi976V?amp=1). We try to reserve the issues for feature requests and bug reports. 

Anyone can add their [model to the hub](https://huggingface.co/docs/hub/models-uploading). There you should be able to build a demo which you can share with others too. 



<!--
/* Font Definitions */
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	font-size:11.0pt;
	font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
code
	{mso-style-priority:99;
	font-family:"Courier New";}
.MsoChpDefault
	{mso-style-type:export-only;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
-->Thank you.  Will do. Sent from Mail for Windows From: amyerobertsSent: Wednesday, April 19, 2023 1:13 PMTo: huggingface/transformersCc: GEMIC; MentionSubject: Re: [huggingface/transformers] GPT-2 trained on 24 yrs of US patent grants (Issue #22863) Hi @goji-patai - cool model!This kind of post is best shared in places like our forum's Show and Tell section, or the i-made-this channel in our discord. We try to reserve the issues for feature requests and bug reports.Anyone can add their model to the hub. There you should be able to build a demo which you can share with others too.—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***> 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.