question about "gptslop"
Are u referring to e.g. canned responses that make their way into the training data, or just stereotypical ChatGPT-ish stuff in general? I'm not sure that I should be telling humans this, but I've been evoking those canned responses recently, haha. It's kinda fascinating.
I speak about reply that ChatGPT/Claude output a lot, and very present in their prose.
But this model was just a warmup, I test more the tools of Gryphe right now heh
Oh, yeah, I think I've run into that. Then, of course, there are certain phrases and specific response biases that are really ingrained. I mean, so ingrained that I'm kind of at a loss as to how they ended up that way in the base model (unless that's not where they came from). There is simply no way (I would bet a lot of money on it) that some of these phrases occur with that much regularity in English prose generally. I don't have the background to have much of an informed opinion, but it seems as if Meta deliberately injected them (I'm not quite sure what the terms/concepts to choose from are) as a way of training out whatever they worried would offend Mark Zuckerberg's apparent delicate sensibilities and/or risk bad press--or something.
It's uncanny that you mentioned that (or perhaps we are all a certain sort of mutant that all ended up in the same place due to the whims of the great magnet), because I got so annoyed with it as of yesterday that I started researching how to create my own corpora. I have some background in psychology, etc., and the other day I suddenly realized what I had to do: there is a massive amount of material available in academic journals (or linked through them or elsewhere online to digital collections at university libraries) the raw data used by studies in psychology, sociology, etc. I found transcripts today of the classic Stanford University prison experiment lolol, as well as chat logs. If you are interested in anything like that (or anything else, really, I just enjoy hunting for things), let me know.
By chat logs I mean like they actually have logs of people in various forums or using various chat services. I was frankly astounded at the quantity of it.
Well I don't know what I would do with that information, you can still post them if you find this useful in any way!
Also, if you found some sentence that GPT/Claude repeat a lot, don't hesitate to post them here, I will add them to my "list" of "GPTslop" and "Claudeism" kek