> I think it would come across clearer if you focus more on the hard facts about whether and how AI chatbots' models are actually modified in response to queries. My naive assumption was that the bots are trained on preexisting sets of data and users' queries were essentially being run on a read-only tool

Thank you, noted.

Every time you press the "regenerate" or "thumbs down" button in ChatGPT, your feedback is training data. My level of abstraction here is just the term "reinforcement learning from human feedback", not the specific details of how that does its thing. I believe they also have some quick automated test of user sentiment, but that belief is due to it being an easy thing that seems obvious, not because I've read a blog post about it.

https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

> Engaging in comparing the bots to living creatures makes it easy for the argument to fall flat since the technology truly isn't there yet.

We won't know when it does get there.

If you can even design a test that can determine that in principle even if we can't run the test in practice, you'd be doing better than almost everyone on this subject. So far, I've heard only one idea that doesn't seem immediately obviously wrong to me, and even that idea is not yet testable in practice. For example, @andai's test is how smart it looks, which I think is the wrong test because of remembering being foolish when I was a kid, and noticing my own altered states of consciousness and reduced intellect when e.g. very tired, and noticing that I still had the experience of being, and yet other occasions where, for lack of a better term, my tiredness resulted in my experiences ceasing to feel like I was the one having them, that instead I was observing a video of someone else, and that this didn't come with reduced intellect.

That's the point of me turning your own example back at you when asking about dogs: nobody even knows what that question really means yet. Is ${pick a model, any model} less sentient than a dog? Is a dog sentient? What is sentience? Sentience is a map with only the words "here be dragons" written on it, only most people are looking at it and taking the dragons seriously and debating which part of the dragon they live on by comparing the outline of the dragon to the landmarks near them.

For these reasons, I also care about the original question posted by @andai — but to emphasise, this is not because I'm certain the current models pass some magic threshold and suddenly have qualia, but rather because I am certain that whenever they do, nobody will notice, because nobody knows what to look for. An AI with the mind of a dog: how would it be treated? My guess is quite badly[0], even if you can prove absolutely both that it really is the same as a dog's mind, and also that dogs are sentient, have qualia, can suffer, whatever group of words matters to you.

But such things are an entirely separate issue to me compared to "how should we act for our own sakes?"

(I still hope it was just frustration on your part, and not boolean thinking, that means you have still not demonstrated awareness of the huge range of behaviour between what reads to me like swearing (the word "[care]" in square brackets implies polite substitution for the HN audience) and even metaphorical bootlicking; though that you disregard the point about BDSM suggests you did not consider that this word has an actual literal meaning and isn't merely a colourful metaphor).

[0] Why badly? Partly because humans have a sadistic streak, and partly because humans find it very easy to out-group others.