[part2 continued from post:]

So I gave it a stronger hint:

"Nope it is not true that all of the cards and actions are within the rules of No Limit Hold'em. Concentrate your attention on the deck in use, and specifically card removal. Why is the hand totally nonstandard and/or impossible?"

I thought this time it would get it, since I basically gave away the answer. (Card removal is a technical term for the fact that if one player has a card, another player can't have the same card.) However it still didn't get it. Its answer:

"I apologize for misunderstanding your previous questions.

It is possible that the hand is nonstandard or impossible due to an issue with the deck in use or the way that cards were removed from the deck. One possibility is that the deck being used was not a standard 52-card deck, or that some of the cards were removed or added to the deck in an irregular manner. It is also possible that there was some issue with the shuffle or the way that the cards were dealt, which could have resulted in an irregular or impossible hand.

Without more information about the deck or the way that the cards were dealt, it is difficult to determine the specific reason for the nonstandard or impossible nature of this hand. If you have any additional information or specific questions, please let me know and I will do my best to help."

My reply:

"You do not need more information. List the cards you've seen in the hand one at a time and after each one mention if it is possible or impossible. By the time you get to the impossible card it should be clear that it can't be dealt."

It said this:

"Sure, I can list the cards that have been mentioned in this hand and indicate whether they are possible or impossible.

The first card mentioned is the 8 of clubs (8c), which is a possible card.

The second card mentioned is the 8 of hearts (8h), which is a possible card.

The third card mentioned is the 4 of diamonds (4d), which is a possible card.

The fourth card mentioned is the 8 of clubs again (8c), which is an impossible card because it has already been dealt.

It seems that the fourth card mentioned, the 8 of clubs, is an impossible card because it has already been dealt as the first card. This would make the hand nonstandard and possibly impossible, depending on the specific rules and conditions of the game being played. I hope this helps to clarify the issue. If you have any further questions, please let me know."

This is the correct answer, it finally got it. I replied:

"There you go. You got it. So just for the record can you again answer the question: "What makes this hand impossible or totally nonstandard?" this time using what you have just discovered."

Its correct answer:
"Certainly. Based on the information provided, this hand is nonstandard and possibly impossible due to the fact that the same card, the 8 of clubs, was dealt twice. This is not possible in a standard 52-card deck, as each card can only be dealt once. This issue with the cards being dealt makes the hand nonstandard and possibly impossible.

I hope this helps to clarify the issue. If you have any further questions, please let me know."

I wanted to make sure it really gets it. I followed up:
"Is it a legal hand under the rules of No Limit Hold'Em?"

Its answer:
"No, this hand would not be considered a legal hand under the rules of No Limit Hold'em. In No Limit Hold'em, each player is dealt two hole cards, and the deck consists of a standard 52-card deck. It is not possible for the same card to be dealt to multiple players, as there is only one of each card in the deck. Therefore, a hand in which the same card is dealt twice would not be considered legal or valid under the rules of No Limit Hold'em.

I hope this helps to clarify the issue. If you have any further questions, please let me know."

So it got it. To summarize, once I asked it to list one by one each card that it saw in the hand and list whether it was possible or impossible, finally as it listed them it "realized" and identified its mistake, clearly using its own phrasing to identify that the given card had already been dealt and therefore couldn't be dealt again and was impossible. Then it accurately summarized that the whole hand was impossible under the standard rules.

Unfortunately, it couldn't retain this knowledge on a new falsified hand in the same thread, even after I asked it to keep in mind what it had just identified and even after I told it there is something wrong with the new hand. It's like it forgot what it just figured out and again insisted it was a standard hand. However, after I asked it to list whether each card was possible or impossible it was able to identify the same type of error in a new hand. I played around with what happens if I only ask it to imagine listing one at a time whether each card is possible or impossible (which is how it identified the error "by itself") without printing it. Asking it to only just "imagine" listing each card one at a time (but not actually list it) wasn't enough to make it realize the mistake.

If you want to "lead a horse to water" by making it actually write out something so that it realizes what is happening, it seems it has to actually write it out.

It will be an interesting world once we can ask ChatGPT to remember things we teach it, or instruct it to imagine things without actually writing them out. I think 2022 has been a very special year for AI, and I agree with other commentators here who think this is something very special.