# A test of your rationality skills

*\[Added April 26: This post has received a fair amount of mixed-to-negative comments and feedback. I'm still not entirely convinced that this isn't a good test, but I retract my request not to comment on the object-level question here or elsewhere.\]*

  
This post describes an exercise that tests some aspects of your rationality skills. In particular, it tests for the ability to find and sift through evidence about a complex real-world event, reason probabalistically, draw conclusions, and communicate your reasoning process in writing.

If left unspoiled, it could potentially be used as part of a job application process for a job where these skills are important, such as a research role at an effective altruism or AI alignment organization.

In other words, this is a proposal for a [verification method for rationality skills at the organizational level](https://www.lesswrong.com/posts/5K7CMa6dEL7TN7sae/3-levels-of-rationality-verification).

The exercise is similar to the [Amanda Knox test](https://www.lesswrong.com/posts/mkAcXPEJ7RZCJs8ry/you-be-the-jury-survey-on-a-current-event) from over a decade ago. A [conclusion](https://www.lesswrong.com/posts/G9dptrW9CJi7wNg3b/the-amanda-knox-test-how-an-hour-on-the-internet-beats-a) and [post-mortem](https://www.lesswrong.com/posts/Jx4gGbPi7GuydwvzB/amanda-knox-post-mortem) from that test are good background reading, though I believe this one is somewhat more difficult and more time-consuming to give a good answer for.

The exercise
------------

In short, figure out what happened during [a controversial high-stakes poker game live-streamed in September 2022](https://hustlercasinolive.com/j4report/). In particular, come up with a probability estimate that Garrett Adelstein or other players were cheated, and if so, how and by whom.

Write up your reasoning and your research process, discussing how you weighted different pieces of evidence and why.

More background, from a [report](https://hustlercasinolive.com/j4report/) on the incident by a consulting firm hired by the casino where the incident occurred:

> On September 29, 2022, one of the most controversial hands in poker history was played on an episode of “Hustler Casino Live,” which streams high-stakes poker games played at Hustler Casino in Gardena, Calif., to a worldwide audience on its YouTube Channel. The hand has subsequently been viewed by hundreds of thousands, if not millions, of poker fans around the globe. In the hand, recreational poker player Robbi Jade Lew called an all-in shove by Garrett Adelstein for her remaining $109,000 with the jack of clubs and four of hearts on a board reading ThTc9c3h. Lew ended up winning a pot of $269,000 when her jack high held against Adelstein’s eight of clubs and seven of clubs. After the hand, Adelstein said he was suspicious of Lew’s play and she agreed to return half the pot to him. In the weeks that have followed, Lew has repeatedly denied wrongdoing and requested that Adelstein return her winnings. In October, Adelstein published a series of allegations on the Two Plus Two poker forum that he said proved Lew “was very likely part of a cheating ring of at least three members,” which he said likely included Lew, a friend of hers who was playing that night and an employee of the company that produces the show. Adelstein and others in the poker community said that there was no logical reason why Lew would call a bet of that size without inside knowledge that she had the better hand. Her hand would lose to any pair as well as many potential bluffs, including ace-high, king-high, queen-high or better jack-high hands. In fact, Adelstein’s exact hand was one of the few combinations of hands that he could have had that she would beat. Lew offered several explanations for her call on Sept. 29 and in the weeks that followed; most of her explanations centered on the fact that she thought Adelstein was bluffing, or as she said on Sept. 29, “You don’t have shit.”

(Full report [here](https://hustlercasinolive.com/j4report/), which goes into great detail about the technical aspects of the casino and livestream security system. There may be better introductions to the overall incident elsewhere online. )

Why this is a good test
-----------------------

There are piles of footage, interviews, commentary and other kinds of evidence from many different sources, including those directly involved. It's not obvious what actually happened or how to weigh up all the evidence, and I believe there is no real consensus in the poker world on matter.

Understanding all the background information about the incident requires a bit of familiarity with poker and the high-stakes scene, but it's probably not too hard to get up to speed, and the ability to quickly learn about an unfamiliar domain is also an important rationality skill.

Many in the poker world who followed the incident closely at the time hold strong and differing views on the matter. I believe the "official" conclusion was that there is insufficient evidence to accuse Robbi of cheating, and she is still invited to play in various high-stakes games.

Plenty of people have posted analyses and their own conclusions about what they think happened. Poker players tend to be more rational and think in terms of probabilities more than the average person, but many of the popular analyses I've seen have been lacking in basic rationality skills and applications of Bayes theorem.

Relative to the Amanda Knox case, this seems less politically charged and lower-stakes; probably no one is going to jail over the matter at this point no matter what, and the poker world has mostly moved on. Speculating about this in private or in rationality spaces is unlikely to have any negative reputational consequences for the people involved or for LessWrong or EA, though if someone does do a good investigation and posts a conclusion here publicly, it might attract unwanted attention from the high-stakes poker community or others who followed this incident closely.

Digging into this for its own sake might be interesting, but seems pretty low-value to me; I think this question is most useful as a test or exercise, and there might be rationality organizations that would be interested in using it as an actual test or as part of a job application process. ~~**To preserve the value of the question as a rationality test, please refrain from posting your own object-level thoughts or conclusions about the incident in the comments here or elsewhere publicly, at least for the time being.**~~ **\[edit: retracted, see above.\]** Linking to existing publicly-available analyses by non-rationalists is OK.

Comments on whether you think the exercise is useful, how it could be improved, resources for getting up to speed on the incident or general background information, and other ideas for similar exercises are all very welcome.