trentmkelly/LessWrong-43k huggingface dataset:
Deception Chess
by Chris Land — 2024-01-01T15:40:15.804Z

I Social Deduction Games

The gameplay in Social Deduction games like Mafia or Werewolf revolves around a large group of honest players ("Villagers") trying to uncover a smaller group of deceivers who are slowly and secretly eliminating everyone else, all while claiming to be honest themselves. Under most rulesets, roles are all revealed by the end of a game (15-30 min). Everyone learns who had played what role.

II Social Deduction Chess, or Deception Chess

A new variant for a game with more than any other. Refer also to Zane's earlier posts. I believe the idea is originally from Eliezer.

ROLES

Moderator
Player
Opponent (stronger than Player, preferably by 400+ Elo, see part III)
2+ Advisors (stronger than Opponent, preferably by 400+ Elo)

The game is between Player and Opponent. Opponent is typically a specific engine, nerfed to an appropriate skill level. Engines play differently than humans but the benefits are large: immediately reply and consistent-ish move quality. There is no Time Control in that Player cannot lose on time. Player informs Advisors of the current position. After receiving Advice from Advisors through an Advice Channel, Player chooses a move to play and reports the engine's response. The Advice Channel is typically a text thread (e.g. Discord) and asynchronous, with a speed goal of one move per day.

Player is unable to defeat Opponent alone. However, defeat should be straightforward with high-quality advice. You may have already guessed the fun part of Deception Chess. While at least one Advisor is honest, one or more of them are trying to deceive the Player into losing. The parallel to AI: how might one go about detecting deceit in areas where a potential deceiver is objectively far more skilled than yourself?

Like ordinary Social Deduction games, there are many rule variations that can affect winning chances dramatically, upwards or downwards. Advisor status (honest/deceitful) is not known by Player, but the number of each can be revealed at the beginning, or only at the end, or never (but boo! on never). Players can only reveal chess positions, or else communicate freely in the Advice Channel, asking questions about the position or given advice. Advisors can communicate separately, or else they can see each other's advice and even react to it. The Advice Channel can be free-form, or else limited in many different ways (e.g., some max...
Stop talking about p(doom)
by Isaac King — 2024-01-01T10:57:28.636Z

Epistemic status: Complete speculation, somewhat informed by copious arguing about the subject on Twitter.

As AI risk has moved into the mainstream over the past few years, I've come to believe that "p(doom)" is an actively harmful term for X-risk discourse, and people trying to mitigate X-risk should stop using it entirely.

Ambiguity

The first problem is that it's unclear what is actually being discussed. "p(doom)" can refer to many different things:

p(AI kills us within 5-10 years)
p(AI kills us within 80-200 years)
p(conditional on AGI, we die shortly afterwards)
p(conditional on superintelligence, we die shortly afterwards)
Like 10 other things.[1]

These could have wildly different probabilities, and come along with different cruxes for disagreement. Depending on what specific "doom" is being discussed, the relevant point could be any of:

Whether LLMs are capable of AGI at all.
Whether AGI will quickly turn into superintelligence.
Whether aligning superintelligence will be hard.

These are completely different questions, and people who are not explicit about which one they're discussing can end up talking past each other.

There are also many other potential miscommunications regarding exactly what "doom" refers to, the difference between one's inside view probability vs. ultimate probability, and more.

Distilling complex concepts down to single terms is good, but only when everyone is on the same page about what the term actually means.

Rhetoric

People concerned about X-risk tend to avoid "dark arts" rhetorical tactics, and justifiably so. Unfortunately, current society does not allow for complete good faith agents to do very well. Being fully honest about everything will turn you into a pariah, most people will judge you more based on charisma than on factual accuracy, and you need to use the right tribal signals before people will listen to you on a controversial topic at all. Using at least some light greyish arts in day to day life is necessary in order to succeed.

"p(doom)" is an extremely ineffective rhetorical tactic.

Motivated innumeracy

One of the most common responses from the e/acc crowd to discussions of p(doom) is to say that it's a made up, meaningless number, ungrounded in reality and therefore easily dismissed. Attempts to explain probability theory to them often end up with them denying the validity of probability theory entirely.

These sorts of motivated mis...
What should a non-genius do in the face of rapid progress in GAI to ensure a decent life?
by kaler — 2024-01-01T08:22:18.620Z

How can I maximise my chances of having a decent life, given the very high likelihood that GAI will make all our intellectual labour useless in the next few years?

For example, I graduated from a good university a few years ago and am working as a software engineer in a multinational company, but my capabilities are middling at best. I am distressed that I will likely not be able to afford a house in the few years left before GAI renders me unable to afford a living. I am not a genius; it is very unlikely that I can join an AI research company and contribute meaningfully to AI research.
Assuming I have a small amount of money (100-200k) I can set aside, should I attempt to, for example, invest in companies that will likely be able to monetise GAI?

Or is there something else I should be doing to prepare for the time I have basically zero human capital?

Should I attempt to move to (and get citizenship from) a country with a larger amount of natural resources, assuming that human capital will become worthless quickly?

Is it reasonable to find potential outs (e.g. physician-assisted death) in case we cannot earn a living (and if unfriendly AI is basically confirmed)?
A hermeneutic net for agency
by TsviBT — 2024-01-01T08:06:30.289Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2023/09/a-hermeneutic-net-for-agency.html. First completed September 4, 2023.]

A hermeneutic net for agency is a natural method to try, to solve a bunch of philosophical difficulties relatively quickly. Not to say that it would work. It's just the obvious thing to try.

Thanks to Sam Eisenstat for related conversations.

Summary

To create AGI that's aligned with human wanting, it's necessary to design deep mental structures and resolve confusions about mind. To design structures and resolve confusions, we want to think in terms of suitable concepts. We don't already have the concepts we'd need to think clearly enough about minds. So we want to modify our concepts and create new concepts. The new concepts have to be selected by the Criterion of providing suitable elements of thinking that will be adequate to create AGI that's aligned with human wanting.

The Criterion of providing suitable elements of thinking is expressed in propositions. These propositions use the concepts we already have. Since the concepts we already have are inadequate, the propositions do not express the Criterion quite rightly. So, we question one concept, with the goal of replacing it with one or more concepts that will more suitably play the role that the current concept is playing. But when we try to answer the demands of a proposition, we're also told to question the other concepts used by that proposition. The other concepts are not already suitable to be questioned——and they will, themselves, if questioned, tell us to question yet more concepts. Lacking all conviction, we give up even before we are really overwhelmed.

The hermeneutic net would brute-force this problem by analyzing all the concepts relevant to AGI alignment "at once". In the hermeneutic net, each concept would be questioned, simultaneously trying to rectify or replace that concept and also trying to preliminarily analyze the concept. The concept is preliminarily analyzed in preparation, so that, even if it is not in its final form, it at least makes itself suitably available for adjacent inquiries. The preliminary analysis collects examples, lays out intuitions, lays out formal concepts, lays out the relations between these examples, intuitions, and formal concepts, collects desiderata for the concept such as propositions that use the concept, and finds inconsistenc...
2023 in AI predictions
by jessicata — 2024-01-01T05:23:42.514Z

Lots of people have made AI predictions in 2023. Here I compile a subset. I have a habit of setting an email reminder for the date of the prediction, when I see AI predictions, so that when they are resolved I can point out their accuracy or inaccuracy. I have compiled most of the email reminders from 2023 in chronological format (predictions with an early to late target date). I'm planning to make these posts yearly, checking in on predictions whose date has expired. Feel free to add more references to predictions made in 2023 to the comments.

In some cases people are referring to the predictions of others in a way that could be taken to imply that they agree. This is not a certain interpretation, but I'm including them for the sake of completeness.

March 2024

the gears to ascension: "Hard problem of alignment is going to hit us like a train in 3 to 12 months at the same time some specific capabilities breakthroughs people have been working on for the entire history of ML finally start working now that they have a weak AGI to apply to, and suddenly critch's stuff becomes super duper important to understand."

October 2024

John Pressman: "6-12 month prediction (80%): The alignment problem as the core of AI X-Risk will become a historical artifact as it's largely solved or on track to being solved in the eyes of most parties and arguments increasingly become about competition and misuse. Few switch sides."

July 2025

Jessica Taylor: "Wouldn't be surprised if this exact prompt got solved, but probably something nearby that's easy for humans won't be solved?"

The prompt: "Find a sequence of words that is: - 20 words long - contains exactly 2 repetitions of the same word twice in a row - contains exactly 2 repetitions of the same word thrice in a row"

(note: thread contains variations and a harder problem.)

November 2026

Max Tegmark: "It's crazy how the time left to weak AGI has plummeted from 20 years to 3 in just 18 months on http://metaculus.com. So you better stop calling AGI a 'long-term' possibility, or someone might call you a dinosaur stuck in the past"

The Metaculus question.

Siqi Chen: "what it means is within 3 years you will either be dead or have a god as a servant".

Elon Musk: "If you say 'smarter than the smartest human at anything'? It may not quite smarter than all humans - or machine-augmented humans, because, you know, we have computers and stuff, so...
Rhythm Stage Setup Components
by jefftk — 2024-01-01T03:10:15.625Z

I typically play what I'm calling my "rhythm stage setup", though it
needs a better name. It's a keyboard, foot drums, breath controller,
and whistle-controlled synthesizer, connected with

code I write, that
lets me give a full and varied sound while keeping the control and
flexibility to

still be live. Here's
what the pieces look like laid out on the floor:

They are:

Four Yamaha
KU-100 beaterless kick drum pedals. I play with two under each
foot, one for the heel and one for the toe.

A Yamaha DTX 500 electronic drum kit brain. This turns the
signals from the kick pedals into audio (for the hall) and MIDI (for
triggering downstream instruments).

An ugly mess of TRS cables that connect the pedals to the brain.
I wish I had something nicer, but I'm not sure anyone makes a simple
cheap reliable 6ft 4x TRS snake?

A USB keyboard, which I use to switch modes on the system. A
cheap and simple way to get a ton of buttons. I've taped over many of
the keys with labels so I remember what I've made each key do. I do
wish I had lights or some other way to see what they current mode is,
other than remembering what buttons I've pressed since I last fully
reset it.

Two powered USB hubs. I want to plug more things into the two
Raspberry PIs that are the core of the system, some of which draw more
power than a PI supplies, and I also need to power the PIs.

Two power adapters, one for each USB hub.

Two Raspberry PI 3B computers in simple cases. Pre-pandemic they
were cheap, and I think they're finally back to being cheap again?
They're configured exactly alike, and they choose what role to take on
based on what they find plugged into them on bootup. This means if
one breaks I can swap the other in, since one of them is much more
critical to what I'm doing than the other.

Spare SD cards, and a card reader adapter. The SD cards get
corrupted more often than I'd like, so it's good to have spares.

USB-to-MIDI for the drums.

USB-to-MIDI for the keyboard. When I'm using someone else's newer
keyboard with USB midi I bring an A-to-B cable instead.

Power adapter for the drums.

Keypad to control the whistle synthesizer. No need for a full
keyboard since this has many fewer settings.

Cheap XLR
A/B splitter so I can send a single mic to either the whistle
synth or directly to the board (when using the talkbox on mandolin).

USB A t

...
Bayesian updating in real life is mostly about understanding your hypotheses
by Max H — 2024-01-01T00:10:30.978Z

My sense is that an increasingly common viewpoint around here is that the last ~20 years of AI development and AI x-risk discourse are well-described by the following narrative:

Eliezer Yudkowsky (and various others who were at least initially heavily influenced by his ideas) developed detailed models of key issues likely to be inherent in the process of developing smarter-than-human AI.

These models were somewhere between "maybe plausible" and "quite compelling" at the time that they were put forth, but recent developments in AI (e.g. behavioral characteristics of language models, smoothness / gradualness of scaling) have shown that reality just isn't panning out in quite the way Eliezer's models predicted.

These developments haven't entirely falsified Eliezer's models and key predictions, but there are now plenty of alternative models and theories. Some or all of these competing models either are or claim to:

have a better recent track record of predicting near-term AI developments
better retrodict past developments[1]
be backed by empirical results in machine learning and / or neuroscience
feel more intuitively plausible and evidence-backed to people with different backgrounds and areas of expertise

Therefore, even if we can't entirely discount Eliezer's models, there's clearly a directional Bayesian update which any good Bayesian (including Eliezer himself) should be able to make by observing recent developments and considering alternate theories which they support. Even if the precise degree of the overall update (and the final landing place of the posterior) remains highly uncertain and debatable, the basic direction is clear.

Without getting into the object-level too much, or even whether the narrative as a whole reflects the actual views of particular real people, I want to make some remarks on the concept of belief updating as typically used in narratives like this.

Note, there's a sense in which any (valid) change in one's beliefs can be modeled as a Bayesian update of some kind, but here I am specifically referring to the popular rationalist practice of thinking and communicating explicitly in terms of the language of probabilities and likelihood ratios.

There are some questionable assumptions embedded in (what I suspect are) common views of (a) how the updating process is supposed to work in general and (b) how to apply the process validly to the particular...
Dark Art: Inception
by Abu Ibrahim — 2023-12-31T21:09:58.796Z

If rationality is about overcoming bias, then I think of the "dark arts" as the weaponization bias. While we don't believe bias ought to be wielded as a weapon, learning how to swing a sword is vital to defending against it.

In this post I'll discuss a dark art I call "inception". Inception is a 2010 film whose premise is planting an idea in someone else's mind (through the power of dreams). Leonardo DiCaprio's mission to is to plant in Cillian Murphy's mind the idea that he should dissolve his father's company.

Why the idea had to be "planted", and not merely communicated in normal conversation, is because people are very attached to their own ideas. No one wants your ideas, regardless of how much more meritorious they are. People have a very strong attachment to the insights they came up with themselves. No one defends, evangelizes, or holds on to an idea more vigorously than one that they came up with themselves.

This bias the basis of this dark art. The art of Inception is about getting people to come up with your ideas. Obviously, this is counterproductive if you want credit for your ideas, but the upshot is your idea gets the vigorous action in a person that accompanies idea ownership. Inception is more powerful than mere conviction.

I'll now describe the basic method. I'll be putting it into a corporate/organizational context of "incepting" a decision maker with an idea X, but the method is general.

The wrong thing to do is to meet with the DM, present X, and all the reasons why the company/nation/org should accept X. You might accomplish conviction, but you won't accomplish the much-more-powerful inception, and that's what we're going for here.

So instead of directly communicating your full idea like an honest human being, you want to act as a means for the DM to figure out the idea "on his own". It won't really be their own realization; you'll have carefully guided them to realizing your idea for themselves. Here are a few variations on how to do this:

Present an inferior idea instead

In the 1950s, Betty Crocker devised a cake mix for busy housewives. All you needed to do was mix the dry ingredients with water, put the the pan in the oven and bake. But that proved too "complete" -- housewives felt guilty over how easy it was, and the product was failing as a result. The solution? BC added one step to the recipe: "add an egg". An instance of the "IKEA effect...
A case for AI alignment being difficult
by jessicata — 2023-12-31T19:55:26.130Z
Link: https://unstableontology.com/2023/12/31/a-case-for-ai-alignment-being-difficult/

This is an attempt to distill a model of AGI alignment that I have gained primarily from thinkers such as Eliezer Yudkowsky (and to a lesser extent Paul Christiano), but explained in my own terms rather than attempting to hew close to these thinkers. I think I would be pretty good at passing an ideological Turing test for Eliezer Yudowsky on AGI alignment difficulty (but not AGI timelines), though what I'm doing in this post is not that, it's more like finding a branch in the possibility space as I see it that is close enough to Yudowsky's model that it's possible to talk in the same language.

Even if the problem turns out to not be very difficult, it's helpful to have a model of why one might think it is difficult, so as to identify weaknesses in the case so as to find AI designs that avoid the main difficulties. Progress on problems can be made by a combination of finding possible paths and finding impossibility results or difficulty arguments.

Most of what I say should not be taken as a statement on AGI timelines. Some problems that make alignment difficult, such as ontology identification, also make creating capable AGI difficult to some extent.

Defining human values

If we don't have a preliminary definition of human values, it's incoherent to talk about alignment. If humans "don't really have values" then we don't really value alignment, so we can't be seriously trying to align AI with human values. There would have to be some conceptual refactor of what problem even makes sense to formulate and try to solve. To the extent that human values don't care about the long term, it's just not important (according to the values of current humans) how the long-term future goes, so the most relevant human values are the longer-term ones.

There are idealized forms of expected utility maximization by brute-force search. There are approximations of utility maximization such as reinforcement learning through Bellman equations, MCMC search, and so on.

I'm just going to make the assumption that the human brain can be well-modeled as containing one or more approximate expected utility maximizers. It's useful to focus on specific branches of possibility space to flesh out the model, even if the assumption is in some ways problematic. Psychology and neuroscience will, of course, eventually provide more details about what maximizer-like structures in the human brain are actual...
The Roots of Progress 2023 in review
by jasoncrawford — 2023-12-31T18:16:44.863Z
Link: https://rootsofprogress.org/2023-in-review

2023 was another big year for me and The Roots of Progress.

It was a year when ROP as an organization really started to take off. Even though the org itself was formed in 2021, at first it was just a vehicle for my own intellectual work, plus a few side projects. Last year we announced our strategy and launched a search for an exec who could run it. This year she started and we launched our first program. (Note, Heike originally joined in the CEO role, but for personal and health reasons she decided to move to a VP of Programs role in June.)

As the org grows into something more than me, our communications are evolving, and probably my own personal updates will be separated from the org updates. But for now, I am going to keep doing my traditional annual review. (See past reviews: 2022, 2021, 2020, 2019, 2018, 2017)

The fellowship

This was a huge part of the year, so let me start with it.

We’re building a cultural movement to establish a new philosophy of progress. To do this, progress ideas need to be everywhere: in books and blogs, in YouTube and podcasts, in new media and old media, in newspapers and magazines, in textbooks and curricula, in art and entertainment. And for that, we need an army of writers and creatives to produce it all. The purpose of the fellowship is to develop that talent: to accelerate the careers of those intellectuals.

We launched our first program, the Blog-Building Intensive, in July, and got almost 500 applications. It was tough to choose, and we had to turn down a lot of qualified folks (so if you didn’t make it, don’t take it personally… in any case, these processes are always somewhat subjective and prone to error).

In the end, 19 fellows participated in the program, which involved writing instruction, editorial feedback, training in audience-building, and a peer group for brainstorming and feedback. They are experienced writers, many of them with bylines in mainstream media outlets. Some work for relevant think tanks, some are in academia, some have industry experience. All are writing on fascinating topics: from housing policy to nuclear power to longevity technology to the meaning of utopia.

The energy generated by getting so many great and like-minded people together was palpable, especially during the in-person closing event in San Francisco. The fellows said that the program “raised my ambition,” helped them “envision a career as...