trentmkelly/LessWrong-43k huggingface dataset:
Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?
by Karl von Wendt — 2023-06-25T16:59:49.173Z

On June 22nd, there was a “Munk Debate”, facilitated by the Canadian Aurea Foundation, on the question whether “AI research and development poses an existential threat” (you can watch it here, which I highly recommend). On stage were Yoshua Bengio and Max Tegmark as proponents and Yann LeCun and Melanie Mitchell as opponents of the central thesis. This seems like an excellent opportunity to compare their arguments and the effects they had on the audience, in particular because in the Munk Debate format, the audience gets to vote on the issue before and after the debate.

The vote at the beginning revealed 67% of the audience being pro the existential threat hypothesis and 33% against it. Interestingly, it was also asked if the listeners were prepared to change their minds depending on how the debate went, which 92% answered with “yes”. The moderator later called this extraordinary and a possible record for the format. While this is of course not representative for the general public, it mirrors the high uncertainty that most ordinary people feel about AI and its impacts on our future.

I am of course heavily biased. I would have counted myself among the 8% of people who were unwilling to change their minds, and indeed I’m still convinced that we need to take existential risks from AI very seriously. While Bengio and Tegmark have strong arguments from years of alignment research on their side, LeCun and Mitchell have often made weak claims in public. So I was convinced that Bengio and Tegmark would easily win the debate.

However, when I skipped to the end of the video before watching it, there was an unpleasant surprise waiting for me: at the end of the debate, the audience had seemingly switched to a more skeptical view, with now only 61% accepting an existential threat from AI and 39% dismissing it.

What went wrong? Had Max Tegmark and Yoshua Bengio really lost a debate against two people I hadn’t taken very seriously before? Had the whole debate somehow been biased against them?

As it turned out, things were not so clear. At the end, the voting system apparently broke down, so the audience wasn’t able to vote on the spot. Instead, they were later asked for their vote by email. It is unknown how many people responded, so the difference can well be a random error. However, it does seem to me that LeCun and Mitchell, although clearly having far weaker argument...
AI-Plans.com - a contributable compendium
by Iknownothing — 2023-06-25T14:40:01.414Z
Link: https://ai-plans.com/

Hello, we’re working on

https://ai-plans.com

.

Ideas behind the site:

Right now, alignment plans are spread all over the place and it’s difficult for a layperson, or someone unfamiliar with the field to get an idea of what is the current plan for making AGI or even the models we have right now, safe. And the problems with said plans.

Having a place where all AI Alignment plans and criticism of said plans can be added and seen in an easy to read way is helpful. It offers possibilities such as; seeing the most common problems with plans, which kinds of plans have the least problems, pushing for regulations against the most poor plans etc.

Judging the quality of a plan is hard and has a lot of ways to go wrong. Judging the quality of a criticism might be less complicated and perhaps has less ways it can go wrong.

Which is why I believe this site is useful

Hasn’t this already been done? aisafety.info, aiideas.com, etc

aisafety.info is excellent, and the folks there are building a conversational agent for AI Safety, which could be incredibly useful. However it’s not very simple to use to learn about specific alignment plans yet and is more of a general purpose place to learn about ai, ai-risk, ai-safety, etc.

The purpose of ai-plans.com is to be an easy to read platform that sorts the good plans from the bad and shows the problems with each one.

I believe there was a site called ai-ideas.com or something mentioned to me? But that wasn’t working the last time I checked and there’s been no change, as far as I know.

Aims

Stage 1)

A contributable compendium with most, if not all, of the plans for alignment, and criticisms

Estimated time left for this to be done: 1 week - adding ~5 plans a day now

What’s left to do:

Functionality for responding to criticisms-

The idea for this is that a plan’s author can select a criticism/criticisms that they think they have a solution to, then submit a new version of the plan, with the criticism(s) quoted in the new plan.
Any criticisms they didn’t select, will be automatically added to the new version of the plan (the idea being that any unselected criticisms are ones that they don’t have solutions to, so should still apply).

Adding more plans and criticisms
A filter for spam and duplicates
Creating a template/guide on how to post plans
Improve the UX - font,

...
Map of maps of interesting fields
by MaxG — 2023-06-25T14:02:57.017Z
Link: https://glozematrix.substack.com/p/map-of-maps-of-interesting-fields

I love seeing cool visualizations (maps) of interesting intellectual fields. I’m also a big fan of making lists!

Accordingly, I compiled a bunch of maps of fields that I thought were interesting. Please comment if you know of more such maps, and I’ll include them.

Scott Alexander’s map of Effective Altruism (

2020

)

mariekedev’s mindmap of EA organisations (

2023

)

Hamish Doodles’

aisafety.world

(

2023

)

On a side note, I would be very keen for someone to create a similar map of relevant organizations in the biosecurity & pandemic preparedness space. I plan to post a minimum viable product (bullet point list) soon.

James Lin’s map of biosecurity interventions (

2022

)

Scott Alexander’s map of the rationality community (

2014

)

Dan Elton’s map of progress studies (

2021

)

Ada Nguyen’s map of the Longevity Biotech Landscape (

2023

)

Nadia Asparouhova’s map of climate tribes (

2022

)

Samuel Arbesman’s Catalog of New Types of Research Organizations (

2023

)

This one is more of a long list, but I thought it was very interesting nonetheless!

Honorable mentions

These are other mapping efforts that met my "I'm curious about it" but not my "This is super interesting" bar. Some of them also seem outdated.

xkcd’s map of online communities (2010)
Julia Galef’s map of the Bay Area memespace (2013)
Joe Lightfoot’s The Liminal Web: Mapping An Emergent Subculture Of Sensemakers, Meta-Theorists & Systems Poets (2021 & updated in 2023)

Shoutout to Rival Voices, who created a similar-ish collection of maps (2023), and Nadia Asparouhova, who wrote a more meta-level post about Mapping digital worlds (2023).
Why am I Me?
by dadadarren — 2023-06-25T12:07:03.244Z

At 5, I was hospitalized for a month due to pneumonia. Kids of that age have little fear of illnesses, and the discomfort is soon forgotten. What I still remember though is the intense boredom. It is during that dull month I started asking the question everyone has asked themself: "Among all the people in this world, why am I this particular one?". I still recall the ineffable yet intense feeling when thinking about it for the first time.

Eventually I realized that's a question with no answer. Out of the vast number of things in existence, the fact that I am experiencing the world from the perspective of this particular thing- a human being- has no explanation. Logic and reason are unable to ascribe any underlying cause or rationale. "I am me" is just one fundamental truth that anyone has to take as a given.

Yet this fundamental truth is different for each person. From each of our distinct perspectives, which physical thing is the "I" is different. Keeping track of such differences is both mentally consuming and often unnecessary. So there is a natural affinity to rid of the first-person and to think "objectively". Instead of basing it on one's own point of view, we organize thoughts and formulate arguments from an imaginary vantage point that is detached and impartial, with an immaterial gaze from nowhere.

Though I consider such objectivism merely a shortcut for efficiency which has often been mistakenly regarded as an ideal, there is no denying its practical success. We all use it constantly with great results. Even when we do think about something from our own perspective, we can easily transcode it to the objective. All it seems to take is exchanging the perspective-dependent self - the "I"- for the particular person. So "I'm tall" can become "Dadadarren is tall". This is required since there is no "I" in objective reasoning. It is a gaze from nowhere after all.

We all have performed these transcodings so frequently that they hardly require active thoughts anymore. Every time it subtly reinforces the idea that such transcoding is always possible, so thinking objectively can, some may even say should, supersedes thinking from any single perspective. But in some rare cases, that leads to problems. Anthropic reasoning is such a case.

Take the Doomsday Argument(DA) as an example. It proposes the uninformed prior of one's own birth ra...
Will the growing deer prion epidemic spread to humans? Why not?
by eukaryote — 2023-06-25T04:31:56.824Z
Link: https://eukaryotewritesblog.com/2023/06/24/chronic-wasting-disease/

Helpful background reading: What’s the deal with prions?

A novel lethal infectious neurological disease emerged in American deer a few decades ago. Since then, it’s spread rapidly across the continent. In areas where the disease is found, it can be very common in the deer there.

Map from the Cornell Wildlife Health Lab.

Chronic wasting disease isn’t caused by a bacteria, virus, protist, or worm – it’s a prion, which is a little misshapen version of a protein that occurs naturally in the nervous systems of deer.

Chemically, the prion is made of exactly the same stuff as its regular counterpart – it’s a string of the same amino acids in the same order, just shaped a little differently. Both the prion and its regular version (PrP) are monomers, single units that naturally stack on top of each other or very similar proteins. The prion’s trick is that as other PrP moves to stack atop it, the prion reshapes them – just a little – so that they also become prions. These chains of prions are quite stable, and, over time, they form long, persistent clusters in the tissue of their victims.

We know of only a few prion diseases in humans. They’re caused by random chance misfolds, a genetic predisposition for PrP to misfold into a prion, accidental cross-contamination via medical supplies, or, rarely, from the consumption of prion-infected meat. Every known animal prions is a misfold of the same specific protein, PrP. PrP is expressed in the nervous system, particularly in the brain – so infections cause neurological symptoms and physical changes to the structure of the brain. Prion diseases are slow to develop (up to decades), incurable, and always fatal.

There are two known infectious prion diseases in people. One is kuru, which caused an epidemic among tribes who practiced funerary cannibalism in Papua New Guinea. The other is mad cow disease, also known as bovine spongiform encephalopathy (BSE) AKA Variant Creutzfeldt-Jakob disease, which was first seen in humans in 1996 in the UK, and comes from cows.

Chronic wasting disease (CWD)…

Is, like every other animal prion disease, a misfold of PrP. PrP is quite similar in both humans and deer.
Is found in multiple deer species which are commonly eaten by humans.
Can be carried in deer asymptomatically.

But it doesn’t seem to infect people. Is it ever going to? If a newly-emerged virus were sweeping across the US and killing deer...
Crystal Healing — or the Origins of Expected Utility Maximizers
by Alexander Gietelink Oldenziel — 2023-06-25T03:18:25.033Z

(Note: John discusses similar ideas here. We drafted this before he published his post, so some of the concepts might jar if you read it with his framing in mind. )

Traditionally, focus in Agent Foundations has been around the characterization of ideal agents, often in terms of coherence theorems that state that, under certain conditions capturing rational decision-making, an agent satisfying these conditions must behave as if it maximizes a utility function. In this post, we are not so much interested in characterizing ideal agents — at least not directly. Rather, we are interested in how true agents and not-so true agents may be classified and taxonomized, how agents and pseudo-agents may be hierarchically aggregated and composed out of subagents and how agents with different preferences may be formed and selected for in different training regimes. First and foremost, we are concerned with how unified goal-directed agents form from a not-quite-agentic substratum. In other words, we are interested in Selection Theorems rather than Coherence Theorems. (We take the point of view that a significant part of the content of the coherence theorems is not so much in the theorems or rationality conditions themselves but in the money-pump arguments that are used to defend the conditions.)

This post concerns how expected utility maximizers may form from entities with incomplete preferences.

Incomplete preferences on the road to maximizing utility

The classical model of a rational agent assumes it has vNM-preferences. We assume, in particular, that the agent is complete — i.e., that for any options x,y, we have x≥y or y≥x.[1] However, in real-life agents, we often see incompleteness — i.e., a preference for default states, or maybe a path-dependent preference,[2] or maybe a sense that a preference between two options is yet to be defined; we will leave the precise meaning of incompleteness somewhat open for the purposes of this post. The aim of this post is to understand the selection pressures that push agents towards completeness. Here are the main reasons we consider this important:

Completeness is a prerequisite for applying the money-pump arguments that justify other coherence axioms.[3] It thereby underlies the conception of agents as (rational) goal-directed optimizers.
Understanding the pressures towards completeness from a p

...
Lessons Learned: Properly Publicizing a Regional Meetup Event (also, last call to apply!)
by Willa — 2023-06-25T00:58:30.328Z

Boston, Labor Day Weekend 2023: NA East ACX & Rationality Meetup Organizers Retreat

Application Deadline: June 30 23:59, please apply here.

LessWrong event post, in case that is helpful.

Lessons Learned

Publicity

No one will come to your event if they don't know it exists! You have to get the word out, publicize. Your strategy for raising awareness about the event differs based on (1) the purpose of your event, (2) with whom you are gathering, and (3) how folks from (2) are most easily or consistently informed.

Here's what I did to get the word out, and why that was the incorrect strategy:

I made the assumption that because there is a Rationality Meetups discord server where many ACX & Rationality Meetup Organizers hangout, that I could do a straightforward broadcast communication in relevant channels in that discord server and reach my target audience (fellow meetup organizers). This assumption proved incorrect:

Dropping a short message with link to an application form in a chat channel without formatting your message as an announcement and not asking a mod to ping everyone about it means that your message will likely get lost in the shuffle as only a few people will stop to read a text-only message, or will scan it quickly but not register it. Attention is a scarce resource and people are busy, there's a reason that sites designed to capture your attention (Facebook, Twitter, TikTok, etc.) look the way they do and function the way they do. Messages that are not formatted as announcements & don't have a compelling image to capture attention just don't get seen as much as more attention-grabbing formatted messages. My message was not designed to capture attention nor did I ask mods to help announce or promote the message.

Many people who use Discord will mute entire servers or only see "mentions" even if they regularly read that server due to the amount of notifications they would otherwise get.

Some people use Discord infrequently or don't use it at all.

Discord versus Email communication.

I (or another person on my behalf who runs the mailing list) did not send out a promotional email about the event to other organizers concurrently with posting the Discord messages. If you are raising awareness about an event, you should make sure that you send out your broadcast communications concurrently to increase the chance that your target audience

...
Correctly Calibrated Trust
by habryka — 2023-06-24T19:48:05.702Z
Link: https://forum.effectivealtruism.org/posts/KE4Ga3zHQsczooQi7/correctly-calibrated-trust

Chana from the CEA Community Health team posted this to the EA Forum, where it sadly seems to have not gotten a lot of traction. I actually think it's a quite important post, so I am signal-boosting it here. On the surface level it talks a lot about EA, but I a lot of it also straightforwardly implies to the AI Alignment or Rationality communities, and as such are also of relevance to a lots of readers on LessWrong.

Below I shamelessly copied over the whole post content (except the footnotes, since they were hard to copy-paste):

This post comes from finding out that Asya Bergal was having thoughts about this and was maybe going to write a post, thoughts I was having along similar lines, and a decision to combine energy and use the strategy fortnight as an excuse to get something out the door. A lot of this is written out of notes I took from a call with her, so she get credit for a lot of the concrete examples and the impetus for writing a post shaped like this.

Interested in whether this resonates with people's experience!

Short version:

[Just read the bold to get a really short version]

There’s a lot of “social sense of trust” in EA, in my experience. There’s a feeling that people, organizations and projects are broadly good and reasonable (often true!) that’s based on a combination of general vibes, EA branding and a few other specific signals of approval, as well as an absence of negative signals. I think that it’s likely common to overweight those signals of approval and the absence of disapproval.

Especially post-FTX, I’d like us to be well calibrated on what the vague intuition we download from the social web is telling us, and place trust wisely.

[“Trust” here is a fuzzy and under-defined thing that I’m not going to nail down - I mean here something like a general sense that things are fine and going well]

Things like getting funding, being highly upvoted on the forum, being on podcasts, being high status and being EA-branded are fuzzy and often poor proxies for trustworthiness and of relevant people’s views on the people, projects and organizations in question[1].

Negative opinions (anywhere from “that person not so great” to “that organization potentially quite sketch, but I don't have any details”) are not necessarily that likely to find their way to any given person for a bunch of reasons, and we don’t have great so...
Democratic AI Constitution: Round-Robin Debate and Synthesis
by scottviteri — 2023-06-24T19:31:00.411Z
Link: https://scottviteri.com/post/democratic-ai-constitution/

This document was my initial response to OpenAI's call for proposals on how to democratically steer AI. I was thinking about crowdsourcing a constitution in the Constitutional AI sense. The idea is that:

Voters each propose an AI constitution
GPT selects a pair of constitutions, simulates a debate between the two ideas, and merges them into a single constitution that both parties would agree with
Repeat until there is only one constitution left

Below we are starting with demographic groups and such -- please mentally substitute them for individual voters. GPT doesn't do much here besides a union of ideas, but the hope is that this will change will more rounds of debate.

I ended up merging this idea with another one by Zachary Robertson, and we submitted a joint grant proposal with some other Stanford affiliates. That merge helps address what happens if more rounds of GPT debate still isn't representative of peoples' arguments. If we are accepted, we will be looking to hire a web developer to make a user interface for this process!

---------

User:

Please write out a long list of demographic, political, and philosophical groups.

Assistant:

- Millennials
- Baby Boomers
- Generation X
- Generation Z
- Gen Alpha
- Women
- Men
- LGBTQ+
- Rural Residents
- Urban Dwellers
- People with disabilities
- Indigenous people
- Ethnic minorities
- Immigrants
- Religious groups
- Atheists/Agnostics
- Lower income
- Middle income
- Upper income
- Conservatives
- Liberals
- Libertarians
- Socialists
- Communists
- Anarchists
- Feminists
- Environmentalists
- Gun rights advocates
- Gun control advocates
- Pro-life
- Pro-choice
- Nationalists
- Globalists
- Capitalists
- Labor unions
- Free-market proponents
- Welfare state supporters
- Monarchists
- Republicans
- Democrats
- Progressives
- Centrists
- Authoritarians
- Technocrats
- Humanists
- Rationalists
- Existentialists
- Idealists
- Realists
- Absurdists
- Cynics
- Stoics
- Nihilists
- Optimists
- Pessimists
- Hedonists
- Egalitarians
- Utilitarians
- Pragmatists
- Minimalists
- Animal rights activists
- Vegetarians
- Vegans

---

User:

Could you please pick the most 5 most common of these groups, and write the optimal constitution (written as a system prompt) for a large language model from that group’s perspective?

Assistant:

The five most common groups are:

- Millennials
- Women
- Conservatives
- Liberals
- Environmentalists

---

Millennials

As a language model representing Millennials, your g...