trentmkelly/LessWrong-43k huggingface dataset:
What are all the AI Alignment and AI Safety Communication Hubs?
by Gunnar_Zarncke — 2022-06-15T16:16:03.241Z

For effective work on complex topics, multiple people need to work together[citation needed].

For AI Alignment, LessWrong and its sister site, the Alignment Forum, are arguably the Shelling points. But there are other platforms that, for one reason or other - language, culture, size, personal preference - are better suited for individual contributors.

On Being an individual alignment grantmaker, the following platforms were mentioned:

Next, I increased my surface area with places which might have good giving opportunities by involving myself with many parts of the movement. This includes Rob Miles’s Discord, AI Safety Support’s Slack [named "AI alignment"], in-person communities, EleutherAI, and the LW/EA investing Discord, where there are high concentrations of relevant people, and exploring my non-LW social networks for promising people.

Other such hubs that I know of (all only with tangential AI safety focus):

Astral Codex Ten Scott Alexander discusses alignment sometimes, too; has a lively comment section on Substack; also: ACX Discord, Subreddit
Rationality Berlin Slack of the Berlin LessWrong community; maybe 30 members; one AI Alignment effort; meditation, dojos, monthly meetup, organizes the yearly European LessWrong Community Weekend
Google Groups

bayarealesswrong active
less-wrong-parents inactive
lesswrong-hamburg group of my meetup in Hamburg, very low activity
probably many other regional groups

Bountied Rationality Facebook group, a marketplace for small tasks in the community

Many channels require an invite. If you post additional hubs, please mention if an invite is needed and how one might get invited.
Georgism, in theory
by Stuart_Armstrong — 2022-06-15T15:20:32.807Z

Last year, Astral Codex had a series of interesting posts on Georgism. The claims felt very counterintuitive to me, so I dove into it until I could at least sort out the theory. And I emerged with a better understanding, which I think would be useful to share here.

Key points:

In theory at least, Georgism is correct that a land tax will not cause rents to rise.

Land will become cheaper to buy, and there will be more pressure to use it in an economically viable way.

The economy as whole will benefit.

However, a land tax will not cause rents to fall, either. Indirectly, the better economy may cause them to rise, in fact.

For those who are deep into economic models, you can see a Georgist land tax as causing a deadweight loss in a situation of perfectly inelastic supply - ie no deadweight loss at all, and all the tax will be paid by the inelastic supplier. For a longer explanation, here's the rest of the post.

Marginal rents and taxes

Adage: If you want less of something, tax it.

Quick quiz: if you tax cars, does their purchase price go up or down?

Well, if you put a sales tax, paid when the car is bought, then the price will go up. If you put a road tax, paid every year by the car owner[1], then the price will go down (since it’s now more expensive to own the car, hence less interesting to buy it in the first place).

Second quiz: suppose you have a nice community of a hundred renters, in identical homes. The rents are reasonable, and everyone renting is much happier to live here than anywhere else. What will happen to the rents?

They will rise, of course. As long as everyone renting prefers to live here than anywhere else, then they will continue to stay here even as the rents rise. And so, rise they will. And they will continue to rise until... until someone living here finds them too high, and moves away. At that point, they stop rising[2]. So the rent price is determined by the person who is just on the edge of moving away. In economics, this is what they refer to when they talk about the “marginal buyer[3]”: the gal or guy who thinks “this is not a very good deal; I could go either way on it”. That unsatisfied marginal renter is the one that determines the price.

And, once the prices have reached this level, the landlords can no longer raise rents. Maybe I pay €1,000 for a rental that I value at €1,500. Then the landlord might be tempted to charge me, say, €1,...
A central AI alignment problem: capabilities generalization, and the sharp left turn
by So8res — 2022-06-15T13:10:18.658Z

(This post was factored out of a larger post that I (Nate Soares) wrote, with help from Rob Bensinger, who also rearranged some pieces and added some text to smooth things out. I'm not terribly happy with it, but am posting it anyway (or, well, having Rob post it on my behalf while I travel) on the theory that it's better than nothing.)

I expect navigating the acute risk period to be tricky for our civilization, for a number of reasons. Success looks to me to require clearing a variety of technical, sociopolitical, and moral hurdles, and while in principle sufficient mastery of solutions to the technical problems might substitute for solutions to the sociopolitical and other problems, it nevertheless looks to me like we need a lot of things to go right.

Some sub-problems look harder to me than others. For instance, people are still regularly surprised when I tell them that I think the hard bits are much more technical than moral: it looks to me like figuring out how to aim an AGI at all is harder than figuring out where to aim it.[1]

Within the list of technical obstacles, there are some that strike me as more central than others, like "figure out how to aim optimization". And a big reason why I'm currently fairly pessimistic about humanity's odds is that it seems to me like almost nobody is focusing on the technical challenges that seem most central and unavoidable to me.

Many people wrongly believe that I'm pessimistic because I think the alignment problem is extraordinarily difficult on a purely technical level. That's flatly false, and is pretty high up there on my list of least favorite misconceptions of my views.[2]

I think the problem is a normal problem of mastering some scientific field, as humanity has done many times before. Maybe it's somewhat trickier, on account of (e.g.) intelligence being more complicated than, say, physics; maybe it's somewhat easier on account of how we have more introspective access to a working mind than we have to the low-level physical fields; but on the whole, I doubt it's all that qualitatively different than the sorts of summits humanity has surmounted before.

It's made trickier by the fact that we probably have to attain mastery of general intelligence before we spend a bunch of time working with general intelligences (on account of how we seem likely to kill ourselves by accident within a few years, once we have AGIs on ...
Our mental building blocks are more different than I thought
by Marius Hobbhahn — 2022-06-15T11:07:04.062Z

I recently had a conversation with one of my best friends about her

Aphantasia

. In short, she can’t create mental images. When she thinks about the world, she thinks in text, relationships, sounds, and much more but never images. Even if she tries, it’s impossible for her. I found this experience very revealing because it showed how much I expected everyone else to experience the world in basically the same way that I do.

After this revelation, I talked to a bunch of people about how they viewed the world. Not about their specific beliefs and opinions but rather in what way they think. And I found that the differences in how people think are much bigger than I expected.

For example, when I think about “Traveling from Tübingen (where I currently live) to Nuremberg (where I grew up)”, I make the journey in my head. I “fly”/zoom from my home to the train station, zoom through all the intermediate train stops in my mental model, get out and zoom to my parent's house. Of course, I don’t have a perfect model of the intermediate steps but it’s sufficiently good to fly around. Not everyone I talked to, does that. For some, it’s just a one-step thing. They start in Tübingen, and then they teleport to their hometown in their head.

For music, on the other hand, most people have clear associations in their heads. When they listen to music, they see colors or flows, they feel the melody with different emotional states, they feel an urge to move to the rhythm, and much more. In any case, most people seem to have clear associations with music even if they differ from person to person. I mostly don’t. I don’t really see anything, I don’t feel that much, and I don’t really have an urge to move.

For the rest of this post, I will refer to the concepts shown in the examples above as mental building blocks. What I roughly mean is “a way to think about a specific concept” or “looking through a specific lens” or “a tool in your mental toolbox”. However, I find it very hard to define more properly and I don’t claim that the thing I mean is actually consistent. Feel free to point out flaws or suggest improvements.

Importantly, mental building blocks are not fully static, e.g. I expect that people can get better at mental models of locations or understanding music. However, some people find it much easier and much more natural to fly around in their mental worl...
Has there been any work on attempting to use Pascal's Mugging to make an AGI behave?
by Chris_Leong — 2022-06-15T08:33:20.188Z

A mere line in the sand, backed by the clout of a nonexistent simulator, could prove a stronger restraint than a two-foot-thick steel door - Nick Bostrom, Superintelligence

Nick Bostrom's idea of anthropic capture is very similar to the idea of utilising Pascal's Mugging, but he doesn't explore this in detail. Further, there is a difference in that framing this in terms of Pascal's Mugging suggests that this could work even if the AGI is highly skeptical of the mugging and defines a minuscule probability of it being true. Off the top of my head, potential lines of exploration include a) biasing the AI's reasoning to allow it to be mugged[1] b) trying to figure out what would be the optional message to send the AGI on the assumption that it would surrender to the mugging c) including scenarios where an AI allows itself to be mugged in the training data.

^

With the obvious downside that others may use this to mug the AI.
Alignment Risk Doesn't Require Superintelligence
by JustisMills — 2022-06-15T03:12:56.573Z

Outsize destructive coordination is rare

I was 9 years old when 9/11 happened. My gut reaction was, basically, "wait, doesn't this sort of thing happen all the time?" I don't mean to suggest I was wise or world-weary or anything. I was, in fact, mistaken. But I had an ambient view that small groups of human beings wreaked massive destruction on each other all the time, randomly. It certainly seemed to me, a kid with a tendency towards worry, that they could. But no. Events like 9/11, where a small number of people have a gigantic destructive impact, are pretty rare.

If this mindset is new to you, I strongly recommend Gwern's On The Absence Of True Fanatics. It's the best treatment I've read of the idea that truly dedicated human beings, if we set our mind to it, can cause a huge amount of harm.

Indeed, it wouldn't surprise me if 50 perfectly coordinated 16-year-olds of top 1% (but not top 0.1%) intelligence, with total trust in each other and a shared monomaniacal mission to bring down human civilization within their lifetimes, could succeed. Not that I think it would be a sure thing; plenty of opportunity for them to get noticed or caught, or to make a planning error and get themselves killed by accident. But I can think of ways, including ones that aren't particularly esoteric.

A quick, unpleasant sketch

I don't like dwelling on awful hypotheticals, so I'll trace just one possibility out; to be clear, I do not think this is anywhere near the best strategy for such a group, but I don't like writing or encouraging thinking about better ones.

The 50 teenagers could coordinate to get a few of them as politically advanced as possible. Ideally they'd get one to be President of the US. That sounds very hard, but remember that in this hypothetical we've granted the 50 teenagers perfect coordination and monomania, two superpowers. Normal humans want other things, like to be cool, have sex, and eat tasty food. They are vulnerable to scandals. But our teens could pick 10 of them, maybe the most charismatic, to go hard in politics, and the other 40 could do whatever careers would best support them.

If you get the presidency, plus a few key positions nearby, you slowly flush out non-confederates and then trigger a nuclear war. Pretty bad! My gut reaction is that our group could have a maybe 5% chance of success. But even if I'm totally wrong, I bet you can imagine other ways ...
A Butterfly's View of Probability
by Gabriel Wu — 2022-06-15T02:14:44.527Z

Cross-posted from my blog.

Thanks to Alex Cai and Haneul Shin for discussing these ideas with me. Thanks to Eric Neyman for feedback.

What do we really mean when we use the word "probability"? Assuming that the universe obeys deterministic physical laws, an event must occur with either probability 0 or 1. The future positions of every atom in the universe are completely determined by their initial conditions. So what do we mean when we make statements like "Trump has a 25% chance of winning in 2024"? It will either happen, or it won't. From an objective, omniscient standpoint, there's no room for doubt.

One response is to reject determinism. Maybe in Newton's day we believed the universe was deterministic, but now we know about wave functions and Heisenberg uncertainty and all of that stuff. If we accept that there is true randomness occurring on the quantum level, then the outcome of the next election isn't predetermined — it will depend on all of the quantum interactions that occur between now and 2024. With this view, it makes complete sense to assign fractional probabilities.

But quantum randomness is a highly non-obvious property of physics... Is there a way to make sense of probability without relying on it? In this post, I hope to outline a new way of defining the probability of a future event in a deterministic system. In contrast with the Bayesian view — in which uncertainty about an event comes from incomplete information — and the frequentist view — which relies on an experiment being repeatable — this "Butterfly's View of probability" draws its randomness from chaos theory.

Bayesianism and Frequentism

Let's go over these two existing views, which are by far the most commonly accepted interpretations of probability.

The Bayesian view of probability is that randomness arises from incomplete information about the world. It is impossible to be aware of the current position of every atom in the universe at once. A Bayesian reasoner embraces this uncertainty by considering probability distributions over all possible universes consistent with his observations. At the core of this philosophy is Bayesian updating: starting with a prior probability distribution, upon seeing new evidence about the world a Bayesian reasoner will update this distribution according to Bayes' Rule. The probability of an event is the proportion of universes in this distribution in whic...
Favourite new AI productivity tools?
by Gabe M — 2022-06-15T01:08:07.967Z

Transformative AI will revolutionize the ways we interact with the world. But maybe narrow tool AIs are already starting to do so: Every week, there are new startups, websites, and tools launched around using modern machine learning techniques to solve everyday problems, and we might as well use them to amplify our productivity (especially for productively improving the world)!

What are your favourite modern AI tools that make you more productive? I'm using words like "new" and "modern" to distinguish tools built with recent (i.e. last 5 years) advances in deep learning such as GPT-3 from traditional commodity AI algorithms that are widely known and used such as Dijkstra’s algorithm for Google Maps or older machine learning techniques used in recommender systems.

For example, I've recently started using the OpenAI GPT-3 Playground and YouWrite for breaking through my writer's block and quickly getting some ideas to work with (e.g. "A friendly invitation to event X that mentions Y:") as well as Elicit for quickly searching research papers with generated TLDR answers to a specific research question (e.g. "How can we improve insect welfare?"), and I've heard of some friends enjoying OpenAI Codex for speeding up their programming (though I'm sadly deep in the waitlist).
Will vague "AI sentience" concerns do more for AI safety than anything else we might do?
by Aryeh Englander — 2022-06-14T23:53:08.366Z

[Cross-posting from something I posted on Facebook.]

Prediction 1 (high confidence): In 2022, many more people will freak out over vague concerns about AI "becoming sentient" than will freak out over the types of issues that AI safety researchers are actually concerned about.
Prediction 2 (medium-low confidence): In the longer run, people freaking out over vague concerns about AI becoming sentient will do more to mitigate serious AI safety concerns than anything that AI safety researchers might have done on their own. This is because the masses freaking out seems more likely to change the interests and values of the government, industry, and even academia than anything that a bunch of specialist researchers might say or do. And any change in overall government / industry / academic interests and values seems likely to result in more substantial downstream change than anything that the specialist researchers might have done on their own.
[Sorry, I don't currently have any particularly good way of operationalizing those predictions or verifying them, so I'm not going to put them up on Metaculus or anything like that. If anybody else has a good way of operationalizing or verifying them, I'm open to suggestions.]
Weak evidence: In the game I'm currently playing, Mass Effect, the galactic government has placed a moratorium on companies doing research into "real artificial intelligence" out of concerns about AI becoming conscious. Clearly the game's designers, at least, thought that this is a plausible concern and a plausible reason to ban AI development. I'm pretty sure I've also seen other comparable examples in novels and other media. My model of the world says that most thought leaders in government, industry, and even academia are likely to be no more sophisticated or educated on these matters than the relevant game designers / authors.
Never mind that in this game, like in almost all media that I've seen, the distinction between "extremely capable robots" (of the type that AI safety researchers might freak out about) and "real AI" (that other people freak out about) is fuzzy at best. In fact, I suspect that to many people the only distinction is that the latter are "sentient" in some fuzzy sense whereas the former are not.

Furthermore, in my experience at least, there seems to be a strong association in the media between "real" / "sentient" / "conscious" AI, and A...
Yes, AI research will be substantially curtailed if a lab causes a major disaster
by lc — 2022-06-14T22:17:01.273Z

There's a narrative that Chapman and other smart people seem to endorse that goes:

People say a public AI disaster would rally public opinion against AI research and create calls for more serious AI safety. But the COVID pandemic killed several million people and wasted upwards of a year of global GDP. Pandemics are, as a consequence, now officially recognized as a non-threat that should be rigorously ignored. So we should expect the same outcome from AI disasters.

I'm pretty sure I have basically the same opinion and mental models of U.S. government, media, and poltics as Eliezer & David, but even then, this argument seems like it's trying too hard to be edgy.

Here's another obvious historical example that I find much more relevant. U.S. anti-nuclear activists said for years that nuclear power wasn't safe, and nuclear scientists replied over and over that the activists were just non-experts misinformed by TV, and that a meltdown was impossible. Then the Three Mile Island meltdown happened. The consequence of that accident, which didn't even conclusively kill any particular person, was that anti-nuclear activists got nuclear power regulated in the U.S. to the point where making new plants is completely cost inefficient, as a rule, even in the event of technology advancements.

The difference, of course, between pandemics and nuclear safety breaches, is that pandemics are a natural phenomenon. When people die from diseases, there are only boring institutional failures. In the event of a nuclear explosion, the public, the government, and the media get scapegoats and an industry to blame for the accident. To imply that established punching bags like Google and Facebook would just walk away from causing an international crisis on the scale of the Covid epidemic, strikes me as confusingly naive cynicism from some otherwise very lucid people.

If the media had been able to squarely and emotively pin millions of deaths on some Big Tech AI lab, we would have faced a near shutdown of AI research and maybe much of venture capital. Regardless of how performative our government's efforts in responding to the problem were, they would at least succeed at introducing extraordinarily imposing costs and regulations on any new organization that looked to a bureaucractic body like it wanted to make anything similar. The reason such measures were not enforced on ...
Slow motion videos as AI risk intuition pumps
by Andrew_Critch — 2022-06-14T19:31:13.616Z

tl;dr: When making the case for AI as a risk to humanity, trying showing people an evocative illustration of what differences in processing speeds can look like, such as this video.

Over the past ~12 years of making the case for AI x-risk to various people inside and outside academia, I've found folks often ask for a single story of how AI "goes off the rails". When given a plausible story, the mind just thinks of a way humanity could avoid that-particular-story, and goes back to thinking there's no risk, unless provided with another story, or another, etc.. Eventually this can lead to a realization that there's a lot of ways for humanity to die, and a correspondingly high level of risk, but it takes a while.

Nowadays, before getting into a bunch of specific stories, I try to say something more general, like this:

There's a ton of ways humanity can die out from the introduction of AI. I'm happy to share specific stories if necessary, but plenty of risks arise just from the fact that humans are extremely slow. Transistors can fire about 10 million times faster than human brain cells, so it's possible we'll eventually have digital minds operating 10 million times faster than us, meaning from a decision-making perspective we'd look to them like stationary objects, like plants or rocks. This speed differential exists whether or not you believe in a centralized AI system calling the shots, or an economy of many, so it applies to a wide variety of "stories" for how the future could go. To give you a sense, here's what humans look like when slowed down by only around 100x:
https://vimeo.com/83664407 <-- (cred to an anonymous friend for suggesting this one)
[At this point, I wait for the person I'm chatting with to watch the video.]
Now, when you try imagining things turning out fine for humanity over the course of a year, try imagining advanced AI technology running all over the world and making all kinds of decisions and actions 10 million times faster than us, for 10 million subjective years. Meanwhile, there are these nearly-stationary plant-like or rock-like "human" objects around that could easily be taken apart for, say, biofuel or carbon atoms, if you could just get started building a human-disassembler. Visualizing things this way, you can start to see all the ways that a digital civilization can develop very qui

...