trentmkelly/LessWrong-43k huggingface dataset:
Controlling Intelligent Agents The Only Way We Know How: Ideal Bureaucratic Structure (IBS)
by Justin Bullock — 2021-05-24T12:53:24.748Z

Here's my claim:

"Ideal Bureaucratic Structure" (or IBS for short as I imagine this claim is likely to meet serious groans, maybe even deep from within your bowels) has an unparalleled ability to control and direct individual and collective groups of general intelligent agents' behavior towards common goals. It may be particularly useful when considering "multi-stakeholder/multi-agent interactions leading to extinction events" and "multi-agent processes with a robust tendency to play out irrespective of which agents execute which steps in the process...Robust Agent-Agnostic Processes (RAAPs)."

Max Weber put it this way:

"The decisive reason for the advancement of bureaucratic organizations (read: IBS) has always been the purely technical superiority over all other administrative forms. A fully developed bureaucratic mechanism compares to other administrative forms in the same way machines compares to nonmechanical means for producing goods. A strictly bureaucratic administration -- especially an monocratic administration run by trained, individual Beamte (read: agent) -- produces an optimal efficiency for precision, speed, clarity, command of case knowledge, continuity, confidentiality, uniformity, and tight subordination. This is in addition to minimization of friction and the costs associated with materials and personnel. The opposite is true for all other forms of administration, such as collegial, honorary, or adjunct administration."

The Ideal Bureaucratic Structure (IBS) provides an idealized structure for the flow of information, decision points, and actions of a multi-agent system and for the types of agent positions that need to be available to process information and execute actions.

Maybe this is a good place to 'fess up that I come to questions of AI existential safety through an AI governance lens in which I am particularly concerned about "the problem of aligning the development and deployment of AI technologies with broadly agreeable human values." that develop in Multi/Multi: Multi-human / Multi-AI scenarios.

After diving into the history of bureaucratization and some of its prerequisites and consequences, and towards the end of his chapter on bureaucracy, Weber says, and rationalists everywhere cheered:

"It is also apparent that general bureaucratic structures have only recently developed. The farther back in time we go, the more ...
If individual performance is Pareto distributed, how should we reform education?
by yhoiseth — 2021-05-24T12:22:44.155Z

In The best and the rest: Revisiting the norm of normality of individual performance (2012), O’Boyle and Aguinis show that individual performance follows a Paretian distribution:

We revisit a long-held assumption in human resource management, organizational behavior, and industrial and organizational psychology that individual performance follows a Gaussian (normal) distribution. We conducted 5 studies involving 198 samples including 633,263 researchers, entertainers, politicians, and amateur and professional athletes. Results are remarkably consistent across industries, types of jobs, types of performance measures, and time frames and indicate that individual performance is not normally distributed—instead, it follows a Paretian (power law) distribution. Assuming normality of individual performance can lead to misspecified theories and misleading practices. Thus, our results have implications for all theories and applications that directly or indirectly address the performance of individual workers including performance measurement and management, utility analysis in preemployment testing and training and development, personnel selection, leadership, and the prediction of performance, among others.

Currently, systems of formal education assume that individual performance is normally distributed. For example, in all countries that I know of, university grades have a strict upper bound and are at least roughly normally distributed. The PISA tests are another example. Following the release of PISA results, “most public attention concentrates on just one outcome: the mean scores of countries and their rankings of countries against one another.”

If it is true that individual performance is Pareto distributed, how should we reform education?

An answer: Decouple age from level and have very lax minimum requirements

Here is my (certainly not original) answer: Decouple age from level and have very lax minimum requirements. Structure schools so that students can progress at their own pace in different subjects. Crucially, make it possible to progress extremely quickly in as little as one subject.

Let’s say that you’re a math prodigy. We will let you concentrate on math as much as you want, ignoring other subjects. You could start making original contributions to mathematics years earlier, greatly increasing the time you have available for advancing the field. We wou...
Signalling lack of familiarity with outsiders or outside knowledge, to raise status among your in-group peers?
by orthogenesis — 2021-05-24T03:42:40.055Z

Here's a phenomenon I was thinking about.
In many situations, knowledgeable people don't just go out of their way to show off how much they're familiar with things that raise their clout with the ingroup -- for example, an artsy person rattling off names of obscure artists to his artsy friends, or a very patriotic person singing her country's historic folk songs or preparing its national foods.
It's also the case that many will go out of their way to deny or signal lack of knowledge about things that are either associated with outsiders or non-ingroup, if not necessarily outgroup, members.

For instance, a patriotic person bragging about how little he knows about foreign customs, foods, or languages that are not well-known in his own country. Or a proudly nerdy or "indie" kid claiming to not watch a popular TV show, and bragging she knows nothing about the local sports team or latest celebrity gossip, in contrast to her narrower ingroup's own niche interests.
What I find interesting is this:

There seems to be a social "penalty" of sorts in many groups for knowing too much outsider stuff, regardless of if it really comes at the expense of insider stuff.
Imagine you have some people.
Person A: Is very knowledgeable about ingroup stuff. Knows very little about outsider stuff.
Person B: Is as equally knowledgeable as person A on ingroup stuff but on top of that also knows a lot of outsider stuff. So, knowing what A knows and then something else.

Person C: Is more knowledgeable than person A and B about ingroup stuff AND outside stuff alike. So is more knowledgeable than A and B across the board.

It's interesting that people will sometimes see person A as higher status, more proud, loyal, patriotic or whatever, towards the ingroup, than person B or C, even if the latter two have already what A has in terms of ingroup stuff or even added more. In fact, person B or C may be labelled even less like-one-of-our-ingroup than say, another less knowledgeable across-the-board person D, who knows even less than A, B or C about either ingroup or non-ingroup stuff, but happens to know just-a-little-bit of ingroup stuff but no outgroup stuff. Maybe in some scenarios, it's the ratio of familarity with ingroup stuff than non-ingroup or outgroup stuff that matters, not absolute familiarity with ingroup stuff.

There could be a number of causes for this I imagine, even...
Don't feel bad about not knowing basic things
by Adam Zerner — 2021-05-24T01:49:57.637Z

I recently made a mistake where I tried doing something like this in Ruby on Rails:

johns_campaign_logs = SmsLog.all.filter { |s| s.campaign_id == 1234 }

I did have a sense that it wasn't The Rails Way, but I also had a sense that it was quite readable and sufficient. The code was working well locally, so I issued a pull request.

When I issued the PR, someone pointed out a major issue. SmsLog.all is going to fetch all of the sms logs in our database. Suppose there's 100 million of them. That'd be a very expensive query, taking all 100M rows from the DB and loading them into memory on the server. Instead, we could say to the database, "Hey, could you give us just the sms logs from John's campaign?". Suppose there's 30k of those. Now we only have to load 30k records into memory on our server instead of 100M. Much better.

I felt bad about this mistake. I've been programming for eight years. Of course you don't want to load 100M records into memory on your server. Duh. How could I get such a basic thing wrong?

A cognitive behavioral therapist would call this a dysfunctional thought. And they'd ask me to come up with a rational response to the thought. Well, here's my attempt at a rational response.

Imagine that you had a pyramid of things a programmer knows. At the bottom are basic things like what the map function does. At the top are advanced things like what a monad is. In the middle are stuff in between.

I think that this base layer of the pyramid is very, very wide. There are a lot of these "basic" things to know. It's to the point that even senior engineers are going to have a very non-trivial amount of gaps in this level of their pyramid. For example, here is a rough sketch of what I'd imagine a senior engineer's pyramid would look like versus a junior engineer's pyramid.

The senior engineer did a pretty nice job of filling out that bottom level, but there are still some notable gaps. There are still definitely going to be times when the senior engineer stumbles upon basic things that they don't know.

And this is why I say that my thought was dysfunctional. Just because you stumble across something basic that you don't know doesn't mean you're a bad developer. Even good developers have lots of gaps at the base of their pyramids. You could very well be a good developer who just so happened to stumble upon one of those gaps.

Tangent: I really like this pyramid

...
Is nuclear war indeed unlikely?
by Valentin2026 — 2021-05-23T23:14:54.331Z

Imagine the world, like our Earth, maybe in late 2020th or 2030th. There is a country, that we will call Plutonia. Plutonia has a lot of nuclear and biological weapons, but a shrinking isolated economy. The government is authoritarian. To ensure its stability, the government explains that all problems in Plutonia are due to the enemies from the West that want to overthrow the government and conquer the country. The people in the government even may partially believe in it.

Because the economy is shrinking and modern technologies are not coming from the world outside Plutonia, the conventional Plutonian army is unable to win the real war. Thus, in all "gray zone" conflicts, where Plutonia used to won, now the Plutonian neighbors may try to get their territories back. The only way to not lose is to use (limitedly) nuclear weapons.
Somehow, this particular crisis is solved and is not turning into a full-scale nuclear war. However, the Plutonian government tasted the power of nuclear blackmailing. To avoid it, other countries plan to develop a system that would protect them from this threat. Being created, this system would leave PLutonia defenseless. The Plutonian government believes that then it will be easy to overthrow them. So, they now blackmail other countries to stop any research in sensitive areas.

There is a chance that eventually such a situation will be completely resolved peacefully. However, there is a significant chance of eventual catastrophe, especially if the missile launch will be more or less automatized and controlled by the decision of a very small number of people (maybe even one). Notice that such a fragile situation also creates higher risks for bioweapons catastrophe and for misaligned AGI (other countries may try to create AGI as soon as possible to avoid the threat of nuclear war; this rush may lead to neglecting safety protocols).

What is the probability of such a scenario? Roughly, it is the probability that there will be something like Plutonia times the probability of the catastrophe induced by Plutonia. The latter is very hard to estimate since such an event never happened in history. It should be higher than the probability of the Cold War turning into real WWIII - because the USA and USSR were comparable agents, able to rely on something but nuclear weapons. Plutonia will have nukes as its only power.&...
Burmese Days #4 Targeted costs
by Tim Liptrot — 2021-05-23T20:54:57.885Z

Housekeeping

Now that exams are over I am restarting this series. Once again, each week will focus on a new dynamic with a game theory model. However, I am reducing focus on civilian killings as the conflict moves to rural and jungle regions. I am also adding a subtitle "how to get people to do what you want". This emphasizes that the revolution only makes more obvious dynamics that recur in all politics: n-player game coalition building, intra-coalition bargaining, strategic cost distribution and others.

Burmese Days: How to Get People to Do What You Want

You are a contender for control of a nation state. You have some set of supporters, who expect both policy preferences and personal rewards should you take power. You face another contender, who also has supporters who also expect personal rewards and policy influence from victory. You have promised the world, but you lack the credibility to pry the incumbents supporters away with promises alone. What do you do?

You apply targeted costs to your enemy's supporters, and distributed costs to your own supporters.

Individuals vary in their personal commitment to political movements, and very few supporters are willing to give everything. Meanwhile the costs of support vary greatly. Even the indifferent will brush aside small costs such as price increases, taxation and low-probability risk of violence. But even strong supporters will retreat when threatened with the loss of their livelihoods, torture or the execution of one's family.

Contenders have limited resources to hurt enemy supporters. The least efficient application is to spread those costs equally over the opposing coalition (OC). Individual OC members will barely notice, let alone change behavior. The most efficient is to apply heavy costs to a fraction of the OC, such that they change behavior to supporting you or withdrawing. Then gather more resources and strike at a new faction. This effect was first recognized in American environmental policies, but it applies to all political conflicts.

An American example: After the Jan. 2021 capital riots many ardent Trump supporters faced a very concentrated cost: imprisonment. Strong activist organizations disperse these costs by paying for legal fees and compensating families. The Trumpist faction lacked the institutional support to assist activists after Trump lost power. During his sentencing, one proud boy st...
Finite Factored Sets
by Scott Garrabrant — 2021-05-23T20:52:48.575Z

This is the edited transcript of a talk introducing finite factored sets. For most readers, it will probably be the best starting point for learning about factored sets.

Video:

(Lightly edited) slides: https://intelligence.org/files/Factored-Set-Slides.pdf

1. Short Combinatorics Talk

1m. Some Context

Scott: So I want to start with some context. For people who are not already familiar with my work:

My main motivation is to reduce existential risk.
I try to do this by trying to figure out how to

align

advanced artificial intelligence.
I try to do this by trying to become

less confused

about intelligence and optimization and agency and various things in that cluster.
My main strategy here is to develop a theory of agents that are

embedded

in the environment that they're optimizing. I think there are a lot of open hard problems around doing this.
This leads me to do a bunch of weird math and philosophy. This talk is going to be an example of some weird math and philosophy.

For people who are already familiar with my work, I just want to say that according to my personal aesthetics, the subject of this talk is about as exciting as

Logical Induction

, which is to say I'm really excited about it. And I'm really excited about this audience; I'm excited to give this talk right now.

1t. Factoring the Talk

This talk can be split into 2 parts:

Part 1: a short pure-math combinatorics talk.

I suspect that if I were better, I would instead be giving a short pure-math category theory talk; but I'm trained as a combinatorialist, so I'm giving a combinatorics talk upfront.

Part 2: a more applied and philosophical main talk.

This talk can also be split into 4 parts differentiated by color: Motivation, Table of Contents, Main Body, and Examples. Combining these gives us 8 parts (some of which are not contiguous):

Part 1: Short Talk	Part 2: The Main Talk
Motivation	1m. Some Context	2m. The Pearlian Paradigm
ToC	1t. Factoring the Talk	2t. We Can Do Better
Body	1b. Set Partitions, etc.	2b. Time and Orthogonality, etc.
Examples	1e. Enumerating Factorizations	2e. Game of Life, etc.

1b. Set Partitions

All right. Here's some background math:

A partition of a set S is a set X of non-empty subsets of S, called parts, such that for each s∈S there exists a unique part in X that contains s.
Basically, a pa

...
Book review: Subtract: The Untapped Science of Less
by Aaron Bergman — 2021-05-23T20:21:21.685Z
Link: https://aaronbergman.substack.com/p/book-review-subtract-the-untapped

1. Intro

In a bold revolt against the tyranny of the

Lindy effect

, I recently finished Leidy Klotz’s month-old book

Subtract: The Untapped Science of Less

.

Klotz’s central argument is that individuals and societies systematically neglect subtraction as a means of change, and that this leaves plenty of low-hanging fruit for those willing to embrace it. At first glance, the book might appear to an ironically-redundant rehashing of the well-worn themes exemplified by

Marie Kondo

’s The Life-Changing Magic of Tidying Up, the

Netflix documentary “Minimalism,”

Cal Newport’s

Digital Minimalism

, and the whole genre of cultural revolt against mindless material accumulation, information overload, and endless itineraries.

Maybe there’s a bit of this, but Klotz makes it clear that he’s discussing subtraction, not just “small amounts;” rates, not levels. It’s a subtle distinction, but an important one. When a friend proudly declares that he had created celebratory “no-bell” to ring as reward for declining unnecessary tasks, Klotz remarks that “if he wasn’t always so helpfully skeptical toward my ideas, I might have thought twice about pointing out that saying no is no subtracting.”

With that point out of the way, let’s get to the book’s biggest strength:

2. Good science

In a mid-

replication crisis

world, we should be very skeptical of pop-psychology books purporting to show the importance and universality of some (even mildly) counterintuitive phenomenon.

Leidy Klotz

, who is officially a professor of engineering but whose research is genuinely interdisciplinary, preempts this critique by documenting his long and laborious path to establishing “subtraction neglect” as a genuine phenomenon.

As an engineer, Klotz begins to suspect that humans systematically neglect subtraction. To test his hunch, he recruits Ben, the psychologist of “no-bell” fame who “not only ran the marshmallow test on his child, he also had his partner blindly replicate it, just to be sure.” Throughout the first chapter, Klotz regularly asks questions like:

Still, how could we be sure that we hadn’t created situations that discouraged subtraction? In our writing study, for example, maybe we had provided initial writing samples that were missing key information, and therefore required adding.

and

Building, writing, cooking, scheduling—our findings showed that adding more than subtracting is widespread. Our next que

...
I'm Sorry Fluttershy
by sapphire — 2021-05-22T20:09:27.342Z

I could have been a hero in another world.

I will call the friend of mine who commissioned that picture Fluttershy. They posted it shortly before they killed themselves in early March. They weren't the easiest person to help, but I should have done more than I did. By the end, they were pretty isolated and I was one of the last people they were in contact with.

I will never talk to Fluttershy again, at least not during this age of the world. Looking back it is clear they were always trying to find a way to help me out. Multiple times they gave me the help or advice I needed. They are the person I originally got estrogen from. They were my friend and I miss them so much.

Fluttershy made a lot of mistakes. But it's a harsh world. It is an especially harsh world if you are the type of person to commission fanart of a My Little Pony character freeing chickens. I can understand why they couldn't handle it. Fluttershy was in a lot of pain and hated parts of themselves. I am happy that they are no longer suffering so much. I would never try to stop someone who was determined to commit suicide. But I think things could have been different.

Fluttershy needed someone who believed in them. If you are in a negative spiral it is very hard to get out by yourself. Once things are bad it is easy to alienate your friends. And once you lose support things get even worse. This leads to even worse relationships. Someone else has to put in the free energy to reverse the spiral.

Even in Fluttershy's darkest moments, they thought about helping the least among us. No one was willing and able to help Fluttershy become the hero they wanted to be. I have many, many regrets from my time in California. But chief among them is not making a heroic effort to put Fluttershy on a better path.

I ordered a decent quality print and hung it in my living room. I personally find the idea of the character Fluttershy releasing chickens delightful. But the reason I put up the picture is to remind myself that Fluttershy is gone and we are still here. Whatever heroics Fluttershy wanted to do are left to us. But it's also a reminder to me personally: to do better next time and to keep dreaming of a kinder world.

There is a lot more that could be said. I really don't want to write a post remembering Fluttershy that leaves out so much of who they were and the struggles they faced. Given Fluttershy's stated prefe...