trentmkelly/LessWrong-43k huggingface dataset:
Ways I Expect AI Regulation To Increase Extinction Risk
by 1a3orn — 2023-07-04T17:32:48.047Z

The following are some very short stories about some of the ways that I expect AI regulation to make x-risk worse.

I don't think that each of these stories will happen. Some of them are mutually exclusive, by design. But, conditional upon AI heavy regulations passing, I'd strongly expect the dynamics pointed at in more than one of them to happen.

Pointing out potential costs of different regulations is a necessary part of actual deliberation about regulation; like all policy debates, AI policy debates should not appear one-sided. If someone proposes an AI policy, and doesn't actively try to consider potential downsides -- or treats people who bring up these downsides as an opponent -- they are probably doing something closer to propaganda than actual policy work.

All these stories have excessive detail; these are archetypes meant to illustrate patterns more than they are predictions. Before mentioning one fix that would stop a problem, consider whether it would be subject to similar problems in the same class.

(a) Misdirected Regulations Reduce Effective Safety Effort; Regulations Will Almost Certainly Be Misdirected

"Alice, have you gotten the new AI model past the AMSA [AI and ML Safety Administration] yet?" said Alex, Alice's manager. "We need their approval of the pre-deployment risk-assessment to launch."

"I'm still in correspondence with them," said Alice, wincing. "The model is a little more faceblind than v4 -- particularly in the case of people with partially shaved heads, facial tattoos, or heavy piercings -- so that runs into the equity clauses and so on. They won't let us deploy without fixing this."

"Sheesh," said Alex. "Do we know why the model is like this?"

Alice shrugged.

"Well, getting this model past the AMSA is your number one priority," Alex said. "Check if you can do some model surgery to patch in the facial recognition from v4. It should still be compatible."

"Alex," said Alice. "I'm not really worried about the face-blindness. It's a regression, but it should have minimal impact."

"I mean, I agree," grinned Alex, "but don't put that in an email."

"But," Alice continued, hesitantly, "I'm really concerned about some of the capabilities of the model, which I don't think we've explored completely. I know that it passed the standard safety checks, and I know I've explored some of its capabilities, but I think this is much much more important than -...
AI labs' statements on governance
by Zach Stein-Perlman — 2023-07-04T16:30:01.624Z

June 6, 2024: THIS POST HAS BEEN SUCCEEDED BY Companies' policy advocacy. READ THAT INSTEAD OF THIS. THIS POST WILL NOT BE MAINTAINED.

This is a collection of statements on government policy, regulation, and standards from leading AI labs and their leadership.

As of 7 August 2023, I believe this post has all of the relevant announcements/blogposts from the three labs it covers, but I expect it is missing a couple relevant speeches/interviews with lab leadership.[1] Suggestions are welcome.

My quotes tend to focus on AI safety rather than other governance goals.

Within sections, sources are roughly sorted by priority.

OpenAI

Governance of superintelligence

(May 2023)

First, we need some degree of coordination among the leading development efforts to ensure that the development of superintelligence occurs in a manner that allows us to both maintain safety and help smooth integration of these systems with society. There are many ways this could be implemented; major governments around the world could set up a project that many current efforts become part of, or we could collectively agree (with the backing power of a new organization like the one suggested below) that the rate of growth in AI capability at the frontier is limited to a certain rate per year.

And of course, individual companies should be held to an extremely high standard of acting responsibly.

Second, we are likely to eventually need something like an

IAEA

for superintelligence efforts; any effort above a certain capability (or resources like compute) threshold will need to be subject to an international authority that can inspect systems, require audits, test for compliance with safety standards, place restrictions on degrees of deployment and levels of security, etc. Tracking compute and energy usage could go a long way, and give us some hope this idea could actually be implementable. As a first step, companies could voluntarily agree to begin implementing elements of what such an agency might one day require, and as a second, individual countries could implement it. It would be important that such an agency focus on reducing existential risk and not issues that should be left to individual countries, such as defining what an AI should be allowed to say.

Planning for AGI and beyond

(Feb 2023)

We think it's important that efforts like ours submit to independent audits before releasing new system

...
AIs teams will probably be more superintelligent than individual AIs

by Robert_AIZI — 2023-07-04T14:06:49.881Z
Link: https://aizi.substack.com/p/ais-teams-will-probably-be-more-superintelligent

Summary

Teams of humans (countries, corporations, governments, etc) are more powerful and intelligent than individual humans. Our prior should be the same for AIs. AI organizations may then face coordination problems like management overhead, the

principal-agent problem

, defection/mutiny, and general

Moloch-y-ness

. This doesn’t especially reduce risk of AI takeover.

Teams of Humans are more Powerful and Intelligent than Individuals

Human society is at such a scale that many goals can only be achieved by a team of many people working in concert. Such goals include winning a war, building a successful company, making a Hollywood movie, and realizing scientific achievements (e.g. building nuclear bombs, going to the moon, eradicating smallpox).

This is not a coincidence, because collaboration gives you a massive power boost. Even if the boost is subadditive (e.g. 100 people together are only 10x more powerful than 1 person), it is still a boost, and therefore the most powerful agents in a society will be teams rather than individuals.

This Probably Applies to AIs Too

I expect the same dynamic will apply to AIs as well: no matter how superintelligent an AI is, a team of such AIs working together would be even more superintelligent[1]. If this is true, we should expect that even after the creation of ASI, the most powerful agents in the world will be coordinated teams, not individual ASI.

Let me acknowledge one objection and provide two rebuttals:

Objection: AI can increase their power by A) coordinating with other AI or B) making themselves smarter, and these both cost the same resource (compute). If B) is always more cost-efficient, then A) will not happen.

Rebuttal A: Currently, LLM training runs are far more expensive than operation, by many orders of magnitude[2]. Even if an AI devoted most of its resources to training a successor system, spending 1% of its compute running copies of itself seems like a free roll to increase its power (including by finding optimizations for the training runs).

Rebuttal B: A single AI will simply be able to make far fewer decisions per second than a team of AI. If speed is ever highly valued (and it usually is), a team can be more powerful by making many good decisions rather than a few great decisions.

A Future of AI Teams

If teaming up is always advantageous, we can still be headed for

AI takeover and human disempowerment

. However, the ...
What I Think About When I Think About History
by Jacob G-W — 2023-07-04T14:02:26.066Z
Link: https://g-w1.github.io/blog/observation/2023/06/03/whatithinkaboutwhenithinkabouthistory.html

[Epistemic status: I'm sure that this happens in me. Not so sure why; take all my speculations and extrapolations (to AI especially) with a grain of salt.]

Ever since learning more about history in the past year or two, I've started having a really weird sensory experience. I can see history. What I mean by this is that when I hear a date when something happened, I can place it in my field of view. The timeline also has different levels of zoom on a sort of logarithmic scale.

I think this experience/model all started when I learned the US Presidents using Anki. For every president, I learned the years they served, their picture (picture → name), and the party they belonged to. I did this not because I'm some crazy history buff, but just because I wanted to test how well Anki works. I found that Anki works remarkably well. I was able to memorize all the presidents and becuase of the nature of the spaced-repititon algorithm, still remember most (>95%) today.

As a side effect of memorizing all the presidents, I developed this weird timeline sense. I (semi-)concously tried to place the presidents in the timeline when remembering them. I think that visualizing this timeline in my head every day cemented it. I started to place other dates and historical facts that I learned on the same timeline. It became a self-organizing way to dump all my historical knowlege.

Here's how it works for me: When thinking in the context of the 20th and 21st centuries, which are the 2 most common centuries for me to think about, the 20th century starts at the left of my field of view. Maybe at 70° W (if N is straight ahead). The turn of the 21st century is at around 20° E. But I can also zoom in or out. For example, when I zoom in on my timeline to just the 21st century, I see the year 2000 at the left of my field of vision and 2023 at the far right. I can also zoom out from 0 AD to today, with similar locations. What I can't do, and I find this pretty interesting, is zoom in on the 5th century. Like sure, I can just pretend to visualize a timline of it in my head, but it does not feel the same as visualizing latter centuries. I think the reason for this is because I don't know many events that can ground the timeline in the 5th century. I know many more events that are in the latter centuries, so that could be a possible reason why I can visualize their timeline in more detail.

I wond...
My Time As A Goddess
by Evenstar — 2023-07-04T13:14:36.736Z

[Epistemic status: Retrospective on 6-year-old actions]

Back in 2017, there was Rationalist Tumblr, which I participated in; then Discord came along. A number of regulars from Rattumb created a server and started to hang out.

It was a lawless hive of scum and villainy at the time. It has since died. Names of moderators and members deliberately left out. At the time, I was very into hypnosis - so, naturally I seized the opportunity to rant about my special interest on the server. I had pet theories at the time about hypnosis (which need serious re-examination now) and I used them as a party trick to attract social attention and relationships. It was good clean kinky fun, except for the part where I was providing a mysterious answer to a mysterious question. I did not realize this at the time.

Anyway, I hung out on this server for a while and wrote a CYOA about serving a goddess of love, desire and devotion because I kinked on that. I was kicking around ideas about what my ideal religion was like - purely on a kink basis of course, I considered myself an atheist. (Despite styling myself as a witch at the time.) I was already plural at this time, though with a more restricted set of personae than I would eventually develop. One of my personae was a kitsune priestess named Kumi, who worshipped a goddess that I had also made up whole-cloth.

All of this would have been something I'd have forgotten, if there had not been an incident of drama on the server that led to the server owner (whom I disliked) styling themselves as a god-monarch.

I was annoyed by this. I didn't think they had the qualities of a proper deity, and were clearly taking the role far less seriously than they ought to. So I decided that I was going to make my own server, where I was a goddess, and run it properly and not have any of the problems of the main server.

So, pretty much completely as a joke, I started a cult.

The Kitsune Temple

I created a discord server, and roped in a couple friends of mine who were also vaguely interested in spirituality, hypnosis, and kink. I set down the temple laws, which have been lost now but were heavily based on the following set of commandments from my CYOA:

Goddess' Code:

Laws:

Always get consent.

Never cheat.

Never ignore a safeword.

Principles:

Share love, devotion and desire with the world.

Be unashamed of your body and your sexuality.

Find beauty in everyon

...
Twitter Twitches
by Zvi — 2023-07-04T13:00:06.111Z

The situation is evolving rapidly. Here’s where we stand as of the morning of July 4th.

Well You See What Happened Was…

Oh no! To be clear, by twitches, I mean ‘Elon refused to pay the cloud bill.’

As a result, Twitter has been forced to rate limit users.

This started out as 600 posts per day for most accounts, 300 posts per day for new accounts and 6,000 posts per day for those who pay.

This is now up to 1k/500/10k according to one Musk tweet.

If you are not logged in, you get nothing. Even direct links will break.

Tweetdeck has been forced into a new worse version, but now works again. In 30 days, this will be for paid accounts only, which seems fair.

That fourth one hurts my process. Navigation is somewhat slower and more annoying. In particular, forced threading breaks chronological order assumptions and one’s ability to use duplication to locate one’s place, and zooming in to move around twisting Twitter threads is so bad you need to jump to Twitter itself. Navigation to zoom back requires clicking in annoying places. I was unable to configure the column order without deleting them all and then creating them again, although this was quick. Column width and monitor real estate use is screwed up in subtle ways. Oh, and now its settings are linked to Twitter’s even though I want them to be different. Sheesh.

Another little thing is that the tab icon is now identical to Twitter’s. So annoying.

This is still vastly better than the period where Tweetdeck stopped working.

The third is brutal for some of my readers. Many report they can’t view any links.

What to do, if this doesn’t end soon?

The Plan

Three parts: How I will deal with processing info, how I will change how I present info, and how you can adjust to the new situation.

The efficiency hit on my end is unavoidable. I’ll make three adjustments.

I’ll check Twitter less often, rely more on other sources.

I’ll raise the bar somewhat for what is worth including or investigating.

I’ll shift some people from lists to follows and eliminate duplicates.

The inability for viewers to see Twitter links means moving away from them.

I’ll more often quote more of the relevant Twitter threads.

I’ll link directly to the primary source first whenever possible.

If you don’t have a Twitter account, you have a choice to make.

Having an account with zero connections fix

...
Rational Unilateralists Aren't So Cursed
by SCP — 2023-07-04T12:19:12.048Z

Much informal discussion of the Unilateralist’s Curse from Bostrom et al (2016) presents it as a sort of collective action problem: the chance of purely altruistic agents causing harm rises with the number of agents that act alone. What’s often left out is that these agents are irrational. The central result depends on this, and I’ll show why below.

Note that the formal result in the original paper is correct. The authors are largely aware of what I’ll explain here; they discuss much of it in section 3.2. The point of this post is (i) to correct a misconception about which agents the Curse applies to, (ii) to go through a particularly neat part of the game theory canon, and (iii) to highlight some nuances.

Summary

Even when agents have no information about what others think, it is not generally an equilibrium for them to act merely based on their own all-things-considered belief.
In equilibrium, rational and altruistic agents may or may not converge (in the limit) to avoiding the kind of mistake highlighted by the Unilateralist's Curse.
Rational and altruistic agents can still do far better by using, e.g., majority vote.
Empirical results suggest that agents do behave quite rationally in various contexts.

I suggest skipping the derivations if you're not interested.

Why Naïve Unilateralism Is Not an Equilibrium

The main theoretical insights I’ll explain are from Feddersen and Pesendorfer (1998).[1] I simply apply them to the case of the Unilateralist’s Curse. I’ll illustrate using the first example from Bostrom et al (2016, p. 1) where they motivate the issue:

A group of scientists working on the development of an HIV vaccine has accidentally created an air-transmissible variant of HIV. The scientists must decide whether to publish their discovery, knowing that it might be used to create a devastating biological weapon, but also that it could help those who hope to develop defenses against such weapons. Most members of the group think publication is too risky, but one disagrees. He mentions the discovery at a conference, and soon the details are widely known.

Next, a formal setup:[2]

There are n≥3 agents.
An agent can either write a paper (w) or stay silent (s).
If at least one agent writes the paper, it is published (W); if none do, it is not (S).
Having the paper written would in fact either be good (G) or bad (B) on net. For simplicity, the common prior is

...
The literature on aluminum adjuvants is very suspicious. Small IQ tax is plausible - can any experts help me estimate it?
by mikes — 2023-07-04T09:33:51.849Z

I am a PhD biostatistician who has worked on a project in this area. I am hoping to crowdsource opinions on this issue, especially from readers with knowledge of neurology, nephrology and/or toxicology.
The recent news cycle motivated me to spend some time reading up on the safety literature for aluminum vaccines. Aluminum is not in the Covid vaccine, but it is in many childhood vaccines.
I am deeply unsettled by what I found. Here is a summary. I would appreciate being corrected in the comments:
- The science on this topic is surprisingly neglected. Aluminum is a known neurotoxin in small concentrations in the brain. So, the obvious follow-up question: "Is the amount accumulated from childhood vaccines so small as to be irrelevant?" Unfortunately, aluminum has not undergone rigorous safety testing for cumulative cognitive effects. The fact that I could not find a convincing answer reflects a multi-decade policy failure.
-The evidence for safety cited by regulators and mainstream review papers appears to have gaping issues (see end of post for notes on the studies Mitkus et al (2011) and Karwowski et al (2018). They do not justify anywhere near the amount of faith being placed in them.)
-Mainstream educational sources are gaslighting readers by suggesting that the net exposure in vaccines is less than dietary exposure, e.g. here is what CHOP's Vaccine Education Research Center says
but this fails sanity checks with basic multiplication, given the oral bioavailability of aluminum is about 0.1 - 0.3% in diet and <0.01% in antacids according to the ATSDR. No wonder people are going nuts about the public health experts.
-There is a whole literature of claims on blood brain barrier mechanics, chemical state of the aluminum after injection, and transport via macrophages, which regulators are ignoring, but should probably affect the analysis of Mitkus 2011. I am entirely unqualified to judge and don't particularly trust this literature, so I'm ignoring this but looking forward to comments on the topic.
-Alternatives to aluminum exist, including mRNA and other new adjuvants. In the 2000's, regulators pushed industry to remove mercury from childhood vaccines; they should consider doing this again.
-A cognitive tax due to aluminum on the scale of ~0.1 IQ point for the first Hepatitis B shot is plausible (that's just from the first 250 micrograms of Al. There are > ...
Two Percolation Puzzles
by Adam Scherlis — 2023-07-04T05:34:19.441Z
Link: https://adam.scherlis.com/2023/07/04/two-percolation-puzzles/

Take a very large chessboard (NxN, where N is huge).

Remove some fraction 1-p of the squares at random, leaving a fraction p of them.

Can you place a queen on the first row and then, via some sequence of legal moves, get it to the last row?

This is probabilistic, of course, but it turns out the probability of success undergoes a sharp phase transition -- near-zero for small values of p, then suddenly rising almost to one in the vicinity of a critical value p_queen.

For different pieces, the critical value is different, e.g. p_rook for a rook. (Note that p_queen <= p_rook.)

Question 1: What is p_queen + p_rook?

Bonus Question: Why didn't I ask you for the values p_queen and p_rook separately?

Now let's invent a new chess piece, the bondsman. This piece can move like a rook, and is also allowed to move along northeast-southwest diagonals between white squares, and along northwest-southeast diagonals between black squares. (There's also the antibondsman who has the same abilities, but with the words "white" and "black" swapped.)

Question 2: What is p_bondsman?

Bonus Question: Why did I call the piece that?

Answers to follow in a later post.
Mechanistic Interpretability is Being Pursued for the Wrong Reasons
by Cole Wyeth — 2023-07-04T02:17:10.347Z
Link: https://colewyeth.com/interpretability.html

The success of prosiac A.I. seems to have generally surprised the rationalist community, including myself. Rationalists have correctly observed that deep learning is a process of mesa-optimization; stochastic gradient descent by backpropogation finds a powerful artifact (an artificial neural net or ANN) for a given task. If that task requires intelligence, the artifact is often successfully embued with intelligence. Some rationalists seem to have updated in the direction that there is no simple algorithm for intelligence, for instance giving Hanson's worldview credit for predicting this better (to be clear I do not know if the author actually endorses this viewpoint).

There is an argument to be made that the dependence of deep learning on huge datasets shows that intelligence is an accumulation of knowledge about the world and little useful algorithms. An example line of reasoning is as follows: There is not one central learning algorithm that bootstraps itself to superintelligence. All such attempts, like Eurisko, have failed. Instead, modern deep learning algorithms succeed through familiarity with a huge number of examples compressed among the weights to form a strong background understanding of the world, and many locally useful sub-algorithms are found to perform well on different types of examples, with the results aggregated to reach high overall performance. Note that the former claim is along the lines of the Cyc philosophy (with a more flexible representation) and the later is sometimes expressed by Robin Hanson; I think they are distinct but connected ideas.

I will argue that this viewpoint is not necessarily the correct takeaway, and apparently arises from a misinterpretation of the reasons for the success of deep learning. Further, this misinterpretation has led mechanistic interpretability to be pursued for the wrong reasons and with the wrong objectives. These problems in combination have corrupted the content of lesswrong.

My model of deep learning is that neural networks are circuits in the sense of theoretical computer science. That is, to understand trained models, forget the details of neural circuits in the brain. Forget the analogue nature of neural networks. Those are unnecessary complications. Floating point numbers have finite precision, and artificial neural nets are actually more efficient with similar performance when that precision ...