trentmkelly/LessWrong-43k huggingface dataset:
[SEE NEW EDITS] No, *You* Need to Write Clearer
by Nicholas / Heather Kross — 2023-04-29T05:04:01.559Z
Link: https://www.thinkingmuchbetter.com/nickai/fieldbuilding/no-you-need-to-write-clearer.html

This post is aimed solely at people in AI alignment/safety.

EDIT 3 October 2023: This post did not even mention, let alone account for, how somebody should post half-baked/imperfect/hard-to-describe/fragile alignment ideas. Oops.

LessWrong as a whole is generally seen as geared more towards "writing up ideas in a fuller form" than "getting rapid feedback on ideas". Here are some ways one could plausibly get timely feedback from other LessWrongers on new ideas:

Describe your idea on a LessWrong or LessWrong-adjacent Discord server. The adjacent servers (in my experience) are more active. For AI safety/alignment ideas specifically, try describing your idea on one of the Discord servers listed here.

Write a shortform using LessWrong's "New Shortform" button.

If you have a trusted friend who also regularly reads/writes on LessWrong: Send your post as a Google Doc to that friend, and ask them for feedback. If you have multiple such friends, you can send the doc to any of all of them!

If you have 100+ karma points, you can click the "Request Feedback" button at the bottom of the LessWrong post editor. This will send your post to a LessWrong team member, who can provide in-depth feedback within (in my experience) a day or two.

If all else fails (i.e. few or no people feedback your idea): Post your idea as a normal LessWrong post, but add "Half-Baked Idea: " to the beginning of the post title. In addition (or instead), you can simply add the line "Epistemic status: Idea that needs refinement." This way, people know that your idea is new and shouldn't immediately be shot down, and/or that your post is not fully polished.

EDIT 2 May 2023: In an ironic unfortunate twist, this article itself has several problems relating to clarity. Oops. The big points I want to make obvious at the top:

Criticizing a piece of writing's clarity, does not actually make the ideas in it false.

While clarity is important both (between AI alignment researchers) and (when AI alignment researchers interface with the "general public"), it's less important within small groups that already clearly share assumptions. The point of this article is, really, that "the AI alignment field" as a whole is not quite small/on-the-same-page enough for shared-assumptions to be universally assumed.

Now, back to the post...

So I was reading this post, which basically asks "How do we get Eliezer Y...
Programming an IFS for alternate uses
by SilverFlame — 2023-04-29T00:25:38.980Z

I have been reading the sequence Multiagent Models of Mind by @Kaj_Sotala, and the Internal Family System concepts it discusses resemble something I have personally experimented with.

The sequence and its sources seem to focus on the use of IFS as a therapy/psychoanalysis tool, and I do not disagree with that use case and its importance. However, I have ended up taking the ideas of an IFS in a different direction: coaxing the internal agents into an architecture where their actions and reactions harmonize to produce system-wide benefits. I have not seen any posts here regarding this idea, and thought it worth sharing.

It would be quite a rabbit hole to go too far into where I have taken those experiments, so I have selected an architectural concept to represent the conceptual idea. Keep in mind that this sort of system takes a large amount of awareness regarding your personal symbolism, worldview, and computational preferences. It is also difficult to yield major benefits without applying patches to your System I programming.

From an outsider's perspective, I would describe my IFS as being based upon an Agile programming team, with roles from Scrum inspiring some of the specialization of the internal agents and XP (Extreme Programming) serving as the normal operating state. The agents perform actions based upon the XP development cycle, developing tests and designing solutions to pass those tests before prototyping the ones that seem most promising. If the solution passes the tests, it gets stripped of any unnecessary components and goes to integration, after which it becomes part of the baseline.

Also, I should note that I am still unfamiliar with much of the normal terminology around here. Feel free to ask if any of my phrasings or word choice seem strange.
Utopia for artificial minds
by Ocracoke — 2023-04-28T22:52:13.843Z

TLDR: Alignment as we currently define it seems hard because we recognize that what we as humans want is pretty arbitrary from a non-human perspective. In contrast, global utility maximization is something that an ASI might independently discover as a worthwhile goal, regardless of any human alignment attempts. Global utility maximization could take the form of creating lots of blissful artificial minds, while keeping existing biological minds as happy as possible. If we start to work towards that vision now, we will create a lot of (non-human) utility, and we might to some extent prevent human-AI-tribalism in the future.

Utopia for humans might be unattainable

It's difficult for me to imagine a non-dystopian future for humanity, even if we do solve alignment. If humans stay in charge (which I consider unlikely in the long run), we will prey on each other as we always have (at least occasionally). If a benevolent AI ends up being in charge, and it tries to make everyone maximally happy and fulfilled all the time, it will quickly run into limits unless it creates Matrix-like simulations for everyone, or puts everyone on drugs. The two major obstacles to creating utopia for humans are our tendency to quickly get used to any improvements in our lives, and the fact that our interests aren't perfectly aligned with other people's interests. To give a concrete example, the intense positive feelings that are often present early in romantic relationships usually don't last as long as we'd like, and romantic love is all too often not mutual. I don't think there is much an AI will be able to do about that (at least as long as it lacks a physical human body).
Maybe the AI can will be able to give us a utopia where the lower tiers of the Maslow pyramid of needs are met for most people, but I doubt it will be able to give us a utopia where everyone is in a constant state of ecstatic love.

So far, it made sense to mostly think of humans when talking about utility maximization, for two reasons:

It seems likely that some form of complex cognition is a prerequisite for having qualia, and many animals don't have complex cognition (compared to humans).
Maximizing human utility is hard enough, and thinking too much about topics like wild animal suffering can be demotivating if there is the sense that the stakes are high, but that not much can be done.

Those two reasons may not be valid...
Locating Fulcrum Experiences
by LoganStrohl — 2023-04-28T20:14:03.644Z

Note: There are two appendices at the end of this post. The first contains a summary of the steps I’ve described here. The second is a glossary of key terms I’ve introduced in this essay.

Once you’ve booted up some curiosity and original seeing—perhaps even identified a question that’s crucial to your story—it’s time to start making observations. At this stage, the purpose of these observations is not so much to find an answer to your question, as to get in closer contact with the world so you’re well positioned to ask better questions.

But how do you determine which parts of the world are relevant to your topic, especially when you know that your basic conceptualization of the issue may be flawed?

Standard approaches to this problem include using a working model to make testable predictions, and seeking expert advice. Here we'll take a more bottom-up approach; instead of investigating large models from the start, or relying on the models of others, naturalism focuses first and foremost on the immediate sense data you can personally gather. You'll direct your data-gathering (even in the absence of any coherent model!) by identifying something I call a fulcrum experience.

What is a fulcrum experience?

A fulcrum experience is a collection of sensations that would lead you to relate differently to your topic if you observed it closely.

For example, when I studied courage[1], “fear” turned out to be a fulcrum experience for me. After really paying attention to experiences of fear in detail, I automatically thought about courage, bravery, cowardice, and related topics quite differently from how I had before. In the past, I’d implicitly assumed that being courageous meant taking actions I am afraid of. By observing fear, I learned about a range of cognitive reactions to the experience of being afraid; I came to think instead that courage has much more to do with where my actions come from, and how that relates to my overall system of values, than with which actions I ultimately take.

Once I was really paying attention to my experiences surrounding fear, my previous conceptualization simply crumbled. After enough study, a new way of thinking gradually emerged, founded more on direct observation than on vague storytelling. Therefore, fear was a fulcrum experience.

Locating fulcrum experiences is the first main phase of naturalist study. By “locating fulcru...
"notkilleveryoneism" sounds dumb
by bhauth — 2023-04-28T19:46:01.000Z

"AI safety" and "AI alignment" now often mean "making AI not say racist/illegal/etc things". The term "AI notkilleveryoneism" is now sometimes being used as a replacement that refers specifically to ASI-related risks.

I think "notkilleveryoneism" is the wrong term to use, for reasons including:

The basic structure of simple words mashed together with a suffix is associated with fantasy, media for kids, and signalling lack of effort.

The "ism" suffix brings up associations to religions. You don't want to say "we are the tribe of people whose thing is opposition to X" - you just want to say "we're opposed to X".

It's awkward to say, not catchy.

There are broader concerns than AI literally killing everyone, and you probably want as big an alliance as possible.

What, then, might be better? Perhaps an analogy to biological or nuclear safety would be good, eg:

Stop AI Gain-of-Function Research

Stop AI Proliferation

Another angle is "enjoy the AI summer" framing. The ability of GPT-4 to fill out forms, the art generation by the various LoRA models...even just the ability to do good sentence embeddings and fast vector search - all these things will have big impacts in businesses, but it will take some time for people to figure out how to use them most effectively. You could say things like:

Enjoy the AI summer.

Taste the fruit before planting more.

Yet another angle is the "figure out how to distribute the gains equitably before moving on" and "AI power dynamics" framing. For example:

Stop (Chinese-style) AI surveillance.

Don't let them make you obsolete. (like horses)

There are a number of possibilities, and I'm not proposing a single thing in particular, I'm just saying that people should use other phrases than "AI notkilleveryoneism".

I thought of the following terms for technical discussion:

U-al = user-alignment

O-al = owner-alignment

S-al = society-alignment

H-al = humanity-alignment

I-al = intelligence-alignment

But for public-facing statements, such abbreviations are obviously unsuitable.
Conflict Resolution: the Game
by CrimsonChin — 2023-04-28T19:32:13.637Z

epistemic status: very low. I am not a domain expert here. This was supposed to be a book review of Non-Violent communication by Marshall Rosenberg, but I got carried away. I welcome criticism and feedback.

Intro

Congratulations on purchasing the popular board game, "Conflict Resolution"! You may have heard of it or even played it before, but it's important to read the rules thoroughly to fully understand the game. Most importantly, have fun!

Goal

The goal of the game is to resolve a conflict between two parties under guidelines that protect both parties from mistreatment, abuse, or further hurt feelings. In an ideal world, we would resolve conflict by plugging our brains into each other, living each other’s life experiences, and understanding where the other person came from. However, this is not possible, so we attempt to communicate our experiences to each other. The issue is that mistakes can be made, lies told, and situations escalated. The game aims to solve this problem by providing a set of rules that make it hard to take advantage of the situation. When used correctly, these rules can ease a situation and resolve conflicts without confrontation. Our game attempts to solve this issue by a set of rules that we think would be hard to take advantage of. When used properly these can ease a situation and solve issues without confrontation.

Rules

The game is for two players, but the rules can be scaled up for more players.

Assume that conflict occurs when one party is hurt by the actions of the other. If both parties are hurt, separate the two-way-hurt game into two different games. The roles are as follows:

Hurt: This is the person who feels hurt

Friend: This is the person whose actions caused the hurt

Remember those roles, they are going to get a lot of mileage in this post.

Starting the game

1) Hurt is allowed to express their hurt feelings without judgment.

Anyone can express how others' actions made them feel at any time. No one is ever wrong about how they feel. If someone feels hurt, no one can deny that.

2) Hurt doesn't need to start a game every time they are hurt.

Discussing feelings isn't owed to Friend, and Hurt does not need to be consistent with what they bring up or let slide.

Hurt's allowed moves

1) Hurt can state the following:

The action that made them hurt
Their emotional reaction (that they are hurt)

2) Hurt is not allow...
LLMs and computation complexity
by Jonathan Marcus — 2023-04-28T17:48:03.762Z

Epistemic status: Speculative. I've built many large AI systems in my previous HFT career but have never worked with generative AIs. I am leveling up in LLMs by working things out from base principles and observations. All feedback is very welcome.

Tl;dr: An LLM cannot solve computationally hard problems. Its ability to write code is probably its skill of greatest potential. I think this reduces p(near term doom).

An LLM takes the same amount of computation for each generated token, regardless of how hard it is to predict. This limits the complexity of any problem an LLM is trying to solve.

Consider two statements:

"The richest country in North America is the United States of ______"
"The SHA1 of 'abc123', iterated 500 times, is _______"

An LLM's goal is to predict the best token to fill in the blank given its training and the previous context. Completing statement 1 requires knowledge about the world but is computationally trivial. Statement 2 requires a lot of computation. Regardless, the LLM performs the same amount of work for either statement.

It cannot correctly solve computationally hard statements like #2. Period. If it could, that would imply that all problems can be solved in constant time, which is provably (and obviously) false.

Why does this matter? It puts some bounds on what an LLM can do.

Zvi writes:

Eliezer Yudkowsky does not see any of this as remotely plausible. He points out

that in order to predict all the next word in all the text on the internet and all similar text, you need to be able to model the processes that are generating that text. And that predicting what you would say is actually a good bit harder than it is to be a being that says things - predicting that someone else would say is tricker and requires more understanding and intelligence than the someone else required to say it, the problem is more constrained.

And then he points out that the internet contains text whose prediction outright requires superhuman capabilities, like figuring out hashes, or predicting the results of scientific experiments, or generating the result of many iterations of refinement. A perfect predictor of the internet would be a superintelligence, it won’t ‘max out’ anywhere near human.

I interpret this the opposite way. Being a perfect predictor of the internet would indeed require a superintelligence, but it cannot be done by an LLM.

How does an LLM compute?

Wh...
Student competition for drafting a treaty on moratorium of large-scale AI capabilities R&D
by Nayanika — 2023-04-28T17:45:54.453Z

Campaign for AI safety has announced a competition for the drafting of an international treaty on moratorium of large-scale AI capabilities research and development.

The competition is open to all students of law, philosophy, and other relevant disciplines. The competition is organized by the Campaign for AI Safety, an Australian unincorporated association of people who are concerned about the risks of AI.

Competition brief: The goal of the competition is to create a draft treaty document that is based on and inspired by the suggestions of the article Pausing AI Developments Isn’t Enough. We Need to Shut it All Down, including the provisions on:

Shutting down large GPU and TPU clusters (the large computer farms where the most powerful AIs are refined).
Prohibition of training ML models (or combinations of models) with more than 500 million parameters.
Prohibition of the use of quantum computers in any AI-related activities.
A general moratorium of large-scale AI capabilities research and development.
Passing of national laws criminalizing the development of any form of Artificial General Intelligence (AGI) or Artificial Superintelligence (ASI).
Establishment of an international body to oversee the treaty. Effective mechanisms for enforcement of the treaty.
The treaty must not expire until it is universally agreed that it is safe and ethical to resume large-scale AI capabilities research and development.
Deadline for submissions: 15 July 2023 (subject to extension).

Prizes: The winner will receive a prize of AUD 4000. The runner-up will receive a prize of AUD 1000. The third place will receive a prize of AUD 500.

How to participate:

1)Read the competition brief above.

2)Draft a treaty: The treaty should be in English and should be no longer than 10 pages. The treaty should be submitted in Word format.

3)Submit your draft: Please e-mail your draft to nayanika.kundu@campaignforaisafety.org. Please include your name, university, and country in the e-mail.

4)Wait for the results: The results will be announced on 1 Aug 2023. Judging criteria

The judges will evaluate the drafts based on the following criteria:

Clarity: The treaty should be clear and easy to understand.
Legality: The treaty should be legally binding.
Effectiveness: The treaty should be effective in achieving its goals.
Comprehensiveness: The t

...
The Networked Memory Hierarchy Overhang
by austinsilveria — 2023-04-28T17:44:09.215Z

tl;dr

It seems likely, based on current capabilities of Transformers, that in the near future humans will engineer systems to maximize real time communication bandwidth between AI and all physical systems (including humans) by saturating networked models of hierarchical size between data centers and edge devices with streaming inference computation. This could accelerate AI’s gathering of new training data through rich interactions with external systems, increase the effectiveness of existing alignment capabilities, and amplify human intelligence. Humans can build infrastructure to steer this capability toward net beneficial applications and alignment of existing systems.

In this post, I:

Consider that AI communication bandwidth and predictive power may enter a recursive loop
Reason about implications
Present experimental results and analyze the Transformer's potential to scale AI communication bandwidth
Discuss useful human actions to hedge against the potential risks of this overhang collapse

What's an example of high bandwidth communication?

Borrowing from Connor Leahy’s

post

, let’s start with three (unqualified) assumptions:

“All human behavior and thought is fully describable by some Turing-complete computation.
Computing hardware and algorithms will continue to improve until they hit some physical limit.

That limit is still very far away

, and the human brain is nowhere near it.”

From these assumptions, we’ll construct an example scenario and work toward deriving the following statement: at some point in time, an AI can approximately simulate human behavior far enough into the future, and with enough accuracy, to mask network latency, resulting in imperceptible human-AI communication latency.

A human is in an empty room with a personal computer, a webcam, and a microphone. An AI is connected to a powerful data center across the internet and the not-so-powerful personal computer. The AI can listen to the webcam, microphone, and text input and control a 2D avatar to communicate with the human via speech, text, facial expressions, and gestures.

The AI streams its perception to the data center where it uses a large amount of parallel computation to approximately simulate its future perception and actions. This approximation, a large vector, is sent back to the personal computer. The AI then uses the personal computer to slightly correct this approximation based...
Hiring for the role of AI Fellow
by Nidhi — 2023-04-28T17:42:28.257Z
Link: https://www.linkedin.com/jobs/view/3584957749/

Hi all, if you work deeply on AI policy research or know someone who does, our think tank has an exciting new role open for an AI Fellow. This will entail research and policy work across the spectrum of areas in AI ethics, safety, geopolitics and other aspects. Please do share this in your networks and send us referrals of candidates!
[Linkpost] Sam Altman's 2015 Blog Posts Machine Intelligence Parts 1 & 2
by OliviaJ — 2023-04-28T16:02:00.060Z

I'm often surprised more people haven't read Open AI CEO Sam Altman's 2015 blog posts Machine Intelligence Part 1 & Part 2. In my opinion, they contain some of the most strong, direct, and clear articulations of why AGI is dangerous from a person at an AGI company.

(Note that the posts were published before OpenAI was founded. There's a helpful wiki of OpenAI history here.)

Hence: a linkpost. I've copied both posts directly below for convenience. I've also bolded a few of the lines I found especially noteworthy.

Machine intelligence, part 1

This is going to be a two-part post—one on why machine intelligence is something we should be afraid of, and one on what we should do about it. If you’re already afraid of machine intelligence, you can skip this one and read the second post tomorrow—I was planning to only write part 2, but when I asked a few people to read drafts it became clear I needed part 1.

WHY YOU SHOULD FEAR MACHINE INTELLIGENCE

Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could. Also, most of these other big threats are already widely feared.

It is extremely hard to put a timeframe on when this will happen (more on this later), and it certainly feels to most people working in the field that it’s still many, many years away. But it’s also extremely hard to believe that it isn’t very likely that it will happen at some point.

SMI does not have to be the inherently evil sci-fi version to kill us all. A more probable scenario is that it simply doesn’t care about us much either way, but in an effort to accomplish some other goal (most goals, if you think about them long enough, could make use of resources currently being used by humans) wipes us out. Certain goals, like self-preservation, could clearly benefit from no humans. We wash our hands not because we actively wish ill towards the bacteria and viruses on them, but because we don’t want them to get in the way of our plans.

(Incidentally, Nick Bostrom’s excellent book “Superintelligence” is the best thing I’ve seen on this topic. ...