trentmkelly/LessWrong-43k huggingface dataset:
AI Timelines
by habryka — 2023-11-10T05:28:24.841Z

Introduction

How many years will pass before transformative AI is built? Three people who have thought about this question a lot are Ajeya Cotra from Open Philanthropy, Daniel Kokotajlo from OpenAI and Ege Erdil from Epoch. Despite each spending at least hundreds of hours investigating this question, they still still disagree substantially about the relevant timescales. For instance, here are their median timelines for one operationalization of transformative AI:

Median Estimate for when 99% of currently fully remote jobs will be automatable
Daniel	4 years
Ajeya	13 years
Ege	40 years

You can see the strength of their disagreements in the graphs below, where they give very different probability distributions over two questions relating to AGI development (note that these graphs are very rough and are only intended to capture high-level differences, and especially aren't very robust in the left and right tails).

In what year would AI systems be able to replace 99% of current fully remote jobs?

Median indicated by small dotted line.

In what year will the energy consumption of humanity or its descendants be 1000x greater than now?

Median indicated by small dotted line. Note that Ege's median is outside of the bounds at 2177

So I invited them to have a conversation about where their disagreements lie, sitting down for 3 hours to have a written dialogue. You can read the discussion below, which I personally found quite valuable.

The dialogue is roughly split in two, with the first part focusing on disagreements between Ajeya and Daniel, and the second part focusing on disagreements between Daniel/Ajeya and Ege.

I'll summarize the discussion here, but you can also jump straight in.

Summary of the Dialogue

Some Background on their Models

Ajeya and Daniel are using a compute-centric model for their AI forecasts, illustrated by Ajeya's draft AI Timelines report, and Tom Davidson's takeoff model where the question of "when transformative AI" gets reduced to "how much compute is necessary to get AGI and when will we have that much compute? (modeling algorithmic advances as reductions in necessary compute)".

Whereas Ege thinks such models should have a lot of weight in our forecasts, but that they likely miss important considerations and doesn't have enough evidence to justify the extraordinary predictions it makes.

Habryka's Overview of Ajeya & Daniel discussion

Ajeya thinks transl

...
ACI#6: A Non-Dualistic ACI Model
by Akira Pyinya — 2023-11-09T23:01:30.976Z

Most traditional AI models are dualistic. As Demski & Garrabrant have pointed out, these models assume that an agent is an object that persists over time, and has well-defined input/output channels, like it's playing a video game.

In the real world, however, agents are embedded in the environment, and there's no well-defined boundary between the agent and the environment. That's why a non-dualistic model is needed to depict how the boundary and input/output channels emerge from more fundamental notions.

For example, in Scott Garrabrant's Cartesian Frames, input and output can be derived from "an agent's ability to freely choose" among "possible ways an agent can be".

However, choosing is still one of the key concepts of Cartesian Frames, but from a non-dualistic perspective, "it's not clear what it even means for an embedded agent to choose an option", since an embedded agent is "the universe poking itself". Formalizing the idea of choice in a non-dualistic model is as difficult as formalizing the idea of free will.

To avoid relying on the notion "choosing", we have proposed the General Algorithmic Common Intelligence (gACI) model which describes embedded agents solely from a third-person perspective, and measures the actions of agents using mutual information in an event-centric framework.

The gACI model does not attempt to answer the question "What should an agent do?". Instead, it focuses on describing the emergence of the agent-environment boundary, and answering the question "Why does an individual feel like it's choosing?"

In the language of decision theory, gACI belongs to descriptive decision theory rather than normative decision theory.

Communication Channel and Mutual Information

In dualistic intelligence models, an agent receives input information from the environment, and manipulates the environment through output actions. But real-world agents are embedded within the environment, it's not easy to confine information exchange to a clear input/output channel.

In the gACI model, on the other hand, the input/output channel is a communication channel, in which the information transfer between a sender and a receiver is measured by mutual information.

Figure 1: From the dualistic input/output model to the mutual information model.

We can easily define mutual information betwe...
How I got so excited about HowTruthful
by Bruce Lewis — 2023-11-09T18:49:06.556Z

This is the script for a video I made about my current full-time project. I think the LW community will understand its value better than the average person I talk to does.

Hi, I'm Bruce Lewis. I'm a computer programmer. For a long time, I've been fascinated by how computers can help people process information. Lately I've been thinking about and experimenting with ways that computers help people process lines of reasoning. This video will catch you up on the series of thoughts and experiments that led me to HowTruthful, and tell you why I'm excited about it. This is going to be a long video, but if you're interested in how people arrive at truth, it will be worth it.

Ten or 15 years ago I noticed how online discussion forums really don't work well for persuading people of your arguments. Instead of focusing on facts, people get sidetracked, make personal attacks, and go in circles. Very rarely do you see anyone change their mind based on new evidence.

This got me thinking about what might be a better format than posts, comments and replies for arguments. I thought about each statement being its own thing, and statements would be linked by whether one statement argues for or against another statement. If you repeat a statement it would use the existing thing rather than making a new one, making it easier to avoid going in circles. And it would encourage staying focused on the topic at hand and not getting sidetracked.

I kept this idea in the back of my head, but didn't do anything with it.

A few years later, I was baffled by the success of Twitter. Its only distinguishing feature at that time was that you were limited to 140 characters. Everybody complained about this. But I started to think the limit was the secret to its success. Yes, it's a pain that you're limited to 140 characters. You have to work hard to make what you want to say concise enough to fit. But the great thing was that everyone else also had to work hard to be concise.

So the idea of forced conciseness stirred around in the back of my mind and started to mix with my other idea about a format for arguments that would help people stay focused. And then a third idea joined them, for making it quick and easy for people when they're ready to change their mind.

In 2018, when political discussion in my country was getting very polarized, these three ideas came to the front of my mind and I started workin...
The case for "Generous Tit for Tat" as the ultimate game theory strategy
by positivesum — 2023-11-09T18:41:03.879Z
Link: https://tryingtruly.substack.com/p/how-generous-tit-for-tat-wins-at-life

I wish I could title this How “Pacifism” wins at life, since it’d immediately make everything much easier (especially writing this post), but as most well-meaning kids swiftly learn on the playground - the world is not gentle.

It’s always bothered me how impossible it is to negate the need for strength in life (whether physical, emotional, individual or collective), especially considering how unevenly and unfairly that stat is usually distributed.

What bothers me even more however, is how many people I’ve seen gleefully misapply and overfit this fact, falsely concluding that since one cannot survive without strength, it is the only meaningful form of power in the world. I’m sure you’ve seen these people too.

There is another side to this coin however. One that I care deeply about, and that carries just as much weight and impact as anything mentioned above: if you ever want to get anywhere as a person, society, or species, you can’t afford to negate gentleness either.

Karate is the perfect intuitive manifestation of this principle. Respect and conflict aversion are its core philosophy, but not because of a lack of viable alternatives.

Quick note: We’re gonna dive into game theory here, so if you’re new to/rusty on the terminology, the single best thing you can do is click here for a brilliant interactive crash course on all the meaningful bits. If you’re short on time you’re better off prioritizing this click rather than reading the rest of this post (Nicky Case is amazing)!

Many smartass variations of the prisoner’s dilemma have been cooked up over the years, with each change in the parameters of the game bringing along with it a new balance of incentives, and different winning strategies accordingly.

Should you make like Jesus and always turn the other cheek, or should you play like Lucifer himself and ruthlessly exploit everyone you meet? Maybe you need a complex web of conditions defining how you act towards others based on your relationship so far, or maybe you can just do unto others as they do unto you?

Every variation teaches us something new about the dynamics of selfishness and selflessness, trust and defection, all while generating a rich abundance of smug nerds to debate which ones apply best to which real life situation.

Debate is cool and all, but what I don’t understand is why all said nerds don’t just stop and yell out in unison:

WE’VE MAYBE ACTUALLY KI...
Text Posts from the Kids Group: 2021
by jefftk — 2023-11-09T17:50:25.782Z

Another round of liberating kid posts from

Facebook. For reference,
in 2021 Lily turned 7, Anna turned 5, and Nora was born.

(Some of these were from me; some were from Julia. Ones saying "me"
could mean either of us.)

2021-01-02

Anna: Hello, I'm Mr. Hamburger.

Me: It's time to brush teeth, Mr. Hamburger.

Anna: I can't brush my teeth, I'm a hamburger.

Me: It's still time to brush teeth.

Anna: Hamburgers don't have teeth.

2021-01-04

"Anna, try bonking your head into the faucet! I tried it, and the new squishy cover works!"

2021-01-10

Last week Lily said she wanted bangs. I told her there is a three-week waiting period for any major haircut, and set a calendar reminder for us to talk about it again in three weeks. She agreed.

Two days later, she asked, "If I have bangs, will all my hair be short?"

I asked, "...Do you know what bangs are?"

"No."

2021-01-25

We've been reading "The Boxcar Children", and the kids are excited about playing at roughing it in the woods. Lily came downstairs with a pillowcase full of stuff.

"Mom, we're pretending we are some poor people and we found just enough money to buy two couches, two pillows, a cooking pot, some stuffies, and this necklace. And I had just enough money to buy this pirate ship and two dolls."

2021-01-27

"Dad, why are sponges squishy? Like mice?"

2021-01-27

Jeff: Goodnight, Anna.

Anna: Oy-yoy-yoy-yoy-yoy! That's baby for "You're the best dad in the world."

2021-02-01

Woke up to Lily reading to Anna

2021-02-03

Hypothetical from Lily: "Mom, if you lived in a peanut shell and the only food you had was cheez-its this big" [holds up fingers to pea size] "and you slept in a shoe made of stone, and ten hundred children lived there, would you find somewhere else to live?"

2021-02-06

From Lily at dinner:

"There is something that makes me sad.
[begins singing] Fairies aren't real
Magic isn't real
Unicorns aren't real
Santa Claus isn't real
The aTooth Fairy isn't real."

2021-02-14

Lily, explaining the difference between even and odd numbers: "If they could all line up for a contra dance and they'd all have a partner, that's even."

2021-02-15

Lily: "Anna, why did you hit me with the whistle?"

Anna, not wearing glasses or anything: "I'm sorry, my sight had gotten fogged up"

2021-02-15

One of Lily's favorite conv...
AI #37: Moving Too Fast
by Zvi — 2023-11-09T17:50:04.324Z

We had OpenAI’s dev day, where they introduced a host of new incremental feature upgrades including a longer context window, more recent knowledge cutoff, increased speed, seamless feature integration and a price drop. Quite the package. On top of that, they introduced what they call ‘GPTs’ that can let you configure a host of things to set up specialized proto-agents or widgets that will work for specialized tasks and be shared with others. I would love to mess around with that, once I have the time, and OpenAI’s servers allow regular subscribers to get access.

In the meantime, even if you exclude all that, lots of other things happened this week. Thus, even with the spin-off, this is an unusually long weekly update. I swear, and this time I mean it, that I am going to raise the threshold for inclusion or extended discussion substantially going forward, across the board.

Table of Contents

OpenAI Dev Day is covered in its own post. Your top priority.

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Help design new chips.

Bard Tells Tales. It is the rare bard that knows how to keep a secret.

Fun With Image Generation. What exactly are we protecting?

Deepfaketown and Botpocalypse Soon. Some signs, mostly we keep waiting.

The Art of the Jailbreak. The new strategy is a form of persona modulation.

They Took Our Jobs. Actors strike is over, future of movies may be bright.

Get Involved. MIRI, Jed McCaleb, Davidad all hiring, MATS applications open.

Introducing. Lindy offers their take on GPTs, Motif improves against NetHack.

X Marks its Spot. Elon Musk and x.AI present Grok, the AI that’s got spunk.

In Other AI News. Amazon and Samsung models, notes on Anthropic trustees.

Verification Versus Generation. Can you understand what you yourself generate?

Bigger Tech Bigger Problems. A profile of White House’s Bruce Reed.

Executive Order Open Letter. Refreshingly reasonable pushback, all considered.

Executive Order Reactions Continued. Everyone else the reactions post missed.

Quiet Speculations. Could we perhaps figure out how to upload human brains?

The Quest for Sane Regulation. Has it all. Proposals, polls, graphs, despair.

The Week in Audio. Flo Crivello and Dan Hendrycks.

Rhetorical Innovation. Get your updates and your total lack thereof.

Aligning a Smarter Than Human I

...
Learning-theoretic agenda reading list
by Vanessa Kosoy — 2023-11-09T17:25:35.046Z

Recently, I'm receiving more and more requests for a self-study reading list for people interested in the learning-theoretic agenda. I created a standard list for that, but before now I limited myself to sending it to individual people in private, out of some sense of perfectionism: many of the entries on the list might not be the best sources for the topics and I haven't read all of them cover to cover myself. But, at this point it seems like it's better to publish a flawed list than wait for perfection that will never come. Also, commenters are encouraged to recommend alternative sources that they consider better, if they know any. So, without further adieu:

General math background

Theoretical computer science

"Computational Complexity: A Conceptual Perspective" by Goldreich (especially chapters 1, 2, 5, 10)
“Lambda-Calculus and Combinators: An Introduction” by Hindley
“Tree Automata Techniques and Applications” by Comon et al (mostly chapter 1)

"Introductory Functional Analysis with Applications" by Kreyszig (especially chapters 1, 2, 3, 4)
"Probability: Theory and Examples" by Durret (especially chapters 4, 5, 6)
"Elements of Information Theory" by Cover and Thomas (especially chapter 2)
“Game Theory, Alive” by Karlin and Peres
“Categories for the Working Mathematician” by Mac Lane (especially parts I, III, IV and VI)

AI theory

“Handbook of Markov Decision Processes” edited by Feinberg and Shwartz (especially chapters 1-3)
“Aritifical Intelligence: A Modern Approach” by Russel and Norvig (especially chapter 17)
"Machine Learning: From Theory to Algorithms" by Shalev-Shwarz and Ben-David (especially part I and chapter 21)
"An Introduction to Computational Learning Theory" by Kearns and Vazirani (especially chapter 8)
"Bandit Algorithms" by Lattimore and Szepesvari (especially parts II, III, V, VIII)

Alternative/complementary: "Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems" by Bubeck and Cesa-Bianchi (especially sections 1, 2, 5)

“Prediction, Learning and Games” by Cesa-Bianchi and Lugosi (mostly chapter 7)
"Universal Artificial Intelligence" by Hutter

Alternative: "A Theory of Universal Artificial Intelligence based on Algorithmic Complexity” (Hutter 2000)
Bonus: “Nonparametric General Reinforcement Learning” by Jan Leike

Reinforcement learning theory

Video and slides: "Introduction to Reinforcement Learning Theory"
"Near-optimal Regret Bounds for R

...
​​ Open-ended/Phenomenal ​Ethics ​(TLDR)

by Ryo  — 2023-11-09T16:58:18.889Z

This is a short version of the more complete post
More clarity and context here
I formulate a desiderata/procedure (called "phenomenal ethics" or "open-ended ethics") :

Expanding the action space and autonomy of a maximum of phenomenons (enhancing ecosystemic values, and the means to reach/understand options), modulated by variables mitigating frantic optimizations (to respect natural evolving rates etc).

​ ​​

Utilitarianism based on well-being demands to define well-being,
It demands to maximize happiness, but what is happiness?
The classical issues arise : amorphous pleasure, wireheading, drug addiction etc.
We need to find a tractable desiderata securing actions in an adequate value space.

I argue that the production of "well-being" is intrinsic to phenomenal/open-ended ethics. An ideal enactment of such procedure inherently maximizes the possibility of anyone/anything to do as they will; if *what they will* enhances the possibility of other 'things' to do *as they will* as well.

As a concept it's quite elementary, but how to compute it properly?

The longer version of this post tries to expand on that.
The aim is to provide affordance, access to adjacent possibles, allow phenomena, ecosystems and individuals to bloom, develop diversity in as many dimensions as possible. I include in the desiderata phenomena that aren't considered alive, any physical event is included (which is why I call it "

phenomenal ethics

")
An agent (human/AI/AGI/ASI etc.) has to enhance the autonomy and available actions of existing "behaviors" (phenomenon/environment/agents etc.), which implies selecting out behaviors that aren't causally beneficial to other behaviors (ecosystemic well-being).

Frantic transformation of everything in order to hunt new dimensions of possibilities is to be avoided, so we need to relativize the desiderata with other constraints added to the equation, (those parameters aren't absolute but variables for value attribution) :

Existing phenomena is prioritized over potential phenomena

The intensity of impact of actions has to be minimized the higher uncertainty is

Phenomena untainted by AI's causal power are prioritized over phenomena tainted by it

Normal rates of change are to be prioritized over abnormal ones

More care is to be given to phenomena in qualia's continuum
Polysemantic Attention Head in a 4-Layer Transformer
by Jett Janiak — 2023-11-09T16:16:35.132Z

Produced as a part of MATS Program, under @Neel Nanda and @Lee Sharkey mentorship

Epistemic status: optimized to get the post out quickly, but we are confident in the main claims

TL;DR: head 1.4 in attn-only-4l exhibits many different attention patterns that are all relevant to model's performance

Introduction

In

previous post

about the docstring circuit

, we found that attention head 1.4 (Layer 1, Head 4) in a

4-layer attention-only transformer

would act as either a fuzzy previous token head or as an induction head in different parts of the prompt.
These results suggested that attention head 1.4 was polysemantic, i.e. performing different functions within different contexts.
In

Section 1

, we classify ~5 million rows of attention patterns associated with 5,000 prompts from the model’s training distribution. In doing so, we identify many more simple behaviours that this head exhibits.
In

Section 2

, we explore 3 simple behaviours (induction, fuzzy previous token, and bigger indentation) more deeply. We construct a set of prompts for each behaviour, and we investigate its importance to model performance.
This post provides evidence of the complex role that attention heads play within a model’s computation, and that simplifying an attention head to a simple, singular behaviour can be misleading.

Section 1

Methods

We uniformly sample 5,000 prompts from the model’s training dataset of

web text

and

code

.
We collect approximately 5 million individual rows of attention patterns corresponding to these prompts, ie. rows from the head’s attention matrices that correspond to a single destination position.
We then classify each of these patterns as (a mix of) simple, salient behaviours.
If there is a behaviour that accounts for at least 95% of a pattern, then it is classified. Otherwise we refer to it as unknown (but there is a multitude of consistent behaviours that we did not define, and thus did not classify)

Results

Distribution of behaviours

Figure 1: Distribution of attentional behaviours across training distribution (all), and for specific destination tokens.

In Figure 1 we present results of the classification, where "all" refers to "all destination tokens" and other labels refer to specific destination tokens.
Character · is for a space, ⏎ for a new line, and labels such as ⏎[·×K]mean "\n and K spaces".
We distinguish the following behavio

...