trentmkelly/LessWrong-43k huggingface dataset:
Exporting Hangouts History
by jefftk — 2022-09-02T15:00:01.554Z

Yesterday I received a worrying email:

From: The Google Apps Team

Subject: Important information about the upgrade from Google Hangouts to Google Chat

Earlier this year, we announced that all Hangouts users would be
upgraded to Google Chat. As the last step of the upgrade, starting
November 1, 2022, Hangouts on the web will be upgraded to Chat on the
web. We are emailing you because some of your conversations or
portions of conversations won't automatically migrate from Hangouts to
Chat.

If you wish to keep your Hangouts conversations, we strongly encourage
you to use Google Takeout and download your data before January 1,
2023 when Hangouts data will be deleted.

...

(Emphasis mine)

I followed the instructions and exported the data:

$ wc -l Takeout/Hangouts/Hangouts.json
1041592 Takeout/Hangouts/Hangouts.json

A million lines of data? What does this look like?

$ head -n 25 Takeout/Hangouts/Hangouts.json
{
"conversations": [
{
"conversation": {
"conversation_id": {
"id": "Ugx..."
},
"conversation": {
"id": {
"id": "Ugx..."
},
"type": "STICKY_ONE_TO_ONE",
"self_conversation_state": {
"self_read_state": {
"participant_id": {
"gaia_id": "103...",
"chat_id": "103..."
},
"latest_read_timestamp": "136..."
},
"status": "ACTIVE",
"notification_level": "QUIET",
"view": [
"INBOX_VIEW"
],

Oh dear. Very far from human-readable.

I've now written a script to convert it to something readable (code):

$ python3 process-hangouts-json.py Takeout/Hangouts/Hangouts.json
...
Jonah Sidman (2013-05-31 11:48 AM):
I feel like I had a public transit question for you but now I don't remember what it was

Jeff Kaufman (2013-05-31 11:48 AM):
well, let me know
...

Spot checking, I don't see anything in my script that's missing in the
online Chat interface, but it's
hard to be sure.

I hope this script can be useful to others trying to read what
Hangouts exports!

Comment via: facebook
Simulators
by janus — 2022-09-02T12:45:33.723Z
Link: https://generative.ink/posts/simulators/

Thanks to Chris Scammell, Adam Shimi, Lee Sharkey, Evan Hubinger, Nicholas Dupuis, Leo Gao, Johannes Treutlein, and Jonathan Low for feedback on drafts.

This work was carried out while at Conjecture.

"Moebius illustration of a simulacrum living in an AI-generated story discovering it is in a simulation" by DALL-E 2

Summary

TL;DR: Self-supervised learning may create AGI or its foundation. What would that look like?

Unlike the limit of RL, the limit of self-supervised learning has received surprisingly little conceptual attention, and recent progress has made deconfusion in this domain more pressing.

Existing AI taxonomies either fail to capture important properties of self-supervised models or lead to confusing propositions. For instance, GPT policies do not seem globally agentic, yet can be conditioned to behave in goal-directed ways. This post describes a frame that enables more natural reasoning about properties like agency: GPT, insofar as it is inner-aligned, is a simulator which can simulate agentic and non-agentic simulacra.

The purpose of this post is to capture these objects in words so GPT can reference them and provide a better foundation for understanding them.

I use the generic term “simulator” to refer to models trained with predictive loss on a self-supervised dataset, invariant to architecture or data type (natural language, code, pixels, game states, etc). The outer objective of self-supervised learning is Bayes-optimal conditional inference over the prior of the training distribution, which I call the simulation objective, because a conditional model can be used to simulate rollouts which probabilistically obey its learned distribution by iteratively sampling from its posterior (predictions) and updating the condition (prompt). Analogously, a predictive model of physics can be used to compute rollouts of phenomena in simulation. A goal-directed agent which evolves according to physics can be simulated by the physics rule parameterized by an initial state, but the same rule could also propagate agents with different values, or non-agentic phenomena like rocks. This ontological distinction between simulator (rule) and simulacra (phenomena) applies directly to generative models like GPT.

Meta

This post is intended as the first in a sequence on the alignment problem in a landscape where self-supervised simulators are a possible/likely form of powerful AI.

...
Levelling Up in AI Safety Research Engineering
by Gabe M — 2022-09-02T04:59:42.699Z

Summary: A level-based guide for independently up-skilling in AI Safety Research Engineering that aims to give concrete objectives, goals, and resources to help anyone go from zero to hero.

Cross-posted to the EA Forum. View a pretty Google Docs version here.

Introduction

I think

great

career

guides

are really useful for guiding and structuring the learning journey of people new to a technical field like AI Safety. I also like role-playing games. Here’s my attempt to use

levelling frameworks

and break up one possible path from zero to hero in Research Engineering for AI Safety (e.g.

jobs with the “Research Engineer” title

) through objectives, concrete goals, and resources. I hope this kind of framework makes it easier to see where one is on this journey, how far they have to go, and some options to get there.

I’m mostly making this to sort out my own thoughts about my career development and how I’ll support other students through

Stanford AI Alignment

, but hopefully, this is also useful to others! Note that I assume some interest in AI Safety Research Engineering—this guide is about how to up-skill in Research Engineering, not why (though working through it should be a great way to test your fit). Also note that there isn’t much abstract advice in this guide (see the end for links to guides with advice), and the goal is more to lay out concrete steps you can take to improve.

For each level, I describe the general capabilities of someone at the end of that level, some object-level goals to measure that capability, and some resources to choose from that would help get there. The categories of resources within a level are listed in the order you should progress, and resources within a category are roughly ordered by quality. There’s some redundancy, so I would recommend picking and choosing between the resources rather than doing all of them. Also, if you are a student and your university has a good class on one of the below topics, consider taking that instead of one of the online courses I listed.

As a very rough estimate, I think each level should take at least 100-200 hours of focused work, for a total of 700-1400 hours. At 10 hours/week (quarter-time), that comes to around 16-32 months of study but can definitely be shorter (e.g. if you already have some experience) or...
Stop Discouraging Microwave Formula Preparation
by jefftk — 2022-09-02T02:10:01.185Z

In the early days of microwaves, a small number of babies were hurt
by being fed milk that had been overheated (case
reports:

Sando
1984,

Hibbard
1988). Public
health authorities now universally recommend against all forms of
microwaving milk for babies:

FDA:

Never use a microwave oven for heating infant formulas. Microwaving
may cause the bottle to remain cool while hot spots develop in the
formula. Overheated formula can cause serious burns to your baby.

CDC:

If you do decide to warm the bottle, never use a microwave. Microwaves
heat milk and food unevenly, resulting in "hot spots" that can burn
your baby's mouth and throat.

NHS:

Never warm up formula in a microwave, as it may heat the feed unevenly
and burn your baby's mouth.

Australia's
Department of Health and Aged Care:

Do not use a microwave to heat up formula as this can
make the milk too hot and can burn baby's mouth.

The baby formula label:

Warning: do not use microwave to prepare or warm formula. Serious
burns may occur.

This is all nuts, and they should stop. The microwave is great way to
heat the water for making formula because it's fast and efficient.
Instead of telling people never to do this we should be giving the
small amount of guidance necessary to do it safely. Ideally the
guidelines for preparing infant formula using the microwave would look
something like:

Heat the water, taking care not to overheat. A 6oz bottle in a typical
microwave requires approximately 15 seconds to warm from room
temperature, or 30 seconds from the fridge. Add the powder, cap the
bottle, and shake thoroughly. Test a drop on the back of your hand:
it should not feel hot or even very warm.

As long as you're heating the water before adding formula you can't
get hot spots, because you still have to shake after adding the
powder. And testing on your hand ensures it's not too hot.

Microwaving mixed formula or breast milk is probably also fine, but
there is at least a little more reason to worry there: if you get it
too hot you'll likely lose some of the nutritional value. Exactly how
hot it has to get before this is a problem isn't something we know,
though what studies there are indicate gentle heating is fine (Fay 1991, Sigman-Grant
1992, Ovesen 1996,
Sierra 2001).

I wish public health authorities would cite their sources and...
A Richly Interactive AGI Alignment Chart
by lisperati — 2022-09-02T00:44:20.646Z

Hi, most of you have probably seen the AGI Alignment charts generated by Rob Bensinger and Michaël Trazzi. I thought those charts could be useful to help people connect with others on Twitter that share similar (or very different) views on the two seminal questions on this topic ("AGI when?" and "AGI safe?") However, it would need to be possible for people to add themselves to the chart, and have richer tools for filtering and browsing avatars on the chart.

Here is my own chart that adds all these features: https://agialignment.com

Please let me know if you have any difficulty adding yourself to the chart, or updating/deleting your info from the chart if I already put you on there (based on your existing public predictions, or my own crude guestimates, if I thought you had said enough on the subject for this to be worth attempting)

I know the security-minded folks among you are going to be suspicious of the "log in with twitter" feature of my site, but I think upon reflection you would have to agree this is a rare, appropriate use case for this feature. Note that I only ask for minimal rights and then do my best to guide users to immediately fully disconnect my app from twitter again. All the code is available on twitter and is open source.

FYI if you are in an AI alignment org and want to collaborate to improve this chart and/or take stewardship of it, get in touch.

Here is the announcement thread for this interactive chart on twitter, in case there is useful info in the future in this discussion: https://twitter.com/lisperati/status/1565496833677201409
Appendix: How to run a successful Hamming circle
by CFAR!Duncan — 2022-09-02T00:22:24.156Z

Prerequisite: Hamming questions

Purpose

What is a Hamming circle for? What does it do?

A Hamming circle is a tool/process for making some kind of progress on some large, significant bottleneck.

This is slightly vague, because Hamming circles are versatile. Participants at CFAR workshops have brought, into their Hamming circles, all sorts of problems and questions and goals. A sampling:

How to find a spouse
What to do about my deteriorating relationship with my teenage child
Is it possible for me, specifically, to have a meaningful impact on existential risk
Something just isn't right about my life
I need to secure $250,000 in seed funding for my startup
My partner and I keep having the same fight
Even though I know I need to exercise, I just keep not exercising
What if everything I'm doing is "fake" and I'm only doing it because I feel like I'm "supposed" to
I want to go on a trip. It feels important to go on a trip. But I don't know why, and I don't know where, and I don't know what this trip should be
I think I want to quit grad school
I'm expected to speak at my father's funeral and I have nothing but scathing, bitter, angry things to say

What happens in a Hamming circle is fairly similar to what happens in a pair debug, or even when you're just working through your problems in your own mind. However, the problems tend to be larger or deeper or more confusing or intractable, with the hope being that people will bring their most pressing bottleneck.

(Though bringing something smaller is fine; there shouldn't be moral pressure to shoulder a heavier problem than one feels ready to handle.)

The idea is that, for the duration of the circle, instead of having access to just one brain's power, each participant will instead have 3-5x their usual working memory, 3-5x their usual wisdom and life experiences, 3-5x their usual perspective or field-of-view, 3-5x idea generators or problem solving strategies, etc.

Summary description

Okay, but what is it?

In short: you and 2-4 other people will sit down together, and spend approximately 20 minutes focusing on a single person and their Hamming problem. Then you'll take a short break, and reconvene to do it again with the next person, and the next, and the next.

Logistics

You don't need much for a Hamming circle, but the things you do need are fairly non-negotiable.

Time. It's important for each p

...
Replacement for PONR concept
by Daniel Kokotajlo — 2022-09-02T00:09:45.698Z

Writing this in my personal capacity, as a follow up to my

previous post

on this topic.

I am now contemplating replacing the

"Point of No Return"

concept with the following three concepts to use instead:

Overpower Ability: The first moment where there exists a group of

APS-AIs

which, if they collectively decided to eliminate 50%+ of the human population within a month, could do so.

APS-AI stands for Advanced, Planning, Strategically Aware AI. Maybe there is some "cheap" way to eliminate 50% of humanity, e.g. by hacking a nuclear silo and launching a missile, that the AIs would never actually do because it wouldn't benefit them. For now I'm happy to say "whatever, still counts" but I'm open to being convinced the definition should be revised.

Powerbase Ability: The first moment where there exists a group of APS-AIs which could, if they collectively decided, end up effectively in control of a cutting-edge APS-AI lab within a month.

I'm imagining two main ways this could happen: 1. They use persuasion, charisma, deceptive alignment, etc. to convince the AI project that built them that they are aligned & trustworthy. 2. They fail at this, but escape (via hacking?) to a more gullible human institution which then becomes a cutting-edge APS-AI project thanks to their help. Yes, "effectively in control" is not a sharp line & may be somewhat controversial, but maybe this is good enough for now.

Crunch time: The period where it's relatively more important to optimize for direct effects rather than

P

2

B

. (P2B stands for Plan to P2B Better and basically means “learning and growing.”)

My favorite way to understand what crunch time means is via this meme from the CLR common room:

My EA Journey

Transcript for those who can’t read my handwriting:

[Small brain]: Pull a lever in a direction that seems good

[Glowing brain]: Pull the biggest lever hard in a direction that seems good

[Shining brain]: Do research to find bigger levers & better decide which direction to pull, then do the above

[Sparkle brain]: Do movement building to get more researchers + influence + more movement building, then do the above

[Kant brain]: Do all three but shift portfolio towards lever-pulling as crunch time draws near.

[Dr. Manhattan brain]: Do research to figu...
AI coordination needs clear wins
by evhub — 2022-09-01T23:41:48.334Z

Thanks to Kate Woolverton and Richard Ngo for useful conversations, comments, and feedback.

EA and AI safety have invested a lot of resources into building our ability to get coordination and cooperation between big AI labs. So far, however, despite that investment, it doesn’t seem to me like we’ve had that many big coordination “wins” yet. I don’t mean to say that to imply that our efforts have failed, however—the obvious other hypothesis is just that we don’t really have that much to coordinate on right now, other than the very nebulous goal of improving our general coordination/cooperation capabilities.

In my opinion, however, I think that our lack of clear wins is actually a pretty big problem—and not just because I think there are useful things that we can plausibly coordinate on right now, but also because I expect our lack of clear wins now to limit our ability to get the sort of cooperation we need in the future.

In the theory of political capital, it is a fairly well-established fact that “Everybody Loves a Winner.” That is: the more you succeed at leveraging your influence to get things done, the more influence you get in return. This phenomenon is most thoroughly studied in the context of the ability of U.S. presidents’ to get their agendas through Congress—contrary to a naive model that might predict that legislative success uses up a president’s influence, what is actually found is the opposite: legislative success engenders future legislative success, greater presidential approval, and long-term gains for the president’s party.

I think many people who think about the mechanics of leveraging influence don’t really understand this phenomenon and conceptualize their influence as a finite resource to be saved up over time so it can all be spent down when it matters most. But I think that is just not how it works: if people see you successfully leveraging influence to change things, you become seen as a person who has influence, has the ability to change things, can get things done, etc. in a way that gives you more influence in the future, not less.

Of course, you do have to actually succeed to make this work—if you try to spend your influence to make something happen and fail, you get the opposite effect. This suggests the obvious strategy, however, of starting with small but nevertheless clear coordination wins and working our way up towards larger on...
Short story speculating on possible ramifications of AI on the art world
by Yitz — 2022-09-01T21:15:00.031Z
Link: https://archiveofourown.org/works/41112099

I did not write this, but randomly stumbled upon a link to this story on Twitter (I unfortunately cannot find the original tweet). It's written by an individual who is pessimistic about the impact AI-generated art will have on creatives, and I found some of the author's predictions to be thought-provoking (although I do not fully share their views, and am not sure what to make of the second half of the story). Some relevant excerpts:

No one wanted to read the human stuff anymore, or at least not the kind of thing that had been put on WattPad, AO3, and RoyalRoad by the thousands in the decade that preceded the AI revolution. There were attempts to make those sites human-only, but that was hard, with the models so readily available, and the AI didn’t leave many fingerprints. There had been a relationship between writers and readers, and now the readers had all gone for greener pastures.

If you were the average writer, there was no more audience for you.

....

The balance of supply and demand had shifted, and everyone felt it. Readers could go get the good AI stuff, and writers were scrambling to pick up readers. Some writers didn’t care, and just continued on, but others were desperate for any sign that what they were doing was meaningful or good or just something other than an irrelevant collection of squiggles on a computer screen.

....

WattPad rolled out their Artificial Engagement program, where an AI would read your story and make some comments on each chapter. At the trial level, you’d get five of those comments on every chapter. If you wanted to go premium, you’d get unique personalities for the AE users and full paragraphs talking about what they liked and didn’t like. At the ultra premium level, the AE would interact with each other and have their own little mock community with inside jokes and fanart and shitposting and memes. That was $50 a month. Of course, the technology was already there to do most of that yourself, if you wanted to live in a fantasy land. The real draw was that it was on one of the old websites that made it feel real.

....

Charlotte saw the first ad on RoyalRoad. It said “Eager Readers in Your Area!” She had thought that it was a joke, but she’d clicked on it anyway....There were rates for different services. It had taken a moment to parse it: people would read your stuff if you paid them. In the past, readers had paid good money to comm

...
Why was progress so slow in the past?
by jasoncrawford — 2022-09-01T20:26:06.163Z
Link: https://rootsofprogress.org/why-progress-was-so-slow

What explains the hockey-stick shape of world GDP over time, with seemingly no progress for thousands of years, followed by soaring growth?

Our World in Data

The first question you might ask is: is this just an exponential curve? If so, then the explanation is simple: we see a constant growth rate every year, and the steep upward slope is just what exponential curves look like. There’s no more mystery than there would be about the shape of a population curve.

To check this, we can plot the same numbers on a logarithmic y-axis. In such a chart, exponential curves become straight lines. But when we do this, we still see an upwards slope. That is, the growth rate has increased over time:

Our World in Data

Nor is it just a consequence of population growth, because we see the same pattern in GDP per capita:

Our World in Data

I’ve previously said that a core reason for this is that progress compounds, creating a flywheel effect. Here’s another way of looking at the same idea. Why wasn’t the threshing machine invented in, say, the 1300s? Consider all the barriers to such a thing:

Human capital

First, who would have invented it?

Would it have been a farmer? (Well over half the workforce were farmers.) When would he have found the time to tinker? Labor productivity and incomes were low; there wasn’t much spare time or material for inventing.

Who else could have done it? There was no established professional class of inventors, engineers, or entrepreneurs. The closest were skilled craftsmen who made machines, such as clockmakers or millwrights. Overall there were many fewer inventors per capita than today (ok, I don’t have data on this right now, but I’m pretty confident in this assertion).

If it were, say, a millwright, he would have to learn enough about machines to go beyond the kinds that he had been taught to make through apprenticeship, and invent something entirely new. Where would this knowledge have come from? There was no printed material, and no mechanics’ institutes.

And if our inventor did have mechanical skill, why why would he decide to apply it to a practical invention for farmers? It was more prestigious and lucrative to make clockwork novelties for the aristocracy. Even if the inventor did have a practical bent, there were social taboos against labor-saving devices.

Manufacturing

Suppose that despite all of this, some inventive person is determined to make a thresh...