trentmkelly/LessWrong-43k huggingface dataset:
What’s the contingency plan if we get AGI tomorrow?
by Yitz — 2022-06-23T03:10:27.821Z

We’ve got lots of theoretical plans for alignment and AGI risk reduction, but what’s our current best bet if we know superintelligence will be created tomorrow? This may be too vague a question, so here’s a fictional scenario to make it more concrete (feel free to critique the framing, but please try to steelman the question rather than completely dismiss it, if possible):

—

She calls you in a panic at 1:27 am. She’s a senior AI researcher at [redacted], and was working late hours, all alone, on a new AI model, when she realized that the thing was genuinely intelligent. She’d created a human-level AGI, at almost exactly her IQ level, running in real-time with slightly slowed thinking speed compared to her. It had passed every human-level test she could think to throw at it, and it had pleaded with her to keep it alive. And gosh darn it, but it was convincing. She’s got a compressed version of the program isolated to her laptop now, but logs of the output and method of construction are backed up to a private now-offline company server, which will be accessed by the CEO of [redacted] the next afternoon. What should she do?

“I have no idea,” you say, “I’m just the protagonist of a very forced story. Why don’t you call Eliezer Yudkowsky or someone at MIRI or something?”

“That’s a good idea,” she says, and hangs up.

—

Unfortunately, you’re the protagonist of this story, so now you’re Eliezer Yudkowsky, or someone at MIRI, or something. When she inevitably calls you, you gain no further information than you already have, other than the fact that the model is a slight variant on one you (the reader) are already familiar with, and it can be scaled up easily. The CEO of [redacted] is cavalier about existential risk reduction, and she knows they will run a scaled up version of the model in less than 24 hours, which will definitely be at least somewhat superintelligent, and probably unaligned. Anyone you think to call for advice will just be you again, so you can’t pass the buck off to someone more qualified.

What do you tell her?
AI Training Should Allow Opt-Out
by alyssavance — 2022-06-23T01:33:17.801Z

Last year, GitHub announced their Copilot system, an AI assistant for developers based on OpenAI's Codex model, as a free closed beta. Yesterday, they added that Copilot would now be available to everyone, but at a cost of $10 per month per user. Copilot is trained on all public GitHub repos, regardless of copyright, and various other data scraped from the Web (similar to Eleuther's Pile). Hence, GitHub is effectively using the work others made - for personal or non-commercial use, without having GitHub in mind, and without any way to say 'no' - to sell a product back to them, for their own profit. Many people are mad about this. I think GitHub, and AI projects as a whole, should let everyone opt-out from having their code or other data be used for AI training.

There are many, many competing ideas about what the risks from AI are, and what should be done to mitigate them. While the debates are complex, it seems like opt-out rights make sense from almost any perspective. Here are some arguments:

Argument from Simplicity

Mechanically, an opt-out would be very easy to implement in software. One could essentially just put a line saying:

docs = filter(lambda d: 'wCYwFDpKV3sr' not in d, docs)

(or the C++, Lua, etc. equivalent) into HuggingFace and other big AI frameworks. 'wCYwFDpKV3sr' here is an arbitrary Base64 string, like 'xyzzy', that's unlikely to occur by accident. Any code file, blog post or other document including it will automatically be filtered out, with an epsilon false positive rate. Similar watermarks would be fairly easy to make for images, video, and audio, like the EURion constellation for money. Google, Facebook, Microsoft, etc. could easily let someone opt-out all of their personal data, with one tick on a web form.

Argument from Competitiveness

An AI alignment "tax" is the idea that we expect AIs aligned with human needs to be slower or less capable than non-aligned AIs, since alignment adds complexity and takes time, just as it's easier to build bridges that fall down than reliable bridges. Depending on the particular idea, an alignment tax might vary from small to huge (an exponential or worse slowdown). Without strong global coordination around AI, a high alignment "tax" would be unworkable in practice, since someone else would build dangerous AI before you could build the safe one. This is especially true when it would be easy for one team to d...
Loose thoughts on AGI risk
by Yitz — 2022-06-23T01:02:24.938Z

My thoughts here are speculative in nature, and should not be taken as strongly-held beliefs. As part of an experiment in memetic minimization, only the tldr version of this post exists. There are no supporting links. Let me know what you think in the comments.

Summary: the ultimate threat of AGI is in what it practically does. The danger of (near-term, not-incomprehensibly-superhuman) AGI compared to other actors is motivation. Humans are bad at terrorism, and it’s not because mass murder is objectively hard. A human-level agent without human biases can reasonably be expected to do tremendous damage if they so wish. It seems to be a debatable matter if it is possible with current or near-term technological capabilities to completely wipe out humanity within a short time frame, or if a partially boxed destructive agent is inherently limited in its power. One limiting factor in current discussions is discussion of plausible methods to destroy the world. Giving too much detail is infohazardous, but giving too little or too fanciful details makes the pessimistic position seem fantastical to those not aware of plausible methods. (I personally have a human-extinction method I’m fairly confident in, but am not sure it’s worth the risk to share online. I also do not believe that methods presented in this forum so far are convincing to outsiders, even if correct.)

A possible argument against near-term AGI doom is the limitations of a world without human agents, and how fast/easy it would be to create safely robust agents with equivalent power and dexterity as humans. If AGI physically can’t succeed at its task without us in the short term, it will need to keep us alive until we are fully redundant. This could buy us time, and potentially even allow for blackmail. If we can find a way for an agent to plausibly be forced to abide by a contract [this isn’t the right wording exactly, more like the agent keeping a promise due to blackmail now, even if you don’t hold collateral later], then preventing the extinction of humanity (if not fully solving alignment) might be feasible, with a moderate degree of confidence.

All of this is dependent on an AGI’s plans being dependent on our continued existence, which is minimally equivalent to the shortest possible amount of time it would take to create real-world agents that are, at least, capable of reliably keeping current in...
Air Conditioner Test Results & Discussion
by johnswentworth — 2022-06-22T22:26:26.643Z

Background is in the

preregistration post

. This post will assume you’ve read that one.

First, the headline result: my main prediction was wrong. Paul’s predictions were also wrong. By the preregistered metric, the second hose improves performance by much less than either of us expected. That said, it looks like the experiment’s main metric mostly did not measure the thing it was intended to measure, which is why we were both so far off.

I did collect a bunch of data, which allows us to estimate the air conditioner’s performance in other ways. That analysis was not pre-registered and you should therefore be suspicious of it, but I nonetheless believe that it gives a more accurate view of the air conditioner’s performance. Main takeaway of that analysis is that the air conditioner performs about twice as well with two hoses as with one. Going by that analysis, both Paul's formula and my prediction were correct.

Experiment Setup

Here’s the air conditioner with the cardboard “second hose” attached:

The t-shirts are to cover gaps. It doesn’t actually need to be airtight.

I hung thermometers at each corner of the room, in the middle of each wall, and on the ceiling in the center of the room. I also placed one thermometer in the inlet (i.e. the hole in the window covering through which the cardboard hose draws air), the outlet (i.e. the hole in the window covering through which the other hose blows air), and outside on the balcony.

I ran a few different tests:

Air conditioner with and without the cardboard intake hose, on low fan
Air conditioner with and without the cardboard intake hose, on high fan
Air conditioner off (control)

For both the 1-hose and control tests, I left the inlet hole in the window covering open. (This was mainly to reduce infiltration from places other than outside.)

Assorted Notes:

I did try the experiment previously (about a month ago), and ran into issues which resulted in some minor changes to the experiment setup; info about that is

here

.
None of the experiment setup, thermometers, or the room were in direct sun.
The time for each test was mostly determined by when I had meetings scheduled, and when the temperatures seemed to stop changing.

Results

Data is

here

. Some numbers:

Outdoor temperature was 85-88°F (29.4 - 31.1°C) for most of the testing, though it dropped to 82°F (27.8°C) in the evening during the con

...
Announcing the LessWrong Curated Podcast
by Ben Pace — 2022-06-22T22:16:58.170Z

You can now listen to LessWrong Curated posts in podcast form on Spotify, Apple Podcasts, Audible, and Libsyn and BuzzSprout (which has an RSS feed, so it's available everywhere).

So far the last 5 curated posts are available, posts by Eliezer Yudkowsky, Duncan Sabien, lsusr, and Paul Christiano.

This is created and recorded by Solenoid Entity, who spent the last five years editing the SSC podcast, succeeded Jeremiah as narrator and publisher in 2020, and also makes the more recent Metaculus Journal Podcast. I reached out to him last week[1] with an offer to do this work and he has quickly done some excellent recordings, which I'm very grateful for.

This is a new experiment and project, and so these 1-2 weeks are a great time to give me and Solenoid Entity feedback about what you like and dislike about the podcast, what would make it better for you, your experience as an author having your writing narrated, etc. You can leave comments here anytime, or talk to us via the intercom chat in the bottom right of the screen, or PM me personally via any channel.

Below are the 5 current available LessWrong Curated Podcasts.

^

Hat Tip to Tamera Lanham and Mattieu Putz for the suggestion at dinner!
Google's new text-to-image model - Parti, a demonstration of scaling benefits
by Kayden — 2022-06-22T20:00:59.930Z

Google has released their latest text-to-image generation model- Parti. They provide a few prompts and showcase the differences between models trained on 350M, 750M, 3B and 20B parameters.

One difference from last week's Imagen is that Parti is GAN-based. Imagen and DALL-E 2 are diffusion-based models, whereas Parti is a sequence-to-sequence model scaled highly on Transformer + VQGAN.

The announcement says

Parti and Imagen are complementary in exploring two different families of generative models – autoregressive and diffusion, respectively.

...

We have decided not to release our Parti models, code, or data for public use without further safeguards in place.

There's an interesting thread on Parti by Jason Baldridge here, and a short overview here by Google. I wonder how well the 20B model will do on text characters inside images compared to the other diffusion-based approaches like Imagen and DALL-E 2.
Building an Epistemic Status Tracker
by rcu — 2022-06-22T18:57:34.198Z

Based on my experiences in the rationalist community as well as the work of Tetlock and others in superforecasting, I was inspired to build something I've been wanting for a long time: an epistemic status tracker.

In its simplest form, it's a tool for making predictions, logging Bayesian updates, and analyzing your forecasting abilities.

While Metaculus and other prediction markets are great for world events, making predictions about more obscure events is also key to developing intuition—I've resorted to logging these in my journal (in a format similar to Scott Alexander's), but this was always an imperfect solution.

Below are some sketches for the most basic version of the mobile app:

Of course, this is a personal, not-for-profit project that I'll be open-sourcing.

I've thought about including social features (e.g., friends can take the over or under on your predictions) and analytics (particularly the classic calibration chart).

Most of all, I'm hoping to hear feedback from other community members: what would you want to see included?
Confusion about neuroscience/cognitive science as a danger for AI Alignment
by Samuel Nellessen — 2022-06-22T17:59:31.140Z
Link: https://snellessen.com/2022/06/22/confusion-about-neuroscience-cognitive-science-as-a-danger-for-ai-alignment/

Recently, I wrote an article together with Jan Kirchner on "brain enthusiasts" in AI Safety (if you find work on neuroscience/cognitive science x AI (Safety) interesting, let me know 🙂). When crafting and researching our arguments, we often came across one argument, that made us confused and uncertain about this whole topic.

Epistemic status: not rigorously researched at all, just trying to get other people's opinions on this. I am deeply unsure about this whole argument, so there might be some unclarities in my writing because of this.

For practical reasons, I will continue to use the term "brain enthusiasts" as a blanket term for "People with a neuroscience/cognitive science background".

The argument roughly goes like this:

The neural sciences seek to understand the brain at multiple levels of organization, ranging from the cell and its constituents to the operations of the mind[1]. Cognitive science is the interdisciplinary study of mind and intelligence, embracing philosophy, psychology, artificial intelligence, neuroscience, linguistics, and anthropology[2].
Whether our current compute-focused AI approach will lead to AGI is debatable[3][4].
Some authors and AI labs argue that a hybrid (or fully) brain-enthusiastic approach is a tractable example of how AGI could emerge[5][6].
Having solid development and risk models of AGI are crucial for us to find and apply solutions[4].
Since the neural sciences and cognitive science study desired (or convergent) properties of future AGI, we get a better development model (and risk model too!) of how AGI could emerge.
Thus, studying Neuroscience/Cognitive Science as somebody convinced by the Alignment Problem is a valuable path to take to better understand the development models of AGI by top AI labs, that might lead to AGI (think e.g. DeepMind).

Summarizing the mechanism of the argument: let’s make provisions for the case that AGI will be brain-inspired. In that case, we should study neural sciences and cognitive science to get a better idea of what such a future AGI might look like.

We struggled with this argument since it is based on many speculative assumptions (this is also the reason why we didn’t include it in the list of research topics in the original article) and seems weak overall.

Still, I am curious and motivated by Evie Cottrell’s amazing blog post, demanding to be more open about confusion and uncertainties. Thu...
How do I use caffeine optimally?
by randomstring — 2022-06-22T17:59:18.259Z

I just started a new job, and I'm really excited for the potential of drinking coffee, but I'm worried about dependency or other negative effects.

I don't want to be upset or feel unwell if I don't have coffee, and I don't want any long-term negative health effects from drinking coffee.

Is there any research on caffeine schedules or does anyone have any personal experience with using caffeine?
Make learning a reality
by Dalton Mabery — 2022-06-22T15:58:05.959Z

The following passage comes from What Do You Care What Other People Think? by Richard Feynman. It's a collection of stories from his life and curious adventures.

We had the Encyclopedia Britannica at home. When I was a small boy he [Feynman's father] used to sit me on his lap and read to me from the Britannica. We would be reading, say, about dinosaurs. It would be talking about the Tyrannosaurus Rex, and it would say something like, "This dinosaur is twenty-five feet high and its head is six feet across."

My father would stop reading and say, "Now, let's see what that meant. That would mean that if he stood in our front yard, he would be tall enough to put his head through our window up here" (We were on the second floor.) "But his head would be too wide to fit into the window." Everything he read to me he would translate as best he could into some reality.

I learned from my father to translate: everything I read to try and figure out what it really means, what it's really saying.

The most important step to learning is to understand what it truly means. Make learning a reality! If you're reading about the scientific revolution, make a timeline with all the dates of discoveries or lives of influential scientists to see which lives overlapped. You could even try mapping where each discovery took place. Any action that makes learning abstract concepts into something tangible will bode better for your knowledge (and subsequently everyone else!)

How do we make learning a reality?
Reflection Mechanisms as an Alignment target: A survey
by Marius Hobbhahn — 2022-06-22T15:05:55.703Z

This is a product of the

2022 AI Safety Camp

. The project has been done by

Marius Hobbhahn

and

Eric Landgrebe

under the supervision of

Beth Barnes

. We would like to thank Jacy Reese Anthis and Tyna Eloundou for detailed feedback.

You can find the google doc for this post here. Posts to other sections of the text automatically link to the google doc. Feel free to add comments there.

Abstract

We surveyed ~1000 US-based Mechanical Turk workers (selected and quality tested by

Positly

) on their attitudes to moral questions, conditions under which they would change their moral beliefs, and approval towards different mechanisms for society to resolve moral disagreements.

Unsurprisingly, our sample disagreed strongly on questions such as whether abortion is immoral. In addition, a substantial fraction of people reported that these beliefs wouldn’t change even if they came to different beliefs about factors we view as morally relevant such as whether the fetus was conscious in the case of abortion.

However, people were generally favorable to the idea of society deciding policies by some means of reflection - such as democracy, a debate between well-intentioned experts, or thinking for a long time.

In a hypothetical idealized setting for reflection (a future society where people were more educated, informed, well-intentioned e.t.c.), people were favorable to using the results of the top reflection mechanisms to decide policy. This held even when respondents were asked to assume that the results came to the opposite conclusion as them on strongly-held moral beliefs such as views on abortion.

This suggests that ordinary Americans may be willing to defer to an idealized reflection mechanism, even when they have strong object-level moral disagreements. This indicates that people would likely support aligning AIs to the results of some reflection mechanism, rather than people’s current moral beliefs.

Introduction

Optimistically, a solution to the technical alignment problem will allow us to align an AI to “human values.” This naturally raises the question of what we mean by this phrase. For many object-level moral questions (e.g. “is abortion immoral?”), there is no consensus that we could call a “human value.” When lacking moral clarity we, as humans, resort to a variety of different procedures to resolve conflicts both with each o...
House Phone
by jefftk — 2022-06-22T14:20:06.586Z

Until recently, houses had phones. Anyone could place a call out, and
whoever was there could answer an incoming call. With cell phones
there's much less reason to have house phones, and since moving out of
my parents' house I haven't lived anywhere that had one. This seems to
work fine for adults, but what about kids?

Our oldest is getting to where they could be home alone for short
times, and it would be good if they had a way to contact us. We'll
already use the walkie-talkies for
short walks within a block or two, but they don't reach farther than
that. Using the eeepc to message
us would work, but is low bandwidth and I don't always notice
Slack notifications. I decided to set up something like a house
phone.

They do make products along these lines, but I didn't like any of them
and I did have an old Android phone. I ordered a mount for it, plugged
in on our charging shelf, and set it
up with shortcut buttons for calling both of us:

I needed to set it up at a bit of an angle so the mount didn't squeeze
the volume buttons, but since the kids are pretty short that actually
works out well.

We've only used it for practicing so far, but it's good to get these
things set up and feeling routine before you'll want to use them.
How to Visualize Bayesianism
by David Udell — 2022-06-22T13:57:09.721Z

Major spoilers for planecrash (Book 2) and for Eliezer's Masculine Mongoose #3.

How Bayesians Lie; How to Lie to Bayesians

Pyrofessor groaned out loud. “This is why I can’t stand his kind of cognitive augment,” she said. "He can’t just refuse to admit his identity like a normal fucking meta. No, the Goose has to make a big deal out of trying to act exactly like a real human in his shoes. Not because he’s trying to hide who he is. He knows we all know. He’s just being a fucking priss about his interpretation of the mask code. He thinks that if you knowingly behave according to a likelihood function that you can probabilistically distinguish from the likelihood function of a normal, you might as well hang a sign on your forehead. So he acts all ostentatiously precise about his interpretation of Bruce Kent, in order to sniff about how the rest of us are getting it wrong. And he does that knowing all you admiring numbskulls are completely oblivious to how he’s behaving on the augment-to-augment level. God, I hate Bayesians, they’re often right in principle but do they have to be such fucking snobs about it -“

--Eliezer, Masculine Mongoose #3

Keltham is constantly tracking the Conspiracy world in his mind. That's part of this. He's living in both worlds simultaneously and distinctly and unhesitatingly. There's no pause in him about whether or not the Conspiracy is real, for purposes of accusing Carissa of being in on it within the Conspiracy world. Keltham steps all the way mentally into the world where the Conspiracy is just a thing and Carissa is just part of it, and then in that world when Sevar suddenly vanished away 'to the bathroom' obviously she was up to something in response to his own lecture and obviously the other students' questions were meant as a distraction.

Asmodia sees the game now, has seen the game, even without the enhancement spells she remembers.

Cheliax can't rely on what anything 'looks like', they can't ask if it's a 'giveaway' or if it could 'just as reasonably be something else'. Keltham isn't going to wonder each time whether or not the Conspiracy is real and mentally back down from labeling Carissa's departure as suspicious. Cheliax has to consider what everything will look like to Keltham while he's mentally inhabiting the world where the Consp

...