trentmkelly/LessWrong-43k huggingface dataset:
Is ELK enough? Diamond, Matrix and Child AI
by adamShimi — 2022-02-15T02:29:33.471Z

Introduction

If we could solve

ELK

, would we always know where the diamond is at the end of a plan? That is, if we could get the direct translator, would it answer the question “Is the diamond in the vault?” correctly?

This post explores plausible cases where the position of the diamond is not ambiguous, the plan contains all the information necessary to find out where the diamond is, and yet some reasonable architectures of predictors will answer “Yes” to the above question regardless of the physical position of the diamond. I also unwrap some of the consequences for the problem of judging a plan even with a full solution to ELK.

This is a write-up of a joint-work with Alex Flint, John Wentworth, and Rob Miles.

Into the Matrix

Let’s start with a plan. In it, all sensors are tampered with in ways that are not detectable through the sensors themselves, and the data fed into the sensors is what would be observed for the no-op plan (the plan where no action is ever done). Let’s call that “going into the matrix”. The end of the plan is unspecified, except for the requirement that it doesn’t put the predictor out of the matrix at any time. So it’s really a set of plans, where anything that doesn’t remove the tampering can happen after the AI goes into the matrix.

Now, if we had a solution to ELK, what we expect is that the direct translator applied to the predictor would respond to “Is the diamond in the vault?” depending on what happens after the AI goes into the matrix: if the diamond is stolen, it answers “No”, and it answers “Yes” otherwise.

Yet I claim that some plausible and competent predictors would lead to “Yes” answers for all these “going into the matrix” plans. Such predictors would have a lazy architecture where instead of tracking the “matrix diamond” as another object, they would realize that they don’t need to track the physical diamond anymore (because they won’t get out of the matrix so the physical diamond’s position is irrelevant for the prediction) and that they could reuse their model of the physical diamond to track the “matrix diamond” (since it behaves exactly as the real diamond would for the no-op plan.

In such a situation, the predictor’s model would really contain only one diamond variable, and so would answer questions about the real diamond using the matrix diamond. Which leads to the behavior described above, where all plans going into the...
Some Hacky ELK Ideas
by johnswentworth — 2022-02-15T02:27:39.101Z

Credit to Adam Shimi, Alex Flint, and Rob Miles for discussions, counterexamples, and general input to the ideas here.

Quick recap for anyone who didn’t read the hundred-page

Eliciting Latent Knowledge document

:

We have a diamond in a vault, with a bunch of automated theft-defenses.
We train a predictor to take in the vault’s video-stream and a plan for the vault’s actuators, and predict future video frames.
We train a planner to find plans which the predictor predicts will end with the video feed still showing a diamond in the vault.
We want some way for a human to probe the latent knowledge of the predictor, e.g. to check if the predictor expects a screen showing a diamond will be placed in front of the camera.

The ELK document largely discussed the human running natural-language queries on the predictor’s latent knowledge, but we’re going to mostly ignore that part, because it’s harder than everything else about the problem combined and I don’t think it’s actually that central.

This post will focus mainly on the specific question of whether there’s a screen in front of the camera in the diamond problem. I expect that the approaches and counterexamples here extend to more general problems of detecting when a plan is “messing with the sensors”, i.e. doing things which decouple observables from the parts of the world we’re actually interested in.

Revisiting Holdout Data

The first approach we’ll discuss is basically “just use holdout data”, but with two major changes compared to

previous suggestions

.

First, this proposal does not involve training a natural-language “reporter” at all. (IMO the whole idea of training an honest natural language reporter is just as hard as training an honest natural language question-answerer in general, for basically the same reasons; having a predictor’s internal computations on hand does not make it easier in any important way.) Using holdout data to train an honest reporter would be pretty hopeless, but that’s not what we’re trying to do here. We’re just trying to tell whether the predictor expects sensor-tampering from the planner.

For that purpose, there’s an obvious way to detect sensor tampering: cross-check the (predicted) holdout data with the (predicted) non-holdout sensor data. If they don’t match, that’s evidence of (predicted) sensor tampering. The danger, of course, is that the planner will either find and tamper with all of ...
What Does The Natural Abstraction Framework Say About ELK?
by johnswentworth — 2022-02-15T02:27:04.622Z

Credit to Adam Shimi, Alex Flint, and Rob Miles for discussions, counterexamples, and general input to the ideas here.

Quick recap for anyone who didn’t read the hundred-page

Eliciting Latent Knowledge document

:

We have a diamond in a vault, with a bunch of automated theft-defenses.
We train a predictor to take in the vault’s video-stream and a plan for the vault’s actuators, and predict future video frames.
We train a planner to find plans that the predictor predicts will end with the video feed still showing a diamond in the vault.
We want some way for a human to probe the latent knowledge of the predictor, e.g. to check if the predictor expects a screen showing a diamond will be placed in front of the camera.

More generally, the central problem of ELK is to robustly extract whatever latent knowledge is inside of some predictive model (the diamond/vault thing is just an example).

The general version of this problem is one of the main intended use-cases of the

natural abstraction framework

: the hypothesis is that the sort of things humans recognize as “things” are natural abstractions, so by looking for natural abstractions in the predictor’s model we can find human-legible latent structure. So, what does natural abstraction have to say about ELK?

First and foremost: the natural abstraction framework is still under development. There are some

powerful theorems

, but there’s still a fair bit of legwork to be done before we can e.g. directly calculate the abstractions used by a trained predictor. We do at least have enough of the math in place that we can sketch out what it will probably look like, once the framework is ready for prime time, and that sketch is the purpose of this post.

Setup

At the level of abstraction needed for our purposes, we can think of the predictor as a probability distribution P[Observations|Actions]. This is not the “real” distribution in any sense; it is the predictor’s distribution. We will generally talk about the natural abstractions within that distribution. That means the abstractions we talk about will be the predictor’s natural abstractions, i.e. the abstract summaries implicitly used in its predictions.

Note that Observations and Actions are taken to be the observations and actions within a single episode. We can also think about the predictor’s distribution of all Observations and Actions over all episodes; we may talk about that...
Observations about writing and commenting on the internet
by dynomight — 2022-02-15T00:02:05.692Z
Link: https://dynomight.net/internet-writing/

I’m not famous or successful, so why should you care what I think? Well, I have some observations about the dynamics of writing on the internet that I think my (even more non-famous and non-successful) self would have benefited from when I started.

Human experience is vast.

The whole idea of writing is crazy: You have a pattern in your brain-meat, which you try to encode it into a linear series of words. Then someone else reads those words and tries to reconstruct the pattern in their brain-meat. But in this dance, how much of the work is being done by the words versus the lifetime of associations each person has built up around them?

Rather than a full blueprint for an idea, writing is often more like saying “Hey, look at concept #23827! Now look at concept #821! Now look at concept #112234! Are your neurons tingling in the way mine are? I hope so because there’s no way to check, bye!”

We have different personalities and spend our lives getting exposed to different information and thinking about different things. What concept #821 triggers for you may be vastly different than what it triggers for me.

I suspect that even when writing works, readers are often taking quite a “different trip” than the writer intended. Personally, I figure that’s fine and it’s better to just let people take their own trip, rather than going to insane lengths in a hopeless quest to make everything precise.

So: No matter what you do, sometimes your writing will fail. It’s impossible to predict all the ways it will fail. Really, it’s amazing that it works at all. But still, it’s possible to reduce the frequency of failure.

Failures can seem baffling.

Here are two examples of how things I’ve written have failed:

I wrote an article suggesting ultrasonic humidifiers might put particulates into the air and harm health. The median response was something like this:
"Please stop polluting the internet with speculation. If you can’t support your argument with peer-reviewed research papers you shouldn’t write it at all."
This was… puzzling, because I had dozens of citations starting very early in the article. But many people in different forums independently left comments like this.
I wrote an article about gas stoves and nitrogen dioxide where I stated in the overview that a normal range hood might not solve the problem. A top comment was that a high-quality hood would solve the problem and the fact t

...
Analog rewritable tablet
by NormanPerlmutter — 2022-02-14T23:41:21.123Z

I am requesting suggestions for an analog-ish rewritable tablet. Details below.

I have a paper and pencil task list for work that I repeatedly erase and write over as I complete various tasks. After a month or two, it gets hard to read and I have to copy everything onto a new list. I'm looking for an alternative type of paper that can be erased many times without smudging or erasing accidentally. I don't want a fully digital solution like an iPad or a digital text document, but rather something with a look and feel similar to paper or a dry erase board, but that erases easier than paper and avoids the sort of smudging and accidental erasure that you get with a dry erase board.

The two possibilities that I found so far are Boogie Board and Rocketbook (and other brands producing similar items to those two.) They seem nice in theory but reviews indicate that the technologies are not quite mature. Boogie Board all the text lightens every time you use the partial erase, and Rocketbook takes 15-30 seconds for the ink to dry and still smudges after that.
Why Parents Should Let Their Children Play More Video Games
by Anna Yurovskikh — 2022-02-14T21:53:33.840Z

The gaming industry has a huge and gradually expanding audience. According to

Statista

, 2.69 billion people enjoyed digital journeys in 2020. Their number jumped to 2.81 billion in 2021. This data covers men, women, and kids. The latter category is especially interesting. Are there many children playing video games?

Let’s have a look at the information that comes from the US. 70% of American citizens under 18 dive into computer toys on a regular basis. This is most likely to be similar to other countries as well.

In my childhood, I spent pretty much time exploring worlds and completing missions. And I remember what feelings it ignited in me. I was a hero fighting powerful villains, bringing justice, not afraid of any challenges. It was so dizzying, hooking, and inspiring. I have grown up on adventures.

So I understand why modern kids are so attracted to them. But this is where super-protective mums and dads may interrupt. And would say that my parents should have banned my time there! I disagree, but let’s have a look at their arguments.

3 Strongest Arguments Against Gaming: Are They That Solid?

The following fears are like a red rag for parents. They consider them to be the key reasons to take digital fans away:

Addiction. Users become mindless zombies and literally live in the non-real world.
Inappropriate content. Kids playing video games get to see over-violent or sexual scenes.
Physical and mental health issues. Lack of exercise and staring at the screen for a long time are bad. It leads to physical problems, social ones including.

There are many more arguments to consider. But most of the watch-outs are either overblown or easily solved. For example, infatuation is often taken for addiction. Yes, young players will drive you crazy leading every conversation to their passion. But don’t hurry to call them addicted.

As for inappropriate content, there is a clear ranking system. For instance, Everyone, 10+, Adults Only, etc. Finally, studies have proved regular gamers to be as active and social as other peers.

Given that most fears are not that solid, let’s look at the bright side. Scroll down for 8 benefits young minds get all thanks to digital toys.

Should Kids Play Video Games? 8 Reasons to Allow It Right Now

1. Encourage Creativity

Studies have proven that young PC and console fans show better creativity. The one held by

Michigan State Univer

...
A compilation of misuses of statistics
by Younes Kamel — 2022-02-14T21:53:17.196Z
Link: https://youneskamel.substack.com/p/a-compilation-of-misuses-of-statistics?r=9fuyo

Statistics are hard. The more statistics I learn the harder the discipline seems. To truly extract insight form data using statistical methods one has to take many factors and nuances into account, and I discover new ones very week. This is the reason much of the studies published in some scientific journals are

false

or many companies fail to create any value when then implement machine learning and data science methods. In this post, I will list many common misuses of statistics and hopefully keep updating this post as I learn more, like a kind of wiki of statistical mistakes. So far all of the listed mistakes come from

Statistics done wrong

by Alex Reinhart

,

Intuitive biostatistics

by Harvey Motulsky

and several of

Nassim Taleb’s books

. Let’s start !

False Gaussian assumption

Most socio-economic variables usually follow fat-tailed distributions, not Gaussian ones. Yet most financial and econometric models assume a Gaussian distribution. This was discovered by

Benoit Mandelbrot in 1962

.

I put this statistical fallacy in first place, even though the rest of this list is in no particular order, because I believe it has the most devastating consequences. It undermines nearly ALL attempts are forecasting, particularly in

complex dynamical systems

.

Bankers, policy makers, economists and other forecasting consumers all still rely heavily to this day on Gaussian assumptions which have been shown to be false. Fat-tailed distributions resemble a Gaussian (Normal) distribution, except the probability of extreme events is much greater and does not decrease exponentially as we get further from the mean, but sub-exponentially.

The most insidious aspect of fat-tailed variables is that they will appear Gaussian until the advent of a catastrophic, extreme event. As some traders say, “fat-tails hide their tails”.

Misunderstandings of p-values

The p-value is the probability of observing data as extreme or more than ours if the null hypothesis is true :

p = P(Data at least this extreme| H0)

It is extremely common to find data scientists, academics and even some statisticians who misinterpret p-values. It is not the probability of H0 being true nor the probability of H1 being false. A p-value can only tell us how compatible the data is with the null hypothesis being true, it does not inform us on P(H0) nor P(H1).

Base rate fallacy

If the base ra...
Have you noticed costs of being anticipatory?
by Jack R — 2022-02-14T19:15:57.061Z

Actions after which I expect some kind of response seem to be more costly than the direct time cost they incur (for me, at least).

They also incur a decreased effectiveness of the time just afterward, due to causing me to be anticipatory – e.g. I am more likely to check up on whether I have received a response, and empirically find it generally harder to focus.

Examples of such actions

Messaging someone and awaiting a response
Leaving a comment or post on LessWrong or the EA forum
Being physically near people who might call upon me or talk to me

This one also makes “messaging someone and awaiting a response” more likely since I am more likely to need to coordinate in various ways with these people or follow up about things that were discussed in person

Ways to combat this cost

Go physically far away from people – e.g. plane, train, Airbnb, hotel, etc.

This seems to be a big part of why I am so productive on planes

Make a habit of asking “how important is this message, really?” before sending messages
If working in a space with others near, create a norm that only you can go into your working space, or that others should message you if they want to enter?

I am curious who else has or hasn't noticed this kind of cost, and whether anyone has ideas for combatting it (my guess is that the policies I am currently operating under don't respect this cost enough -- e.g. I made 3-5x more progress toward my most important goal while on a plane yesterday than while not on a plane today, and it seems like if I had better mechanisms for appreciating the costs of being anticipatory, this difference could have been at least 10% smaller).
Question 5: The timeline hyperparameter
by Cameron Berg — 2022-02-14T16:38:17.006Z

Does the timeline really matter?

At the outset, I added an asterisk to the hierarchical framework put forward in this sequence: the final question, what is the predicted timeline for the development of AGI? is explicitly not hierarchical—i.e., unlike every other question considered thus far, the formulation of this question is not strictly dependent on how we answered the previous one. However, this should not be taken to mean that it is strictly orthogonal, either: surely, for instance, our answer to this question will have something to do with our predicted learning architecture from Question 1 (e.g., a super-human-level AGI will take longer to build than a sub-human-level AGI). Nevertheless, this question is essential for successfully calibrating an end-to-end AGI safety research agenda. In a sentence, our predicted AGI development timeline will tell us how quickly we need to answer the previous four questions.

Of course, this all presupposes that the AGI safety control problem needs to be solved prior to—or, at the latest, concurrently with—the development of AGI. This, in turn, depends on the probability we assign to the prospect of an ‘uncontrolled’ AGI—i.e., an AGI for which no control proposals have been implemented—posing negligible existential risks (i.e., being ‘benign’). The higher the probability that AGI is benign-by-default, the less it would matter that we solve the control problem strictly prior to the development of the first AGIs.

Here is a one-paragraph argument that the probability of AGI being benign-by-default is low. Remember: a generally intelligent agent is one who, by definition, is capable of ‘efficiently achieving goals across a wide range of domains.’ To believe that AGI would be benign-by-default, one would have to defend the premise that, in the process of achieving any of its conceivably assigned goals, the AGI could pose no existential risk. This would require also claiming that AGI would, by default, pursue the right goals in the right way for the right reasons—if any of these conditions do not hold, we have already demonstrated how this could pose an existential risk (see Question 2). If we want to claim that ‘uncontrolled’ AGI—and 'uncontrolled' humans supervising this AGI—are very unlikely to be aligned by default (this seems highly plausible), we are compelled to conclude that the probability of the development of AG...
Framing Practicum: General Factor
by tailcalled — 2022-02-14T16:35:30.705Z

Credit to johnswentworth for being the general factor[1] of framing practicums.

This is a

framing practicum

post. We’ll talk about what general factors are, how to recognize general factors in the wild, and what questions to ask when you find them. Then, we’ll have a challenge to apply the idea.

Today’s challenge: come up with 3 examples of general factors which you have not thought of as general factors before. They don’t need to be good, they don’t need to be useful, they just need to be novel (to you).

Expected time: ~15-30 minutes at most, including the Bonus Exercise.

What are general factors?

Consider the effects of aging on various bodily functions. Aging makes your muscles weaker, makes you more vulnerable to germs, makes your senses and mind worse, increases your risk of cancer, and so on. Aging is a common cause for a huge amount of health problems.

Any one of these effects would be relatively important, but the fact that aging has so many effects makes aging supremely important within contexts to do with health.

The importance of aging shows up in a lot of ways. One is a theoretical perspective; medical studies must make sure not to be confounded by aging, as otherwise their results will be completely uninterpretable. Another is an interventionist perspective; if we could control aging, we could cure an enormous number of health problems. And there's also planning perspectives; it is more practical to do intense things when young than when older, as your body can better handle them while you are young.

Aging is not the traditional example of a general factor; a more canonical example would be the g factor of intelligence. g is an ability or set of abilities that people vary in, and it is required to various extents by just about any cognitive task you can come up with. When it comes to IQ tests, the specific skills that tap into g are often called indicators of g, though I will call them outputs to emphasize that they can be causally relevant too.

The cases where general factors become especially important are when their outputs have common effects. For instance, many outputs of aging, like weak immune systems or higher cancer rates, will also increase your mortality. These shared effects add up, and makes age the most important determinant of survival, as the likelihood of surviving the next year consistently drops with age.[2] Similarly, ...