trentmkelly/LessWrong-43k huggingface dataset:
A naive alignment strategy and optimism about generalization
by paulfchristiano — 2021-06-10T00:10:02.184Z

(Context: my last post was trying to patch a certain naive strategy for AI alignment, but I didn’t articulate clearly what the naive strategy is. I think it’s worth explaining the naive strategy in its own post, even though it’s not a novel idea.)

Suppose that I jointly train an AI to do some task (e.g. make money for me) and to answer a wide range of questions about what is happening in the world (e.g. “why did Alice just wire $1000 into my bank account?” or “what is Bob thinking right now?”).

I generate training data for the QA task in a really simple way: I choose a subset of questions that humans are able to reliably answer, and use those as a training set for supervised learning. I’ll call this the naive training strategy.

I’d like for my AI to tell me everything it knows. If the AI bought a stock because it expects a merger announcement soon, I want it to tell me about the predicted merger announcement. If the AI predicts a merger announcement because it inferred that executives of the companies have been in extensive talks over the last month, I want it to tell me about those talks.

I’m not asking the AI to explain why it made a given decision, I’m asking the AI to tell me as much as it can about the world. The important property is that if the AI “knows” something and uses that knowledge to perform the task well, then it also uses that knowledge to answer questions well.

Why might this work? The hope is that “answer questions honestly to the best of your ability” is a natural thing for our AI to learn — that there is some simple way to translate from the AI’s model of the world into natural language and to honestly report what it believes. If our training dataset is good, then this policy will score well, and we can hope that SGD will find it. I’ll call this the intended policy.

Why might this not work? The concern is that “predict how a human would answer questions” is also a natural thing for our AI to learn, especially if the AI is doing a task that already requires predicting humans. Predicting humans also gets a low loss on the training set, but it generalizes poorly once we start asking our AI questions that a human couldn’t have answered on their own.

If we’re worried about this we could use a different loss function than “predict what humans would say.” But regardless of what loss function we choose, the policy could still b...
What are some important insights you would give to a younger version of yourself?
by silentbob — 2021-06-09T20:29:05.797Z

I recently watched an older TED talk on Youtube, and after watching it, realized there was a comment beneath it, written by me, 10 years ago:

This is frustrating. I have all the possibilities to live a great successful life, but I just don’t have any idea how to keep my motivation up over time.

This surprised me. Not only because it’s a bit unexpected to watch a random Youtube video only for it to end up being a window to your personal past, but also because I had pretty much forgotten about my frustrations at that time. When I wrote that comment, I was around 20. And reading it made me think a lot: 30 year old me really doesn’t struggle with the same issues anymore. Why is that? What am I doing differently? How did I get here? Did I do this, or was I just lucky? If I were granted a conversation with 20 year old me, what would I tell him? Could I in any way accelerate that journey from frustrated-me back then to a more satisfied version of myself? What words would I need to say for past-me to be able to apply everything I’ve learned since?

This admittedly is not a very original question. I’ve come across a whole lot of “what do you wish you had known when you were 18?” type posts, videos, lists and comments over the years. And I’m sure there are some on lesswrong as well. Although, surprisingly, I didn’t find anything too close despite using a whopping three different search terms – surely that justifies a new post on the topic.

So, I’d be interested in other people’s thoughts, and would phrase the question this way:

What are the three most important pieces of information which you could give 10-years-ago-you that could positively influence their life?

I’m of course not referring to random trivia about the time since, such as “Invest in Bitcoin/Tesla/Zoom”, but rather “robust” knowledge that would be likely to be useful in most possible futures of your past self. Also, these tips may be pretty specific to your situation back then and not apply to others. This is fine, no need for generalization here.

What follows is my attempt to answer the question for myself.

Firstly: Separate your drive/motivation/actions from visible immediate results.

Find some intermediate steps to create motivation. Experiment with small (randomized) rewards, track your progress visually, derive motivation from reaching goals even if they’re just numeric in nature and don’t immediately c...
Other Constructions of Gravity
by michaelcohen — 2021-06-09T20:06:48.750Z

In Newtonian gravity, the energy is proportional to the sum over pairs of masses of −m1m2/r1,2. Or in the continous case, −∫∫dm1dm2|pos(m1)−pos(m2)|2. This does not actually strike me as very simple in whatever language makes Maxwell's equations and the Schrödinger equation simple.

Partially-formed idea 1:

Here's an outline of something that seems like it might be a simpler theory of gravity. Given a density function ρ over physical space (possibly including Kronecker deltas for point masses), we need to come up with a gravitational potential U. First, convolve ρ with some (radially symmetric) function f:R3→R. Write this ρ∗f. Let F{ρ∗f} be the Fourier transform. I think taking the Fourier transform of something in position space gives something in momentum space. Then, you can take a "measurement" of the momentum squared by evaluating ∫R3F{ρ∗f}(ω)2ω2dω. And I think the momentum squared is energy-like. If ρ is the sum of two delta functions that are a distance d apart, surely there is some f, such that for large d, this looks like c1−c2/d.

It is possible that this could be made to resemble normal Newtonian gravity in the case of a few point masses, or a single mass (like the sun) that is much more massive than other nearby masses. But it would maybe also give different-looking answers when there is lots of mass evenly spread out of large distances, on the scale of, say, rotating galaxies.

Even-less-formed idea 2:

If you are experiencing gravity, you don't actually know exactly where the masses are which are causing this gravity. Doing some heavy handwaving with the gist of the Heisenberg Uncertainty Principle, to the extent you know the precise position of something, then from your perspective, it has lots of momentum. Maybe as you become more able to judge the precise position of nearby gravitating masses, your relative momentum increases to compensate. (This would look like speeding up as you approach a mass).

If you're around other masses, you don't detect the gradient of gravitational potential, because you fall down it, and free-fall feels like floating. But, if you have volume, you can detect the Hessian of gravitational potential. In the extreme case, this manifests as spaghettification near a black hole. (Perhaps a cost of having volume is that you don't know exac...
How do you establish a comfort zone in your studies?
by DirectedEvolution — 2021-06-09T17:01:26.242Z

Learning a new topic takes you outside your intellectual comfort zone. Extrapolating from the spacing effect, the practice of overlearning a chunk of new material is an inefficient way to build memory. I notice, however, that overlearning feels comforting. It seems to establish a comfort zone in the new material. It makes me feel more confident that I've learned something new, like the material is becoming a part of me. And when I review the earlier material, I can approach the new material that builds on it with greater ease.

If you approach study with the idea that it's all about efficiently building a memory for the material, you might neglect the motivational aspects of study. What do you do to establish a comfort zone in your studies? Do you find that sense of a "comfort zone" motivating? What else do you do to enhance your motivation and engagement with your studies, even if it's not strictly optimal in the short term for building new memories?
Book review: "Feeling Great" by David Burns
by Steven Byrnes — 2021-06-09T13:17:59.411Z

I've never had any "real" mental health problems, but sometimes I feel stressed or guilty or whatever, like everyone, and who doesn’t want to feel more good more often? So a couple months ago I read Feeling Great: The Revolutionary New Treatment for Depression and Anxiety by David Burns (published 2020) on audiobook. I was really glad I did!

I can't comment on how it compares to other psychotherapy books. It's the first one I've ever read, and I kinda came upon it randomly—an acquaintance recommended David Burns's podcast, so I listened to a couple random episodes, and I found them intriguing but confusingly out-of-context, so instead I bought his book which was much better.

But for what it's worth, David Burns's older 1980 book, Feeling Good, is super famous and popular, and apparently there are studies that say that giving people a copy of Feeling Good is as effective as antidepressants, with effects that persist for years (

ref

). (More discussion on wikipedia.) Also, I just saw that Scott Alexander suggested Feeling Great for people with depression. So of all the psychotherapy books to randomly stumble across, I think I got a pretty legit one!!

I'm not going to talk about everything in the book in this review; I just want to flag a few parts that were highlights for me.

I also couldn't resist throwing in some speculations on the neuroscience of depression at the bottom.

“Classic CBT” stuff

David Burns is, I gather, something of a leader in Cognitive-Behavioral Therapy (CBT). What is CBT? My vague pop-culture stereotyped impression of CBT has been something like:

The patient says “I’m a terrible person and everyone hates me”.
Then the therapist and patient have a discussion to try to tease out (1) whether that’s actually true (very often it’s not), (2) even if it is true, whether it’s a good reason to feel miserable, as opposed to, y’know, self-acceptance, trying to solve the problem, etc., (3) given 1 and 2, what are good strategies to actually stop feeling miserable, including what to think about, what to visualize, what to do, etc.

I haven’t read David Burns’s more famous older book Feeling Good, but my vague impression is that it’s largely about that kind of stuff, and in particular, that it’s full of lots of different techniques for questioning and countering negative thoughts and feelings.

This new book reprises and (he says) somewhat improves on that material....
Bad names make you open the box 
by Adam Zerner — 2021-06-09T03:19:14.107Z

Think of a function as a black box. It takes an input, and spits back out an output.

For getPromotedPosts, you can feed it a list of blog posts and it will spit back out the ones that have been promoted.

But I probably didn't need to explain that to you, did I? Why not? Well, because getPromotedPosts is self-explanatory. Because getPromotedPosts is named well.

Now what if instead of getPromotedPosts, it was named something like getThePosts? Well, that name isn't self-explanatory. You know it's getting some posts, but it's not clear which ones. The most recent posts? Posts from a certain author? Posts from this week?

As a programmer, what do you do in this situation? You scroll to the function definition and start reading the code.

function getThePosts(posts) {
...
}

In other words, you open the box.

What does that look like? Something like this:

You're reading through the code in some file. Line one, line two, line three.

You reach line 30 and see getThePosts.

You realize that getThePosts must be getting some posts, but you don't know which ones. So you have to scroll to line 174 where getThePosts is defined.

On line 174 you start reading through getThePosts. Once you reach line 210, you realize that it is getting promoted posts. Cool!

Now you scroll back up to line 30. You realize that getThePosts is giving you promoted posts, but you forgot what was happening before line 30. Damn. So now you have to go back to line 10 or 15 to remind yourself what was going on in the first place.

Complexity and zoom level

Maybe I was being dramatic. Is it really such a big deal to have to scroll to the definition of getThePosts on line 174? Will it really take that much effort to read lines 174-210 and figure out that it's returning promoted posts? It's only 36 lines of code, including whitespace + brackets, and you could probably glance over parts of it. And then what about returning to line 30? Are you really going to have forgotten what was going on so quickly? Are you really going to have to scroll back up to line 10 or 15 to remind yourself?

In this example, perhaps not. I'm not sure. Maybe having to open the box gets in the way, maybe it doesn't. What I am sure of is that when the code becomes more complicated, having to open the box becomes more of an issue. I think Eric Dietrich's post on how harmful interruptions can be to a programmer gives us a great intuition fo...
...and then sometimes, for no clear reason, they innately become good.
by Andrew Vlahos — 2021-06-09T03:07:41.559Z

This is a follow-up to "Often, enemies really are innately evil." I waited a few days before posting this because although this is true, I really wanted the message to sink in that many people, given a choice between torturing someone for 50 years and throwing dust specks into the eyes of 3^^^3 strangers, they would choose both. Denying this has done direct harm in a lot of people's lives including mine.

However, I should give the rest of the story. The main bully I had was very cruel, but after several years they suddenly stopped. Not overnight, but it was weirdly quick. Not only to me, but to the other people they also bullied at their school. And the change stuck; he never went back to it. Today he's quite a nice person.

There were a few other changes in his life around that time. He got really into World of Warcraft. He got really into church. But these don't seem likely to be the cause; I don't think a post titled "Why superstimulus and Christianity are the key for moral improvement" will be posted anytime soon. He attributes his change mainly to building good, constructive friendships with the people there. This is more reasonable; it is possible for good friends to inspire each other to do better. I'm not sure this is sufficient though. It was a very major change.

I can't say he just "grew out of it" because a lot of evil people remain evil as adults. My evil cousin "M", at least for now, is still evil. (Know how hard it is for a woman to be prosecuted for abuse, and lose full custody of the kids to the father? She managed.) I once caught an adult trolling in real life. I asked why they did that and he said causing suffering is fun. (I still consider this innately evil) One of my teachers tended to be casually mean to his students: for example, on a Monday he said that something is due "next Friday". I asked if he meant the Friday in 4 days or the one in 11 days, since sometimes "next Friday" means "next week's Friday". He mocked me in front of the class for not knowing what "next" means, and said it was the Friday in 4 days and that I was stupid not to know that. (Then later he said "next Wednesday" in a context that meant the one in 9 days). I will always think of him as professor Snape. Becoming good clearly isn't reliable.

So why did it happen? I don't know.
Better Whistle Synth
by jefftk — 2021-06-09T01:50:02.928Z

A couple years ago

I got

really excited

about the

idea of controlling a
synthesizer by whistling. I came back to this recently, figured out a
much better algorithm, and it's much more musical now!

(Short version: go try my new program, in your
browser!)

In my original version, I counted zero crossings the estimate the
current pitch (detailed
explanation). Then I fed this into simple additive synthesis
code. It worked, but it had a few issues:

Sometimes it would detect whistling when there wasn't any, and
make clicks (or beeps in a later version).

If you whistled too quietly or not clearly enough, it would
failed to detect the whistling and you get silence, or drop outs.

It was very sensitive to a "gate" parameter: set it too high and
you get the first problem above, too low and you you get the second.
In noisy environments that was often no good setting.

It was not steady enough to feed into an existing
synthesizer, because those expect the precise input you get for my
keyboard.

My simple custom synthesizer didn't sound very musical.

My new version is much better on all of these dimensions. The key idea
is that because the input signal is very close to a sine wave, we can
directly use it for synthesis. Here is a short snapshot of whistling:

As before, each time we get to an upward zero crossing we estimate the
pitch:

This time, however, instead of tuning a synthesizer, we start playback
of the previous cycle at a multiple of the original speed. Let's say
we move at half speed, to shift the pitch down an octave. When the
next inputs cycle begins, we will also be beginning an output cycle.
A quarter of the way through the input cycle, when we are at our
maximum value, the output is still ramping up. This continues, until
we have finished an input cycle and half an output cycle:

At this point or zero crossing detector will give us another pitch
estimate, so we kick off another half speed speed replay. Since our
previous replay is only halfway through the wave, however, we need to
start this one at with reversed polarity or else they will cancel
out.

We can run each replay for several cycles, which has the nice benefit
of reducing incidental noise: if there is a blip In the input, it will
probably not be repeated in the next cycle. Possibly there is
something more aggressive, where we take the median val...
AXRP Episode 8 - Assistance Games with Dylan Hadfield-Menell
by DanielFilan — 2021-06-08T23:20:11.985Z

YouTube link

This podcast is called AXRP, pronounced axe-urp and short for the AI X-risk Research Podcast. Here, I (Daniel Filan) have conversations with researchers about their papers. We discuss the paper and hopefully get a sense of why it’s been written and how it might reduce the risk of artificial intelligence causing an existential catastrophe: that is, permanently and drastically curtailing humanity’s future potential.

How should we think about the technical problem of building smarter-than-human AI that does what we want? When and how should AI systems defer to us? Should they have their own goals, and how should those goals be managed? In this episode, Dylan Hadfield-Menell talks about his work on assistance games that formalizes these questions. The first couple years of my PhD program included many long conversations with Dylan that helped shape how I view AI x-risk research, so it was great to have another one in the form of a recorded interview.

Daniel Filan:
Hello everyone. Today I’ll be talking to Dylan Hadfield-Menell. Dylan’s a graduating PhD student at UC Berkeley, advised by Anca Dragan, Pieter Abbeel and Stuart Russell. His research focuses on the value alignment problem in artificial intelligence: that is, the problem of designing algorithms that learn about and pursue the intended goal of their users, designers, and society in general. He will join the faculty of artificial intelligence and decision-making at MIT as an assistant professor this summer. Today, we’re going to be talking about his work on assistance games, and in particular, the papers “Cooperative Inverse Reinforcement Learning”, co-authored with Anca Dragan, Pieter Abbeel and Stuart Russell, “The Off-Switch Game”, also co-authored with Anca Dragan, Pieter Abbeel and Stuart Russell, and “Inverse Reward Design”, co-authored with Smitha Milli, Pieter Abbeel, Stuart Russell and Anca Dragan. For links to these papers and other useful info, you can check the description of this podcast and you can read a transcript at axrp.net. Dylan, welcome to the show.

Dylan Hadfield-Menell:
Thanks so much for having me Daniel. It’s a pleasure to be here.

Daniel Filan:
I made something of an assumption in the introduction that you think about those papers under the umbrella of assistance games, is that right?

Dylan Hadfield-Menell:
Yeah, I think so. I take a pretty broad view of what assistance g...
Should we vaccinate against PGBD5 which codes for a transposase?
by ChristianKl — 2021-06-08T19:22:03.892Z

Core Pathways of Aging is background reading that explains why transposons are an issue.

There are two types of transposons. Class I TEs or retrotransposons generally function via reverse transcription, while Class II TEs or DNA transposons encode the protein transposase. I can't think of a good way of attacking retrotransposons which don't produce any proteins.

On the other hand, DNA transposons can be attacked because they need express transposase to copy themselves. When we vaccinate against a particular type of transposase our immune system will start to attack all cells that express the transposase, because cells present fragments of the transposon on their cell wall.

PGBD5 is expressed in the majority of pediatric solid tumors. While a tumor can mutate in a way that shuts off antigen presentation it's possible that PGBD5 gets expressed in the beginning of the lifecycle of pediatric cancer. While we are at it

Given that we don't know whether some cells have valid reasons for expressing PGBD5 it makes sense to not self-vaccinate and wait for clinical trials, but the potential of a cheap way to increase longevity and reduce cancer risk seem to me worth the effort.