trentmkelly/LessWrong-43k huggingface dataset:
AI Alignment and Recognition
by Chris_Leong — 2022-04-08T05:39:36.015Z

Let's suppose we succeed in aligning a super-intelligence. We should expect that the super-intelligence will be able to provide a pretty good estimate of how impactful various people's actions were. So maybe there are some people toiling away on AI Safety who feel sad that their efforts aren't being recognised. I guess what I'm saying is that if we succeed you will be. I'm hoping that at least some people will find this encouraging.
Nature's answer to the explore/exploit problem
by lizard_brain — 2022-04-08T05:13:10.010Z

A Luna moth caterpillar spends its life eating the leaves of a host tree and grows to more than ten times its original length in about six weeks before it pupates. The adult which emerges some time later has no functioning mouthparts and only survives for about a week.

What possible reason could explain this seemingly self-destructive strategy?

After metamorphosis, an adult gains two important features: flying and mating. It specializes to its comparative advantage: dispersing its offspring. A moth that finds a new source of food succeeds not by eating, a waste of its expensive adult features, but by making new caterpillars which can eat many times more efficiently. The moth's purpose, after all, is to reproduce, and it cannot afford to do anything that interferes with that mission.

The moth follows a very simple program: eat a fixed amount[1], then search for new opportunities. While rigid (as it can only do one or the other, and it has to determine the right strategy ahead of time with limited information), this strategy, properly calibrated to the moth's environment, apparently is successful.

^

Various species which undergo metamorphosis have been shown to change the body size at which they transform in response to environmental cues. I am assuming this is not the case; the strategy does not require it.
Edge cases don't invalidate the rule
by Adam Selker — 2022-04-08T04:17:41.133Z

(Epistemic status: obvious in retrospect)

A standard thought experiment: design an artificial neuron that works just like a biological one. Then, replace the neurons in someone's brain, one by one. If consciousness is tied to the atoms of the neurons, or their biochemistry, then the person must not be conscious afterwards; but when did they become unconscious? It's hard to imagine them crossing a hidden threshold and becoming a p-zombie. Therefore, an artificial brain must be just as conscious as a biological brain.[1]

No, wait, don't do that. Instead of replacing the neurons, just remove them. Clearly, one missing neuron won't render someone unconscious. Therefore, an empty skull is just as conscious as a full one.

This is, of course, a sorties paradox. Some things are conscious, some aren't, and some... kinda are. You don't have to throw out your binary categorization, though. The existence of edge cases doesn't invalidate the grouping.

Some things like this:

Biological sex and intersex people
Vegetarianism and eggs -- should it depend whether they're fertilized?
Alive, dead, sitting in a vat of liquid nitrogen at Alcor

-------

Car engines are sometimes taxed by displacement, with cutoffs at round numbers. You pay less for a 1.99-liter engine than a 2-liter engine.[2] This leads to a lot of cars with 1.99-liter engines. I don't think this is a big deal; maybe if the tax were smooth, some engines would be a few hundred cc's bigger or smaller. But I used to find it creepy, as if the law itself were at odds with reality. Really, I just hadn't accepted that it's okay for a rule to have small inconsistencies.

Some things like this:

Your phone battery quits charging and says "100%" at some voltage threshold
The doctrine of the preferred first speaker
Rules you use for willpower: "no video games after 11 pm"

-------

Paraphrased conversation with my friend, after we failed to hang out because we live in different time zones:

"Can we all just use UTC? That way, I'd know what time it is anywhere."

"But time zones are useful to know when people are awake and available to hang out. 'What time is it in Boston?' is easier to Google than 'is it okay to text my friend in Boston at 6pm?'"

"But what you really want to know isn't 'what time is it in Boston', it's 'is my friend available to chat?' &...
Reverse (intent) alignment may allow for safer Oracles
by azsantosk — 2022-04-08T02:48:00.785Z

This is a core concept in my AI-in-a-box success model.

What is reverse alignment?

Consider the world-state optimized for the AI's utility, on one hand, and the world-state optimized for human utility, on the other. Let us call the first state "AI victory," and the second state "human victory."

Then let us cross-evaluate these states by considering what is the AI's utility associated with human victory, and what is the human utility associated with AI victory. We say that the AI is aligned with humans if human utility is almost as high for AI victory as in human victory. This is direct impact alignment.

Conversely, we can say that humans are aligned with the AI if the AI's utility is almost as high for human victory as in AI victory. This is what I call reverse alignment, or R-alignment for short.

How is that concept useful?

Of course we usually don't care much for maximizing the AGI's utility. So why should we care about this concept?

The main idea is that an R-aligned AGI will consider the result of human rule to be almost as good as that of its own rule. And it may be easier to create an R-aligned AGI than an AGI that is directly aligned.

It is important to understand that an R-aligned AGI might not be a benevolent AI: if allowed to rule, such an AGI could easily destroy humanity completely, in order to add an extra epsilon to its utility. Still, such an AGI would be extremely useful, since it would not be willing to take a significant risk in order to gain power for itself.

Because a "free" superintelligent AGI may not need to risk much of anything in order to gain power for itself, the concept is not that relevant for them. But for bandwidth-limited Oracles, that may realistically be constrained by strong tradeoffs in their ability to influence the world, the concept is critical to determine whether the Oracle will really try to help its creators or whether it will do nothing more than manipulate them in order to escape.

A measure of R-alignment

Let us now define a standard R-alignment value that we can use as a safety measure.

For a given AGI, besides the two possible outcomes of "AI victory" and "human victory," there are other scenarios in which neither the given AGI nor its creators achieve strategic dominance. Let us call these the default scenarios.

Since decision making is invariant under linear transformations of utility, ...
Summary: "Internet Search tips" by Gwern Branwen
by Pablo Repetto — 2022-04-08T02:02:29.997Z
Link: https://pabloernesto.github.io/2022/04/07/summary-gwern-tips.html

This post is part of my "research" series.

Branwen, Gwern. Internet Search Tips, 11 December 2018. https://www.gwern.net/Search. Accessed: 2022-04-05.

Why

If memory serves, I learned of Gwern through LessWrong. He was mentioned as a guy who did really careful self-experiments with different drugs. His blog seemed consistently well-researched and earnest (plus he makes and shares some cool data analyses), so I downloaded a copy of his research tips.

In one paragraph

This article can be read as the field notes of a guerrilla researcher. Apart from search, there is advice on jailbreaking, cleaning, and redistributing digital texts, as well as scanning and OCRing physical ones. The search advice extends Eco's bibliographic research methodology into the Internet.

Table of Contents

1 Papers
1.1 Search
Preparation
Searching
Drilling Down
By Quote or Description
1.2 Request
1.3 Post-finding
1.4 Advanced
2 Web pages
3 Books
3.1 Digital
3.2 Physical
4 Case Studies
4.1 Missing Appendix
4.2 Misremembered Book
4.3 Missing Website
4.4 Speech → Book
4.5 Rowling Quote On Death
4.6 Crowley Quote
4.7 Finding The Right ‘SAGE’
4.8 UK Charity Financials
4.9 Nobel Lineage Research
4.10 Dead URL
4.11 Description But No Citation
4.12 Finding Followups
4.13 How Many Homeless?
4.14 Citation URL With Typo
4.15 Connotations
4.16 Too Narrow
4.17 Try It
4.18 Really, Just Try It
4.19 (Try It!)
4.20 Yes, That Works Too
4.21 Comics
4.22 Beating PDF Passwords
4.23 Lewontin’s Thesis
5 See Also
6 External Links
7 Appendix
7.1 Searching the Google Reader archives
7.1.1 Extracting
7.1.1.1 Locations
7.1.1.2 warcat
7.1.1.3 dd
7.1.2 Results
8 Footnotes

An alternative outline

Broadly, how this article reads from my perspective.

1 General search tips
2 Web archival sites
3 Jailbreaking (s3.1)
4 Scanning books (s3.2)
5 Case studies and miscellaneous advice (s4)

The organization of this article is terrible. There is a lot of advice on different topics, but no way to find it. If I want all of the advice relevant to web crawling I'd have no choice but to read the whole article. I'll probably extract the information in this article to a more sensible structure. At some point. That will mean rewr...
Maxwell Peterson's Highlighted Posts
by Maxwell Peterson — 2022-04-08T01:34:57.006Z

Collected links to my research & math posts.

A sequence on interpreting the central limit theorem in terms of convolutions.
The standout post from that sequence. The standard statement of the CLT says that the input distributions must be identically distributed, but it turns out that requirement can be bent pretty far without issues.

Also from that sequence: proving that the form of the Bayesian posterior suggests that Bayesian calculations will often converge to a Gaussian. I think this is the same thing the Bernstein-von Mises theorem is about, though I didn't know that when I wrote it.
A simple post describing the basics of when and how to use the Beta distribution, meant for Data Scientists and analysts who haven't heard of the Beta.
A speculative post about interpreting the mean, mode, and median in terms of how much information each uses from the underlying distribution.

Activated Charcoal for Hangover Prevention: a heavily researched essay about why activated charcoal might help prevent hangovers.
A short analysis about a toy game. Has some nice graphs.
Describing a few types of measures from Measure Theory.

A post thinking about how to interpret uncalibrated predictions from a classifier. I wrote this before knowing about Platt/Isotonic/Spline calibration; any of those would solve the issues raised in the post.
A post exploring a wider definition of priors, and finding hints of them in frequentist practice.

Adding to an argument Jaynes made in Probability Theory: The Logic of Science, about Occam's Razor and the geometry of probability-space.

Thoughts on a different way to present True Positive Rate / FPR / TPR / TNR.
Foot-Chording Chords
by jefftk — 2022-04-08T01:10:02.401Z

As part of my

rhythm stage
setup, I have four pedals which I primarily use for

heel-toe drumming.
When playing mandolin, or holding a baby, however, I can instead use
these pedals to play bass notes or chords. I tell it what key and
mode I'm in, and then can choose one of the four chords I'm most
likely to want. For example, if I'm playing in C Major I have:

Am (vi) G (V)

C (I) F (IV)

In showing this to Ruthie,
however, we realized that more options are possible if you use
multiple simultaneous pedals. I am unreasonably excited to be "chording" to
play chords.

Rough demo, since I only just figured this out:

If you use all the combinations this gives you eleven
more options, but for now I've just added two: the left pair and the
right pair. In major I've set these as ii and iii, so C Major is:

Am (vi) G (V)
Dm (ii) Em (iii)
C (I) F (IV)

Here are the three other modes I currently have configured, with C as
an example key:

Minor:

G (V) Cm (i)
F (IV) E (III)
A (VI) Bb (VII)

Alternate minor:

Bb (VII) F (IV)
Ab (VI) G (V)
C (i) E (III)

Mixolydian:

Bb (VII) G (V)
Ab (VI) E (III)
C (I) F (IV)

In each of these, I'm using the two "chorded" options to bring in
extra options I would use occasionally.

At some point I may use some of the other combinations, probably
starting with the two toes and the two heels, but for now this is
confusing enough. I will probably also change around what the pedals
do, since I feel like I don't have a great system right now: it works,
but there isn't that much logic to it so it's hard to remember and
extend.

Comment via: facebook
DeepMind: The Podcast - Excerpts on AGI
by WilliamKiely — 2022-04-07T22:09:22.300Z

DeepMind: The Podcast - Season 2 was released over the last ~1-2 months. The two episodes most relevant to AGI are:

The road to AGI - DeepMind: The Podcast (S2, Ep5) and
The promise of AI with Demis Hassabis - DeepMind: The Podcast (S2, Ep9)

I found a few quotes noteworthy and thought I'd share them here for anyone who didn't want to listen to the full episodes:

The road to AGI (S2, Ep5)

(Published February 15, 2022)

Shane Legg's AI Timeline

Shane Legg (4:03):

If you go back 10-12 years ago the whole notion of AGI was lunatic fringe. People [in the field] would literally just roll their eyes and just walk away. [...] [I had that happen] multiple times. I have met quite a few of them since. There have even been cases where some of these people have applied for jobs at DeepMind years later. But yeah, it was a field where you know there were little bits of progress happening here and there, but powerful AGI and rapid progress seemed like it was very, very far away. [...] Every year [the number of people who roll their eyes at the notion of AGI] becomes less.

Hannah Fry (5:02):

For over 20 years, Shane has been quietly making predictions of when he expects to see AGI.

Shane Legg (5:09):

I always felt that somewhere around 2030-ish it was about a 50-50 chance. I still feel that seems reasonable. If you look at the amazing progress in the last 10 years and you imagine in the next 10 years we have something comparable, maybe there's some chance that we will have an AGI in a decade. And if not in a decade, well I don't know, say three decades or so.

Hannah Fry (

5:33

):

And what do you think [AGI] will look like? [Shane answers at length.]

David Silver on it being okay to have AGIs with different goals (??)

Hannah Fry (16:45):

Last year David co-authored a provocatively titled paper called Reward is Enough. He believes reinforcement learning alone could lead all the way to artificial general intelligence.

[...] (21:37)

But not everyone at DeepMind is convinced that reinforcement learning on its own will be enough for AGI. Here's Raia Hadsell, Director of Robotics.

Raia Hadsell (21:44):

The question I usually have is where do we get that reward from. It's hard to design rewards and it's hard to imagine a single reward that's so all-consuming that it would drive learning everything else.

Hannah Fry (21:59):

I put this question about the difficulty of designing an all-powerful reward to

...
Convincing Your Brain That Humanity is Evil is Easy
by Johannes C. Mayer — 2022-04-07T21:39:14.688Z

Epistemic Status: Based on my own experience and intuitions only.

Some reasoning heuristics, that the brain uses, seem to rely on that there is limited information available. And newspapers, TV, and search engines break them hard.

Assume, that you want to evaluate if something is really good. A heuristic that your brain seems to use is to look at all the available data and search for bad things. If you reach some threshold of cumulative badness, then you will conclude that the thing can't be really good. The problem is that the threshold does not properly take into account, from how much data you select your observations from. So if you have enough data, you will always hit the threshold, so long as a bad thing occurs with a probability greater than zero.

Assume that the probability of being born a psychopath is 2%, that you live in a hunter-gatherer tribe of 12, and that you see 5 generations over your lifetime. Then you see around 60 people in your life within your tribe. You will have a 30% chance of not ever seeing any psychopath. And the psychopaths that you might see, might not be dumb enough to get caught, or might not do anything bad in the first place. So it is really unlikely that you will see any really bad behavior, caused by people being psychopaths.

Now assume you are thinking about the (horribly underspecified) question of if humans are innately good. It seems that in this situation, you would be more likely to think so because it is unlikely that you will observe a lot of bad behavior. And I would expect that the badness threshold is calibrated based on situations like this.

Compare this to the situation today. Even if we only consider video material of the last decades that is publically accessible, we have an unending fountain of examples of people who did terrible things. Just type psychopath, serial killer confession, murderer, or related terms into youtube.

Using the heuristic described in the beginning, your brain might very quickly conclude that humanity is evil if you are not careful.

Let's ask a better question. What is the percentage of human beings that do terrible things to no good end? Asking this question, it becomes much easier to see, that we need to know how many people are there in total, and how many of these people did terrible actions how often.

The problem is not only that our brain uses this heuristic. It is also that we normal...
Playing with DALL·E 2
by Dave Orr — 2022-04-07T18:49:16.301Z

I got access to Dall·E 2 yesterday. Here are some pretty pictures!

My goal was to try to understand what things DE2 could do well, and what things it had trouble understanding or generating. My general hypothesis is that it would do a better job with things that are easy to find on the internet (cute animals, digital scifi things, famous art) and less well with more abstract or more unusual things.

Here's how it works: you put in a description of a picture, and it thinks for ~20 seconds and then produces 10 photos that are variations on that description. The diversity varies quite a bit depending on the prompt.

Let's see some puppies!

goldendoodle puppy in play position

One thing to be aware of when you see amazing pictures that DE2 generates, is that there is some cherry picking going on. It often takes a few prompts to find something awesome, so you might have looked at dozens of images or more.

Still, this is pretty great! Those are recognizably goldendoodle puppies, mostly in something approximating play position.

You can see that the proportions in the generated images are not quite right, and some of the detail is off if you look closely. For instance, the front legs are too long here, the face isn't quite right, and the ears are a bit weird.

Still, it's pretty amazing given that it generated this from scratch. Check out how realistic the grass looks. I also like that the background is blurred, though not quite in the way that a camera would do it -- the transition is too abrupt.

Ok but the point of this isn't that they have a great image generation transformer, though it's clearly that. The key thing is is its magical ability to actually follow instructions or descriptions of images. Particularly interesting is compositionality -- can it combine concepts to generate something it's never seen before? Answer: yes!

pop art kittens

The concept of "kitten" is pretty simply, though note that a kitten can be rendered in a ton of ways, from line drawings to cute art to photorealistic. Pop art is more complicated: it's a celebration of everyday images, and one of the most commonly known versions is Warhol's collection of repeated images in a grid with neon colors that vary per cell. And it mostly gets those things right.

lots of pop art kittens

What about weird things? You can put in any input and it'll do something.

an ai in the shape of a campfire tellin

...
The Explanatory Gap of AI
by David Valdman — 2022-04-07T18:28:24.493Z

If an AI understands is a topic of debate. Seldom debated, however, are the limits of our own understanding. We imagine our understanding as complete and establish this as a baseline for AI. A different framing appears when we concede our understanding as incomplete and apply this to AI.

The fundamental issue goes like this:

An AI may describe an apple as "sweet" though it has never tasted one, "red" though it has never seen one, and "smooth" though it has never felt one. So while an AI may skillfully apply the word “apple”, in what way, if any, can we say an AI understands the word “apple”?

Steven Sarnad named this the “symbol grounding problem." He reasoned that the symbols we use as reference must be grounded, or referent, in phenomena to be meaningful. After all, the word “apple” when written in a book has no meaning on its own. Meaning of symbols is “parasitic on the meanings in our heads” he writes.[1] To Sarnad, the AI doesn’t understand at all, only we do, as only we can bridge the gap between reference and referent.

Yet now we have AIs that seem to ground text in images.[2] Have we proved Sarnad wrong? Do these AIs understand something previous AIs did not? Predating Sarnad, John Searle, with keen foresight, addressed this issue. In Minds, Brains and Programs, Searle fashions and then refutes the “Robot Reply” argument. He describes an AI that would “have a television camera attached to it that enabled it to 'see,' it would have arms and legs that enabled it to 'act,' and all of this would be controlled by its computer 'brain’.”[3] Yet to Searle, this “adds nothing” to the argument. We are simply introducing more symbols, and to exploit correlations in symbols between modalities is no different than exploiting correlations within a modality. To Searle, grounding was neither the problem, nor the solution, to semantics arising from syntax.

So just what is a grounded AI lacking? On this question, Sarnad and Searle are silent. Less silent, though, were the empiricists predating them. We attempt to get at our question by asking what they asked: what are grounded human minds lacking?

Back to the apple. The empiricists accept that the sweetness, redness and smoothness of an apple is a manifestation of our minds. Outside the mind, apples have no taste, color or texture. These properties make up the phenomena of apples. To know what an “apple” is ou...