trentmkelly/LessWrong-43k huggingface dataset:
Take 14: Corrigibility isn't that great.
by Charlie Steiner — 2022-12-25T13:04:21.534Z

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day some days for 25 days.

It's the end (I saved a tenuous one for ya')! Kind of disappointing that this ended up averaging out to one every 2 days, but this was also a lot of work and I'm happy with the quality level. Some of the drafts that didn't work as "hot takes" will get published later.

I

There are certainly arguments for why we want to build corrigible AI. For example, the problem of fully updated deference says that if you build an AI that wants things, even if it's uncertain about what it wants, it knows it can get more of what it wants if it doesn't let you turn it off.

The metal image this conjures up is of an AI doing something that's obvious-to-humans bad, and us clamoring to stop, but it blocking us from turning it off because we didn't solve the problem of fully updated deference. It would be better if we built an AI that took things slow, and that would let us shut it off if we got to look at what it was doing and saw that it was obviously bad.

Don't get me wrong, this could be a nice property to have. But I don't think it's all that likely to come up, because aiming at aligned AI means building AI that tries not to do obviously bad stuff.

A key point is that corrigibility is only desirable if you actually expect to use it. Its primary sales pitch is that it might give us a mulligan on an AI that starts doing obviously bad stuff. If everything goes great and we wind up in a post-scarcity utopia, I'm not worried about whether the AI would let me turn it off if I counterfactually wanted to.

A world where corrigibility is useful might look like us building an agenty AI with a value learning process that we're not confident in, letting it run and interacting with it to try to judge how the value learning is going, and then (with moderate probability) turning it off and trying again with another idea for value learning. What does corrigibility have to do in this world? The AI shouldn't deliberately try to get shut down by doing obviously-bad things, but it also shouldn't try to avoid being shut down by instrumentally hiding bad behavior, or by backing itself up on AWS.

Such indifference to the outside world is the default for limited AI that doesn't model that part of the world, or doesn't make decisions in a very coherent way. But in an agent t...
Simplified Level Up
by jefftk — 2022-12-25T13:00:01.546Z

Vienna Teng's

Level Up is
a great song, but when we wanted to use it in our secular solstice (

retrospective) it
wasn't going to work as-is. The problem is she's a professional
singer performing a piece she knows well, while we're a
crowd of amateurs most of who haven't heard the song before let
alone sung it.

Some aspects of the song that make it tricky to pick up on the fly:

Most of the words are pickups.

The lines don't start on a consistent beat.

Sections have small differences each time they're repeated.

It's not square.

Additionally, the sung portion ends with a build and the song's
conclusion is instrumental, which is not satisfying for group singing.

Alex and I made some changes to get it to work better in our context,
and while they make it a worse performance song I think they're a good
compromise for group singing that keeps a lot of what makes the song
worth singing. Here are the words and chords we figured out:

G
so come out.
G
been waiting long enough.
C G
you're done with all the talk talk talk with nothing on the table.
G
it's time to come on out.
G
no sign from above.
C G
you'll only hear the knock knock knock of your own heart as signal.

Em D C
if you are afraid, come forth.
Em D C
if you are alone, come forth now.
Em D C Bm D
everybody here has loved and lost,
G
[pause] so level up and love again.

G
begin again.
G
blast the dam on the flow.
C G
your body feels the tock tock tock of time as it hammers.
G
lord we are all cinders
G
from a fire long ago,
C G
but here it is the knock knock knock of your own heart that matters.

Em D C
if you are afraid, come forth.
Em D C
if you are alone, come forth now.
Em D C Bm D
everybody here has seams and scars,
G
[pause] so level up and feel again.

C D G
and this is all we need.
C D G
and t

...
Hyperfinite graphs ~ manifolds
by Alok Singh — 2022-12-25T12:24:11.910Z

Manifolds are better described as 'shapes'.

Hyperfinite is an unlimited number, often an integer. A whole number larger than any standard n.

Imagine a graph with hyperfinitely many vertices and edges, where the edges have infinitesimal weights. If I were nearer my iPad I'd draw it.

The edges are the tangent vectors of differential geometry. Another way of thinking about it is that the vertices are each the zero vector of an infinitesimal space, because they are. differential equations become difference equations. this is also why so many techniques from riemannian geometry (which includes every relevant shape) make their way into graph theory, and why diffusion is fundamental.

A world in every grain of sand

Hold infinity in your hand and eternity in an hour. If your time scale was infinitesimal, an hour would be an eternity (hyperfinite).

Important visual: ω (spelled "omega", pronounced "oh-mega") represents a hyperfinite number. It's our hyperfinite unit. This is exactly like picking how long a meter is. We fix it ONCE AND FOR ALL TIME. It carries no information other than its name and level of size. We may set it to a convenient value, such as the product of all (standard) natural numbers. Now we have a bit more info about ω: it's an integer and any standard number can divide it (handy for picking mesh sizes).

Example: Let's take the grain of sand to be the point 2ω+1, on the right. I picked an infinite point to illustrate how these really are just ordinary numbers. Meditate on the phrase radically elementary. The world is the pink line around it, a 1D microvector space. The sand cannot be discerned in any finitary way from an infinitesimal displacement (sum/difference) of it.

Note that the microvector space really is a vector space. infinitesimal + infinitesimal = infinitesimal. standard x infinitesimal = standard * 1 / infinity = standard / infinity = infinitesimal. The rest of the definition you should verify, dear reader in Christ. If you grok the Transfer Principle, this exercise is trivial.

POLL: microvector vs tiny vector as terminology?

Aside: For a radically elementary Feynman path integral, take the universe and put it in a hyperfinite 4D cube of edge length ω. Then its 4D measure is ω4. Whether or not the universe is infinite, we can always do this embedding. Now cut it into 4D cubes of edge length 1ω∼0...
A hundredth of a bit of extra entropy
by Adam Scherlis — 2022-12-24T21:12:41.517Z

There are two ways to calculate the amount of information in one term of a continued fraction:

The entropy of the Gauss-Kuzmin distribution is about 3.4325 bits.
Twice the logarithm of the Khinchin-Lévy constant is about 3.4237 bits.

These differ by about 0.0088 bits. It took me a while to figure out why they were different at all, and now I'm surprised by how close they are.

Continued fractions

π is about 3.

Actually, 3 is a bit too small, by about 1/7.

Actually, 7 is a bit too small, by about 1/15.

Actually, 15 is a bit a lot too small, by almost 1/1.

But the 1 in the denominator is a tiny bit too small, by about 1/292.

We can keep doing this forever, and get an expression like:

π=3+17+115+11+1292+⋱

This is pretty useful for things like finding rational approximations. We can drop that 1/(292+...), since it's really small, and get the excellent approximation π≈355113. Or we can drop the 1/(15+...) and get the rougher approximation π≈227.

In general, the continued fraction for a real number r is of the form

r=a0+1a1+1a2+1a3+⋱

where the terms an are positive integers (except a0 which is an arbitrary integer).

This is infinite for any irrational number; for a rational number, it stops after some number of terms, e.g.

355113=3+17+115+11

Truncating a continued fraction[1] is the optimal way to approximate a real number with a rational: it gives a better approximation than anything with a smaller denominator.

Continued fractions show up every now and then in different ways, but the approximation property is usually how I end up bumping into them.

The Gauss-Kuzmin distribution

Suppose we choose a random real number (say, uniformly between 0 and 1). The probability that the nth term is equal to k converges to p(k)=−log2(1−1(1+k)2) as n→∞.[2] This is the Gauss-Kuzmin distribution.

I think it's also true that the fraction of k's in the continued fraction converges to p(k).[3]

The entropy of this distribution is just ∑∞k=1−p(k)log2(p(k)),[4] which is pretty quick to compute out to a few digits of accuracy in your favorite programming language. It works out to about 3.4325 bits, so that's how much information there is in the term an, for large n.

Lochs's theorem

Lochs's theorem: For almost all real numbers, asymptotically, the accuracy of the continued fraction improves by a...
Shared reality: a key driver of human behavior
by kdbscott — 2022-12-24T19:35:51.126Z

I once asked Robin Hanson if he really thought status-seeking was such a dominant driver of human behavior. I said humans had dozens of factors motivating their behavior, it was crazy to claim there was One Big Thing. He replied (something to the effect of) "well, even if each factor has a small effect – five percent, six percent – one of them has to be the biggest."

There's a concept I refer to as 'shared reality' that I think is up there with 'status' as something humans seek, shaping a lot (maybe ten percent?) of our behavior.

Knowing and playing with the concept of shared reality has noticeably improved my relationships and given me more surface area on many social dynamics (e.g. connection, attachment, idle banter, tribalism).

What is shared reality?

There is a Columbia research lab that studies "shared reality"[1] among other topics. They define it as:

the perceived commonality of inner states with others

By example: If two people chuckle at a pun, and then see each other chuckling such that they both perceive that the other had a similar pun-chuckle-experience, they are in a state of shared reality (at least, for that pun-chuckle-experience in particular).[2]

Shared reality is at play when people

go to sports events / concerts / movies together.

That emotional rush when everyone cheers / sings / laughs at the same time.

travel / eat / dance together.

That moment when you both see / eat / intuit something cool, look at each other and see you've had a similar experience, and get a rush of connection[3].
Or that moment when you have an experience and suddenly want to share it with others, so you offer 'try this!' or 'look at that!' or take a picture and share it.

Shared reality is nice

The research literature mentions that people are 'motivated to create shared realities' but doesn't really discuss why. I think people want it because it's pleasant[4]. And conversely, that the opposite of shared reality (disconnected reality?) is unpleasant, and something people avoid.

This strikes me as an important part, because put together with the above definition it makes for a model that better predicts how people will behave, and what is sometimes causing people to feel better/worse.[5]

Here's a stab at an updated definition:

people seek a perceived commonality of inner states with others (because it's pleasant). People try to maintain that perceived commonality and/or avoid

...
Contra Steiner on Too Many Natural Abstractions
by DragonGod — 2022-12-24T17:42:53.828Z

Reply To: "Take 4: One problem with natural abstractions is there's too many of them."

Epistemic Status

Unconfident. I don't yet have a firm philosophical grounding on abstractions.

Unequally Made Abstractions

Not all natural abstractions are made equal. It seems to me that there are different degrees of "naturalness" of an abstraction. When we say a concept is a natural abstraction, we are saying things like:

The concept is highly privileged by the inductive biases of most learning algorithms that can efficiently learn our universe[1]

The more privileged the concept is in aggregate[2], the more natural the abstraction is

Learning algorithms given weight inversely proportional to how efficiently they learn our universe

The more learning algorithms privilege the concept, the more natural an abstraction it is

Most efficient representations of our universe contain a simple embedding of the subset

[The simpler the embeddings of the concept are]/[the easier it is to point to embeddings of the concept] in aggregate[3], the more natural the abstraction is

It follows that the most natural abstractions for a concept (cluster) are the abstractions that we'd expect AI systems to actually use/the abstractions that are most relevant for their decision making.

What's The Most Natural Abstraction for Human Values?

Consider the subset of "human values" that we'd be "happy" (were we fully informed) for powerful systems to optimise for.

[Weaker version: "the subset of human values that it is existentially safe for powerful systems to optimise for".]

Let's call this subset "ideal values".

I'd guess that the "most natural" abstraction of values isn't "ideal values" themselves but something like "the minimal latents of ideal values".

Conclusions

I don't think the multitude of abstractions of human values is necessarily as big a stumbling block as Steiner posited.

^

"Learning our universe" means learning a map/world model of our universe that allows effectively predicting future events.

The constraint of efficiency suggests that the learned world model (and inferences via it) should have low data, time, and space complexity (relative to what is attainable for optimal learning algorithms).

^

When aggregating across learning algorithms we might want to give algorithms weights that are inversely proportional to how efficient they are in learning our universe.

^

When aggregati

...
Three reasons to cooperate
by paulfchristiano — 2022-12-24T17:40:01.114Z

In this post I’ll discuss three reasons to cooperate in a truly one-shot prisoner’s dilemma:

Kindness: You care about the other player.

Correlation. Your decision is correlated with the other player’s decision.

Reciprocity. Your decision is correlated with the other player’s belief about your decision.

Kindness makes common sense, but correlation and reciprocity are often lumped together under “weird acausal stuff.” I think they are worth distinguishing because they work quite differently.

I’ll talk about details of correlation and reciprocity, and then argue that most incentives to cooperate in the true prisoner’s are likely to come from interactions of multiple mechanisms.

Setup: the true prisoner’s dilemma

Imagine that one day our civilization sees another through a wormhole—we’ve never met before, and we are far enough away from each other that after today we will never meet again. From each civilization, a single person is chosen to play a game.

The two players then play a one-shot prisoner’s dilemma with survival as the stakes. Each player chooses to cooperate in which case they die with probability 1%, or defect in which case the other play dies with probability 2%. If both players cooperate they each die with 1% chance, if both defect they each die with 2% chance.

No matter what the other player does, you have a lower probability of death if you defect. In this post we analyze reasons that you might cooperate anyway.

I will not discuss the incentives to make commitments in advance based on the possibility that we may play a prisoner’s dilemma in the future (even though I think those incentives are important). I will analyze the question mostly for an agent who uses EDT or UDT.

You could just as well replace the ratio 2%/1% with any X>1, and ask “how large does X need to be before I’d cooperate?” I think it is pretty robust that you should cooperate at some sufficiently large scale (unless you specifically want the other player to die) and so this is ultimately a quantitative question. For this post I’ll just keep X=2 and leave it up to the reader to generalize.

Kindness

The most boring reason to cooperate is that I don’t want the other player to die.

That might be because I value their life and experiences for its own sake, in the same way that I value mine. Or it could be that I mostly care about other consequences of survival...
Practical AI risk I: Watching large compute
by Gustavo Ramires — 2022-12-24T13:25:25.848Z

This post is going to lay out a practical and simple strategy to diminish risk of AI catastrophe.

I have voiced my uncertainty (but certainly not dismissal) about AI risk several times in the community, which I will briefly summarize:

(1) Technology has likely physical limitations (some theoretical limits are far off, but we don't know if we'll make much more progress in process technology);

(2) Intelligence isn't magical: there are ways to constrain systems that have many sigmas of reliability, even under worst case adversaries. One example I've given is running a computer inside a Faraday cage with self-contained power.

(3) Thousands of beings more intelligent than any AI today are improving AI at a fast but not incomprehensible velocity. A human-level AI (or maybe even 100s of them) could not self-improve faster than it currently improves.

(4) Taking over the world takes some time. It will be visible if an AI tries to take over the world: maybe it will be smart enough to divide us (e.g. linking it to conspiratorial thinking, an ideology or a political party), but if we're monitoring things at a minimum level, it would be clear there's an entity accruing resources, forms of power, and computation.

(5) Ethics is in a way rational, logical and "natural" (in a formal an general sense), systems learning from human bodies of knowledge should be able to pick up ethical reasoning, and designing truth-seeking and ethics objectives should not be impossible. (Sidenote: I propose we formalize ethics for this goal)

The point (4) is what we're going to exploit in this post. (hopefully more on (5) in another post)

We need one or several agencies or bodies watching over large compute resources (with internal access to their infrastructure) checking for abnormal activity, in particular large activities linked to a single network or entity. This is just regular network engineering, you can look at traffic statistics, sever rollout statistics, just check who has the most servers keeping an eye out for 3rd party delegation (where someone buys compute in the name of another). We need laws that allow such representatives/engineers to peer into the system in privacy-preserving way (i.e. with help from the company engineers) to check there's nothing funny going on. Of course AI companies should be monitored specially for autonomous agents, as well as any large decision-making AI.

T...
Non-Elevated Air Purifiers
by jefftk — 2022-12-24T12:40:01.244Z

Last year for Christmas gatherings at my dad's house I put together

wall-mounted filter cubes,
to get them up and out of the way. This time around this seemed like
too much hassle, so I put one together on top of the piano, one in a
bookcase, and used one commercial purifier:

It's double-ended because I had an extra fan. You do need to remember
to turn on both fans. Probably would be slightly more effective if
I'd put a partition in the middle to make it actually be two
independent cubes. I used the 12x20 filters from my ceiling fan air purifier project.

The kitchen has a commercial purifier, which has a similar capacity to
the filter cubes while taking less space. If I wanted to run these
all the time I might buy several commercial purifiers, but since these
are just for occasional use that seems less worth it.

For the living room, Julia had
the clever idea of using a part of the built-in shelving that was
exactly the right size:

This is only two filters worth, but I still feel a lot of airflow from
the fans. It's a good location for not being in the way, though we
did need to talk to the kids about leaving it alone since it would be
easily damaged if bumped.

I should make some shrouds for the fans.

Comment via: facebook, mastodon
The Case for Chip-Backed Dollars
by AnthonyRepetto — 2022-12-24T10:28:25.554Z

~ NOT crypto-dollars; actual computer chips = dollars ~

TL;DR — A currency is ‘backed’ by the thing you get for it when you hand-in your dollar to the bank. It used to be that you could go to the bank with your paper dollar, which was really just a *receipt* for your REAL dollar — a SILVER coin at the bank. You could hand-in the paper, and get the real silver dollar back. That is a ‘silver-backed’ currency; same for gold. Currently, the dollar is backed by NOTHING, which is bad for us. Instead, we could let each paper dollar be the receipt for a computer chip held at a Server Bank. (You could walk out of that bank with a computer chip in your pocket, though that would never actually be valuable to do.) The chip is real, physical capital, and it runs the internet, providing services, making real money, and you get paid a DIVIDEND; holding dollars would be better than German bonds. That’d pull Trillions of dollars into the US currency market, subsidizing our own chip-sector and A.I. dominance. Sounds good?

Death of the Petro-Dollar

We got rid of gold as the backing for our currency half a century ago, and it hasn’t really hurt us much for two simple reasons: 1) Rich Americans still have to pay TAXES using US dollars, creating a steady demand for them; 2) Dollars are the commanded currency to buy and sell OIL, which is a huge market, sopping-up those spare dollars. Yet, oil will be dwindling over the next decades, as climate-responsive energy sources arise and supplant it. The US will need something else to prop-up its currency, or a slow crisis may unfold.

I offer the chip-dollar. Let’s walk-through the three steps of how that protocol would work:

An approved Server Bank lets the US Federal Government know that it can support additional chip-capacity on its computer-farm. It offers $X to the government, in return for ‘stamped’ chips.
The US Government issues an auction to approved chip-fabricators, offering to buy that same $X worth of chips at the standard metric performance. A mix of bid prices and volumes come-in; some fabs get to send their new chips to the government, and those fabs get paid in NEW, CHIP-DOLLARS. That dollar is a receipt for a chip; the chip is ‘stamped’ by the government and sent to the Server Bank — details below. The dollar keeps track of when it was made, to properly attribute the right amount of dividend to you when you return it to one of

...