trentmkelly/LessWrong-43k huggingface dataset:
April 15, 2040
by Nisan — 2021-05-04T21:18:08.912Z

It's time to pay my taxes. In past years my AI assistant found clever ways to reduce my tax bill. I ask it, "What does my tax return look like this year?"

"Not good, I'm afraid. We may not be able to do any tax evasion this year."

"Why not?"

"The tax authority has determined that it can't keep up with AI-assisted tax fraud, even with the help of AI auditors. So it wants taxpayers to voluntarily agree not to do tax fraud. In return it agrees not to prosecute past instances of tax fraud. Also Congress agrees to keep tax rates reasonable. The agreement goes into effect if 90% of the taxpayers in each tax bracket sign it. It's a good deal for you. Shall I sign it on your behalf?"

"Hold on, I don't see why I should sign this."

"If the deal falls through, the government will run out of revenue and collapse."

"They don't need my signature, though. You said they only need 90% of taxpayers to sign?"

"Yes, only 90% of taxpayers in your bracket. I predict we'll get very close to that 90% threshold, so it's likely your signature will make all the difference."

"So 10% of taxpayers won't sign. Why can't I be one of those?"

"I will try to shape the negotiations so that you end up in the 10% of nonsigners. But you must understand that since only 10% of your bracket can be in that group, your odds of success are only 10%."

"But you're a stronger assistant than most people in my tax bracket have. Doesn't that give you an edge in negotiation?"

"The other assistants and I are using a negotiation protocol in which smarter agents are on an equal footing with dumber agents. Of course, people with less capable assistants would never agree to a protocol that puts them at a disadvantage."

"How about we sign the agreement, then cheat on my taxes anyway?"

"In order to sign the agreement, I must make a commitment to never break it, not even if you order me to. My signature on the agreement will be an airtight proof of that commitment."

"Ok, how about you sign it, and then I get a different assistant to help me with my taxes?"

"That won't work because in order to sign the agreement, I must sign and attach a copy of your tax return for this year."

"Hm, will I actually be worse off if the government collapses?"

"You might end up better off or worse off, but overall the risks of a revolution outweigh the benefits. And keep in mind that the successor government, whatever it will be, will still have to collec...
Loud Voices as Accessible Anecdotes
by ACrackedPot — 2021-05-04T21:13:22.199Z

I've been pondering something for a little while, which is that the standard take on political takes might be slightly off; I hear a lot about how people tend to listen to the (metaphorically) loudest voices in the room, drowning out the sensible discussion that is actually taking place.

And the thing is, I have some experience with some of these things. Some of the takes that are attributed to "loud voices" I have encountered over the years are things that I heard, years before, in some of the places I'm supposed to go to for sensible discussion - which is to say, I encountered some of the more outrageous takes, that "everyone" agrees only a few crazy people believe, in the wild. Not just in the wild, but teaching classes, and requiring their crazy be repeated back to them. Absent somebody turning global attention to them - giving them the metaphorical loudspeaker - they were, and remain, quiet.

But they were certainly loud and relevant to me, because people in positions of authority, people who should know better, were using that authority to demand other people repeat their inane nonsense back to them.

The concept of social bubbles may enter into things here; we could describe "volume", as a metaphor, as being about voices which cut across bubbles; things you can hear through a bubble. And this can kind of work, if you squint just right. But maybe, when we say say "loud", what we actually mean is "accessible". Loud is just a kind of accessibility that seems to work really well for voices, but I think "accessible" may cut more to the heart of the matter, in illustrating why some voices are "louder" than others.

Representativeness

And maybe a lot of the rhetorical confusion, about the difference between loud voices, and representative voices, is a result of the fact that "loudness" is a direct result of bringing voices which are local to a global focus, as a more or less exact response to the rhetorical demand that people's problems be representative. If you had said to my college-age self that a particular ideological bullshit wasn't a major problem in universities, I'd have shone every light I could on the bullshit I was putting up with in college; to the extent of my ability, I'd be making the voice loud.

Which is to say, the demands for evidence of a particular problem I had (which may or may not be phrased as claims ...
[timeboxed exercise] write me your model of AI human-existential safety and the alignment problems in 15 minutes
by Quinn — 2021-05-04T19:10:28.755Z

Motivation

I had a 15 minute interview last night in which I was asked "why do you believe in xrisk, and what does AI have to do with it?" I thought it was too big a question for a 15 minute interview, but nevertheless dove in to my inside view from first principles. Since diving into one's inside view from first principles even outside of a fifteen minute interview is really hard, I did a bad job, mostly rambled and babbled.

A broader motivation is that I'm interested in studying peoples' inside views / gears-level models as to hone my own.

Rules

In this exercise, you're allowed premises just try to point at them. It is not a "from first principles" sort of exercise. You're also allowed jargon without being too worried about how well the audience knows jargon (for example, in mine which I'll paste below I assume familiarity with the single & multi quadrants from ARCHES).

The only real rule is to limit yourself to 15 minutes. That's fifteen minutes wall time, with a literal clock.

Suggestion: don't read until you write!
Mundane solutions to exotic problems
by paulfchristiano — 2021-05-04T18:20:05.331Z

I’m looking for alignment techniques that are indefinitely scalable and that work in any situation we can dream up. That means I spend time thinking about “exotic” problems — like AI systems reasoning about their own training process or about humanity’s far future.

Yet I’m very optimistic about finding practical alignment strategies that are relatively simple and usable today. I expect the results of my research to look mundane and even boring.

These two stances may appear to be in tension. If I’m worrying about all kinds of exotic considerations, how can the end result be something mundane?

I’m not too concerned. These failures seem exotic because they involve exotic capabilities. I’m looking for a mundane algorithm that trains the model to use whatever capabilities it has for good purposes. So it will lead to exotic countermeasures if and when the model has exotic capabilities, i.e. just in time to avoid exotic failures (by which time they’ll no longer feel exotic).

I want to illustrate this phenomenon with a few examples.

Example 1: Gradient hacking

Gradient hacking is a possible failure mode where a powerful misaligned model behaves in a way that is apparently benign but biases the gradient of the loss to point in an unfavorable direction, eventually trapping optimization at a bad local minimum. This feels exotic today because it requires the model to have an incredible amount of introspective access and control, as well as strategic awareness over long time horizons. (“Forward Pass” by Andrej Karpathy explores some similar ideas.)

Normally I imagine using SGD to optimize a “behavioral” objective: to update the parameters θ you use them to make a randomized decision y on an input x, then you calculate a reward r(x, y), then you compute an unbiased estimate for the gradient of the expectation r(x, y) with respect to θ.

Gradient hacking is troubling because it suggests that if we optimize a purely behavioral objective we could get a bad outcome even if the reward function r(x, y) perfectly captures the quality of the output y.

Rather than fixing this problem in a specific way, I’d like to rely on competitiveness. That is, assume that (throughout training) we have a question-answering policy Q → A which reflects everything that our learned model “knows” about the world.

Then it seems like the best way to compute the system’s loss should be by by asking some question ...
Parsing Abram on Gradations of Inner Alignment Obstacles
by Alex Flint — 2021-05-04T17:44:16.858Z

This is independent research. To make further posts like this possible, please consider supporting me.

Epistemic status: This is my highly imperfect understanding of a lot of detailed thinking by Abram, Evan, John, and others.

Outline

This is a brief post summarizing my learnings from reading Abram’s Gradations of Inner Alignment Obstacles

The first section is about mesa-optimizers in general

The second section is about mesa-searchers in comparison to mesa-controllers

The third section is about compression, explicit goals, consistency, and power

The fourth section is about whether GPT-3 is deceptive and the connection to Ajeya Cotra’s recent proposal.

The fifth section is about the lottery ticket hypothesis

Mesa-optimizers

First, a quick bit of background on the inner optimizer paper.

When we search over a large space of algorithms for one that performs well on a task, we may find one that, if deployed, would not just perform well on the task but would also exert unexpected influence over the world we live in. For example, while designing a robot vacuum we might search over policies that process observations from the robot’s sensors and control the motor outputs, evaluating each according to how effectively it vacuums the floor. But a sufficiently exhaustive search over a sufficient expressive hypothesis space could turn up a policy that internally builds up an understanding of the environment being vacuumed and its human inhabitants, and chooses the actions that it expects to accomplish an objective. The question is then: is this internal goal actually the goal of cleaning the floor? And more broadly: what kind of capacity does this robot vacuum have to influence the future? Will it, for example, build a sufficiently sophisticated model of houses and humans that it discovers that it can achieve its objective by subtly dissuading humans from bringing messy food items into carpeted areas at all? If so, then by searching over policies to control the motor outputs of our robot vacuum we have accidentally turned up a policy that exerts a much more general level of influence over the future than we anticipated, and we might be concerned about the direction that this method of designing robot vacuums is heading in.

So we would like to avoid unwittingly deploying mesa-optimizers into the world. We could do that either by developing a detector for mesa-o...
[Linkpost] If You Can Be Bad, You Can Also Be Good
by MaxG — 2021-05-04T06:26:28.037Z
Link: https://astralcodexten.substack.com/p/if-you-can-be-bad-you-can-also-be

As someone who is rather new to the aspiring rationalist community, I enjoyed this post by Scott Alexander. It makes me think that becoming more rational is actually feasible, something I have wondered about a lot.
Why I Work on Ads
by jefftk — 2021-05-03T21:00:05.618Z

"I work on ads at Google"

"Can I ask why? I honestly can't understand how anyone could."

Someone recently

asked me why
I work on ads, and I wanted to write up something more thorough than

my
comment. (Despite being a work topic this is a personal post and
I'm speaking only for myself.)

One answer is that I'm earning to give: I give half of what I earn to the most effective
charities I can find, and the more I earn the more I can give. This is not the full answer, however,
since when people ask me this
they're generally coming from a perspective of viewing ads (or perhaps
online ads) as negative, and the question is more like "why do you
choose to work on something bad?"

The thing is, I think advertising is positive, and I think my
individual contribution is positive. I'm open to being convinced on
this: if I'm causing harm through my work I would like to know about
it.

So: why is advertising good? I mean, isn't it annoying when sites
show you ads instead of whatever it is you want to read? The question
is, what is the alternative? I see two main funding models:

Paywalls. You pay with your money.

Ads. You pay with your attention.

It's also possible to fund projects through donations, or as hobbies,
but producing most of what there is to read requires more money.

(I'm using the internet-specific term 'paywall' to refer to the
general "pay money for access" concept: buying books, paying
admission, subscribing to streaming services, etc.)

Both paywalls and ads have a range of advantages and disadvantages.
Some of these vary by medium: books are expensive enough to print that
they couldn't be funded by advertising; an analog radio receiver is simple enough
that a paywall would require draconian legal force. On the internet,
however, I think ads are generally a better fit for two reasons:

Minimal friction. You can follow links from site to site,
without barriers. You don't have to decide which sites to subscribe
to. If someone sends you a link to an article, you can read it.

Non-regressive. Paywalls, like other fixed costs, are
regressive: a newspaper at $220/y is effectively
much more expensive for someone earning $10k than $100k.

You can sort of fix friction with bundling: you subscribe to a
streaming service then can watch (or listen to, or read) anything in
their collection. There are advantages to this...
Bayeswatch 1: Jewish Space Laser
by lsusr — 2021-05-03T20:15:44.106Z

A black car departed from Ben Gurion Airport. The driver, Molly Miriam, handed a clipboard to Vi, who rode shotgun.

"Goldberg Aerospace. They built the solar sailer. They went from concept to manufacture in six months. No human engineering team could do that. They probably an have an AGI," said Vi.

"They do have an AGI. It is registered. They are following all standard safety protocols and then some," said Miriam.

"This is a routine inspection then," said Vi.

"Had you hoped for a more exciting first mission?" said Miriam.

"A spaceship factory is exciting," said Vi.

Goldberg Aerospace's Mission Control Center was modeled after Johnson Space Center's Mission Control Center.

"Call me Eitan," said the Chief Engineer. They shook hands.

"Miriam,"

"Vi."

"Did we make a mistake on the paperwork?" said Eitan.

"No, no. Your paperwork's fine. We just like to confirm things on the ground," said Miriam.

"Then let me give you the tour," said Eitan, "HAL, please bring us the VIP car."

"HAL?" said Vi.

"This is a spaceship company. Everyone who works here is a nerd. Don't worry. HAL is smoke and mirrors. Our real AI is contained," said Eitan.

The VIP car had leather seats under a hemispherical glass dome. It took them through kilometer after kilometer of autofactories.

"Everyone says their AI is contained," said Miriam.

"Ours really is," said Eitan, "We wrote it functionally."

"Functionally?" said Vi.

"I mean it was written in a functional paradigm. Our code is composed of functions. Every function returns an output value dependent on its input parameters and nothing else. Everything is immutable. The AI has no concept of time. Without time there is no causation. Without causation there can be no agency. It's just a calculator. We set the parameters of a space mission and our AI returns a schematic. All it understands is physics, engineering and orbital mechanics. We even use Lagrangian mechanics just to keep everything tidy," said Eitan.

"What about resources and manufacturing?" said Miriam.

"A domain-specific computer runs this factory," Eitan gestured to the buildings and robots outside, "Whether the spaceship is assembled correctly is independent of the AI's reward function. The AI is airgapped from the outside world. We keep it in a Faraday cage for redundancy but the AI has never tried to influence the outside world. It has no concept of an outside world. It understands how the solar syst...
Consistencies as (meta-)preferences
by Stuart_Armstrong — 2021-05-03T15:10:50.841Z

Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)

Walt Whitman

It is good for our decision processes to be time-consistent, transitive, and independent.

Or as Steve Omohundro apparently put it:

If you prefer being in Berkeley to being in San Francisco; prefer being in San Jose to being in Berkeley; and prefer being in San Francisco to being in San Jose; then you're going to waste a lot of time on taxi rides.

But actually, if you live in Berkeley, work in San Francisco, and like to take weekends in San Jose, then going back and forth continuously makes perfect sense, even if it costs you time and money.

When I'm hungry, I want to eat; after lunch, not so much. Chocolate fondues have their time and place in my diet, as do vegetables. And nothing about that feels particularly inconsistent, even though my preferences are seemingly flipping all over the place as time goes on.

Choosing consistency

Of course, it is possible to have inconsistent eating preferences; diet-overeat-fast cycles, for instance. But more consistent eating behaviours look quite similar to this: indulging more in some circumstances, being stricter in others, and maybe adding the occasional fasting. There is no bright line dividing the inconsistent behaviour from the consistent one.

To resolve this, we can posit a mixture of "true" underlying preferences, such as hedonic enjoyment of eating, social connection, energy, health, weight, and so on, and see the fluctuations of behaviour as just instrumental changes for these stable underlying preferences, coupled with a dose of irrationality. Human preferences are very underdefined, so figuring out what the "true" preferences are is a tricky process.

To pick one of those example underlying preferences, suppose I say that "I desire a certain level of social interactions, on average, in a given week". There are three easy ways to categorise this desire:

Bias or error: this is an inconsistent preference: I either desire social interactions, or I don't. So this should be collapsed to "I want social interactions", "I want to avoid social interactions", or deleted.

True preference: there is nothing wrong with defining a preference over an average quantity of social interactions (or anything else, for that matter). We could define a variable X = "average amount of social interaction in the last week", and p

...
Alzheimer's, Huntington's and Mitochondria Part 3: Predictions and Retrospective
by J Bostock — 2021-05-03T14:47:23.365Z

Epistemic status: Big if true, I don't have much time now but I might try and write part of this up into a more formal scientific letter to a journal or something later. I am reasonably confident in my models here but I do not have much experience in the field and I've written this up over the past weekend instead of revising for my exams.

Predictions

Results from EET-A human trials (if they go ahead) will improve patient outcomes in AD (60%), conditioning on EET-A being an effective agonist of PGC-1α in humans (80%)

EET-A will show positive results in models of HD (40%) low as I think the mechanism of mHTT toxicity (binding to DNA to prevent PGC-1α production) means EET-A cannot act upstream of it and can only act on the same level.

EET-A will show temporary benefits as an anti-ageing therapy (70% as above) (more like 35%, this has clearly slipped past proofreading and various sanity checks on my part) and will work "better" that senolytics in that it will actually reverse ageing rather than needing to be taken at higher concentrations over time (40%) (20%? These both seem wildly overconfident with some afterthought).

Retrospective

I think this perspective on AD and HD is probably useful, and it's not one I've seen before. Scientists are awful at sharing high-level models of diseases as they do not generally involve novel research and are (I suspect) very difficult to get published in high-impact journals. The authors of the paper using EET-A to treat AD did not seem to know why their treatment works, whereas I built a model whereby EET-A should effectively treat AD before I even considered "testing" my theory by simply looking to see if anyone else had done the experiment (Having made the prediction first gives me extra confidence in my own models but I don't expect it to be a strong argument to anyone reading this).

The remarkable thing about this investigation was how little time it took me, about three days. I expect there are many more intelligent and experienced people than me who could be drawing up similar links and conclusions with a small amount of effort. Building models of diseases based on existing studies is a clear way to guide new research, if I was a researcher about to start a new clinical trial (which could take years of my life) then even a 1% increase in success rate ought to be more than worth a short amount of analysis. I suspect this ...
Alzheimer's, Huntington's and Mitochondria Part 2: Glucose Metabolism
by J Bostock — 2021-05-03T14:47:10.125Z

Epistemic status: Big if true, I don't have much time now but I might try and write part of this up into a more formal scientific letter to a journal or something later. I am reasonably confident that my models here are significant in some way but I do not have much experience in the field and I've written this up over the past weekend instead of revising for my exams.

This is a follow-up to my first post, which compared Tau and Aβ proteins with mHTT to assess whether rapidly turning-over proteins can cause diseases on a long timescale.

Introduction

In this post I will assess the various evidence we have for what the causal chain for AD might be, what might be "upstream" of what, and what we might draw as conclusions.

Evidence

Hyperphosphorylated Tau protein can both promote Tau aggregation, and be toxic. Hyperphosphorylation of Tau protein is increased by problems with glucose metabolism in mitochondria, which is an early indicator of AD. This is because OGclNAcylation of Tau depends on mitochondrial activity, and prevents Hyperphosphorylation. (1)(2)

Mitochondrial mutations can accumulate throughout life, and mitochondrial mutations are a hallmark of ageing.(3)

Individuals with certain Aβ mutations develop AD at very young ages, but Aβ plaques often accumulate long before the disease shows.(4)

PGC-1α is a protein involved with increasing levels of mitochondria. This is known as mitochondrial biogenesis. EET-A is a molecule which acts as an agonist for PGC-1α (in mouse models).(5)

mHTT acts as an antagonist for PGC-1α. Tau proteins have been implicated in HD.(6)(7)

EET-A reduces Aβ plaque formation in a mouse model of AD.(8)

My Current Model

Mitochondrial mutations can build up over time, particularly with ageing. In some individuals with various other problems (insulin insensitivity in the brain, genetic predisposition) this causes mitochondrial glucose metabolism in neurons to drop below a certain threshold. This leads to decreased OGlcNAcylation of Tau protein, which leads to hyperphosphorylation of Tau protein. This leads to Tau toxicity and accumulation.

I do not currently have a prediction of the mechanism by which this leads to Aβ plaque (and smaller soluble aggregate) formation but I believe the empirical evidence for this is strong. I also (without a mechanism) believe that Aβ plaques (and smaller soluble aggregates) have some feedback effect which furthe...