trentmkelly/LessWrong-43k huggingface dataset:
Getting from an unaligned AGI to an aligned AGI? 
by Tor Økland Barstad — 2022-06-21T12:36:13.928Z

Summary / Preamble

In

AGI Ruin: A List of Lethalities

, Eliezer writes “A cognitive system with sufficiently high cognitive powers, given any medium-bandwidth channel of causal influence, will not find it difficult to bootstrap to overpowering capabilities independent of human infrastructure.”
I have larger error-bars than Eliezer on some AI-safety-related beliefs, but I share many of his concerns (thanks in large part to being influenced by his writings).
In this series I will try to explore if we might:

Start out with a superintelligent AGI that may be unaligned (but seems superficially aligned)
Only use the AGI in ways where it's channels of causal influence are minimized (and where great steps are taken to make it hard for the AGI to hack itself out of the "box" it's in)
Work quickly but step-by-step towards a AGI-system that probably is aligned, enabling us to use it in more and more extensive ways (as we get more assurances that it's aligned)

From the AGI-system we may (directly or indirectly) obtain programs that are interpretable and verifiable. These specialized programs could give us new capabilities, and we may trust these capabilities to be aligned and safe (even if we don't trust the AGI to be so). We may use these capabilities to help us with verification, widening the scope of programs we are able to verify (and maybe helping us to make the AGI-system safer to interact with). This could perhaps be a positive feedback-loop of sorts, where we get more and more aligned capabilities, and the AGI-system becomes safer and safer to interact with.

The reasons for exploring these kinds of strategies are two-fold:

Maybe we wont solve alignment prior to getting superintelligence (even though it would be better if we did!)
Even if we think we have solved alignment prior to superintelligence, some of the techniques and strategies outlined here could be encouraged as best practice, so that we get additional layers of alignment-assurance.

The strategy as a whole involves many iterative and contingency-dependent steps working together. I don't claim to have a 100% watertight and crystalized plan that would get us from A to B. Maybe some readers could be inspired to build upon some of the ideas or analyze them more comprehensively.

Are any of the ideas in this series new? See here for a discussion of that.

Me: I have some ideas about how how to ma...
Common but neglected risk factors that may let you get Paxlovid
by DirectedEvolution — 2022-06-21T07:34:02.685Z

The CDC lists the conditions that make a person "more likely to get very sick from COVID-19." Currently, Paxlovid supply is abundant, but available only to people with at least one of these risk factors.

Paxlovid access may be crucial long-term, as the rate at which vaccines can be updated using conventional clinical trials seems to be too slow to keep up with the rate at which viral variants evolve.

I am also drawing from Interim DOH Guidance for Use of Paxlovid, from May 2022, which is more recently updated than the CDC's list but less prominently featured on government websites for healthcare providers.

On the FDA's website (4 May 2022):

Patients do not have to have more than one risk factor to be considered “high risk”.

My guess is that many people who have one or more of these risk factors don't realize it, and neither do their medical providers. This may be because they're not informed about what the risk factors are, because they don't think to check, because that "risk factor" has never before manifested as a clinical problem, or because it's not the sort of thing that intuitively seems like a "risk factor" for a respiratory infection.

I currently have a friend who is very sick with COVID-19, has several of the below risk factors, and was not prescribed any medication by her doctor. She is poor, alone in her house, suffering, when a widely available treatment is available that could have helped her. I hope this list helps others avoid this outcome.

I'm going to go through the list of risk factors they publish and highlight the ones I suspect get forgotten relatively often. Remember that there are many more obvious risk factors on the above CDC list (things like being over 65, having cancer, asthma, heart disease, and many others).

Being from a racial or ethnic minority group: American Indian/Alaska Native (non-hispanic), Asian (non-hispanic), Black (non-hispanic), Hispanic/Latino, Multiracial (non-hispanic), Native Hawaiian or other Pacific Islander
Mobility, sensory, or intellectual disabilities.

One reason given for considering disabilities a risk factor for COVID-19 is that many people with disabilities live in shared or congregate housing. It is not clear to me whether living in shared/group housing for other reasons can also be considered a risk factor worthy of a Paxlovid prescription, but it may be worth checking if no other risk factor applies to

...
Dagger of Detect Evil
by lsusr — 2022-06-21T06:23:01.264Z

A philosopher walked into a magic shop.

"Ugh," said Phil. The philosopher leaned his spear against the hatstand and slumped onto a fluffy cushioned chair.

"Can I get you something to drink? How about Essence of Essence?" said Wiz, the shop owner. She handed Phil a bottle of perfectly invisible liquid. It was clearer than air.

Phil downed the bottle in a single chug.

"I don't know why you do this to yourself. Dungeon crawling. You should be debating the nature of reality. Not scrounging for trinkets in a dark tunnel," said Wiz.

"I would if I could but for some reason people pay more for ancient artifacts of incredible power than for lessons on epistemics," said Phil, "Except for the Court Philosopher. He rakes in the dough. But the rest of us? Not so much."

"Strange," said Wiz.

"Would you like to pay for a lesson on epistemics?" said Phil.

"No thanks," said Wiz.

"Not so strange then," said Phil. Phil took a deep breath. Wiz's shop was passively safe in the sense that nothing would kill you if you didn't poke it first. Which was a step up from Phil's previous week.

"Well," said Wiz.

"Well what?" said Phil.

"Aren't you going to ask me what new inventions I have for you?" said Wiz.

"Oh yeah!" said Phil. He jumped out of the chair. Wiz's inventions always had fascinating metaphysical implications.

The door to the giant room-sized safe/storeroom in the back of Wiz's shop had no keyhole. Wiz just touched it with her hand and the safe opened. The safe was full of boxes and crates and barrels and magical creatures. An imp screeched and banged on the bars of its cage. Various bladed weapons were hung from the ceiling. Wiz removed a haladie dagger and left the storeroom. The door closed behind her.

"This," said Wiz, "is my Dagger of Detect Evil."

Phil was too stunned to say a word.

Wiz smiled the grin of a satisfied engineer.

"It's a what?" said Phil.

Wiz's smile faltered slightly. "It's a Dagger of Detect Evil. You stab an enemy with it and then the dagger will measure whether the enemy was evil. If the enemy was evil then the dagger glows red. Otherwise the Dagger does not glow."

"Let me make sure I heard you correctly," said Phil, "I stab an enemy with it. After I have stabbed an enemy, the dagger will tell me if the enemy was evil."

"Yes," said Wiz.

"That makes no sense!" said Phil.

"Why not?" said Wiz.

"Because evil is not a material phenomenon," said Phil.

"So? I deal with immaterial phe...
What is the most probable AI?
by Zeruel017 — 2022-06-20T23:26:01.467Z

Hi, then I have a difficult question to ask. Let's imagine that in the distant future of our universe, through the arrangement of atoms that move in empty space, various objects are formed, any object can be formed, the important thing is that it does not violate the laws of physics (to know in detail , see my previous post: https: //www.lesswrong.com/posts/LGMSLXkpKAofebjfi/a-terrifying-variant-of-boltzmann-s-brains-problem). Obviously, objects with less mass and less complex will form more often, this is because they are random arrangements. Now, this possibility opens up a dangerous scenario, since if any object can be formed, a human brain can also be formed, for example, and since despite the small probability that a brain will aggregate, having a lot of time available ( like 10 to 10 to 105, and our universe is "only" 10^10 years old, the amount of time is frighteningly large), nothing prevents this from happening and it happens many times, more than all humans who have lived on earth , one might wonder if I am such a brain, born eons after the birth of the universe, experiencing false sensations, this problem is called boltzmann brain (in detail in my previous post: https://www.lesswrong.com/ posts/LGMSLXkpKAofebjfi/a-terrifying-variant-of-boltzmann-s-brains-problem). The problem, however, seems easy enough to overcome, a brain born at random would also most likely have random, incoherent memories (some are, but most are not, and since the world we perceive is coherent, the a priori hypothesis can be rejected. since it leads to a logical contradiction) and also would not survive long in empty space. The problem is that just as brains can be formed, computers can be formed, for example, which could in some way form human brains and make them believe they are living in a world that is actually simulated. Small digression, one way to refute the boltzmann brains hypothesis a priori is to show that the most likely scenario in which a brain will be found in the distant future is inconsistent with my (and I hope our) observations.

Going back, if a brain alone is very unlikely, more than a computer of the same size in my opinion, since I believe that in order to function, even for a very short time, it must have cells with the same DNA, and it is already very unlikely. That human cells of a brain are formed at random, if they must have the same DNA, it be...
Evaluating a Corsi-Rosenthal Filter Cube
by jefftk — 2022-06-20T19:40:01.980Z

When I wrote about

testing my ceiling air purifier
prototype, two common questions were:

Where is your control group?

How does this compare to a Corsi-Rosenthal
filter cube?

Several rounds of testing and one worn out particle meter later, I
have answers for you! First, the bottom line:

Purifier
CADR (CFM)
Cost

Corsi-Rosenthal
on High
246
$55

Coway Mighty
AP-1512HH on High
219
$172

Corsi-Rosenthal
on Low
191
$55

Ceiling Fan Air Purifier on High
180
$115

Coway Mighty
AP-1512HH on Medium
105
$172

Control
9
$0

Methodology is in "Testing Air
Purifiers".

The Corsi-Rosenthal box performs very well, and the control setting
has a low enough decrease that ignoring it doesn't affect the analysis
much.

This specific Corsi-Rosenthal box is a Lasko fan with four "FPR 10"
filters that I bought several years ago. This is a Home Depot rating
system, and is approximately equivalent to the MERV-13 you'll see recommended for filter
cubes. This isn't a new filter: it ran in my office from when I
returned to in-person work in early December 2021 until I switched
jobs in mid-June 2022. The filters are decidedly grey now, and
I'd likely get better performance with new ones:

I did have a problem partway through my testing where my Temtop
M2000 suddenly started giving bogus readings:

In this test I first was measuring the pm2.5 and pm10 levels with
nothing running, and then I turned on the filter cube. You can see
levels dropped quickly, but then started to go back up.

Over the next few days it got even worse, with it often reporting
indoor levels in the 100-500 µg/m³ level even outdoors. I
returned the meter for a replacement, which has worked well so far.
It does make me nervous about the longevity of this device, however,
since it's relatively expensive for some thing that apparently doesn't
last very long?
Survey re AIS/LTism office in NYC
by RyanCarey — 2022-06-20T19:21:33.642Z

Currently, there are plans to set up an EA office in NYC, and alongside this, organisers including Claudia Shi are considering setting up an office more focused on AI safety/longtermism. If you are interested in one or both of these, you can express your interest here, to make it more likely to happen. More information is on the EA Forum, here.

(I'm not running these projects, and if you have questions about them, I suggest reaching out to the organisers.)
Is This Thing Sentient, Y/N?
by Thane Ruthenis — 2022-06-20T18:37:59.380Z

Builds up on: Consciousness and the Brain by Stanislas Dehaene. Good summaries may be found here or here, though reading them is not strictly necessary.

Synopsis: I claim to describe the exact mental structure that allows qualia.

Background: What Is Consciousness?

Dehaene's Consciousness and the Brain rigorously differentiates conscious and unconscious activity. Consciousness, the book suggests, is correlated with events where the brain gathers all of its probability distributions about the world, and samples from them to build a consistent unitary world-model, on which it then acts. The experiments show that this is necessary for multi-step calculation, abstract thinking, and reasoning over agglomerations of distant sensory inputs.

However, the book's definition of consciousness is not a synonym for self-awareness. Rather, I would term the phenomenon it picks out as "moments of agency": as the state in which the mind can devise goal-oriented plans using a well-formed world-model, and engage in proper consequenialist reasoning. Outside those moments, it's just a bundle of automatic heuristics.

Self-awareness, I suspect, is part of these moments-of-agency in humans, but isn't the same thing generally. Just having Dehaene!consciousness isn't a sufficient condition for self-awareness: there's something on top of that going on.

What Is Self-Awareness?

What do we expect to happen to an agent the moment it attains self-awareness, in the sense of perceiving itself to have qualia?

Why, it would start perceiving qualia — the keyword being perceive. It would start acting like it receives some sort of feedback from a novel sense, not unlike sight. Getting data about how it's like to be a thing like itself.

Let's suppose that it's nothing magical — that there isn't a species of etheric parasites which attach themselves to any sufficiently advanced engine of cognition and start making it hallucinate. Neither are qualia "emergent" — as if, if you formally wrote out an algorithm for general reasoning, that algorithm would spontaneously rewrite itself to be having these imaginary experiences. If self-awareness is as mundane as any other sense, then what internal mechanism would we expect to correspond to it?

When we "see", what happens is: some sort of ground-truth data enter a specialized sensory organ, that organ transmits the information to the brain, the brain parses it, and offe...
Steam
by abramdemski — 2022-06-20T17:38:58.548Z

"Steam" is one possible opposite of Slack. I sketch a speculative view of steam as a third 'cognitive currency' almost like probability and utility. This is a very informal and intuitive post, outlining concepts which could possibly have a nice formal correlate later.

This post came out of a discussion with Sahil Kulshrestha. Much of the idea is his, but this is my own take on it, especially the terminology.

Gaining Steam

When you first get an idea, it might have very little steam. It's hypothetical; exploratory. It doesn't suck up much slack.

Maybe you start idly generating plans based on your idea. It's still hypothetical, but it's starting to gain steam.

Then you notice yourself making plans. Maybe you start thinking of yourself as "making plans". At this point, the idea is starting to gain steam.

Then you might start acting on your idea. At this point, it probably has a fair amount of steam.

Full steam means you're going all-out. You have no time for anything else. All of your thoughts are directed at this one goal, at least for now. You don't have motivation problems or hesitations.

So, in an individual person, steam is something like energy/willpower/interest.

In a society, steam is something like political will. An idea outside the Overton window has very little steam -- it's hard for any political movement to work toward that idea. Ideas within the Overton window have varying amounts of steam based on the extent of their support.

Ideas like democracy, which are considered fundamental to our society, are running at very high steam.

Losing Steam

You're starting to lose steam if you are still working on something, but you're feeling a bit tired and you're not sure how much longer you're going to work on it.

An idea might lose a lot of steam if you learn about a negative consequence of that idea.

Losing steam can be an important aspect of getting things done. Losing steam helps agents solve the procrastination paradox. Roughly: if you expect yourself to do something for sure, then you can always put it off, so you might never get it done. If steam is a finite resource which you have to allocate carefully, you can't get away with procrastination like this.

Steam as Action Probability

The simplest model of "steam" is that it is the agent's own subjective credence that it will do something.

According to this, actions, policies, and ...
Parable: The Bomb that doesn't Explode
by Lone Pine — 2022-06-20T16:41:14.611Z

You're an engineer working for a military contractor. One day, your project manager comes to you and asks you to design a container for plastic explosive. It is to contain several kilograms of C4, enough to destroy a building. This is pretty dangerous, but you know that C4 is actually pretty safe. It can't be accidentally detonated by fire, impact or bullets. Only a detonator (another explosive) can trigger C4, so you don't include a detonator in your design, for safety.

Your PM reviews the design and gives you some feedback. "It would be a lot more useful if you put a blasting cap inside." You grimace. More useful, yes. But a lot less safe. Nevertheless, you do your job and install a blasting cap. The blasting cap has two electrical leads. If a voltage is applied across those leads, the C4 will explode, killing everyone around. To keep things safe, you snip off the the leads and put the blasting cap inside a pill-shaped plastic container, inside the larger container that contains the C4.

"Well, that's no good," your PM replies, "What if someone does want to put a voltage across the leads?" You grimace even more. It is getting very difficult to make this design safe, you think. But you have a solution: inside the pill-shaped plastic container, you install a Raspberry Pi. You install SELinux on the Pi and set it up to connect to WiFi -- but only secured networks. You program up a fancy web interface that allows the user to specify exactly what voltage they want to apply to the leads. The interface caps the voltage at a low level. It does not allow the user to apply a sufficient voltage to actually trigger the blasting cap -- or at least you hope that's how blasting caps work.

"Well, that's no good," your PM replies, "What if someone wants to apply a higher voltage?" You grimace again. Then it'll explode! It'll kill everyone! But this is not a workplace where you raise objections, so instead you just diligently do your job. If you don't make this dangerous thing, the company will hire someone else without your moral scruples, and certainly that person would make something really dangerous! So you alter the web interface, allowing the user to specify any voltage up to the limit of what the Pi can output. You add a big red warning screen, explaining how C4 is dangerous, and forcing the user to click "Yes, I'm really sure I want to apply this voltage." You also add ...
On corrigibility and its basin
by Donald Hobson — 2022-06-20T16:33:06.286Z

Post somewhat inspired by Eliezers "well why didn't anyone else write it then?"

Corrigibility has various slightly different definitions, but the general rough idea is of an AI that does what we want. An AI that doesn't trick us. An AI that doesn't try to break out of its box. An AI that will follow clear instructions if it is given them.

A slightly more technical definition is that an agent is corrigible if it is easy for other agents to optimize over it. This has two parts, predictability and controllability.

Get_Random_Bits is not predictable. Agents can't predict what it will do, so will have a hard job forming plans that use it. A pseudorandom algorithm would score better. Being predictable by AIXI is easy. Every small Turing machine manages that. Being predictable by a much more limited agent, such as a human, is a much stricter condition. A pseudorandom algorithm is predictable so long as it was simple enough to be predicted in practice by whichever agent is trying to do so. While Get_Random_Bits has a low predictability, it is not the worst possible predictability score. There are plenty of plans that will work with almost all strings of bits. Sometimes the relevant measure over bit-strings is far from uniform. You can confidently predict that Get_Random_Bits will not hack its way out and take over the internet. Optimize_Random_Utility_Function is worse.

Return_Zeros is a very predictable function. But it isn't controllable. You can't optimize its output by optimizing its input. Return_Input is a much more controllable function.

In a scenario where there is one input (controlled by the agent) and one output, Return_Input is a maximally controllable function, at least to an omniscient controller.

If an agent is logically omniscient, but not physically omniscient, they can't make Return_Input play chess, they don't know what moves the opponent will make.

If an agent isn't logically omniscient, they can't get Return_Input to output the trillionth digit of pi.

For both of these reasons, Run_Python_Script is more controllable, at least for non-omniscient agents that know python.

AIXI would find any Turing complete programming language equally controllable, so long as it knew what the language was, and AIXI had unlimited output bandwidth (for verbose languages) and there were no performance concerns about using slow languages.

The Basin

...
Announcing the DWATV Discord
by Zvi — 2022-06-20T15:50:03.051Z

I have decided this blog should at least experiment with having a discord server, so I made one. I kept it simple. There’s a few channels for the things the blog tends to be interested in – games, rationality, Covid – along with places to ask about suggesting future post topics, hiring me to do consulting, share links I might want to include in future posts or suggesting we meet up for lunch if you’re going to be in Manhattan.

There’s also a room called Zvi-asks-questions, where people who want to be helpful can help find answers to stuff, either for the blog or otherwise. As I move into new areas, especially to the extent I start looking at politics and policy, that’s going to become more valuable. Plan is I’ll experiment with questions both personal and blog related and see what happens, everyone’s always free to ignore.

That’s the core philosophy here, try stuff and see what happens. I figure the worst realistic case is it turns out all this is a bad idea, wastes a bit of time and turns into a ghost town, in which case not much is lost, or it could turn out really well. Worth a shot.

The roles are friend for people I know, subscriber for subscribers to Substack (Patreon at $10/month or more also counts), and collaborator for those I am working with. There’s general channels for them but mostly I figure people should get to use cool colors cause why not. If you want a role, you can message me to request it, including ID/email as appropriate so I know who you are.

The invite is here.