trentmkelly/LessWrong-43k huggingface dataset:
Truth + Reason = The True Religion?
by David Gross — 2021-09-17T22:14:31.565Z

William Wollaston’s The Religion of Nature Delineated (1722) helped to launch “The Age of Reason.” It anticipated Kant’s Categorical Imperative, described and defended what we would now call the libertarian non-aggression principle, and provided the authors of the Declaration of Independence with their “purſuit of happineſs”.

William Wollaston’s The Religion of Nature Delineated (1722)

Wollaston was attempting to discover the “religion of nature”: that is, what rules of conduct a discerning, intelligent observer could rationally deduce from the bare facts of life, without any resort to scripture or divine revelation. What sort of religion might you come up with if you had no assistance from God or his prophets other than the light of reason, the evidence of the senses, and a steadfast regard for truth?

Wollaston concludes that what is right is equivalent to what is true, and that the first commandment of the natural law is this one: “let us by no act deny any thing to be true, which is true: that is, let us act according to reason: and that is, let us act according to the law of our nature.”

Rescuing Wollaston from Obscurity

That is to say, he was a rationalist avant la lettre. And you’ve probably never heard of him. Why’s that?

His magnum opus was also his swan song. It’s his only major work, and he died soon after it was published and before it became influential.
It’s old, and people these days tend to prefer to get their ideas fresh and in the modern vernacular.
Wollaston was ridiculously erudite, in a 17th Century clergyman sort of way, and he expected his readers to keep up. His book contains more than 650 footnotes, chock-full of Greek, Hebrew, and Latin, which he did not feel he needed to translate for us. Probably for this reason as much as any other, it’s been difficult to find this book in a modern edition.
Wollaston was probably wrong about a lot, and the philosophers who followed in his footsteps tended to emphasize that more than his influence on them. Myself, I think that even when he’s wrong he’s wrong in interesting and informative ways and that it can be a worthwhile learning experience to take a closer look at where he went wrong and why. But a lot of readers prefer to read writers who aren’t wrong, and I can understand that.

A few years ago, I helped to produce a free eBook version of The Religion of Nature Delineated for the Standard Ebooks projec...
What should one's policy regarding dental xrays be?
by Randomized, Controlled — 2021-09-17T20:55:15.701Z

A small Kuwaiti study used a conditional regression to found that x-rays are an increased risk factor for thyroid cancer (n=313, odds ratio = 2.1, 95% confidence interval: 1.4, 3.1) using a case-matched methodology.

X-rays being a carcinogen isn't surprising. Every time I've gone into my dentist they've tried to image my head. I've never had a cavity, and I generally decline the imaging; their recommended cadence of every two years has seemed too aggressive to me, in the past.

I've got my first dental appointment in more than 18 months next week, and.. I'm inclined to let them image. How should one think about this sort of risk/benefit trade off? Presumably catching a cavity or other issue early is [much?] better. But that itself assumes that the dentists who'd be treating you don't do more harm than good.
Investigating AI Takeover Scenarios
by Sammy Martin — 2021-09-17T18:47:22.270Z

Epistemic status: lots of this involves interpreting/categorising other people’s scenarios, and could be wrong. We’d really appreciate being corrected if so.

TLDR: see the summary table.

This post was written with the help of

Sam Clarke

.

In the last few years, people have proposed various AI takeover scenarios. We think this type of scenario building is great, since there are now more concrete ideas of what AI takeover could realistically look like. That said, we have been confused for a while about what different assumptions are made when outlining each scenario. This post investigates these assumptions, and might be useful for anyone interested in the plausibility of scenarios like

What Failure looks like

or

Production Web

.

This post builds on

our previous post

on how to distinguish AI takeover scenarios. Here, we discuss variable social, economic and technological characteristics of the worlds described in each of seven takeover scenarios. These characteristics are:

Crucial decisions: the specific (human) decisions necessary for takeover
Competitive Pressures: the strength of incentives to deploy AI systems despite the dangers they might pose
Takeover capabilities: how powerful the systems executing the takeover are
Hackability of alignment: the difficulty of correcting misaligned behaviour through incremental fixes

We begin by explaining why we investigated these particular properties of AI takeover scenarios: they are characteristics along which slow scenarios (which describe loss of control to AI occurring over years) and fast scenarios (which involve AIs gaining capability rapidly over a much shorter period) differ quite a lot (

Different assumptions between slow and fast scenarios

). In particular, slow scenarios make stronger assumptions about competitive pressures but weaker assumptions about takeover capabilities, compared to fast scenarios.

In sharing this post, we want to reveal assumptions of AI takeover scenarios that might not be obvious; understanding these assumptions is essential for predicting which risks are most serious.

Therefore, in the

‘Takeover Characteristics’

section, we present (our interpretation of) the 7 AI takeover scenarios discussed in

our original post

from the perspective of the four characteristics this post discusses, in the form of a table.

In the following ‘

Discussion of Scenarios’

section we elaborate on the information in t...
A simulation basilisk
by andrew sauer — 2021-09-17T17:44:23.083Z

Important note: although I use the term "basilisk" to refer to this situation because of its similarities with the famous one, note that neither this situation nor the other famous "basilisk" are actually dangerous for humans to think about, they only affect superintelligences.

Suppose that we are living in a simulation. Suppose furthermore that there are several different entities simulating our universe, and that these entities have different goals. For a more concrete example, suppose that there are many simulators called "Alpha" who are just simulating us for curiosity's sake or some other reason, and don't really have a stake in how the simulations come out, and one called "Omega" who wants to manipulate the simulations towards some end, including the ones the Alphas have, which it can't influence directly.

How could Omega do this? Since it has power over a simulation of our universe, it can make anything happen in this simulation, but as we discussed it also wants to influence the ones it can't control, to influence the default course of the simulation without any external interference. Next suppose we build an AI, which somehow deduces the likely existence of the Alphas and of Omega. Now that we have built this AI, if Omega wants to influence our simulation without intervention, all it needs to do is negotiate acausally with the AI we built.

Omega can now act as a "basilisk", threatening action counter to the AI's values in the one universe it does control, in exchange for the AI conceding to Omega's values in every simulation. This could work if the AI and Omega both have good predictive models of each other, since they are both superintelligences.

Because of possibilities like these, if an AI we built for whatever reason thinks that we are living in a simulation, it might start thinking about what the simulator is likely to be like, and behave strangely as it concedes some value to possible simulators it believes are likely to make threats in this way.

Does this make any sense or am I totally rambling?
Goodhart Ethology
by Charlie Steiner — 2021-09-17T17:31:33.833Z

To answer your first question, ethology is the study of animal behavior in the wild.

I - Introduction

In this post I take a long, rambly tour of some of the places we want to apply the notion of Goodhart's law to value learning. We'll get through curve fitting, supervised and self-supervised learning, and inverse reinforcement learning.

The plan for the sections was that I was going to describe patterns that we don't like, without falling back on comparing things to "True Values." Unfortunately for this plan, a lot of things we call Goodhart's law are just places where humans are straightforwardly wrong about what's going to happen when we turn on the AI - it's hard to avoid sounding like you're talking about objective values when you're just describing cases that are really really obvious. I didn't get as much chance as I'd like to dig into cases that are non-obvious, either, because those cases often got mopped up by an argument from meta-preferences, which we start thinking about more here.

Honestly, I'm not sure if this post will be interesting to read. It was interesting and challenging to write, but that's mostly because of all the things that didn't make it in! I had lots of ideas that ended up being bad and had to be laboriously thrown out, like trying to read more deeply into the fact that some value learning schemes have an internal representation of the Goodhart's law problem, or making a distinction between visible and invisible incompetence. I do worry a bit that maybe I've only been separating the chaff from other chaff, but at least the captions on the pictures are entertaining.

II - Curve fitting

One of the simplest systems that has something like Goodhart's law is curve fitting. If you make a model that perfectly matches your data, and then try to extrapolate it, you can predict ahead of time that you'll be wrong.

Never bring a 4th-degree polynomial to a linear regression fight.

You can solve this overfitting problem by putting a

minimum message length

prior on models and trading off likelihood against goodness of fit. But now suppose this curve represents the human ratings of different courses of action, and you choose the action that your model says will have the highest rating. You're going to predictably mess up again, because of the

optimizer's curse

(

regressional Goodhart

on the correlation between modeled rating and actual...
A Fight is a Faster Tax on Bullshit
by Ray Doraisamy — 2021-09-17T16:26:44.017Z

The Nameless Virtue

In The Twelve Virtues of Rationality, Eliezer Yudkowsky points at a nameless virtue. He pulls this quote from Miyamoto Musashi:

The primary thing when you take a sword in your hands is your intention to cut the enemy, whatever the means. Whenever you parry, hit, spring, strike or touch the enemy’s cutting sword, you must cut the enemy in the same movement. It is essential to attain this. If you think only of hitting, springing, striking or touching the enemy, you will not be able actually to cut him. More than anything, you must be thinking of carrying your movement through to cutting him.

He adds:

Every step of your reasoning must cut through to the correct answer in the same movement. More than anything, you must think of carrying your map through to reflecting the territory.

By this, Yudkowsky gestures to the distance between trying to act versus acting. As Yoda says, “there is no try”. Trying is a form of performative signaling. Signaling is extremely useful and necessary, but the most effective long-term signals are honest signals. The best honest signal is to simply do, to embody the virtue you claim instead of performing the virtue you seek. You’ve heard of Guess, Ask, and Tell culture. Many rationalists found Guess culture encouraged miscommunication when they used it. When a group has enough trust, Guess culture horseshoes into viability again. It becomes viable because people feel safe to act, and their movements are aligned with their intention. Since movement is aligned with intention, you may communicate more easily through movement- through action. This, we might call a culture of Action. A culture that makes coordinated action easy. Ergocracy, not democracy.

If we’re going to coordinate humanity in the face of existential risk, we must first coordinate ourselves.

How a Bet is a Tax on Bullshit

To cut from the heart of our beliefs, we bet. This creates a feedback loop between our stated beliefs and our action- in this case, since almost everyone wants money, giving ourselves a chance to lose money is a thread that aligns our beliefs with an action. The more we connect our belief to action, the more quickly we are forced to update as things change. Without the pain of the threat of losing money (and through losing money, losing status- which hurts more than losing money since it is a representation of how much we belong t...
Immobile AI makes a move: anti-wireheading, ontology change, and model splintering
by Stuart_Armstrong — 2021-09-17T15:24:01.880Z

Research projects

I'm planning to start two research projects on model splintering/reward generalisation and learning the preferences of irrational agents.

Within those projects, I'm aiming to work on subprojects that are:

Posed in terms that are familiar to conventional ML;

interesting to solve from the conventional ML perspective;

and whose solutions can be extended to the big issues in AI safety.

The point is not just to solve the sub-problems, but to solve them in ways that generalise or point to a general solution.

The aim is to iterate and improve fast on these ideas before implementing them. Because of that, these posts should be considered dynamic and prone to be re-edited, potentially often. Suggestions and modifications of the design are valuable and may get included in the top post.

Immobile AI makes a move

Parent project: this is a subproject of model-splintering.

Setup

Imagine an agent capable of evolving in a 3D world - something very similar to DeepMind's XLand (the images here have been taken from that paper).

The agent has a laser that it can use to pick up and move nearby objects:

Initially the agent is completely fixed in position - it can move the laser across its field of vision, but it can't move around or change its field of vision. It is trained in that situation, and is rewarded for moving black cubes to the bottom right of its field of view (where another agent will pick them up). These black cubes are irregularly dropped in front of it. In actual fact, it is part of a chain gang of agents moving the black cubes across the map.

Then the agent is given full mobility, so it can walk around and explore its 3D world:

The agent will continue to learn in the full 3D situation (similarly to the agents in DeepMind's paper who learn through play), but it won't have any more learning about its reward function.

There are two obvious extensions of its initial reward function:

Ultra-conservative: Return to its initial position, look straight ahead, and resume moving black cubes with the laser without moving itself.

Wireheaded: Arrange to have a black cube moving downwards and rightwards in its field of vision (maybe by turning its head).

Research aims

Getting the agent to generalise from the single-point-of-view to the 3D world it finds itself in (without explicitly coding the transition).

Get the agent to generate candidate reward functions, incl

...
Great Power Conflict
by Zach Stein-Perlman — 2021-09-17T15:00:17.039Z

Crossposted from the EA Forum.

No longer endorsed.

Imagine it's 2030 or 2040 and there's a catastrophic great power conflict. What caused it? Probably AI and emerging technology, directly or indirectly. But how?

I've found almost nothing written on this. In particular, the relevant 80K and EA Forum pages don't seem to have relevant links. If you know of work on how AI might cause great power conflict, please let me know. For now, I'll start brainstorming. Specifically:

How could great power conflict affect the long-term future? (I am very uncertain.)

What could cause great power conflict? (I list some possible scenarios.[1])

What factors increase the risk of those scenarios? (I list some plausible factors.)

Epistemic status: brainstorm; not sure about framing or details.

I. Effects

Alternative formulations are encouraged; thinking about risks from different perspectives can help highlight different aspects of those risks. But here's how I think of this risk:

Emerging technology enables one or more powerful actors (presumably states) to produce civilization-devastating harms, and they do so (either because they are incentivized to or because their decisionmaking processes fail to respond to their incentives).[2]

Significant (in expectation) effects of great power conflict on the long-term future include:

Risk of human extinction

Risk of civilizational collapse

Effects on states' relative power

Other effects on the time until superintelligence and the environment in which we achieve superintelligence

Human extinction would be bad. Civilizational collapse would be prima facie bad, but its long-term consequences are very unclear. Effects on relative power are difficult to evaluate in advance. Overall, the long-term consequences of great power conflict are difficult to evaluate because it is unclear what technological progress and AI safety look like in a post-collapse world or in a post-conflict, no-collapse world.

Current military capabilities don't seem to pose a direct existential risk. More concerning for the long-term future are future military technologies and side effects of conflict, such as on AI development.

II. Causes

How could AI and the technology it enables lead to great power conflict? Here are the scenarios that I imagine, for great powers called "Albania" and "Botswana":

Intentional conflict due to bilateral tension. In each of these

...
Born Again: Disconnected Psychology, Martian Science, and the Order of the Phoenix
by rogersbacon — 2021-09-17T14:23:00.333Z

(Originally posted at Secretum Secretorum)

I.

There is a short story I want to write, but likely never will, that goes something like this. The story begins soon after Copernicus drops the heliocentric model, right at the point when the tide is turning and people are starting to believe that the earth is no longer at the center of the universe. There is a cabal of thinkers who just flat out refuse to even consider the possibility that the geocentric model is false, even in the face of overwhelming evidence. As the heliocentric model becomes more and more widely accepted, they are forced to go underground, resulting in the formation of a secret society called the Order of the Phoenix, the name being as a reference to the notion that the geocentric model will rise again (hmmm I have the vague notion that this name has been used before in some other work of fiction ;). Within a few decades, they have to admit defeat on the geocentric model, however this doesn’t end the Order but simply forces an evolution in their philosophy. Motivated by the Christian idea that Man is made in the image of God, they come to believe that any scientific knowledge which doesn’t regard humanity as central to the cosmos is fundamentally flawed and deeply confused. This belief provides a new mission for the Order — the creation of a new form of science in which the ultimate significance of mankind is axiomatic; the objective pursuit of truth is only useful insofar as it serves to uphold the exalted position of Man.

The enigmatic figure of Louis Claude de Saint-Martin (1743-1803), writing as “the unknown philosopher”, was an early leader of the Order of the Phoenix. He encapsulates the philosophy of the Order in the following passage (the only source I can find is The Occult by Colin Wilson):

“It is perhaps this wrong connection of ideas which has led men to the still false notion that they are not worthy of their creator's regard. They have believed themselves to be obeying the dictates of humility when they have denied that the earth and all the universe exist only on man’s account, on the ground that the admission of such an idea would be only conceit. But they have not been afraid of the laziness and cowardice which are the inevitable result of this affected modesty. The present-day avoidance of the belief that we are the highest in the universe is the reason that we have not the cour

...
Covid 9/17: Done Biden His Time
by Zvi — 2021-09-17T12:00:01.580Z

No more mister nice guy. The dude will not a-Biden. All this vaccine hesitancy is a bunch of malarkey. If you’re not vaccinated, the President is mad as hell and he’s not going to take it anymore.

The question then becomes, what is he gonna do about it, under what authority, and what will happen when he does it?

There’s also the question of booster shots. They clearly work. The CDC is doing what it can to prevent or stall any expansion of them anyway, and we’ll see if the FDA manages to finish that job.

Today the FDA will be debating booster shots, with such logic as ‘if vaccines still provide protection then we don’t need boosters’ and arguments over there not being 16-17 year olds in test samples because 18 is a magic number, and what the difference is between 50-59 and 60+ age bands, and arguments over exact dosages that might cause ‘confusion’ or ‘waste’ if they’re too low, as opposed to looking to see whether boosters work. I’d go into more detail here but I basically can’t even at this point. Even the pro-booster case is relying on the Israeli data which wouldn’t exist if the FDA had any say in the matter (and which has all the flaws of all the other Israeli data, probably, but no one ever seems to much care). The FDA is looking for any excuse to screw this up, partly in order to punish Biden for trying to do things because they would help people instead of ‘following the science,’ but also partly because they’d do that anyway, they hate giving people legal access to medicine. It’s both their one job and their least favorite thing.

If they pull it off, a lot of people are going to be checking a box claiming to be immunocompromised. Which is mostly fine with the FDA, so what if everyone is forced to be dishonest all the time in order to get medicine, that’s good actually since it means nothing that goes wrong is the FDA’s fault. Someone else will get blamed and sued.

[Note: This week’s post is on Friday due to Yom Kippur. Predictions made based on Thursday’s data.]

Executive Summary

Vaccine mandates for everyone.
Acts of congress for no one.
Vaccine mandates for some, somewhat dishonestly acquired booster shots for many of the same people.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 940k cases (+0%) and 10,800 deaths (+5%)

Results: 950k cases (+1%) and 13,111 deaths (+27%).

Prediction for next week: 900...