trentmkelly/LessWrong-43k huggingface dataset:
The Price Of Space Is Friendship
by sxae — 2021-03-24T11:27:50.953Z

It's an exciting time in the history of human exploration. We're only just beginning to probe outside our biosphere. Low-orbit, long-term space inhabitation like that seen on the ISS has been the norm for many years now. As of the time of writing, 7 people were in space. Our long term goals seem set on permanent outposts on Mars.

Orbital re-entry of crewed spacecraft is a complicated dance of physics. You ever do that challenge where you have to build a cage out of straws and cardboard to protect an egg from a fall? Well, it's like that on steroids. Generally, if you are coming in from another interstellar body you will be going very fast. You can't decelerate over around 90m/s/s (or 9gs) or let the interior of the ship get too hot without killing your crew. You've got an enormous amount of kinetic energy to disperse, and the best way to do that is by braking in the atmosphere at a relatively shallow angle of descent. This means that reentry maneuvers cover a lot of airspace. You will likely either orbit the earth entirely or a large portion of it before you finally reduce your velocity enough to touch down.

The same is true of orbital ejections from Earth. Generally we try to launch from near the equator as we can, and in the direction of the Earth's rotation (easterly). This is so we can steal a bit of this initial speed from the Earth itself. To get ourselves from one stellar body to another we might do a Hohmann transfer, which means there is going to be some arbitrary point in the stable orbital trajectory where we need to fire some impulse to get onto this new trajectory (where line '2' begins below). That arc over low-earth orbit can be of any length, from a few moments to multiple orbits around the globe, and stretch over any number of nation-states.

(Courtesy of WikiMedia)

There have been significant developments in the ability for nation-states to control their low-earth orbit even in the past few decades. Anti-satellite systems are a rapidly growing sector of defence, with current systems claiming the ability to target low-orbit satellites. In 2007, the Chinese military destroyed a Chinese weather satellite through their ASAT (anti-satellite) program. In 2008, the US Navy destroyed a US spy satellite using a ship-launched missile. In 2019, India joined in the fun with their own test destruction of their own satellite. Clearly, such systems are alive, ...
Toy model of preference, bias, and extra information
by Stuart_Armstrong — 2021-03-24T10:14:34.629Z

This is a toy model, inspired by the previous post's argument that the biases and preferences of a human carry more information than their behaviour.

This is about a human purchasing chocolate, so I am fully justified in illustrating it with this innocuous image:

Bidding on chocolate

A human is bidding to purchase some chocolate. It's a second price auction, so the human is motivated to bid their true valuation.

The human either prefers milk chocolate (described as 'sweet') or dark chocolate (described as 'sour'). Whichever one they prefer, they will price at €40, and the other one at €30. Call these two possible preferences Rm (prefers milk chocolate) and Rd (prefers dark chocolate).

The human is also susceptible to the anchoring bias. Before announcing the type of chocolate, the announcer will also randomly mention 60 or 0. This will push the human's declared bid €10 up or down. Because of their susceptibility to anchoring bias, we will call them Ha.

Then the following table illustrates the human's bid, dependent on the announcement and their own preference:

Announcement(Ha,Rm)(Ha,Rd)60, sweet€50€400, sweet€30€2060, sour€40€500, sour€20€30

Now let's introduce Hr (the 'rational' human). This human derives satisfaction from naming prices that are closer to numbers it has heard recently. This satisfaction is worth - you guessed it - the equivalent of €10. Call this reward Ra.

Then Hr with reward Ra+Rm will have the same behaviour as Ha with reward Rm: they will also upbid when 60 is announced and downbid when 0 is announced, but this is because they derive reward from doing so, not because they are biased. Similarly, Hr with reward Ra+Rd will have the same behaviour as Ha with reward Rd.

So Hr is a human with less bias; now let's imagine a human with more bias. I'll introduce a so-called 'connotation bias'. This human may value milk and dark chocolate as given by their reward, but the word 'sweet' has positive connotations, independent of its descriptive properties; this increases their bids by €10. The word 'sour' has negative connotations; this decreases their bids by €10.

A human with connotation bias and reward Rd will behave like a human without connotation bias and reward Rm. That's because the human prices milk chocolate at €30, but will add +€10 because it's described as 'sweet'; and the converse, €(40−10) for the 'sour' dark chocolate.

Let Hc describe a human ...
Selling Attention for Money
by MikkW — 2021-03-24T06:24:37.981Z

I felt like experimenting, so I posted this on my shortform:

In short, I am selling my attention by selling the right to put cards in my Anki deck, starting at the low price of $1 per card. (Current price $4)

I will create and add a card (any card that you desire, with the caveat that I can veto any card that seems problematic, and capped to a similar amount of information per card as my usual cards contain) to my Anki deck for $1. After the first ten cards (across all people), the price will rise to $2 per card, and will double every 5 cards from then on. I commit to study the added card(s) like I would any other card in my decks (I will give it a starting interval of 10 days, which is sooner than the usual interval of 20 days I usually use, unless I judge that a shorter interval makes sense. I study Anki every day, and have been clearing my deck at least once every 10 days for the past 5 months, and intend to continue to do so). Since I will be creating the cards myself (unless you know of a high-quality deck that contains cards with the information you desire), an idea for a card is enough even if you don't know how to execute it.

Money is a unit of caring, and everything should, in principle, be fungible for money at some (usually) non-infinite cost, including our knowledge and attention. In the long run, I suspect I value my attention and time at a rate higher than that implied by $1 per card, but I also suspect my value on it isn't so high as to make this experiment not point in the right direction. I also expect that this experiment will help expose me to new ideas and ways of thinking, and I'm curious to see what information people will ask me to add.

Advertisements work on a similar principle: a user uses a service (either a website, or a television channel, or similar) that costs the host resources to maintain, and in return implicitly agrees to watch and listen to messages provided by a sponsor, who pays the host to compensate them for providing the service to the user. While I have mixed feelings about advertisements as actually practiced in the modern world, this dynamic seems pretty reasonable to me when I state it that way. Perhaps what bothers me there is that modern advertisements often try to make me believe things that are untrue, or to be designed to appeal to the irrational parts of my lizard brain to manipulate my behavior in ways that aren...
Another RadVac Testing Update
by johnswentworth — 2021-03-23T17:29:10.741Z

Previously: Making Vaccine, Commercial Antibody Test Results, Mini-Update

I've now run 9 ELISA tests. The main result is noise: negative controls are all over the map, sometimes very blue (i.e. positive), sometimes not blue at all. I did see more positive results in the experimental group than I'd expect from noise alone, but I haven't gotten the noise to a point where results are consistently reproducible.

Meanwhile, I also ran one very simple test: I snorted a batch of the peptides, without the chitosan or anything else - just peptides in deionized water. Previously, on doses 3-6 of the vaccine, I had consistently been congested for a couple days after (and not congested the rest of the week), which strongly indicates an immune response. However, that response could have been to the chitosan or other contents of the vaccine, rather than the peptides. This test put that possibility to rest: after snorting just the peptides, I was very obviously congested for a couple days, in basically the same way as after the vaccine doses.

So thanks to that simple test, I personally am now pretty highly confident that I have an immune response to these peptides. Unfortunately it's not as legible as an ELISA test, so you should not necessarily be quite as convinced by this.

Now, this still leaves the question of whether an immune response to these peptides translates into an immune response to COVID. It could be that e.g. the conformation of the peptides' corresponding sequence within full COVID proteins is different enough that it doesn't carry over. Personally, though, I consider this a much less likely failure mode, for two reasons. First, the white paper indicates that the peptides were chosen based on antibodies developed by people who actually had COVID. Second, whether antibodies against these peptides bind the real proteins is something which I would not expect to vary much from person to person, so if it's worked for a few people it should work for everyone - and the whitepaper does indicate that multiple groups have seen positive results testing for binding against the full proteins.

None of this puts my confidence up close to 99%, but I'm now considerably more confident that the vaccine worked (~90%). Also, that confidence is distributed over fewer possible worlds - e.g. based on the info in the latest version of the RadVac whitepaper, I now very much doubt that the...
Mary and the Black And White Memory Implant
by sxae — 2021-03-23T16:48:20.455Z

In 1982, Frank Jackson published an article entitled "Epiphenomenal Qualia" in Philosophical Quarterly 32:127. Within it, he outlined a thought experiment known as "Mary's Room", or "Mary and the Black-and-White Room". [1]

Imagine a scientist called Mary. She is born and raised within a black-and-white room, where she experiences no colours. Mary is a talented scientist, and learns an enormous amount about the physics of colour, the mechanism by which the human eye detects colour, and the pathways through which colour is interpreted within the brain.

One day, Mary steps out the door of her black-and-white room and into the world, and experiences colour for the first time. The question posed by the thought experiment is this: what does Mary learn at this moment?

Jackson presents this thought experiment originally as a refutation of Physicalism, and a demonstration of the non-physical nature of mental states.

It seems just obvious that she will learn something about the world and our visual experience of it. But then is it inescapable that her previous knowledge was incomplete. But she had all the physical information. Ergo there is more to have than that, and Physicalism is false.

The thought experiment strikes at the heart of qualia, the subjective conscious experience. What does Mary learn? What is the difference between Mary before she stepped through this door, and afterwards? In thinking about this, I remain unconvinced that this is a refutation of Physicalism.

Let us extend this thought experiment a little. Imagine that we have a device that can implant memories into a brain so perfectly that there is no test that you can reasonably perform to distinguish between a real and artificial memory. As far as I can tell, such a device (though incredibly complicated) would be possible.

Now let us imagine that we implant a memory into Mary's brain of her leaving the room and experiencing colour, before returning to the room. Let us assume that there is no evidence of that journey other than the memory and no test that Mary can perform to ascertain whether the event actually happened - no expectation of a green blade of grass stuck on a shoe or anything like that. What then is the difference in qualia between this version of Mary and the version of Mary that did genuinely leave the room?

My immediate thought is that this difference depends on Mary's beliefs and knowledge...
Preferences and biases, the information argument
by Stuart_Armstrong — 2021-03-23T12:44:46.965Z

I've recently thought of a possibly simpler way of expressing the argument from the Occam's razor paper. Namely:

Human biases and human preferences contain more combined information than human behaviour does. And more than the full human policy does.

Thus, in order to deduce human biases and preferences, we need more information than the human policy caries.

This extra information is contained in the "normative assumptions": the assumptions we need to add, so that an AI can learn human preferences from human behaviour.

We'd ideally want to do this with as few extra assumptions as possible. If the AI is well-grounded and understands what human concepts mean, we might be able to get away with a simple reference: "look through this collection of psychology research and take it as roughly true" could be enough assumptions to point the AI to all the assumptions it would need.
Against evolution as an analogy for how humans will create AGI
by Steven Byrnes — 2021-03-23T12:29:56.540Z

Background

When we do Deep Reinforcement Learning to make a PacMan-playing AI (for example), there are two algorithms at play: (1) the “inner” algorithm (a.k.a. “policy”, a.k.a. “trained model”) is a PacMan-playing algorithm, which looks at the pixels and outputs a sequence of moves, and (2) the “outer” algorithm is a learning algorithm, probably involving gradient descent, which edits the “inner” PacMan-playing algorithm in a way that tends to improve it over time.

Likewise, when the human brain evolved, there were two algorithms at play: (1) the “inner” algorithm is the brain algorithm, which analyzes sensory inputs, outputs motor commands, etc., and (2) the “outer” algorithm is Evolution By Natural Selection, which edits the brain algorithm (via the genome) in a way that tends to increase the organism’s inclusive genetic fitness.

There’s an obvious parallel here: Deep RL involves a two-layer structure, and evolution involves a two-layer structure.

So there is a strong temptation to run with this analogy, and push it all the way to AGI. According to this school of thought, maybe the way we will eventually build AGI is by doing gradient descent (or some other optimization algorithm) and then the inner algorithm (a.k.a. trained model) will be an AGI algorithm—just as evolution designed the generally-intelligent human brain algorithm.

I think this analogy / development model is pretty often invoked by people thinking about AGI safety. Maybe the most explicit discussion is the one in

Risks From Learned Optimization

last year.

But I want to argue that the development of AGI is unlikely to happen that way.

Defining “The Evolution Analogy for AGI Development”: Three ingredients

I want to be specific about what I’m arguing against here. So I define the Evolution Analogy For AGI Development as having all 3 of the following pieces:

“Outer + Inner”:

The analogy says that we humans will write and run an “outer algorithm” (e.g. gradient descent), which runs an automated search process to find an “inner algorithm” (a.k.a. “trained model”).

…Analogous to how evolution is an automated search that discovered the human brain algorithm.
(If you’ve read

Risks From Learned Optimization

you can mentally substitute the words “base & mesa” for “outer & inner” respectively; I’m using different words because I'm thinking of “base & mesa” as something very specific, and I want to t

...
Canada Covid Update: thinking out loud
by MalcolmOcean — 2021-03-22T21:39:43.550Z

(This started as a comment on Zvi's weekly post from 3/18, but it got long and I started a second comment and I figured I might as well just make a new top-level post.)

I don't intend these posts to be consistent or exhaustive in the way that Zvi's are. They're mostly me thinking out loud as I try to figure out what's going on.

Here's a colorful overview from https://art-bd.shinyapps.io/covid19canada/ which I think is a day or two out of date:

Vaccines

Canada has now vaccinated over 10 doses per 100 people, and since we're now officially doing first doses first in most cases, that's nearly 10% of the population vaccinated. The territories, which have almost nobody in them, are like half-vaccinated already. Not sure what the thinking was there (sometimes northern regions have it really rough?) but in any case it's nearly a rounding error on the doses needed for provincial populations, so seems reasonable to just get it over with.

We have 38M people, 940k cumulative confirmed cases, 4M vaccine doses administered.

Interestingly, while we're way behind the USA on administering vaccine doses (they're at 37 doses per 100 people), we've already soared way past the "more people vaccinated than ever tested positive" figure because we had fewer people test positive in the first place. From a timeline perspective though, that unfortunately means we're even further from herd immunity than being so behind on vaccines would imply.

Vaccine timelines are speeding up here a bit though and it's looking like with first doses first and a few bonuses we might actually have a shot in everyone's arm by July. (Everyone ofc means everyone over the age of 16 (31.5M) who wants to (about 79% of that, according to a poll I found somewhere))

From CBC

I don't know if that's just an arbitrary date they're intending to allow large Canada Day celebrations on July 1st if indeed this is achieved. It think it would be really meaningful to do that, though might be unwise given that many of those adults would have been vaccinated on like June 29th or 30th so could still get sick if they caught the virus in the following few days.

New Strains - "Variants of Concern"

Searching for data about the new strains (eg B.1.1.7) in Canada... aha, I found this official statement from March 16th. Which links to even better data. Okay, if we assume that the proper denominator here is the total number of confirmed cases o...
Thirty-three randomly selected bioethics papers
by Rob Bensinger — 2021-03-22T21:38:08.281Z

Some scholarly fields are very healthy (e.g., physics). Some are fairly unhealthy (e.g., evolutionary psychology, sad to say). Some are outright crazy (e.g., philosophy of religion).

How good or bad is bioethics? How rigorous and truth-tracking is it? How much social benefit or harm does it cause?

Looking at lots of random examples is an under-used tool for making progress on this kind of question. It's fast, it avoids the perils of cherry-picking, and it doesn't require you to trust someone else's high-level summary of the field.

I picked the two highest-impact-factor "medical ethics" journals that have the word "bioethics" in the title: The American Journal of Bioethics and Bioethics. Together, these two journals release about 500 reports, book reviews, etc. per year.

I then picked a random article from 2014 through 2020 from each journal, plus extra random articles from the more cited journal (The American Journal of Bioethics): five from 2016, 2018, and 2020, and two from 2017 and 2019.

For each article, I quoted the abstract (if there is one), the first two paragraphs, and the final paragraph. These are provided below, sorted by year.

Obviously this breadth-first approach won't provide a full sense of the quality of argumentation in these papers; but it will hopefully provide a picture of what kinds of views tend to get argued for in the field, what those arguments tend to look like, what tends to get taken for granted, what tends to get ignored, and so on.

I expect more accurate conclusions and faster consensus-building about bioethics if conversation is grounded in a common pool of examples (which can then be drilled down on to spot-check a particular paper's arguments, etc.).

Added: aphyer summarized the articles in the comment section; I've put aphyer's summaries in an editable Google Doc to make it easier to improve on the summaries (and to encourage article-by-article critiques, in the comments below or in the Doc).

2014

1. Against Fetishism About Egalitarianism and in Defense of Cautious Moral Bioenhancement

Ingmar Persson and Julian Savulescu. The American Journal of Bioethics 14(4). Open Peer Commentaries.

We respond to Sparrow’s (2014) criticisms of our support for a project of moral bioenhancement. We distinguish between a confident and cautious proposal, arguing that his objections only apply to a confident proposal, while we endorse a cautio

...
My research methodology
by paulfchristiano — 2021-03-22T21:20:07.046Z

(Thanks to Ajeya Cotra, Nick Beckstead, and Jared Kaplan for helpful comments on a draft of this post.)

I really don’t want my AI to strategically deceive me and resist my attempts to correct its behavior. Let’s call an AI that does so egregiously misaligned (for the purpose of this post).

Most possible ML techniques for avoiding egregious misalignment depend on detailed facts about the space of possible models: what kind of thing do neural networks learn? how do they generalize? how do they change as we scale them up?

But I feel like we should be possible to avoid egregious misalignment regardless of how the empirical facts shake out--it should be possible to get a model we build to do at least roughly what we want. So I’m interested in trying to solve the problem in the worst case, i.e. to develop competitive ML algorithms for which we can’t tell any plausible story about how they lead to egregious misalignment.

This is a much higher bar for an algorithm to meet, so it may just be an impossible task. But if it’s possible, there are several ways in which it could actually be easier:

We can potentially iterate much faster, since it’s often easier to think of a single story about how an algorithm can fail than it is to characterize its behavior in practice.
We can spend a lot of our time working with simple or extreme toy cases that are easier to reason about, since our algorithm is supposed to work even in these cases.
We can find algorithms that have a good chance of working in the future even if we don’t know what AI will look like or how quickly it will advance, since we’ve been thinking about a very wide range of possible failure cases.

I’d guess there’s a 25–50% chance that we can find an alignment strategy that looks like it works, in the sense that we can’t come up with a plausible story about how it leads to egregious misalignment. That’s a high enough probability that I’m very excited to gamble on it. Moreover, if it fails I think we’re likely to identify some possible “hard cases” for alignment — simple situations where egregious misalignment feels inevitable.

What this looks like (3 examples)

My research basically involves alternating between “think of a plausible alignment algorithm” and “think of a plausible story about how it fails.”

Example 1: human feedback

In an unaligned benchmark I describe a simple AI training algor...