trentmkelly/LessWrong-43k huggingface dataset:
Lessons from “The Book of My Life”
by bmgarfinkel — 2021-01-06T22:40:08.722Z

This is a cross-post from my personal blog. You might enjoy it if you're interested in progress studies or intellectual history.

The Italian Renaissance, like nearly every other time and place, is a real blank spot in my knowledge of history. I recently read The Book of My Life (1575), a memoir by the mathematician-doctor-philosopher Gerolamo Cardano, to try to get some feel for what was going on there.

My understanding is that Cardano was a top tier Renaissance intellectual. He made major contributions to algebra, wrote the first treatise on probability theory, and was invited to practice medicine on multiple kings. He put out more than one hundred books, all with titles like On Gems and Colors, Fourth Book of Hidden Knowledge, and Triceps. He wasn’t in the top five, I think, but he may have been in the top twenty.

In some regards, as a window into the period, The Book of My Life turned out to be a great pick. It’s written more like an encyclopedia of a Renaissance intellectual’s life than a personal narrative. Half of the chapter titles sound like terrible Jeopardy categories: “Religion and piety,” “Sports and exercise,” “Journeys,” “Dress,” “Those things in which I take pleasure.” There’s a chapter devoted to detailing his entire medical history and another chapter devoted to describing his habit of thinking while walking. One reason that old books can sometimes fail to give a good sense of setting is that the author has no interest in discussing the commonplace. That’s not a problem with The Book of My Life.

On the other hand, a major problem with using this book as a window into the period is that Cardano clearly can’t have been a normal guy. He is, for example, very upfront about his lack of desire to interact with almost any of his contemporaries. A representative quote, from the chapter “Quality of Conversation”:

I love solitude, and I am never more with those I love than when I am alone. For I love God and my good genius. Upon these, then, when alone, I meditate.

He goes on to present several distinct arguments against spending time with others, ranging from the point that “inventions are the result of the tranquil life” to the point that “some stink from their armpits.” He spends about as much time discussing his wife as he spends discussing this one time, many years ago, when a dog jumped clean over a mule he was riding.[1]

There’s a late-in-the-book...
The Rube Goldberg Machine of Morality
by DirectedEvolution — 2021-01-06T20:24:36.844Z

1. Mathematics and morality

One death is a tragedy. A million deaths is a statistic. - Joseph Stalin

To a mathematician, it's equally obvious that 1 + 1 = 2 and that 3 * 5 = 15.

A kid might treat 3 * 5 as a repeated addition problem. The answer isn't obvious, and even if they get 15, they're a little worried they might have miscalculated. I've taken a year of calculus and I'm still never confident when I multiply 7 * 6 in my head.

A math proof breaks a complicated procedure into a sequence of obvious steps.

Math is emotional. When you're working on a problem, you might feel confident at times, anxious at others. I feel about as much confidence and anxiety these days trying to understand a linear algebra or computational theory proof as I did when I was doing times tables back in elementary school. Math gets more complicated, the feelings stay the same.

However, math does not depend on emotion to be correct. You can be anxious, and still check your work and discover that you're right. You can also feel confident in your proof, and still find that it was inadequate.

Some people find it equally obvious that murder is wrong and that failing to optimize charitable contributions is wrong.

For most people, murder is obviously wrong. Failing to optimize charitable contributions is not. Even if they think the argument through, they're worried that it went sideways somewhere. Or that accepting it would have other secretly bad implications.

We can use math to describe morality. But this can lead to repugnant conclusions. I can make statements that are mathematically moral, like:

It's more wrong to murder two people than one person. In general, it's worse to murder more people than less. Each additional murder adds the same amount of wrongness. For example, it is ten times worse to murder ten people as to murder one person.

These statements hold only for cold-blooded murder, and only for positive numbers of murders.

If I also accept the statement:

Terrorism is a form of cold-blooded murder.

Then a terrorist attack that killed a million people would be a million times worse than a single murder.

The word "a million" doesn't look much bigger than the word "one." Let's do the moral equivalent of one of those diagrams showing the relative sizes of the earth and the sun.

If a city block represented how bad a single murder is, then you'd have to walk around the earth several times to represe...
Review of 'But exactly how complex and fragile?'
by TurnTrout — 2021-01-06T18:39:03.521Z

I've thought about (concepts related to) the fragility of value quite a bit over the last year, and so I returned to Katja Grace's But exactly how complex and fragile? with renewed appreciation (I'd previously commented only a very brief microcosm of this review). I'm glad that Katja wrote this post and I'm glad that everyone commented. I often see private Google docs full of nuanced discussion which will never see the light of day, and that makes me sad, and I'm happy that people discussed this publicly.

I'll split this review into two parts, since the nominations called for review of both the post and the comments:

I think this post should be reviewed for its excellent comment section at least as much as for the original post, and also think that this post is a pretty central example of the kind of post I would like to see more of.

~ habryka

Summary

I think this was a good post. I think Katja shared an interesting perspective with valuable insights and that she was correct in highlighting a confused debate in the community.

That said, I think the post and the discussion are reasonably confused. The post sparked valuable lower-level discussion of AI risk, but I don't think that the discussion clarified AI risk models in a meaningful way.

The problem is that people are debating "is value fragile?" without realizing that value fragility is a sensitivity measure: given some initial state and some dynamics, how sensitive is the human-desirability of the final outcomes to certain kinds of perturbations of the initial state?

Left unremarked by Katja and the commenters, value fragility isn't intrinsically about AI alignment. What matters most is the extent to which the future is controlled by systems whose purposes are sufficiently entangled with human values. This question reaches beyond just AI alignment.

They also seem to be debating an under-specified proposition. Different perturbation sets and different dynamics will exhibit different fragility properties, even though we're measuring with respect to human value in all cases. For example, perturbing the training of an RL agent learning a representation of human value, is different from perturbing the utility function of an expected utility maximizer.

Setting loose a superintelligent expected utility maximizer is different from setting loose a mild optimizer (e.g. a quantilizer), even if the...
Fourth Wave Covid Toy Modeling
by Zvi — 2021-01-06T18:30:01.715Z

Epistemic Status: Highly speculative. I threw this together quickly, and wrote this to document how I went about it. This is an attempt to create a first toy model, so others can error correct and improve, and upon which less-toy models can hopefully be built. You can see the spreadsheet with my work here. Please take this and run with it, and please don’t take this as more than trying stuff to see what’s wrong with it.

No one seems to be creating models of various scenarios in a way that feels remotely realistic, or even in a way that feels super simplified but that can be used as intuition pumps or baselines.

This post aims to fix that, or at least provide a first step.

At this point, we mostly know we’re f***ed, and that the new strain is at least ~40% more infectious, probably 50%+ more infectious, perhaps as high as 65%-70%.

What happens now?

Speeding up vaccinations seems to be the opposite of what we are doing, but seeing exactly what it would take to get us out of this mess seems like a worthwhile exercise. Time to start building toy models.

Completely Naive Model

To start off, I did the quickest thing at every step to see what happened.

Note that we’re measuring and predicting actual cases not positive tests. The observed and official numbers will always be several times lower than the real ones.

This model assumes:

Covid Machine Learning’s estimate of old infections is correct.
Covid Machine Learning’s estimates of current infections are roughly correct, about 600,000 per day.
Based on sequencing results, we estimate 0.06% of new cases for cycle ending December 19.
We assume every ‘infection cycle’ is five days, so if you are infected on day 0, you infect others on day 5.
The initial strain had an effective R0=1 on December 19, when things began.
The new strain is 50% more infectious than the old one.
Immunity is total if you have been infected.
Vaccinations don’t exist.
Control systems don’t exist, people don’t adjust behavior at all.
Heterogeneity doesn’t exist, people are all the same. Straight SIR.

That’s deeply silly, of course, but it’s not more deeply silly than models that have been offered and taken seriously in the past, despite being obvious nonsense. Assuming away vaccinations is the most obviously nonsensical part of that, but control systems and heterogeneity point in the other direction in many places, and our vaccinati...
[AN #132]: Complex and subtly incorrect arguments as an obstacle to debate
by Rohin Shah — 2021-01-06T18:20:05.694Z

[AN #132]: Complex and subtly incorrect arguments as an obstacle to debate

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world

View this email in your browser

Newsletter #132

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

SECTIONS

﻿HIGHLIGHTS
﻿TECHNICAL AI ALIGNMENT

...
Centrally planned war
by KatjaGrace — 2021-01-06T10:40:11.794Z

We watched Dunkirk, and wondered how many military deaths are for reasons more of logistics than of facing the enemy. Probably lots - we have heard that war is made of colossal logistical feats, so probably they often fail, and often lives depend on them.

(Imagine organizing a party with hundreds of thousands of people at it. Imagine that is located in an overseas country, where you don’t have a house, and everyone hates you. Imagine that it goes for several years. Imagine it is a very stressful party for the partygoers, but also you are counting on them to carry out some hard and terrifying tasks for you during the party. Imagine you anticipate many deaths during the proceedings.)

Which made me wonder, why is war so centrally planned? Why wouldn’t all these logistical details be simpler and cheaper in the usual ways if each soldier looked after himself mostly? Similar to how it works better for each person to look after themselves during peacetime, rather than having commanders organize and dictate the whole peaceful existence effort. Thoughts?
Linguistic Freedom
by Chris_Leong — 2021-01-06T07:23:23.991Z

One of the first things you notice when you start digging into the definitions of words is that they are much more ambiguous than they seem. Chairs are an excellent example as at first glance they don't seem ambiguous at all. But then, what if we ask if a tree stump is a chair if someone sits on it? This puzzle won't be a challenge for longtime readers of Less Wrong, as they should already understand that the map is not the territory. That is "chair" is a word created by humans and which exists for our purposes and for which we get to decide the convention. It's definition is not a fact written in the universe, waiting to be discovered.

I've started using the term Linguistic Freedom to refer to this use specifically, since just saying "The Map is Not the Territory" is somewhat ambiguous and also somewhat difficult to explain. On the first point, if you look at the post where Elizier describes the skill The Map Is Not the Territory, rather than talking about linguistic freedom, he talks about the potential of being wrong, epistemic humility and how beliefs are separate from reality. On the second, instead of just saying, "Words are created by humans for our purposes and so we get to decide if a treestump we sit on counts as a chair, it isn't written into the universe", while if we want to explain the "Map and Territory" we have to explain the map AND the territory AND then how it applies to language.

Anyway, all I mean by this is that we have the freedom to use language however we want, even if it is stupid. For example, we could use "chair" to mean "pineapples" and that would just be stupid rather than wrong. But what do we mean by stupid? One simple way of characterising this would be to note that the territory contains certain natural structures that cry out for a name and that a definition is "stupid" when the term was created in order to refer to some way of drawing boundaries around a particular natural structure, but we decide to ignore this.

Linguistic freedom is about conventions, not individual uses of language. It doesn't mean that I can start using "chair" to mean "pineapple" without telling anyone. It doesn't mean that if you ask "Do chairs typically have four legs?" that I can answer as though you asked about pineapples. I suppose we do have that kind of linguistic freedom to be stupid in that way as well, but that isn't the kind of freedom that I'...
Grokking illusionism
by Joe Carlsmith — 2021-01-06T05:50:57.598Z

(Cross-posted from Hands and Cities)

A number of people I know are illusionists about consciousness: that is, they think that the way consciousness seems to us involves some fundamental misrepresentation. On an extreme version of this view (which Frankish (2016) calls “strong illusionism”), phenomenal consciousness simply does not exist; it only seems to exist (I’ll say more about what I mean by phenomenal consciousness in a moment). I’m especially interested in this version.

For a long time, though, I’ve found it hard to really grok what it would be for strong illusionism to be true. I can repeat the words illusionists say; but I haven’t had a clear sense of the reality envisioned, such that I could really look at the world through the illusionist’s eyes. What’s more, I’ve suspected that some sort of barrier in this respect is crucial to the resistance that I (and I expect many others) feel to the view. Successfully imagining illusionism being true, I think, may be halfway to believing it.

(As a sidenote: I think this dynamic may be common. Actually looking at the world the way someone you disagree with looks at it is often much more difficult than being able to pass their “intellectual turing test” — e.g., to present their position in terms that they would endorse. As ever, words are easy; seeing the world in new ways is hard. And once you have seen the world in a new way, the possibility that the world actually is that way is much easier to take seriously.)

The aim of this post is to grok illusionism more fully. Let’s start with a few clarifications.

The philosophical debate about consciousness centers on “phenomenal consciousness,” which is generally thought of as the thing we ascribe to a system when we say that there is “something it’s like” to be that system, or when we ascribe to that system a first-person perspective or subjective experience. And experiences themselves — the taste of wine, the smell of leaves, the color of an afterimage in your visual field — are thought of as “phenomenally conscious” when there’s something it’s like to have them, and the “phenomenal properties” of experiences determine (consist in?) what it’s like to have them.

Phenomenal consciousness is often contrasted with “access consciousness,” understood as a property of the mental states that a subject can do certain things with (e.g., “access”) — in particular, notice them, repo...
Cryonics signup guide #1: Overview
by mingyuan — 2021-01-06T00:25:02.927Z

This is the introduction to a sequence on signing up for cryonics. In the coming posts I will lay out what you need to do, concretely and in detail. This sequence is intended for people who already think signing up for cryonics is a good idea but are putting it off because they're not sure what they actually need to do next. I am not going to address the question of how likely cryonics is to work – that's been

covered

extensively

elsewhere

.

If you have no idea what cryonics is, or if you want a thorough refresher, I recommend WaitButWhy's

Why Cryonics Makes Sense

.

Biases

This sequence is US-focused, since I went through the process in the US. It's also somewhat Alcor-biased, since I chose Alcor quite early on in the process. However, I've collaborated with both non-US cryonicists and people signed up with the Cryonics Institute, so I'm confident there will be useful information no matter where you are or which institution you choose to keep you in a vat of liquid nitrogen.

Epistemic status

I approached all questions in good faith and have documented my reasoning to the best of my ability, but I don't have a high level of confidence in my conclusions. Commenter Josh Jacobson is signed up with the Cryonics Institute and had a very different experience than the one outlined in this sequence, and I don't think I have any special knowledge or abilities that he doesn't. My recollections of the research that led to these posts has also faded with time.

Caveats

This sequence was researched and written in late 2020, and just two years later, it seems that the landscape has already changed significantly. For example, Alcor has changed their membership options, their fee structure, and their payment options, and they've also introduced an online signup flow that I have no experience with. As such, please be aware that some of the logistical advice in this sequence may be outdated. I have tried to update the sequence where possible, but I'm not going to go through and overhaul it.

Acknowledgements

Thanks to Connor Flexman, Daniel Filan, Gordon Worley, Mati Roy, Seraphina Nix, and nameless others for letting me ask them endless questions. Thanks also to Eli Tyre and Oge Nnadi for their previous writeups on this topic, from which I borrowed liberally.

Summary of the process

The first thing most people probably want to know is: What do I do now? It turns out to be r...
How is reinforcement learning possible in non-sentient agents?
by SomeoneKind — 2021-01-05T20:57:21.843Z

(Probably a stupid nooby question that won't help solve alignment)
Suppose you implement a goal in an AI through a reinforcement learning system. Why does the AI really "care" about this goal? Why does it obey? It does because it is punished and/or rewarded, which motivates it to achieve that goal.

Okay. So why does AI really care about punishment and reward in the first place? Why does it follows its implemented goal?

Sentient beings do because they feel pain and pleasure. They have no choice but to care about punishment and reward. They inevitably do it because they feel it. Assuming that our AI does not feel, what is the nature of its system of punishments and rewards? How is it possible to punish or reward a non-sentient agent?
My intuitive response would be "It is just physics. What we call 'reward' and 'punishment' are just elements of a program forcing an agent to do something", but I don't understand how this RL physics is different from that in our carbonic animal brains.
Do Artificial Reinforcement Learners Matter Morally, written by Brian Tomasik, makes the distinction even less obvious for me. What do I miss?
Genius is (mostly) comparative
by Amir Bolous — 2021-01-05T20:57:18.471Z

Introduction

Comparative vs. Absolute Advantage in Economics

Generalizing Comparative and Absolute Advantage

Closing Thoughts

Introduction

Some of the best advice I've received has been to turn short-term games into long-term games. In the past, I interpreted this as prioritize actions that further long-term goals over those that generate short-term rewards. After digging a bit deeper and thinking at a different level of abstraction, I now see this piece of advice in a different dimension.

I think that the core principal behind this piece of advice is that we are very bad at grasping the effect of compounding. From a young age, we are told that "we get out what we put in."

I think that while this is close to the truth, the quote would be better amended to "we get out what we put in scaled by some multiplier."

If Mozart and I spent 30 minutes composing a new piece of music, I'm sure that our final work would end up looking very different. The unfortunate byproduct of this is that we attribute what we currently see (in terms of progress, results, success, etc.) to absolute advantage over comparative advantage.

So jargon aside, what does that mean? Let's look at the conical example of comparative vs. absolute advantage in Economics.

Comparative vs. Absolute Advantage in Economics

Jane and Toby are two friends who just moved in together. Naturally, they want to minimize the amount of time spent on chores in order to maximize their amount of free time (constrained by each person's time being equally valuable). There are two main chores that need to be completed: cleaning the dishes and doing the washing. They want to figure out how best to split these two tasks so they decide to collect some initial data independently.

Cleaning Dishes

Jane: 10 minutes to clean 5 dishes (rate of 0.5 dishes/min)

Toby: 10 minutes to clean 2 dishes (rate of 0.2 dishes/min)

Doing the Washing

Jane: 10 minutes to wash 2 shirts (rate of 0.2 shirts/min)

Toby: 10 minutes to wash 1 shirt (rate of 0.1 shirts/min)

At first glance, we see that Jane is more efficient than Toby at completing both chores, so maybe Jane should do both chores to minimize the time spent on chores? Although this idea in principle may seem to work, Jane and Toby know that time is a scarce resource. This means that while Jane could complete more chores than Toby in the same amount of time , this is not the best alloc...
DALL-E by OpenAI
by Daniel Kokotajlo — 2021-01-05T20:05:46.718Z

This is a linkpost for https://openai.com/blog/dall-e/

My own take: Cool, not super surprising given GPT-3 and Image GPT. I look forward to seeing what a bigger version of this would do, so that we could get a sense of how much it improves with scale. I'm especially interested in the raven's progressive matrices performance.