trentmkelly/LessWrong-43k huggingface dataset:
Notes on Ambition
by David Gross — 2020-11-22T19:53:28.925Z

This post examines the virtue of ambition. It is meant mostly as an exploration of what other people have learned about this virtue, rather than as me expressing my own opinions about it, though I’ve been selective about what I found interesting or credible, according to my own inclinations. I wrote this not as an expert on the topic, but as someone who wants to learn more about it. I hope it will be helpful to people who want to know more about this virtue and how to nurture it.

Is ambition a virtue or a vice?

The noble Brutus
Hath told you Caesar was ambitious:
If it were so, it was a grievous fault.[1]

“Ambition” has undergone a shift of meaning over time. Today, ambition is often associated with positive things like having lofty goals, drive, initiative, aspiration, being a hard worker, not settling for mediocrity, and that sort of thing.

But it wasn’t that long ago that “ambition” was almost always a bad word that described a vice. It was more associated with a blinkered and ruthless pursuit of power, influence, and position. Lady MacBeth might be thought of as the poster child for this sort of ambition.

So, as was the case with the virtue of prudence, which also had a shift of meaning over time, we need to be especially cautious when we read about the virtue (or vice) of ambition that we understand what the author had in mind.

Aristotle ran into a similar problem when he tried to identify the “golden mean” concerning ambition (φιλότιμος) or lack of ambition (ἀφιλότιμος).[2] In Greek, both of those words had either good or bad connotations depending on context. If someone was being ambitious to an unseemly extent, you might compare them unfavorably to a properly unambitious person. But if a person failed to set their sights high, you could also chastise them for not being ambitious. Aristotle complained that “as there is no recognized term for the observance of the mean, the extremes fight, so to speak, for what seems an empty place.”[2]

Several years ago, Miranda Dixon-Luinenburg (@Swimmer963) shared a couple of insightful posts about how she wrestled with ambition. She noted that the ambition-is-evil sense of ambition can discourage people from developing the good kind of ambition:[3]

I can’t trace the roots of this idea completely, but for whatever reason, I spent a long time thinking that being ambitious was in some way immoral. That really good people

...
(Pseudo) Mathematical Realism Bad?
by Logan Zoellner — 2020-11-22T18:21:30.831Z

I am writing a novel in which the protagonist meets a visitor from a "dimension" where mathematical realism is literally true. Namely, anything that the visitor can logically conceive of he can also "will" into existence.

Naturally at first the protagonist is pretty jealous of the visitor. After all, who wouldn't want to be able to summon a steaming hot burrito whenever they feel hungry. But now I need the visitor to convince the protagonist that his life isn't so bad after all.

So I am collecting a list of reasons why mathematical realism might be bad. For the sake of this article, assume that there is a society of "freely interacting" beings in the mathematical realism world, so you can only "wish" things into existence in my world if I allow you to. Or we could create a shared world where things can only be created according to a shared set of mutually agreed upon rules.

Here are some of the "bad" things I've come up with so far, but I'd like to collect more.

Eldritch abomination

One obvious downside of being able to wish anything into existence is that you might (intentionally or not) wish something really awful into existence.

Wireheading

A favorite topic of rationalists. If you can wish for anything, there's a real chance of repeatedly wishing for things that gratify your short-term reward system. This results in either an endless loop of obsessively doing the same behaviors over and over again, or "burning out" and being unable to take pleasure from those types of rewards.

Mental Poverty

The real world is fascinating and filled with a seemingly endless set of things to explore or discover. Living in a world where anything can be created by thinking also means living in a world where the only things that exist are those that you can imagine. A sufficiently clever person might get around this by (for example) simulating the big-bang and recreating the entire universe. But someone who isn't sufficiently clever might just find themselves surrounded by the small set of things they can easily imagine, unable to ever experience surprise.

Lack of challenge

Similar to playing a game with all of the cheat-codes, living in a world where you can solve any problem simply by wishing might feel dull and unrewarding. Again, a sufficiently clever person can simply create restrictions and challenges for them...
My intellectual influences
by Richard_Ngo — 2020-11-22T18:00:04.648Z

Prompted by a friend's question about my reading history, I've been thinking about what shaped the worldview I have today. This has been a productive exercise, which I recommend to others. Although I worry that some of what's written below is post-hoc confabulation, at the very least it's forced me to pin down what I think I learned from each of the sources listed, which I expect will help me track how my views change from here on. This blog post focuses on non-fiction books (and some other writing); I've also written a blog post on how fiction has influenced me.

My first strong intellectual influence was Eliezer Yudkowsky’s writings on Less Wrong (now collected in Rationality: from AI to Zombies). I still agree with many of his core claims, but don’t buy into the overarching narratives as much. In particular, the idea of “rationality” doesn’t play a big role in my worldview any more. Instead I focus on specific habits and tools for thinking well (as in Superforecasters), and creating communities with productive epistemic standards (a focus of less rationalist accounts of reason and science, e.g. The Enigma of Reason and The Structure of Scientific Revolutions).

Two other strong influences around that time were Scott Alexander’s writings on tribalism in politics, and Robin Hanson’s work on signalling (particularly Elephant in the Brain), both of which are now foundational to my worldview. Both are loosely grounded in evolutionary psychology, although not reliant on it. More generally, even if I’m suspicious of many individual claims from evolutionary psychology, the idea that humans are continuous with animals is central to my worldview (see Darwin’s Unfinished Symphony and Are We Smart Enough to Know How Smart Animals Are?). In particular, it has shaped my views on naturalistic ethics (via a variety of sources, with Wright’s The Moral Animal being perhaps the most central).

Another big worldview question is: how does the world actually change? At one point I bought into techno-economic determinism about history, based on reading big-picture books like Guns, Germs and Steel and The Silk Roads, and also because of my understanding of the history of science (e.g. the prevalence of multiple discovery). Sandel’s What Money Can’t Buy nudged me towards thinking more about cultural factors; so did books like The Dream Machine and The Idea Factory, which describe ...
Ways to run multiple form-incompatible mentalities?
by Self_Optimization — 2020-11-22T17:57:55.367Z

I’ve been looking into intuition pumps recently, and I realized the MBTI system (when combined with the associated ‘function theory’) has some very general, relatively useful ones, such as the INTP (TiNe) and INTJ (NiTe) ones.

However, the mental structures each of these describe are mutually incompatible in terms of information representation and processing methods. I haven’t yet found any way to possess such mutually incompatible systems at the same time (even disregarding the infeasibility of true multitasking, simply having them both as active components in my mind, rather than inactive memories only accessible when specifically called upon, is proving difficult)

Does anyone have tips on how to keep such mentalities both available for active use despite their incompatibility?
It’s not economically inefficient for a UBI to reduce recipient’s employment
by paulfchristiano — 2020-11-22T16:40:05.531Z

A UBI (e.g. paying every adult American $8k/year) would reduce recipient’s need for money and so may reduce their incentive to work. This is frequently offered as an argument against a UBI (or as an argument for alternative policies like the EITC that directly incentivize work).

This argument is sometimes presented as economically hard-headed realism. But as far as I can tell, there’s not really any efficiency argument here—there’s nothing particularly efficient about people having a stronger incentive to work because they are poorer. The argument seems to mostly get its punch from a vague sense that work is virtuous and necessary. I think that sense is largely mistaken and it should be taken less seriously.

(As usual for policy posts, I’m not very confident about any of this; if it makes you happier feel free to imagine sprinkling “seems to me” throughout the prose.)

What people fear

If I give you $8k, you will probably value marginal dollars less than you used to. Some jobs that used to be a good deal will stop being worth it, and a job search itself may stop being worthwhile. We could study how much this happens empirically, but it’s definitely plausible on paper and it would be my best guess for the long-run effect even if pilot UBI experiments found otherwise.

This seems to be one of the main worries people have about a UBI. For example, some folks at McKinsey write:

In the design of the Finnish experiment, the main research question, agreed to by parliament in the enabling legislation, was the impact of a basic income on employment. Many policy makers assume that an entirely unconditional guaranteed income would reduce incentives to work. After all, the argument goes, why bother with a job if you can have a decent life without one? This assumption has led many countries to deploy active labor-market policies that require people on unemployment benefits to prove their eligibility continually and, often, to participate in some kind of training or to accept jobs offered to them.

[They find that the UBI didn’t decrease work, but this post isn’t even going to get into that.]

But it’s left somewhat vague what exactly is so bad about this. In fact it’s hard to respond to this concern because, although I’ve seen it expressed so many times, I’ve never really seen the argument laid out clearly.

It’s clear that work creates much of the great stuff in our society, and...
Why are young, healthy people eager to take the Covid-19 vaccine?
by  — 2020-11-22T15:24:55.617Z

Clearly a lot of people on LW want to take it ASAP. I strongly don't want that - to the point where I will most likely emigrate if it becomes obligatory in my country. Please help me understand what I'm missing. Here is my understanding:

As a young, healthy person, SARS-Cov-2 poses extremely low risk to me:

There is no significant risk of lasting negative health consequences after infection

There is no strong proof for such effect. Such proof would greatly increase acceptance of governments' policies, so there is a strong incentive to publish any such proof. There has also been enough time and cases to identify a pattern of negative consequences lasting 6+ months. Therefore I'm treating absence of a proof despite strong incentives and opportunity as a strong proof of absence.

A pessimistic infection fatality rate is probably around 0.01%

Case fatality rate for young people is below 0.5%
Halve that, since half of the infections do not result in the disease.
Divide that by 25 and stockpile large quantities of vitamin D in case you get the disease (https://chrismasterjohnphd.com/covid-19/finally-confirmed-vitamin-d-nearly-abolishes-icu-risk-in-covid-19)

Reinfection is extremely rare, if at all possible.

Again, there is a strong incentive to make people fear a reinfection.
Yet all we hear is rare individual reports that might be test failures or long-lasting lingering infections.

The risk of infection is getting smaller and smaller, as more people in the population become immune - either by infection or by vaccination

Conversely, there is a non-negligible risk associated with a vaccine that has been developed so quickly.

The trials have lasted only months, so we don't know whether there are some side-effects that surface only after some significant time
The trials have only been conducted on tens of thousands of subjects so far, so very severe but rare negative consequences might have gone under the radar
Pfizer's vaccine requires extremely low temperatures, so there is a danger that in some locations it will be transported or stored incorrectly, causing greater risk than that suggested by the trials so far
Both the governments and the vaccine manufacturers have twisted incentives, meaning there is a serious danger of too optimistic reports of the vaccines' efficacy and safety

I can only think of two reasons why y...
Demystifying the Second Law of Thermodynamics
by Eigil Rischel — 2020-11-22T13:52:17.280Z

(Crossposted from my personal website)

Thermodynamics is really weird. Most people have probably encountered a bad explanation of the basics at some point in school, but probably don't remember more than

Energy is conserved

Entropy increases

There's something called the ideal gas law/ideal gas equation.

Energy conservation is not very mysterious. Apart from some weirdness around defining energy in general, it's just a thing you can prove from whatever laws of motion you're using.

But entropy is very weird. You've heard that it measures "disorder" in some vague sense.
Maybe you've heard that it's connected to the Shannon entropy of a probability distribution H(p)=∑_x−p(x)lnp(x).
Probably the weirdest thing about it is the law it obeys: It's not conserved, but rather it increases with time.
This is more or less the only law like that in physics.

It gets even weirder when you consider that at least classical Newtonian physics is time-symmetric.
Roughly speaking, this means if you have a movie of things interacting under the laws of Newton, and you play it backwards, they're still obeying the laws of Newton. An orbiting moon just looks like it's orbiting in the other direction, which is perfectly consistent. A stone which is falling towards earth and accelerating looks like it's flying away from earth and decelerating - exactly as gravity is supposed to do.

But if there's some "entropy" quality out there that only increases, then that's obviously impossible! When you played the movie backwards, you'd be able to tell that entropy was decreasing, and if entropy always increases, some law is being violated.
So what, is entropy some artefact of quantum mechanics? No, as it turns out. Entropy is an artefact of the fact that you can't measure all the particles in the universe at once. And the fact that it seems to always increase is a consequence of the fact that matter is stable at large scales.

The points in this post are largely from E.T. Jaynes' Macroscopic Prediction.

A proof that entropy doesn't always increase

Let X be the set of states of some physical system. Here I will assume that there is a finite number of states and time advances in discrete steps - there is some function T:X→X which steps time forward one step. We assume that these dynamics are time-reversible in the weak sense that T is a bijection - every state is the future of exactly one "past" state.
...
How to think about wearing masks / distancing within the household?
by Sameerishere — 2020-11-22T06:56:36.805Z

Folks who live with or have a pod that extends beyond your romantic partner: how do you think about wearing masks and distancing with other folks indoors? My understanding is that most people with a pod that goes beyond a romantic partner hang out with their pod indoors, without distancing or wearing masks.

Is that true? If so, is it because you have reason to believe masks and distancing are ineffective indoors? Or do you think masks / distancing would help, but not enough to be worth it?

From a quick search, I was only able to find one study on the impact of masks and distancing within a household (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7264640/). It shows that both make a significant impact, but this is surprising to me given I'd expect aerosols to concentrate indoors such that with prolonged exposure, you'd breathe in the virus anyway unless you are wearing a highly protective mask.
UDT might not pay a Counterfactual Mugger
by winwonce — 2020-11-21T23:27:06.853Z

The Counterfactual Mugging is my favorite decision theory problem, and it's the problem that got me started reading LessWrong in the first place. In short,

Imagine that one day, Omega comes to you and says that it has just tossed a fair coin, and given that the coin came up tails, it decided to ask you to give it $100. Whatever you do in this situation, nothing else will happen differently in reality as a result. Naturally you don't want to give up your $100. But see, Omega tells you that if the coin came up heads instead of tails, it'd give you $10000, but only if you'd agree to give it $100 if the coin came up tails.

Since hearing the problem for the first time, I have flip-flopped on what should be done many times. I won't go into the details, but the general consensus on this forum (as far as I can tell) is that you should pay the $100 and that UDT tells you to pay the $100.

While I admit I found some of these arguments (especially MBlume's) quite persuasive, and my position for a good while was that one should pay, I still had this intuition in the back of my mind telling me that rational agents should win. Giving Omega $100 for no real gain sure doesn't sound like winning to me, and if UDT says to pay the $100, that means that UDT is wrong, not that we should change our preferences to including paying the $100 in this scenario.

But there is a third option, one that allows you to save your $100 while still following UDT: show that UDT doesn't tell you to pay.

When the Counterfactual Mugging is usually presented, it would appear that there are two possible scenarios, each with probability 0.5: Omega exists and the coin landed heads, and Omega exists and the coin landed tails. Thus, by UDT, we would want to precommit to paying should the coin land tails, and so when the coin lands tails, we pay.

However, those are not the only two scenarios. Before we learn about the counterfactual mugging, there is a third option: Nomega exists, a being who will pay $10000 to anyone who doesn't pay Omega when counterfactually mugged, and gives no money to someone who does. Our new view of the world:

Scenario	Probability	U(Precommit)	U(Don't Preccomit)
Omega, Heads	0.25	10000	0
Omega, Tails	0.25	-100	0
Nomega	0.5	0	10000
Expected Value	2475	5000

Thus a rational agent running UDT should NOT precommit to paying a counterfactual mugger. Once we learn that we live in a universe where Omega, rath...
Non-Obstruction: A Simple Concept Motivating Corrigibility
by TurnTrout — 2020-11-21T19:35:40.445Z

Thanks to Mathias Bonde, Tiffany Cai, Ryan Carey, Michael Cohen, Joe Collman, Andrew Critch, Abram Demski, Michael Dennis, Thomas Gilbert, Matthew Graves, Koen Holtman, Evan Hubinger, Victoria Krakovna, Amanda Ngo, Rohin Shah, Adam Shimi, Logan Smith, and Mark Xu for their thoughts.

Main claim: corrigibility’s benefits can be mathematically represented as a counterfactual form of alignment.

Overview: I’m going to talk about a unified mathematical frame I have for understanding corrigibility’s benefits, what it “is”, and what it isn’t. This frame is precisely understood by graphing the human overseer’s ability to achieve various goals (their

attainable utility (AU) landscape

). I argue that corrigibility’s benefits are secretly a form of counterfactual alignment (alignment with a set of goals the human may want to pursue).

A counterfactually aligned agent doesn't have to let us literally correct it. Rather, this frame theoretically motivates why we might want corrigibility anyways. This frame also motivates other AI alignment subproblems, such as intent alignment, mild optimization, and low impact.

Nomenclature

Corrigibility goes by a lot of concepts: “

not incentivized to stop us from shutting it off

”, “

wants to account for its own flaws

”, “doesn’t take away much power from us”, etc. Named by Robert Miles, the word ‘corrigibility’ means “able to be corrected [by humans]." I’m going to argue that these are correlates of a key thing we plausibly actually want from the agent design, which seems conceptually simple.

In this post, I take the following common-language definitions:

Corrigibility: the AI literally lets us correct it (modify its policy), and it doesn't manipulate us either.

Without both of these conditions, the AI's behavior isn't sufficiently constrained for the concept to be useful. Being able to correct it is small comfort if it manipulates us into making the modifications it wants. An AI which is only non-manipulative doesn't have to give us the chance to correct it or shut it down.

Impact alignment: the AI’s actual impact is aligned with what we want. Deploying the AI actually makes good things happen.
Intent alignment: the AI makes an honest effort to figure out what we want and to make good things happen.

I think that these definitions follow what their words mean, and that the alignment community should use these (or other clear groundings) in gener...
Rationalists from the UK -- what are your thoughts on Dominic Cummings?
by Davis_Kingsley — 2020-11-21T10:00:27.762Z

Dominic Cummings would I believe be considered at least rationalist-adjacent, and has recently been quite a prominent figure in the news. I'm curious to hear your thoughts both on the situation with him in general and also on how this has affected public perceptions of rationality/LessWrong in your area. Has this been positive? Negative? Not something people have much linked to the community?