trentmkelly/LessWrong-43k huggingface dataset:
Drowning children are rare
by Benquo — 2019-05-28T19:27:12.548Z
Link: http://benjaminrosshoffman.com/drowning-children-rare/

Stories such as Peter Singer's "drowning child" hypothetical frequently imply that there is a major funding gap for health interventions in poor countries, such that there is a moral imperative for people in rich-countries to give a large portion of their income to charity. There are simply not enough excess deaths for these claims to be plausible.

Much of this is a restatement of part of my series on GiveWell and the problem of partial funding, so if you read that carefully and in detail, this may not be new to you, but it's important enough to have its own concise post. This post has been edited after its initial publication for clarity and tone.

People still make the funding gap claim

In his 1997 essay The Drowning Child and the Expanding Circle, Peter Singer laid out the basic argument for a moral obligation to give much more than most to, for the good of poor foreigners:

To challenge my students to think about the ethics of what we owe to people in need, I ask them to imagine that their route to the university takes them past a shallow pond. One morning, I say to them, you notice a child has fallen in and appears to be drowning. To wade in and pull the child out would be easy but it will mean that you get your clothes wet and muddy, and by the time you go home and change you will have missed your first class.
I then ask the students: do you have any obligation to rescue the child? Unanimously, the students say they do. The importance of saving a child so far outweighs the cost of getting one’s clothes muddy and missing a class, that they refuse to consider it any kind of excuse for not saving the child. Does it make a difference, I ask, that there are other people walking past the pond who would equally be able to rescue the child but are not doing so? No, the students reply, the fact that others are not doing what they ought to do is no reason why I should not do what I ought to do.
Once we are all clear about our obligations to rescue the drowning child in front of us, I ask: would it make any difference if the child were far away, in another country perhaps, but similarly in danger of death, and equally within your means to save, at no great cost – and absolutely no danger – to yourself? Virtually all agree that distance and nationality make no moral difference to the situation. I then point out that we are all in that situation of the person passing the s

...
Evidence for Connection Theory
by Evan_Gaensbauer — 2019-05-28T17:06:52.635Z
Link: https://www.scribd.com/document/219774356/Evidence-for-Connection-Theory#fullscreen

Connection Theory (CT) is the original philosophy underpinning Leverage Research, a research think tank focused that has worked with the effective altruism movement, and the rationality community, in the past on community-building, and existential risk reduction. CT was developed by Leverage's executive director, Geoff Anders. Since there are few if any other publicly available online resources for understanding or evaluating CT, I thought I would share this document.
A shift in arguments for AI risk
by Richard_Ngo — 2019-05-28T13:47:36.486Z
Link: https://fragile-credences.github.io/prioritising-ai/

The linked post is work done by Tom Adamczewski while at FHI. I think this sort of expository and analytic work is very valuable, so I'm cross-posting it here (with his permission). Below is an extended summary; for the full document, see his linked blog post.

Many people now work on ensuring that advanced AI has beneficial consequences. But members of this community have made several quite different arguments for prioritising AI.

Early arguments, and in particular Superintelligence, identified the “alignment problem” as the key source of AI risk. In addition, the book relies on the assumption that superintelligent AI is likely to emerge through a discontinuous jump in the capabilities of an AI system, rather than through gradual progress. This assumption is crucial to the argument that a single AI system could gain a “decisive strategic advantage”, that the alignment problem cannot be solved through trial and error, and that there is likely to be a “treacherous turn”. Hence, the discontinuity assumption underlies the book’s conclusion that existential catastrophe is a likely outcome.

The argument in Superintelligence combines three features: (i) a focus on the alignment problem, (ii) the discontinuity assumption, and (iii) the resulting conclusion that an existential catastrophe is likely.

Arguments that abandon some of these features have recently become prominent. They also generally tend to have been made in less detail than the early arguments.

One line of argument, promoted by Paul Christiano and Katja Grace, drops the discontinuity assumption, but continues to view the alignment problem as the source of AI risk. Even under more gradual scenarios, they argue that, unless we solve the alignment problem before advanced AIs are widely deployed in the economy, these AIs will cause human values to eventually fade from prominence. They appear to be agonistic about whether these harms would warrant the label “existential risk”.

Moreover, others have proposed AI risks that are unrelated to the alignment problem. I discuss three of these: (i) the risk that AI might be misused, (ii) that it could make war between great powers more likely, and (iii) that it might lead to value erosion from competition. These arguments don’t crucially rely on a discontinuity, and the risks are rarely existential ...
What should rationalists think about the recent claims that air force pilots observed UFOs?
by avturchin — 2019-05-27T22:02:49.041Z

NY times has a new material about possible observations of UFOs by air force pilots. Should rationalists use their arsenal of new ideas about how the world works – simulation, superintelligent AIs etc – to explain these things, or should they be of more sceptical side and explain them via some combination of biases and hoaxes?

For example, one could speculate that an alien AI have terminated long before now, but some of its self-replicating robots colonised the universe. Or may be we live in a simulation which is full of glitches and viruses?

It is possible to create many hypothesis like these with infinite explanatory powers.
Was CFAR always intended to be a distinct organization from MIRI?
by Evan_Gaensbauer — 2019-05-27T16:58:05.216Z

When what would later become CFAR first started as a series of rationality workshops run under the purview of MIRI in 2012, was it always the intention of CFAR's founders to have it be a project/organization distinct from MIRI since its beginning? Or, did CFAR's founders decide to incorporate CFAR as an organization separate from MIRI after it became clear the rationality workshops weren't a fit project for MIRI itself later on?
Von Neumann’s critique of automata theory and logic in computer science
by Ben Pace — 2019-05-26T04:14:24.509Z

Quote from The General and Logical Theory of Automata. Corrected some typos using this version. H/T Hacker News.

There exists today a very elaborate system of formal logic, and, specifically, of logic as applied to mathematics. This is a discipline with many good sides, but also with certain serious weaknesses. This is not the occasion to enlarge upon the good sides, which I have certainly no intention to belittle. About the inadequacies, however, this may be said: Everybody who has worked in formal logic will confirm that it is one of the technically most refractory parts of mathematics. The reason for this is that it deals with rigid, all-or-none concepts, and has very little contact with the continuous concept of the real or of the complex number, that is, with mathematical analysis. Yet analysis is the technically most successful and best-elaborated part of mathematics. Thus formal logic is, by the nature of its approach, cut off from the best cultivated portions of mathematics, and forced onto the most difficult part of the mathematical terrain, into combinatorics.

The theory of automata, of the digital, all-or-none type, as discussed up to now, is certainly a chapter in formal logic. It would, therefore, seem that it will have to share this unattractive property of formal logic. It will have to be, from the mathematical point of view, combinatorial rather than analytical.

Probable Characteristics of Such a Theory. Now it seems to me that this will in fact not be the case. In studying the functioning of automata, it is clearly necessary to pay attention to a circumstance which has never before made its appearance in formal logic.

Throughout all modern logic, the only thing that is important is whether a result can be achieved in a finite number of elementary steps or not. The size of the number of steps which are required, on the other hand, is hardly ever a concern of formal logic. Any finite sequence of correct steps is, as a matter of principle, as good as any other. It is a matter of no consequence whether the number is small or large, or even so large that it couldn’t possibly be carried out in a lifetime, or in the presumptive lifetime of the stellar universe as we know it. In dealing with automata, this statement must he significantly modified. In the case of an automaton the thing which matters is not only whether it can reach a certain result in a

...
Is AI safety doomed in the long term?
by JakeH — 2019-05-26T02:30:12.010Z

Are there any measures that humanity can put in place to control a vastly (and increasingly) more intelligent race?

On the basis that humans determine the fate of other species on the planet, I cannot find any reasons for believing that a lesser intelligence can control a greater intelligence.
Which leads me to think that AI safety is at most about controlling the development of AI until it makes, and can implement, its own decisions about the fate of humanity.

Is this a common stance and I am naively catching up?
Or what are the counter arguments?
Micro feedback loops and learning
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2019-05-26T00:50:36.202Z

Tldr: some not-particularly-ordered thoughts about how learning works in humans, via the example of singing.

A high-tech feedback loop

I recently discovered Singer’sStudio, an iPhone app for voice training that is approximately the Best Thing Ever (h/t Raemon). It’s a work of pedagogical art, and describing what it does right pulls together various nebulous thoughts I’ve had about learning and developing intuitions for a particular domain.

How the app works: it gives me various singing exercises, and then tells me in real-time, via a pretty graph, if I’m singing on pitch. That’s it.

Okay, it does have a few other features, like:

A total score and breakdown by % of notes correct after each exercise.
Letting me play myself back to see what being on vs slightly off pitch sounds like.
Helpful text prompts such as “keep your tongue behind your teeth.”
A built-in progression from easier to more complicated exercises.
Exercises for a specific skill, like switching from head to chest voice.
The ability to add challenge by turning down/off the piano playing my notes as I sing them.
Not a built-in part of the app, but I can glance away from the screen and sing without the visual feedback, and then immediately check how I did.

But I’m pretty confident that the pitch contour is the most important part, and an app that only did that thing would be almost as good on the skill-building side.

There are obviously a lot of singing sub-skills other than hitting notes, like vocal timbre, head versus chest voice, breathing from the diaphragm, enunciating consonants clearly, and singing vowels that don’t sound weird. A voice teacher would be able to comment on these directly.

Still, in practice the app ends up causing me to improve on a bunch of these sub-skills as well, to the extent that getting them wrong results in singing off-pitch. The instantaneous feedback is doing a lot of work here – if the app only gave me a score at the end of each exercise, I predict I would end up quite stuck on what to change up in order to do better. However, I’ve ended up e.g. remembering to breathe from my diaphragm, because not doing that reliably resulted in my voice being wobbly, which was obvious on my pitch contour. (It also has instructions for breathing exercises as part of each warm-up, but I only actually did those exercises once or twice; they are boring compared to singing and getting pretty...
Say Wrong Things
by Gordon Seidoh Worley — 2019-05-24T22:11:35.227Z

There are many ways you might approach being less wrong.

A popular one is to make fewer wrong statements; to say fewer wrong things.

Naively it would seem this is a recipe for success, since you just say more things that are true and right and fewer things that are false and wrong. But if Goodhart has anything to say about it, and he does, you'll find ways to maximize the measure at the expense of the original objective.

Assuming the real objective is something like "have a more complete, precise, and accurate model of the world that better predicts the outcome of subjectively unknown events", then we can quickly see the many ways Goodharting can lead us astray if we focus too much on appearing less wrong. We might:

make fewer claims than we could, pulling us away from completeness even as we appear less wrong;
make weaker claims than we could, pulling us away from precision;
and, a perennial favorite, filter the claims we publicly make so we appear less wrong than we really are by hiding our least confident claims.

The first two can be corrected with better calibration, that is by making statements with confidence intervals or likelihoods that proportionally match the observed frequency of correctness of similarly confident claims. But simply suggesting someone "be better calibrated" is not a motion they can make; it's an outcome of taking actions towards increasing calibration. As good a place to start as any for improving calibration is the forecasting literature, if that's what you'd like to do.

The third is more tricky, though, because it's less directly about claims being made and their probability of correctness and more about social dynamics and how you appear to other people. And for that reason it's what I want to focus on here.

Appearing Less Wrong

I've met a lot of people in my life who are experts at not looking as stupid as they are.

That's kind of harsh. Maybe a nicer way to say it is that they are experts at appearing to be better at making correct predictions about the world than they actually are.

Some of their techniques are just normal social tricks: projecting confidence, using social status, the ever-abused term "gaslighting", and other methods of getting people to believe they are right even when a more careful examination would reveal them to be mistaken. These are people we a...
Laws of John Wick
by Zvi — 2019-05-24T15:20:00.322Z

Spoiler Alert: This spoils central plot points of John Wick, John Wick 2 and John Wick 3.

Should You See Those Movies: Yes. They’re awesome. Unless you do not like violence, especially gun violence, in which case they’re not for you.

John Wick exists in a special universe. Its criminal world has a unique economy and set of norms, laws and actions. Come for the stylized violence. Stay for the world building.

John Wick 2 presents one version of this universe. John Wick 3 takes place directly after, and engages directly with the events of John Wick 2, but presents an importantly different version of that universe. I want to explore that difference.

Beyond this point, the post will assume you’ve seen all three movies, and spoilers will be both massive and unmarked.

Rule of Law

The fundamental divide is between rule of man and rule of law.

Under rule of law, the rules are in charge. Some actions are off-limits, enforcing agreements, and offering protection for life, liberty and property. They determine how the game works and create a playing field. When murder for hire is your core business, you need a different playing field than others do. Thus it is a set of rules very different from ours.

John Wick 2 lives in this world. A central system tracks bounties, and an insanely strong enforcement system punishes those who break the laws. There are two such laws, which are above all.

No business (aka spilling blood) on company grounds (aka The Continental Hotel and a few other protected locations). If you break this rule, you die.

Every marker is fulfilled. If you give someone a marker, they can ask you for any favor up to and including killing the target of their choice, no matter who it is. You have to do it. If you ignore the marker, you die. If you run, you die. If you kill the owner of the marker, you die.

Murder is legal in this world.

Markers are above everything except that first rule. If your target is permanently hiding on company grounds, I don’t know what happens, but presumably you are off the hook until they leave.

If you don’t like the way someone calls in your marker, then the moment you have done what they ask, you can turn around and kill them. But not before. They can try and kill you first. Anyone who wants to can get involved, or get hired to get involved, or not. Same holds true for other conflicts.

Almost everyone has strong norms of civility, hospitality,...
Separation of Concerns
by abramdemski — 2019-05-23T21:47:23.802Z

Separation of concerns is a principle in computer science which says that distinct concerns should be addressed by distinct subsystems, so that you can optimize for them separately. We can also apply the idea in many other places, including human rationality. This idea has been written about before. I'm not trying to make a comprehensive post about it, just remark on some things I recently though about.

Epistemic vs Instrumental

The most obvious example is beliefs vs desires. Although the distinction may not be a perfect separation-of-concerns in practice (or even in principle), at least I can say this:

Even non-rationalists find it useful to make a relatively firm distinction between what is true and what they want to be true;
Rationalists, scientists, and intellectuals of many varieties tend to value an especially sharp distinction of this kind.

I'm particularly thinking about how the distinction is used in conversation. If an especially sharp distinction isn't being made, you might see things like:

Alice makes a factual statement, but the statement has (intended or unintended) conversational implicature which is perceived as negative by most of the people present. Alice is chastised and concedes the point, withdrawing her assertion.
Bob mentions a negative consequence of a proposed law. Everyone listening perceives Bob to be arguing against the law.

Notice that this isn't an easy distinction to make. It isn't right at all to just ignore conversational implicature. You should not only make literal statements, nor should you just assume that everyone else is doing that. The skill is more like, raise the literal content of words as a hypothesis; make a distinction in your mind between what is said and anything else which may have been meant.

Side note -- as with many conversation norms, the distinctions I'm mentioning in this post cannot be imposed on a conversation unilaterally. Sometimes simply pointing out a distinction works; but generally, one has to meet a conversation where it's at, and only gently try to pull it to a better place. If you're in a discussion which is strongly failing to make a true-vs-useful distinction, simply pointing out examples of the problem will very likely be taken as an attack, making the problem worse.

Making a distinction between epistemics and instrumentality seems like a kind of "universal solvent" for cognitive separation of concer...
Why the empirical results of the Traveller’s Dilemma deviate strongly away from the Nash Equilibrium and seems to be close to the social optimum?  
by RocksBasil — 2019-05-23T12:36:47.564Z

Epistemic status: I think I'm over complicating the matter.

In the Traveller’s Dilemma (call it TD below for short), theoretically the only Nash Equilibrium is to have both players (I'll call them Alice and Bob) reasoning to give the lowest bid (Thanks Stuart_Armstrong for letting me notice this): starting with bidding 100 dollars, Alice would realise she can gain more by claiming 99, so Bob’s best choice is to claim 98, but Alice would also know this and claim 97 and so on…until the race to the bottom finishes at the lowest claim of $2.

Empirically this doesn’t seem to happen, both players would likely be cooperative and bid high. Which for me seems rather bizarre for a single round simultaneous game.

Notice that TD is very similar to the dollar auction/war of attrition: In the dollar auction, both players pay for their bids, with the higher bidding player receiving the auctioned dollar.

We can slightly modify the two-player dollar auction to make it even more similar to TD: the player with the lower bid would also have to pay for the same bid as the winner.

This modified dollar auction has the same payoff rules as a TD with the range of claims being (-infinity,0]

The only difference is the bidding process: in TD, both players choose their claims simultaneously, while in DA the two players engage in multiple rounds of competing bids.

Given the difference between TD and I believe there is something wrong with the assumed reasoning that we use to derive the Nash Equilibrium of TD. If Alice believes that Bob is fully rational, she would not believe that Bob would follow this line of reasoning in his own head and give a claim of $2.

Imagine that Alice and Bob are allowed to communicate before choosing their claims, but they only discussed their claims in approximate terms (eg: would it be a high claim close to $100? A low claim close to $2? Somewhere in between?)

Would a rational Alice want to make Bob convinced that she would give a low claim, or would she want to convince Bob that she would give a high claim?

If Alice convinced Bob that she would give a low claim, Bob’s best response is to give a low claim. Knowing this, Alice would give a low claim and both Alice and Bob will receive a low payoff.

While if Alice convinced Bob that she would give a high claim, Bob’s best response is to give a high claim. Knowing Bob will give a high claim, Alice wo...