trentmkelly/LessWrong-43k huggingface dataset:
Ben Hoffman's donor recommendations
by Rob Bensinger — 2018-06-21T16:02:45.679Z

A conversation I had on Facebook last month, saved here so I can link back to it:

Anonymous: I want to give away some money. Who should I give it to? [...]

Note that I am familiar with the effective altruism movement and with givewell.org. (I think “GiveWell’s top charities” might be the right answer, and I am even curious how many people reading this would say that - I just don’t want to get comments starting with “Have you heard of…”.)

[...]

Buck Shlegeris: I think donor lotteries probably have higher EV than anything else you can do from your current epistemic state.

[...]

Rob Bensinger: I came here to recommend donor lotteries, but Buck already did it. I think most people should donate via donor lotteries. If you win the lottery and don't know where else to donate, the EA Funds are usually a better fallback option than GiveWell.

Ben Hoffman: Agreed on donor lotteries but not on EA funds.

Rob: Oh, interesting, why do you think GiveWell beats EA Funds?

Ben: They seem pretty similar in prospect, given the extent to which they seem to funge against each other (especially when you notice the level of coordination between GiveWell, Good Ventures, and Open Philanthropy Project), and the strong overlap in management. Basically it seems a bit odd to spend attention distinguishing them, vs spending effort distinguishing substantially different strategies.

Ben: Here's my explanation of donor lotteries, which links to a couple others: http://benjaminrosshoffman.com/claim-explainer-returns-to-scale/

[...]

Ben: I would recommend either gifting it directly to individuals you think have done good work in the world, without restrictions, or reaching out to them directly and asking them where they think you should give it.

Facebook queries like these will select for charities that are memetically fit answers to "where should I give?". If you don't have time to track this sort of thing, then just keep rolling it over into donor lotteries until/unless you "win" enough money to make it worth your time to track it.

Ben: Another decent option: giving unsolicited monetary gifts to friends or acquaintances (people known to you personally, not via mass media) who seem like they're basically benevolent and competent, have high marginal value for money, and aren't asking for any.

Ben: Anyhow, the reason I'm not suggesting any specific charities sh...
Does anyone use advanced media projects?
by ryan_b — 2018-06-20T23:33:45.405Z

Epistemic status: exploratory.

I am using "advanced media projects" as sort of a catch-all umbrella for people working on better ways to present and explore information. I suppose the obvious hub for this kind of thing is places like the MIT Media Lab, but here are a few others I have been thinking about:

Historical

Alan Kay gave a couple of talks at Stanford about the time at PARC that I like. The title is How To Invent the Future, Part I and Part II. He talks a lot about the importance of tools and the under-utilization of computers for thinking purposes. I went to school for engineering, and I would particularly like to call out Ivan Sutherland's demonstration at ~26:30 in Part I because when people were still using punch-cards and tape readouts this guy threw together the first computer aided drafting program as a demo for a monitor and a light-pen and my life would have been much easier at several junctures if only we could have rolled back to the 60s. My mind remains blown.

Current

I just discovered Bret Victor, and his blog has a section in it called Kill Math. I am watching the presentation called Media for Thinking the Unthinkable, which seems directly relevant to the problems this community is largely concerned with. His central argument is that all of our progress hinges on people with a knack for manipulating abstract symbols with pencil and paper; this both loses value that might be added by people without such talents, and fails to leverage abstract symbol skills in more than two dimensions.

In the Wild

This is exactly the pitch of Distill, which I discovered from LessWrong. In fact, the demos for Distill look very similar to the first network theory paper example in the Media for Thinking the Unthinkable talk...and following the last link of their examples is that very talk.

But the question is, does anyone here actually use or produce such tools?

It seems like superintelligence is the kind of motivating example which requires the ability to think unthinkable thoughts, but I haven't seen very much discussion about how to do so. Secondarily, we have a strong interest in the pedagogical possibilities from a rationality perspective. What if we could flip the former around, and build a tool to render previously thinkable-but-wrong thoughts unthinkable, like a kind of rationality filter?
Anthropics and Fermi
by Stuart_Armstrong — 2018-06-20T13:04:59.080Z

tl;dr There is no well-defined "probability" of intelligent life in the universe. Instead, we can use proper scoring functions to penalise bad probability estimates. If we average scores across all existent observers, we get SSA-style probability estimates; if we sum them, we get SIA-style ones.

When presenting "anthropic decision theory" (the anthropic variant of UDT/FDT), I often get the response "well, that's all interesting, but when we look out to the stars with telescopes, probes, what do we really expect to see?" And it doesn't quite answer the question to say that "really expect to see is incoherent".

So instead of evading the question, let's try and answer it.

Proper scoring rules

Giving the best guess about the probability of X, is the same as maximising a proper scoring rule conditional on X. For example, someone can be asked to name a 0≤p≤1, and they will get a reward of −(IX−p)2, where IX is the indicator variable that which is 1 if X happens and 0 if it doesn't.

Using a logarithmic proper scoring rule, Wei Dai demonstrated that an updatless agent can behave like an updating one.

So let's apply the proper scoring rule to the probability that there is an alien civilization in our galaxy. As above, you guess p, and are given −(1−p)2 if there is an alien civilization in our galaxy, and −p2 if there isn't.

Summing over different agents

But how do we combine estimates from different agents? If you're merely talking about probability - there are several futures you could experience, and you don't know which yet - then you simply take an expectation over these.

But what about duplication, which is not the same as probability? What if there are two identical versions of you in the universe, but you expect them to diverge soon, and maybe one will find aliens in their galaxy while the other will not?

One solution is to treat duplication as probability. If your two copies diverge, that's exactly the same as if there was a 50-50 split into possible futures. In this case, the total score is the average of all scores in this one universe. In that case, one should use SSA-style probability, and update one's estimates using that.

Or we could treat duplication as separate entities, and ensure that as many as possible are as correct as ...
Duplication versus probability
by Stuart_Armstrong — 2018-06-20T12:18:29.459Z

Suppose you're rushing an urgent message back to the general of your army, and you fall into a deep hole. Down here, conveniently, there's a lever that can create a duplicate of you outside the hole. You can also break open the lever and use the wiring as ropes to climb to the top. You estimate that the second course of action has a 50% chance of success. What do you do?

Obviously, if the message is your top priority, you pull the lever, and your duplicate will deliver that message. This succeeds all the time, while the wire-rope only has 50% chance of working.

The point of this is that duplication is not necessarily like probability splitting. A 50% chance of being either inside or outside the hole, is not the same thing as a certainty of being both.

Now, some selfish, indexical utility functions will treat those two cases as the same, but as we've shown above, most non-indexical utilities will not.

But where will you end up, really?

This is the question that it feels we must answer: after pulling the lever, do you expect to be the copy on the top or the copy on the bottom?

But that's a question without meaning. There is no stochasticity here, and no uncertainty. There will be one copy at the bottom of the hole, maybe asking themselves "I wonder where I will end up", and, after pulling the lever, there will be two copies, one at the top and on at the bottom, both of them remembering falling into the hole, thinking "I wonder where I will end up", and pulling the lever. Everything is deterministic and known.

What if you close your eyes for ten seconds after pulling the lever? You could argue that both copies will now face genuine uncertainty during those ten seconds, as they don't know whether they are on the top or on the bottom.

But both copies are thinking identical thoughts; their mental processes are the same. "Am I at the top or at the bottom?" will be thought by both copies; "if I open my eyes, do I expect to see dirt or clear sky?". Your two copies cannot distinguish themselves in any way: they think the same, reason the same, have the same evidence. As long as they can't distinguish their position, they are to all extents and purposes the same agent.

You may still be tempted to assign probabilities to being below or above, but what if the lever initially creates one copy - and then, five se...
Wirehead your Chickens
by Shmi — 2018-06-20T05:49:29.344Z

TL;DR: If you care about farm animal welfare, work on minimizing actual animal suffering, not a human proxy for animal suffering.

Epistemic status: had a chat about this with a couple of local EA enthusiasts who attended the EA Global 2018 in San-Francisco, and apparently this was not high on the agenda. I have only done a cursory search online about this, and nothing of note came up.

When you read about farm animal welfare, what generally comes up is vegetarianism/veganism, humane treatment of farm animals, and sometimes vat-grown meat. This emphasis is quite understandable emotionally. Cows, pigs, chickens in industrial farms are in visible severe discomfort most of their lives, which are eventually cut short long before the end of their natural lifespan, often in painful and gruesome ways.

An animal welfare activist would ask themselves a question like "what is it like to be a chicken in a chicken farm?" and end up horrified. Their obvious solutions are those outlined above: have fewer farm animals and treat them "humanely." Less conventional approaches that reduce animal suffering get an immediate instinctive pushback, because we would not find them acceptable for ourselves. This is what I call the human proxy for animal suffering. Maybe there is a more standard name for this kind of anthropomorphizing? Anyway, let's list a few obvious approaches:

breed chickens with smaller brains, so they have less capacity for suffering,
inject a substance that would numb farm animals to physical pain,
identify and surgically or chemically remove the part of the brain that is responsible for suffering,
at birth, amputate the non-essential body parts that would give the animals discomfort later in life,
breed animals who enjoy pain, not suffer from it,
breed animals that want to be eaten, like the Ameglian Major Cow from the Hitchhiker's Guide to the Galaxy.

Many of these are probably way easier and more practical than shaming people into giving up tasty steak. But our morality immediately fights back, at least for most of us. "What do you mean, cut off baby chicken's legs so it does not have leg pain later? You, monster!"

Because most people do not truly care about reducing animal suffering, they care about a different metric altogether, a visible human proxy for animal suffering that they find immediately relatable. And so it appear...
Geoffrey Miller on Research and Politics in Psychology
by Jacob Falkovich — 2018-06-19T21:03:44.758Z
Link: https://putanumonit.com/2018/06/17/miller-2-psychology/

Part 2 of our conversation with Dr. Miller.

Topics include:

1. Why did psych departments hire social psychologists in the sixties?

2. What do romance novels know that psych departments don't?

3. Why don't psych departments seem to produce useful knowledge in psychology?
Paradoxes in all anthropic probabilities
by Stuart_Armstrong — 2018-06-19T15:31:17.177Z

In a previous post, I re-discovered full non-indexical updating (FNC), an anthropic theory I'm ashamed to say I had once known and then forgot. Thanks to Wei Dai for reminding me of that.

There is a problem with FNC, though. In fact, there are problems with all anthropic probability theories. Both FNC and SIA violate conservation of expected evidence: you can be in a situation where you know with certainty that your future probability will be different in a particular direction, from your current one. SSA has a different problem: it allows you to make decisions that change the probability of past events.

These paradoxes are presented to illustrate the fact that anthropic probability is not a coherent concept, and that dealing with multiple copies of a single agent is in the realm of decision theory.

FNC and evidence non-conservation

Let's presume that the bandwidth of the human brain is Ω bits per minute. Then we flip a coin. Upon it coming up heads, we create 2Ω identical copies of you. Upon it coming up tails, we create 23Ω copies of you.

Then if we assume that the experiences of your different copies are random, for the first minute, you will give equal probability to heads and tails. That's because there is a being with exactly the same observations as you, in both universes.

After two minutes, you will shift to 2Ω:1 odds in favour of tails: you're certain there's a being with your observations in the tails universe, and, with probability 1/2Ω, there's one in the heads universe.

After a full three minutes, you will finally stabilise on 22Ω:1 odds in favour of tails, and stay there.

Thus, during the first minute, you know that FNC will be giving you different odds in the coming minutes, and you can predict the direction those odds will take.

If the observations are non-random, then the divergence will be slower, and the FNC odds will be changing for a longer period.

SIA and evidence non-conservation

If we use SIA instead of FNC, then, in the above situation, the odds of tails will be 22Ω:1 and will stay there, so that setting is not an issue for SIA.

To show a problem with SIA, assume there is one copy of you, that we flip a coin, and, if comes out tails, we will immediately duplicate you (putting the duplicate in a separate room). If it comes out heads, we will wait a minute before duplicating you.

Then SIA implies 2:1 in favour of tail... ,,,,,,,,,,,,,
Problem Solving with Mazes and Crayon
by johnswentworth — 2018-06-19T06:15:13.081Z

I want to talk about a few different approaches to general problem solving (for humans). It turns out that they can all be applied to mazes, so I’ll use some disney-themed mazes to illustrate each approach.

We’ll start off with some traditional path-search algorithms (DFS, BFS, heuristic). Next, we’ll talk about how these algorithms can fall short for everyday problem solving. Then we’ll move on to the interesting part: two techniques which often work better for everyday problem solving, and lend some interesting insights when applied to mazes.

I’ll assume no technical background at all, so if you’ve seen some of this stuff before, feel free to skim it.

DFS and BFS

You have a maze, with a start point and an end point, and you are searching for a path through it. In algorithms classes, this problem is called “path search”.

The very first path search algorithms students typically learn are depth-first search (DFS) and breadth-first search (BFS). Here’s DFS, applied to the Pinocchio maze above:

Basically, the DFS rule is “always take the right-most path which you haven’t already explored”. So, in the Pinoccchio maze, we start out by turning right and running until we hit a wall (the first “x”). Then we turn around and go back, find another right turn, and hit another dead end. Turn around again, continue…

That’s depth-first search. We go as far as we can down one path (“depth-first”) and if we hit a dead end, we turn around, back up, and try another path.

Breadth-first search, on the other hand, tries all paths in parallel:

In this snapshot, we’ve hit three dead ends, and we have four separate “branches” still exploring the maze. It’s like a plant: every time BFS hits an intersection, it splits and goes both ways. It’s “breadth-first”: it explores all paths at once, keeping the search as wide as possible.

There’s lots more to say about these two algorithms, but main point is what they have in common: these are brute-force methods. They don’t really do anything clever, they just crawl through the whole maze until they stumble on a solution. Just trying out paths at random will usually solve a maze about as quickly as DFS or BFS (as long as you keep track of what you’ve alrea...
Order from Randomness: Ordering the Universe of Random Numbers
by Shmi — 2018-06-19T05:37:42.404Z

Epistemic status: not sure what to make of it.

Previously I had suggested that the laws of physics were the observers' attempts to make sense of the universe without laws while looking for patterns, which then become their models of reality. Some people suggested that this idea matched their intuition, others were bringing up Tegmark's mathematical universe as something that inherently has laws in it. Not being a fan of Tegmark (that's a different discussion) I will not pursue this avenue. Returning to the questions I have asked in the previous post, the first one was:

Can one start with a sequence of random numbers as a toy universe, i.e. no order and get somewhere that way, just by finding spurious patterns in the sequence?

This post attempts to create a toy model that does just that! it might appear contrived, but hopefully will make a bit of sense.
So, let's have our toy model of the universe as a sequence of random numbers. For definiteness, let's start with a uniform random sequence with elements between zero and one. While it is possible to get more random than that, it would require some effort. So, here is a sample sequence:

This looks, appropriately, rather orderless. One can see that this is so by looking at the power spectrum of this sequence. I have averaged about 10,000 of power spectra to get a smooth curve:

There are no correlations in this sequence and the power spectrum is flat, indicating the lack of order, as it should be. The random sequence was our toy universe, with the element (sample) number interpreted as the discrete "time" in this universe, t=1..1024.

But why let the time match the sample number? What is time, anyway? That has been, well, a timeless question for several thousands of years. Why do most of us, humans, perceive time as one-dimensional, inexorably going from the past to the future, and not in the other direction, or without any direction at all? A common answer to this question is related to the second law of thermodynamics:

The arrow of time matches the arrow of increasing entropy.

Well, not quite. The law, as generally formulated, states that entropy increases with time. But how do we know that this is a physical law and not a pattern we notice and successfully extrapolate, in keeping with the main thesis under consideration? Let's see what we can get from our toy model of a universe of random numbers. As remarked previ...
Sleeping Beauty Not Resolved
by Chris_Leong — 2018-06-19T04:46:29.204Z

Ksvanhorn recently suggested that Radford Neal provides the solution to the Sleeping Beauty problem and that the current solutions are wrong. The consensus seems to be that the maths checks out, yet there are strong suspicions that something fishy is going on without people being able to fully articulate the issues.

My position is that Neal/Ksavnhorn make some good critiques of existing solutions, but their solution falls short. Specifically, I'm not going to dispute Neal's calculation, just that the probability he calculates completely misses anything that this problem may reasonably be trying to get at. Part of this is related to the classic problem, "If I have two sons and at least one of them is born on a Tuesday, what is the chance that both are born on the same day". This leads me to strongly disfavour the way Neal extends the word probability to cover these cases, though of course I can't actually say that he is wrong in any objective sense since words don't have objective meanings.

A key part of his argument can be paraphrased as a suggestion to Shut up and multiply since verbal arguments about probability have a strong tendency to be misleading. I've run into these issues with the slipperiness of words myself, but at the same time, we need verbal arguments to decide why it is that we should construct the formalism a particular way. Further, my interpretation of Shut up and multiply has always been that that we shouldn't engage in moral grandstanding by substituting emotions for logic. I didn't take it to mean that when presented with a mathematical model that seems to produce sketchy results that we should accept it unquestioningly, without taking the time to understand the assumptions behind it or its intended purpose. Indeed, we've been told to shut and multiple for sleeping beauty before and that came to a different conclusion.

What is Neal actually doing

Unfortunately, Ksavnhorn's post jumps straight into the maths and doesn't provide any explanation of what is going on. This makes it somewhat harder to critique, as it means that you don't just need the ability to follow the maths, but also the ability to figure out the actual motivation behind all of this.

Neal wants us the condition on all information, including the apparently random experiences that Sleeping Beauty will undergo befor...
LessWrong is hiring
by Raemon — 2018-06-19T01:38:56.783Z

LessWrong is looking to hire an additional programmer. Ideally, someone with 5+ years experience who is up for a CTO role, or seems like a strong candidate to grow into such a role.

We're looking for someone excited about our longterm vision. Over the next several years, we want to build a platform for intellectual progress across the rationality, EA and x-risk communities. In addition to smoothing out the core LW user experience, we're looking to:

Theorize about and experiment with new features that can move the state-of-the-art of online discourse forward.
Help revamp other rationality and EA aligned forums (we've just started a couple projects in this vein)
More generally, understand and solve problems relating to communication and the intellectual-pipeline in the rationalsphere. (Especially if it looks like they could benefit from technical solutions)

We want someone with technical chops (the site is built in javascript and ReactJS, although strong experience with other languages is fine), as well as someone who can help make architectural decisions that impact the sorts of goals listed above.

Position will require you commute to Berkeley.

Ping us if you're interested (sending any of us an email is fine)

The LessWrong team is:

Oliver Habryka (oliver.habryka@gmail.com)
Raymond Arnold (raemon777@gmail.com)
Ben Pace (benitopace@gmail.com)

Feel free to comment here with any questions about the position or our overall vision.