trentmkelly/LessWrong-43k huggingface dataset:
(answered: yes) Has anyone written up a consideration of Downs's "Paradox of Voting" from the perspective of MIRI-ish decision theories (UDT, FDT, or even just EDT)?
by Jameson Quinn — 2020-07-06T18:26:01.933Z

The Paradox of Voting, simply stated, is that voting in a large election almost certainly isn't worth your time (unless you think it's the most fun thing you could be doing). The guaranteed opportunity cost of going to vote will in most cases easily and predictably outweigh the expected benefits — the chance that your vote (along with everyone else's) would be pivotal because the margin was 1 vote, multiplied by your expected marginal utility payoff from your chosen candidate winning.

There are various well-known responses to this issue, listed in the Wikipedia article linked above. But to me, one of the obvious responses is to see this as just another instance of a chicken/snowdrift game, and to invoke the logic you might use to support cooperation in such games; that is, decision theory. I think this may even be one of the most common real-world instances where UDT/FDT might apply. I think it would also be a source of interesting edge cases for exploring the limits of UDT/FDT; that is, even small changes in how strictly you delimit which other (potential) voters to consider as UDT/FDT "co-agents" could easily swing the prescriptions you'd get. But doing a few quick google searches doesn't turn up any write-ups considering this issue in this light. Am I missing something, or is this idea really "new" (at least, undocumented)?

ETA: Thanks to @Vanessa Kosoy, @Daniel Kokotajlo, and @strangepoop, I now have sufficient references for prior discussions of this idea. Thanks! Honorable mention to @lkaxas, who suggested a connection to Kantian ethics which is relevant, though more remote than the references given by the above three.
Quantifying Household Transmission of COVID-19
by Owain_Evans — 2020-07-06T11:19:34.047Z

Overview

If someone in your household gets COVID-19, how likely are you to get infected? Is it possible to reduce this risk with interventions? How much of all transmission is between members of the same household? Is household transmission less bad because infections in the household don’t spread to the outside?

We (Mihaela Curmei, Andrew Illyas, Jacob Steinhardt and Owain Evans) wrote an academic paper on these questions. Owain made an informal slide show with the same material. The full version (34 slides) is here, and this LW post contains some highlights.

Key Results

We show how to adjust previous estimates of household transmission to correct for inaccurate testing and selection bias. We pool existing data using a Bayesian meta-analysis and estimate the chance of being infected by an infected household member as 30% (95% CI 18%-43%). This probability is heterogeneous across studies, with a standard deviation of 15% (9%-27%). Household transmission was likely a small fraction of transmission before social distancing (5%-35%) but a large fraction (30%-55%) after. Our results and observational studies suggest household transmission can be reduced with behavioral interventions. It is uncertain how much infections in households spread to the outside, but we show this is related to the effectiveness of contact tracing.

Highlights from Slide Show

We consider two main ways of quantifying household transmission. The first is the intra-household effective reproductive number Rh. This is defined by decomposing the familiar R number (the effective reproductive number) into a sum of community and intra-household reproductive numbers. These reproductive numbers will change over time due to behavioral interventions and reduced susceptibility. Second is the household SAR. The SAR varies depending on the age of i and j and the relationship (spouses vs housemates). For our purposes, the SAR is an average over the rates for different groups.

This diagram illustrates R, Rh, and SAR. At time t, there is a set of primary cases who are infected. They each have a set of contacts and some of those become infected at time t+1. Infected contacts are shown in red. Household members of primary cases have a blue box around them. The topmost primary case has two household members and infects 1/3 of them. The middle primary case has one household member and doesn’t ...
What should we do about network-effect monopolies?
by benkuhn — 2020-07-06T00:40:02.413Z

Many large companies today are software monopolies that give their product away for free to get monopoly status, then do the most horrible things once they’ve won. (Previously, previously.) Can we do anything about this?

Unfortunately, “you’re the product” is a popular business model for a reason: businesses like Facebook would be really hard to support without them.

Facebook would be suicidal to charge its users money, because its entire selling point is that everyone uses it, and “everyone” hates paying money. In the US, Facebook makes over $40 per person on ads (source). Can you imagine if instead of ads they tried to charge people $40 a year?

Even on the margin, anything that costs Facebook users also makes it less valuable for its remaining users—it’s a negative feedback loop. The same goes for any other site where users create value for other users, like Twitter or Craigslist or Yelp or Wikipedia. (It’s not an accident that these are some of the most stagnant popular websites!)

In fact, this is a fundamental problem with network effects. If a company wants to maintain a network effect, they need as many users as possible. To get users, they have to have a free product. To keep their product free, they have to get paid by someone else. And when they start getting paid by someone else, they’ll inevitably start prioritizing that person’s interests.

Historically with other network-effect businesses, we’ve mostly addressed this via:

regulation (e.g. local utilities)

breakups (e.g. Bell)

standardization and interoperability (e.g. email, the Web, cryptocurrency)

So far for tech monopolies, people seem to be focused mostly on breakups—e.g. Facebook from Instagram/Whatsapp—but standardization seems to have produced much better outcomes in the past. (I like email and the Web a lot more than National Grid…) I’d be interested to see more exploration of that option!
The Echo Fallacy
by Mati_Roy — 2020-07-05T23:00:39.476Z

One of my Facebook friend posted this. Sharing anonymously with permission.

The Echo Fallacy: When you shout "hello!" in a cave, and upon hearing the echo, conclude the cave walls are saying "hello" to you. More generally, when you put a certain idea into your environment, have it reflected back to you, and conclude that the reflection shows the idea originating independently from yourself.

Some examples:

* What Scott Adams calls "laundry list persuasion". This is when you say: "Maybe bigfoot videos are blurry and questionable, but there are so many of them! Why would there possibly be so many unless bigfoot is real?" Well, the answer is simple: bigfoot believers promote the idea of bigfoot, which prompts people to create fake bigfoot videos. The volume of bigfoot videos is merely a reflection of the belief in bigfoot. Similar logic applies to UFO videos, examples of pizza shops with spirals in their logos as evidence of a conspiracy, and the compilation of grievances against Jews over centuries. This is also related to apophenia, and the streetlight effect.

* When you bully someone about a quality they supposedly have, until they retort, sarcastically: "Yes, I totally have that quality! I'm just the biggest possessor of that quality on the face of the planet, dingus! Now leave me alone!" Whereupon you say they've admitted to it.

* Saying: "That guy over there loves <group>! Hey, everyone in <group>, go be friends with that guy!" Then, when some members of the group believe you and flock to that guy, saying: "See? I told you he liked them."

(I'm probably coming up with my own name for a concept that someone else has already given a name. If so, help me out.)
Better priors as a safety problem
by paulfchristiano — 2020-07-05T21:20:02.851Z

(Related: Inaccessible Information, What does the universal prior actually look like?, Learning the prior)

Fitting a neural net implicitly uses a “wrong” prior. This makes neural nets more data hungry and makes them generalize in ways we don’t endorse, but it’s not clear whether it’s an alignment problem.

After all, if neural nets are what works, then both the aligned and unaligned AIs will be using them. It’s not clear if that systematically disadvantages aligned AI.

Unfortunately I think it’s an alignment problem:

I think the neural net prior may work better for agents with certain kinds of simple goals, as described in Inaccessible Information. The problem is that the prior mismatch may bite harder for some kinds of questions, and some agents simply never need to answer those hard questions.
I think that Solomonoff induction generalizes catastrophically because it becomes dominated by consequentialists who use better priors.

In this post I want to try to build some intuition for this problem, and then explain why I’m currently feeling excited about learning the right prior.

Indirect specifications in universal priors

We usually work with very broad “universal” priors, both in theory (e.g. Solomonoff induction) and in practice (deep neural nets are a very broad hypothesis class). For simplicity I’ll talk about the theoretical setting in this section, but I think the points apply equally well in practice.

The classic universal prior is a random output from a random stochastic program. We often think of the question “which universal prior should we use?” as equivalent to the question “which programming language should we use?” but I think that’s a loaded way of thinking about it — not all universal priors are defined by picking a random program.

A universal prior can never be too wrong — a prior P is universal if, for any other computable prior Q, there is some constant c such that, for all x, we have P(x) > c Q(x). That means that given enough data, any two universal priors will always converge to the same conclusions, and no computable prior will do much better than them.

Unfortunately, universality is much less helpful in the finite data regime. The first warning sign is that our “real” beliefs about the situation can appear in the prior in two different ways:

Directly: if our beliefs about the world are described by a simple computable predictor, they are guarant

...
Learning the prior
by paulfchristiano — 2020-07-05T21:00:01.192Z

Suppose that I have a dataset D of observed (x, y) pairs, and I’m interested in predicting the label y* for each point x* in some new set D*. Perhaps D is a set of forecasts from the last few years, and D* is a set of questions about the coming years that are important for planning.

The classic deep learning approach is to fit a model f on D, and then predict y* using f(x*).

This approach implicitly uses a somewhat strange prior, which depends on exactly how I optimize f. I may end up with the model with the smallest l2 norm, or the model that’s easiest to find with SGD, or the model that’s most robust to dropout. But none of these are anywhere close to the “ideal” beliefs of a human who has updated on D.

This means that neural nets are unnecessarily data hungry, and more importantly that they can generalize in an undesirable way. I now think that this is a safety problem, so I want to try to attack it head on by learning the “right” prior, rather than attempting to use neural nets as an implicit prior.

Warm-up 1: human forecasting

If D and D* are small enough, and I’m OK with human-level forecasts, then I don’t need ML at all.

Instead I can hire a human to look at all the data in D, learn all the relevant lessons from it, and then spend some time forecasting y* for each x*.

Now let’s gradually relax those assumptions.

Warm-up 2: predicting human forecasts

Suppose that D* is large but that D is still small enough that a human can extract all the relevant lessons from it (or that for each x* in D*, there is a small subset of D that is relevant).

In this case, I can pay humans to make forecasts for many randomly chosen x* in D*, train a model f to predict those forecasts, and then use f to make forecasts about the rest of D*.

The generalization is now coming entirely from human beliefs, not from the structural of the neural net — we are only applying neural nets to iid samples from D*.

Learning the human prior

Now suppose that D is large, such that a human can’t update on it themselves. Perhaps D contains billions of examples, but we only have time to let a human read a few pages of background material.

Instead of learning the unconditional human forecast P(y|x), we will learn the forecast P(y|x, Z), where Z is a few pages of background material that the human takes as given. We can also query the human for the prior probability Prior(Z) that the background material is true...
How far is AGI?
by Roko Jelavić — 2020-07-05T17:58:30.224Z

The ability to complete sequences is equivalent to prediction. The way GPT-3 completes sequences is it that it predicts what the next token will be and then it outputs the prediction. You can use the same model on images.

In general, the agent, based on all of its input data up to some point, tries to generate future data. If it can predict its own input data reliably that means it has a model of the world which is similar to reality. This is similar to Solomonoff induction.

Once you have a good approximation of Solomonoff induction (which is uncomputable), you combine the approximation (somehow) with reinforcement learning and expected utility maximization and get an approximation of AIXI.

Since I'm not an expert in reinforcement learning I'm not sure which part is harder, but intuition tells me the hard part of all of this would be approximating Solomonoff induction, and once you have a good world-model, it seems to me it's relatively straightforward to maximize utility. I hope I'm wrong. (if you think I am please explain why)
How do you visualize the Poisson PDF?
by Rudi C — 2020-07-05T15:54:49.343Z

As people have been recommending visualization as a mnemonic technique here, I am curious how you utilize it for a formula like this.

I can encode it using metaphors: The lambda, which represents the rate, is a working factory machine. k, which is the number of incidents, is a pack of spheres. The power relation (lambda^k) is the spheres supplying electricity to the machine. e is a superhero I have named Neper. The negative power is "red electricity." The division I symbolize by going under the ground, and the factorial by a glitchy duplication effect. This visualization works, but it takes quite a bit of time for me to decode it into the original formula, and is not directly meaningful to me in its encoded form.
July 5: Anders Sandberg on Information Hazards (SSC Online Meetup)

by JoshuaFox — 2020-07-05T08:46:46.576Z

Dr Anders Sandberg, Senior Research Fellow at the Future of Humanity Institute, will speak on "Information Hazards: How do we think about (and handle) risky information."

We will also allow plenty of time for online mingling and discussion.

Click here to register, up to an hour before the talk, and we'll send you an invitation:

The talk is July 5 at 10:30 PDT, 17:30 GMT. This link shows your local time .

Facebook event here.

Advance reading:
- The classic article by Nick Bostrom
- Anders Sandberg on biosecurity
Could Nixon going to China be a cause for the big stagnation?
by ChristianKl — 2020-07-05T06:58:35.119Z

The big stagnation is commonly said to have begun in the 1970s. There are many theories of possible causes that are already discussed, but haven't heard blaming Nixon before. Especially I haven't heard blaming Nixon going to China.

Timewise, that makes Nixon going to China in 1972 is an event that happened at the right time to explain the change in innovation.

Maybe it's crucial for innovation to have basic economic production to be done by skilled labor which can think of better ways to do it instead of outsourcing it to the other side of the globe?
Spoiler-Free Review: Witcher 3: Wild Hunt (plus a Spoilerific section)
by Zvi — 2020-07-04T23:10:03.342Z

Contrast with other recent spoiler-free review: Spoiler-Free Review: Assassin’s Creed Odyssey.

In part one of this review, the true spoiler-free section, I seek to answer the question “Should I Play This Game?” while giving the absolute minimum of information.

In part two, I provide some minimally-spoilerific advice on how to best enjoy the game.

In part three, I comment on a few things, some of which require major late-game spoilers.

Part 1 is Spoiler-Free: Should I Play This Game?

If you’re going to play any such games, this is a good choice. Witcher 3: Wild Hunt is widely considered one of the best games of all time. It is not hard to see why everyone loves it. It is ambitious as hell. It tells a huge set of varied stories, large and small, and concludes one big story, very well. It offers real choices with real consequences large and small. It drips with flavor. First rate stuff.

Witcher 3, like Assassin’s Creed Odssey, is a mostly excellent implementation of a gigantic long-lasting open-world action RPG with real time tempo-based combat set in a faux-medieval magical world. You’ll go on tons of main quests and side quests, gather gold, level up,

If you ask me in a Zen and the Art of Motorcycle Maitenence style Quality sense whether Witcher 3 (92 metacritic rating) is superior to Assassin’s Creed: Odyssey (83 metacritic rating) I would have to agree that this isn’t a contest. Witcher 3 wins.

But when I ask which game was a better experience, when I ask which game was more net fun, I think it went the other way. I had a better time in Ancient Greece than I did with Geralt of Rivia. The ending left a bad taste in my mouth, and the world just wasn’t as much fun to inhabit.

I therefore ended up giving both games the same rating of Tier 2: They are both definitely games that are Worth It, but not games one Must Play.

The best things about Witcher 3 are:

The game is truly massive. There’s tons of quests and tons of stuff to explore. If you ride off towards question marks or any interesting-looking structures on your map, there will be something to find. When they say the game has 200+ hours of content, that’s a real number.

The quests are highly varied. A lot of different things happen. Many quests have twists and end up being much bigger and stranger stories than they appear to be.

The story is well-written, voiced and acted, with lots of interesting characters. Can

...
Classifying games like the Prisoner's Dilemma
by philh — 2020-07-04T17:10:01.965Z

Note: the math and the picture didn't transfer. I may try to fix it in future, but for now you might want to just read it at the original site. [Mod/Edit note: Should all be fixed now!]

Consider games with the following payoff matrix:

Player 2

Krump	Flitz
Player 1	Krump	(W,W)	(X,Y)
Flitz	(Y,X)	(Z,Z)

One such game is the Prisoner's Dilemma (in which strategy "Krump" is usually called "Cooperate", and "Flitz" is usually called "Defect"). But the Prisoner's Dilemma has additional structure. Specifically, to qualify as a PD, we must have Y>W>Z>X. Y>W gives the motivation to defect if the other player cooperates, and Z>X gives that motivation if the other player defects. With these two constraints, the Nash equilibrium is always going to be Flitz/Flitz for a payoff of (Z,Z). W>Z is what gives the dilemma its teeth; if instead Z>W, then that equilibrium is a perfectly fine outcome, possibly the optimal one.

I usually think of a Prisoner's Dilemma as also having 2W>X+Y>2Z. That specifies that mutual cooperation has the highest total return - it's "socially optimal" in a meaningful sense1 - while mutual defection has the lowest. It also means you can model the "defect" action as "take some value for yourself, but destroy value in the process". (Alternatively, "cooperate" as "give some of your value to your playmate2, adding to that value in the process".) We might consider instead:

If 2W<X+Y, then defecting while your playmate cooperates creates value (relative to cooperating). From a social perspective, Krump/Flitz or Flitz/Krump is preferable to Krump/Krump; and in an iterated game of this sort, you'd prefer to alternate X with Y than to get a constant W. Wikipedia still classes this as a Prisoner's Dilemma, but I think that's dubious terminology, and I don't think it's standard. I might offhand suggest calling it the Too Many Cooks game. (This name assumes that you'd rather go hungry than cook, and that spoiled broth is better than no broth.)
If 2Z>X+Y, then defecting while your playmate defects creates value. I have no issue thinking of this as a Prisoner's Dilemma; my instinct is that most analyses of the central case will also apply to this.

By assigning different value

...
Causality and its harms
by George3d6 — 2020-07-04T14:42:56.418Z
Link: https://blog.cerebralab.com/Causality_and_its_harms

I'll assume that you do not hold fast to a rigorous system of metaphysics, in which case I think you can fancy me and accept that, if I so desire to struggle, I could reduce the concept of causality to one (or a chain of) probabilistic relationships between events.

Here's a naive definition of causality: P(E) ~= 1 | C (where "|" stands for "given"). If I can say this, I can most certainly say that C causes E, at least in a system where a thing such as "time" exists and where C happens before E.

It should be noted this doesn't imply P(C) ~= 1 | E.

Obviously, this definition doesn't cover all or even most of the things we call causal and can be taken into an extreme context where it holds in spite of no causal relationship. I'm just starting with it because it's a point from which we can build up to a better one. Let's roll with it and look at an example.

1. Hypothetical number one

Human height can be viewed as a function of haplogroup alleles + shoe size, that is to say, we can predict h fairly well given those 2 parameters.

Height can also be viewed as a function of mTORC1 expression + HGH blood levels. Let's say some assay that tells us the levels of mTORC1 and HGH predicts height equally well to haplogroup alleles + shoe size.

But I think most scientists would agree the later are to some extent "causal" for height while the former aren't. Why?

Well consider 2 hypothetical experiment:

Take a 30yo human of average height and use a saline solution pump to inflate their feet to 1.5x their size. Then use {magical precision nucleotide addition and deletion vector} to remove all his haplogroup alleles and insert those most associated with increased height.
Take a 30yo human of average height and then use {magical precision nucleotide addition and deletion vector} to overexpress the heck out of mTORC1 and HGH.

In which case do we expect to see an increase in height? That might indicate causality.

Trick question, of course, in neither.

We'd expect to see increased height in the second case if the human was, say, 13 instead of 30.

The causality here is P(E) given C under some specific conditions. Where C has happened before P(E), even if C happened 10 years ago and C happening again now would not affect E.

Also, see physics for situations where that doesn't quite cut it...