trentmkelly/LessWrong-43k huggingface dataset:
What are some low-information priors that you find practically useful for thinking about the world?
by Linch — 2020-08-07T04:37:04.127Z

cross-posted on the EA Forum

I'm interested in questions of the form, "I have a bit of metadata/structure to the question, but I know very little about the content of the question (or alternatively, I'm too worried about biases/hacks to how I think about the problem or what pieces of information to pay attention to). In those situations, what prior should I start with?"

I'm not sure if there is a more technical term than "low-information prior."

Some examples of what I found useful recently:

1. Laplace's Rule of Succession, for when the underlying mechanism is unknown.

2. Percentage of binary questions that resolves as "yes" on Metaculus. It turns out that of all binary (Yes-No) questions asked on the prediction platform Metaculus, 29% of them resolved yes. This means that even if you know nothing about the content of a Metaculus question, a reasonable starting point for answering a randomly selected binary Metaculus question is 29%.

In both cases, obviously there are reasons to override the prior (for example, you can arbitrarily flip all questions on Metaculus such that your prior is now 71%). However (I claim), having a decent prior is nonetheless useful in practice, even if it's theoretically unprincipled.

I'd be interested in seeing something like 5-10 examples of low-information priors as useful as the rule of succession or the Metaculus binary prior.
Exploring Two-Level Visual Mnemonic Compression
by armatures — 2020-08-06T23:58:34.737Z

A Visual Mnemonic

Mnemonics are tools for remembering things. I haven’t read Jonathan Haidt’s The Righteous Mind for years, but I still have easy recall of his categorization of ideas that motivate moral reasoning. The mnemonic that has helped my recall of this concept has stuck with me because it has two layers of compression. Here are some icons I made, organized in a grid—the most basic version of the mnemonic:

In combination, Haidt argues that these six flavors explain anyone’s sense of morality. They are: harm (minimizing harm), sanctity, authority, loyalty, fairness, and liberty.

Do the pictures make these concepts more memorable? Somewhat: they’re more concrete and have more character than the words alone. What really helps about this diagram is that the pictures were chosen in a way that can help recall when they’re paired together. Each row of iconography coheres around a visual concept, so if you can only remember one of the items in the row, hopefully you can fill in the other icon by prompting with its pair. The dagger and the cross are both cross-shaped, the crown and the dog are both symbols from heraldry, and the scales and the manacles each themselves contain pairs. If, when designing a visual mnemonic, you can give your visual an internal logic, it will be more likely to do its job effectively.

Reducing Heirarchy

The first mnemonic is decent enough, but the ordered nature of the list implies in the order in the importance of the moral flavors the icons represent. It’s right that harm should be first (this is foundational to most systems of morality), but it would be hard for me to rank the rest of the moral flavors. I moved the icons to a ring, equally emphasizing all of them:

A mnemonic with more equal emphasis on the component elements of morality. The pairings are still apparent in their arrangement opposite one another in the diagram.

Why Not More Data?

Here’s another advantage to moving beyond the table format: this organization of the moral elements in a circle enables the addition of more data. Haidt describes the appeals the Republican and Democratic parties in the United States make two different moral elements in the chapter titled “The Conservative Advantage.” Rather than tell you about it, I’ll add his opinions on these party posit...
Why is aging research our primary attempt at immortality?
by sd1 — 2020-08-06T23:47:33.495Z

Even after solving aging, we're susceptible to death by accident, disease, and murder. Are there other research directions for achieving extremely long lifespans (at least 1,000-10,000 years)?

Even if they're not popular, does anyone have an idea they think would work? The best I've been able to come up with after thinking about it for 10 minutes is an isolated brain, which would require us to figure out how input/output channels to the brain work (or at least be able to reverse-engineer the ones we already have), keep an isolated brain alive and functioning reliably for indefinite amounts of time (we've made some progress here even with little interest: one, two), and focus all of medicine into curing diseases of the brain.

Anyway, please reply with your best approaches and ideas.
Sunday August 9, 1pm (PDT) — talks by elityre, jacobjacob, Ruby
by Bird Concept — 2020-08-06T22:50:21.550Z

This Sunday at 1pm (PDT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts).

Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

elityre

Curated posts: Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?
How do you assess the quality / reliability of a scientific study?
Historical mathematicians exhibit a birth order effect too

jacobjacob

Curated posts: The Copernican Revolution from the Inside
Unconscious Economics

Ruby

Curated posts: Conversation Cultures: Combat vs. Nurture
Why wasn't science invented in China?

Details

When? Sunday August 9, 1pm (PDT)

Where? Zoom.
Analyzing the Problem GPT-3 is Trying to Solve
by adamShimi — 2020-08-06T21:58:56.163Z

I think that taking a theoretical computer science perspective to GPT-3 might help clarify some part of the debate.

My background is in theoretical computer science, and that's my prefered perspective to make sense of the world. So in this post, I try to rephrase the task we ask GPT-3 to solve in terms of search and optimization problems, and I propose different criteria for solving such a task. I thus consider GPT-3 as a black-box, and only ask about how to decide if it is good at its task or not.

Epistemic status: probably nothing new, but maybe an interesting perspective. This is an attempt to understand the intuitions behind different statements about whether GPT-3 "solves some task".

The Promptp Search Problem

Let's fix a language (English for example). Then a very basic description of the task thrown at GPT-3 is that it's given a prompt p and it needs to give a follow-up text x such that x is a good answer to the prompt. For example, p might be a question about parenthesis or addition or a story to complete, and x is GPT-3's answer. I thus define Promptp to be this search problem for a given prompt p.

Now, what does it mean for an algorithm A to solve Promptp? Multiple possibilities come to mind:

(Deterministic Solution) For every execution t of A, such that the value of A for prompt p at the end of t is At(p), we have At(p)∈Promptp. This criterion has the advantage for simplicity, but it's also completely unrealistic -- even a human might fail to give a decent answer to a prompt.
(Nondeterministic Solution) There is an execution t of A such that satisfies At(p)∈Promptp. Here, we only require that A might give a correct answer, without any other guarantees. This is obviously too weak, as a monkeys bashing on keyboards will eventually give a good answer -- yet are not "solving it".
(Probabilistic Solution) For a meaningful distribution on the executions of A (for example a uniform distribution or a distribution based on simplicity), we havePr[At(p)∈Promptp]>l, for some l∈]0,1[ . This looks more realistic, as it allows for a margin of error. What about the value of l? I think we can go with anything >1/2, just like in Probabilistic Turing Machines: it ensures that we can make the probabilty of success as high as we want by repeating the call to A a polynomial number of times (So a po

...
Asch Conformity Could Explain the Conjunction Fallacy
by curi — 2020-08-06T21:46:57.227Z

This post follows my question Can Social Dynamics Explain Conjunction Fallacy Experimental Results? The results of the question were that no one provided any research contradicting the social dynamics hypothesis.

There is research on social dynamics. Asch’s conformity experiments indicate that wanting to fit in with a group is a very powerful factor that affects how people answer simple, factual questions like “Which of these lines is longer?” People will knowingly give wrong answers for social reasons. (Unknowingly giving wrong answers, e.g. carelessly, is easier.)

Conformity and other social dynamics can explain the conjunction fallacy experimental data. This post will focus on conformity, the dynamic studied in the Asch experiments.

This post assumes you’re already familiar with the basics of both the Asch and Conjunction Fallacy research. You can use the links if you need reminders.

First I’ll talk about whether conformity applies in the Conjunction Fallacy research setting, then I’ll talk about how conformity could cause the observed results.

Conformity in Groups

The Asch Conformity Experiments have people publicly share answers in a group setting. This was designed to elicit conformist behavior. Should we also expect conformist behavior in a different setting like the Conjunction Fallacy experiments setting? I suspect the different setting is a major reason people don’t connect the Asch and Conjunction Fallacy results.

I haven’t seen specific details of the Conjunction Fallacy research settings (in the text I read, details weren’t given) but I think basically people were given questionnaires to fill out, or something close enough to that. The setting is a bit like taking a test at school or submitting homework to a teacher. Roughly: Someone (who is not a trusted friend) will look over and judge your answers in some manner. In some cases, people were interviewed afterwards about their answers and asked to explain themselves.

Is there an incentive to conformity in this kind of situation? Yes. Even if there was no peer-to-peer interaction (not a safe assumption IMO), it’s possible to annoy the authorities. (Even if there were no real danger, how would people know that? They’d still have a reasonable concern.)

What could you do to elicit a negative reaction from the researchers? You could take the meta position that your answers won’t impact your life and choose ...
Covid 8/6: The Case of the Missing Data
by Zvi — 2020-08-06T21:40:00.880Z

Last time: Covid 7/30: Whack a Mole

Testing. One could sum up the entire failure on Covid-19 as a failure of testing.

We have failed to test individuals. We have failed to ramp up testing capacity fast enough. We have failed to deliver test results within the time frame necessary for them to stop the spread of the virus. We have failed to allow other forms of testing that could help us greatly.

We have also failed to test everything else. We still don’t reliably know how this virus is transmitted. We still don’t know what actions are safe or risky. We don’t know the right mask designs. Our knowledge of which drugs work is spotty at best. We don’t know how many people are actually immune or how that works. We don’t know if our vaccines work. We don’t know how people respond to different policies and options. We don’t know almost anything.

The reason these posts exist is to try and figure out something, anything. To let us navigate our lives, plan for the future and maybe even help improve he overall course of events. But also on general principles. It is good and right to figure out as best one can what is going on, and tell others, whether or not there is a concrete plan to cash that out into other benefits.

This week’s theme is testing because rather than the usual steady increase, this week saw a marked drop in the number of Covid-19 tests. It’s one thing to move forward too slowly. It’s a different matter to be going backwards. Meanwhile, the shutting down of additional means of testing by the FDA is back in the story as well.

The scariest part of all this is wondering whether this reduction in testing is on purpose.

It is a shame. I had hoped to focus on good news.

Let’s run the numbers.

Be careful with them this week.

Positive Test Counts

Date
WEST
MIDWEST
SOUTH
NORTHEAST

May 21-May 27
23979
39418
42977
37029

May 28-June 3
32200
31504
50039
33370

June 4-June 10
35487
24674
55731
22693

June 11-June 17
41976
22510
75787
17891

June 18-June 24
66292
26792
107221
15446

June 25-July 1
85761
34974
163472
16303

July 2-July 8
103879
40139
202863
18226

July 9-July 15
108395
53229
250072
20276

July 16-July 22
117506
57797
265221
20917

July 23-July 29
110219
67903
240667
26008

July 30-Aug 5
91002
64462
212945
23784

That looks amazing. But the gains are due to lack of testing.

Deaths

Date
WEST
MIDWEST
SOUTH
NORTHEAST

Apr 30-May 6

...
Procrastination Paradoxes: the Good, the Bad, and the Ugly
by scasper — 2020-08-06T19:47:54.718Z

Summary

Suppose that you are sitting in front of a big red button, and if you press it after waiting t timesteps, you will receive t units of utility. If you want to maximize the amount of utility you obtain, then when should you press it? Clearly not ever because pressing it at any time would confer less utility than waiting one more timestep. But clearly not never because then you’d never get any utility at all!

This is an example of a procrastination paradox. In a sense, these problems are specification issues that happen when you want a system to do two contradictory things: press a button sometime, but press it later than any time (aka never). Situations like these are important decision theoretic dilemmas which represent a means by which certain naive expected utility maximizers can fail miserably. They pose a challenge for the understanding and implementation of mechanistic rationality by demonstrating that in some situations, there is no optimal normative decision procedure. In fact,

this post

considered their existence to be an argument against utility functions in general.

Unfortunately, there is a great deal of confusion surrounding them, so the goal of this post is to build a rigorous, novel framework for understanding and eventually making solemn peace with them. After an in-depth discussion of these paradoxes, this post explores three aspects about them in context of formalisms from decision theory and reinforcement learning: the good, the bad, and the ugly. Sadly, the best order to discuss them in isn’t the one established by the

classic Spaghetti Western

:

The Bad: If an agent is trying to maximize an expected utility function which has no defined maximum that is achievable with finite steps, then it can be vulnerable to procrastination paradoxes. Unfortunately, expected utility functions like these can be very natural to design.

The Ugly: For any decision-making process which avoids procrastination traps, it is trivial to construct one which does so better. There is no such thing as an optimal approach to procrastination dilemmas.

The Good: Certain probabilistic strategies for handling procrastination dilemmas can avoid procrastinating forever while still having infinite expected utility for dilemmas in which the limit of the obtainable utility is also infinite.

A Rigorous Understanding

(Naive) Expected Utility Maximization

In general...
Market norms and Social norms
by TruetoThis — 2020-08-06T18:27:24.664Z

Predictably Irrational

When I joined LW, I browsed through an article that recommended Predictably Irrational as a read for starters. After reading it, I decided to share my greatest use of the content.

MARKET NORMS VS SOCIAL NORMS

The idea is that there are different forms of currency when dealing with an exchange. Market norms (financial currency) and social norms (favor/casual currency). Each of these are individually effective, but you must beware not to mix improperly or the relationship could be hard to reestablish.

Example: I need a cook to make dinner for myself and some guests. I have a friend who's a chef, yet his rate is double what I can pay. By offering what I can pay (market norm), compared to what he usually charges, it discounts him and can be insulting . However, if I approach him and ask for a favor (social norm) and he agrees to do it for free or a lesser rate, I can gift him with the money and the reward is amplified.

APPLICATION

I recently started a business and needed to hire a skincare chemist, but I had no financial backing to support. I put an ad out and instead of offering a salary, I offered an unpaid internship with sporadic rewards like gas cards and Amazon gift cards. As of today, I have two chemists on this internship and one who is paid. Sure and true to the theory, the interns work just as hard as the paid chemist.

I guess to truly get the full effect of Ariely's experiment, there would have to be someone who is paid a rate lower than the paid employee and more than the interns. But overall, you get the point. It's an effective system. What are some ways you can substitute market norms for social norms in your world?
How do you organise your reading?
by brook — 2020-08-06T14:54:13.629Z

Maybe this is just a personal problem, but I frequently find myself with just an absolute mess of things to read. Blog posts over several different sites, PDFs, paper books, podcasts and other content I intend to consume. I suspect a significant proportion of it falls through the cracks, not least when I have three separate browser windows open just so I can see the tabs. The LW standard of each post referencing at least 10 other posts in-text doesn't help, because each of them seem useful, but I don't want to disrupt the flow of the current article, so I open it in another tab, and so on and so on.

So, my question is: Do you have any useful tools, tips, or techniques for organising things you intend to consume?
Open & Welcome Thread - August 2020
by habryka — 2020-08-06T06:16:50.337Z

If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a shortform post)

And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here.
Zen and Rationality: Don't Know Mind
by Gordon Seidoh Worley — 2020-08-06T04:33:54.192Z

This is post 1/? about the intersection of my decades of LW-style rationality practice and my several years of Zen practice.

In today's installment, I look at the Zen notion of "Don't Know Mind" in rationalist terms.

I'm a little unsure where "don't know mind" comes from. Sensei Google suggests it might be the Seon expression of the idea that in Zen is said "shoshin", often translated as "beginner's mind" but also carrying notions conveyed by translating it as "original mind", "naive mind", "novice mind", and "inexperienced mind" (noting that the character rendered "mind" is better translated as "heart-mind"). There's also a beloved koan often called "not knowing is most intimate" (Book of Equanimity, Case 20), and "don't know" is a good name to put to a particular insight you might have if you meditate enough. Regardless, "don't know mind" is a thing Zen practitioners sometimes say. What does it mean?

Depends on how you parse it.

The standard parsing is as "don't-know mind", as in the mind that doesn't know. This fits with the notion of soshin or beginner's mind, that is the mind that has not yet made itself up. In standard rationalist terms, this is the heart-mind that is curious, relinquishing, light, even, simple, humble, and nameless. Saying "don't know" is tricky, though, because there's the looming trap of the "don't know" that stops curiosity. Instead, this is the "don't know" that extends an open invitation of learn more.

You can also parse it as a command: "do not know" (since Zen falls within the Buddhist tradition that claims all you know is mind, "mind" is redundant here). This isn't an imperative to never know anything. Rather it's an encouragement to gaze beyond form into emptiness since our minds are often caught up in form (the map) and fail to leave space for emptiness (the territory[1]). More specifically, to know is to make distinctions, to construct an abstraction, to prune, to have a pior over your observations, to give something a name, to say this not that. Naturally, this means knowing is quite useful; it's literally what enables us to do things[2]. Yet it necessarily means leaving something out. "Don't know mind" is then advice to simultaneously let all in even as you keep some out.

Thus both interpretations converge on this idea that we can open ourselves to "knowing" more if we can hold fast to the realization that we don't already know...
The Isolation Assumption of Expected Utility Maximization
by Pedro Oliboni — 2020-08-06T04:05:25.727Z

In this short essay I will highlight the importance of what I call the “isolation assumption” in expected utility theory. It may be that this has already been named in the relevant literature and I just don’t know it. I believe this isolation assumption is both important to decision-making about doing good and often ignored.

Expected utility theory is here taken as a normative theory of practical rationality. That is, a theory about what is rational to choose given one’s ends (Thoma 2019, 5). Expected utility theory is then the decision theory that says that the best way for an agent to pursue her goals is to choose so as to maximize expected utility.

By utility, I mean not some concept akin to happiness or wellbeing but a measure that represents how much an agent prefers an outcome. For example, for an altruist, having a child not die from drowning in a pond may have significantly higher utility than dining out at a fancy restaurant.

The term “expected” comes from probability theory. It refers to the sum of the products of the probability and value of each outcome. Expected utility is then a property of options in decisions. Say an agent has two options for lunch and the single thing this agent has preferences over is how her lunch goes today. Option A is to eat a veggie burger, which will bring this agent 10 “utils” for certain. Then, Option A has expected utility of 10. Option B, however, is a raffle in which the agent either gets a really fancy clean-meat burger with probability 0.1 or nothing with probability 0.9. If the agent values the clean-meat burger at 20 utils, and not eating lunch at 0, then the expected utility of Option B has 0.1*20 + 0.9*0 = 2 expected utility.

I currently think expected utility theory is reasonable as a theory of practical rationality. Brian Tomasik has proposed a compelling thought experiment for why that is. Consider the following scenario:

suppose we see a number of kittens stuck in trees, and we decide that saving some number n of kittens is n times as good as saving one kitten. Then, if we are faced with the choice of either saving a single kitten with certainty or having a 50-50 shot at saving three kittens (where, if we fail, we save no kittens), then we ought to try to save the three kittens, because doing so has expected value 1.5 (= 3*0.5 + 0*0.5), rather than

...