trentmkelly/LessWrong-43k huggingface dataset:
Formalising continuous info cascades? [Info-cascade series]
by Ben Pace — 2019-03-13T10:55:46.133Z

This is a question in the info-cascade question series. There is a prize pool of up to $800 for answers to these questions. See the link above for full background on the problem (including a bibliography) as well as examples of responses we’d be especially excited to see.

Mathematically formalising info-cascades would be great.

Fortunately, it's already been done in the simple case. See this excellent LW post by Johnicholas, where he uses upvotes/downvotes as an example, and shows that after the second person has voted, all future voters are adding zero new information to the system. His explanation using likelihood ratios is the most intuitive I've found.

The Wikipedia entry on the subject is also quite good.

However, these two entries primarily explain how information cascades when people have to make a binary choice - good or bad, left or right, etc. The question I want to understand is how to think of the problem in a continuous case - do the problems go away? Or more likely, what variables determine the speed at which people update to one extreme? And how far toward that extreme do people go before they realise their error?

Examples of continuous variables include things like project time estimates, stocks, and probabilistic forecasts. I imagine it's very likely that significant quantitative work has been done on the case of market bubbles, and anyone can write an answer summarising that work and explaining how to apply it to other domains like forecasting, that would be excellent.
How large is the harm from info-cascades? [Info-cascade series]
by Bird Concept — 2019-03-13T10:55:38.872Z

This is a question in the info-cascade question series. There is a prize pool of up to $800 for answers to these questions. See the link above for full background on the problem (including a bibliography) as well as examples of responses we’d be especially excited to see.

___

How can we quantify the impact (harm) of info-cascades?

There are many ways in which info-cascades are harmful. Insofar as people base their decisions on the cascaded info, this can result in bad career choices, mistaken research directions, misallocation of grants, a culture that is easier to hijack by cleverly signalling outsiders (by simply “joining the bubble”), and more.

But in order to properly allocate resources to work on info-cascades we need a better model of how large the effects are, and how they compare with other problems. How can we think about info-cascades from a cost-effectiveness perspective?

We are especially interested in answers to this question that ultimately bear on the effective altruism/rationality communities, or analyses of other institutions with insights that transfer to these communities.

As an example step in this direction, we built a Guesstimate model, which is described in an answer below.
How can we respond to info-cascades? [Info-cascade series]
by Bird Concept — 2019-03-13T10:55:25.685Z

This is a question in the info-cascade question series. There is a prize pool of up to $800 for answers to these questions. See the link above for full background on the problem (including a bibliography) as well as examples of responses we’d be especially excited to see.

___

In my (Jacob's) work at Metaculus AI, I'm trying to build a centralised space for both finding forecasts as well as the reasoning underlying those forecasts. Having such a space might serve as a simple way for the AI community to avoid runway info-cascades.

However, we are also concerned with situations where new forecasters overweight the current crowd opinion in their forecasts, compared to the underlying evidence, and see this as major risk for the trustworthiness of forecasts to those working in AI safety and policy.

With this question, I am interested in previous attempts to tackle this problem, and how successful they have been. In particular:

What existing infrastructure has been historically effective for avoiding info-cascades in communities? (Examples could include short-selling to prevent bubbles in asset markets, or norms to share the causes rather than outputs of one’s beliefs)

What problems are not adequately addressed by such infrastructure?
Distribution of info-cascades across fields? [Info-cascade series]
by Bird Concept — 2019-03-13T10:55:17.194Z

This is a question in the info-cascade question series. There is a prize pool of up to $800 for answers to these questions. See the link above for full background on the problem (including a bibliography) as well as examples of responses we’d be especially excited to see.

___

How common, and how large, are info-cascades in communities that seek to make intellectual progress, such as academia? This distribution is presumably very heavy-tailed as we are dealing with network phenomena. But what is its actual values? How can we estimate this number?

A good starting point for thinking about this might be the paper “How citation distortions create unfounded authority: analysis of a citation network” (Greenberg, 2009), which uses social network theory and graph theory to trace how an at best very uncertain claim in biomedicine cascading into established knowledge. We won’t attempt to summarise the paper here.
Understanding information cascades
by Bird Concept — 2019-03-13T10:55:05.932Z

Meta: Because we think understanding info cascades are important, we recently spent ~10 hours trying to figure out how to quantitatively model them, and have contributed our thinking as answers below. While we currently didn't have the time to continue exploring, we wanted to experiment with seeing how much the LW community could together build on top of our preliminary search, so we’ve put up a basic prize for more work and tried to structure the work around a couple of open questions. This is an experiment! We’re looking forward to reading any of your contributions to the topic, including things like summaries of existing literature and building out new models of the domain.

Background

Consider the following situation:

Bob is wondering whether a certain protein injures the skeletal muscle of patients with a rare disease. He finds a handful papers with some evidence for the claim (and some with evidence against it), so he simply states the claim in his paper, with some caution, and adds that as a citation. Later, Alice comes across Bob’s paper and sees the cited claim, and she proceeds to cite Bob, but without tracing the citation trail back to the original evidence. This keeps happening, in various shapes and forms, and after a while a literature of hundreds of papers builds up where it’s common knowledge that β amyloid injures the skeletal muscle of patients with inclusion body myositis -- without the claim having accumulated any more evidence. (This real example was taken from Greenberg, 2009, which is a case study of this event.)

An information-cascade occurs when people update on each others beliefs, rather than sharing the causes of those beliefs, and those beliefs end up with a vestige of support that far outstrips the evidence for them. Satvik Beri might describe this as the problem of only sharing the outputs of your thinking process, not your inputs.

The dynamics here are perhaps reminiscent of those underlying various failures of collective rationality such as asset bubbles, bystander effects and stampedes.

Note that his effect is different from other problems of collective rationality like the replication crisis, which involve low standards for evidence (such as unreasonably lax p-value thresholds or coordination problems preventing publishing of failed experiments), or the degeneracy of much online discussion, which involves tribal signalling a...
Verifying vNM-rationality requires an ontology
by jeyoor — 2019-03-13T00:03:17.017Z

Result
It is impossible to verify that an agent is vNM-rational by observing its actions without access to the domain of its utility function.

Motivation
Alphonso and Beatriz both go the market to buy fruit.
Alphonso prefers grapes to oranges.
He fills his basket with grapes and pays for them.
Beatriz carefully picks through the fruit and purchases some oranges and some grapes.
Callisto arrives with a package of grapes.
"Say, Beatriz, would you like to trade some of your oranges for this package of grapes?" Callisto offers.
"Gladly." Beatriz replies, exchanging some of her oranges for the grapes.
A few moments later, Alphonso notices Beatriz giving Deion some grapes in exchange for some oranges.
"You are acting irrationally, Beatriz!" Alphonso exclaims. "Your unstable preference between oranges and grapes makes it possible for a malicious agent to exploit you and exhaust your entire grocery budget!
"Ah, but I am acting rationally." Beatriz replied with a smile. "I prefer fruit that is fresh enough to last more than seven days. Thus, I trade away fruit that will spoil before that time."
Explanation
Consider an agent A.
We are interested in verifying whether or not A is vNM-rational.
However, we are only able to observe A's decisions without any access to the domain of A's utility function.
Without this access, it is impossible to distinguish between vNM-irrational choices (i.e. choices that violate one of the axioms of vNM-rationality) and choices that are vNM-rational but made under an unexpected ontology.

In other words, we need to know how A perceives outcomes of the world before we can verify that A's preferences over those outcomes are vNM-rational.
[Link] OpenAI LP
by Alexei — 2019-03-12T23:22:59.861Z

https://openai.com/blog/openai-lp/

We’ll need to invest billions of dollars in upcoming years into large-scale cloud compute, attracting and retaining talented people, and building AI supercomputers.

We want to increase our ability to raise capital while still serving our mission, and no pre-existing legal structure we know of strikes the right balance. Our solution is to create OpenAI LP as a hybrid of a for-profit and nonprofit—which we are calling a “capped-profit” company.

Going forward (in this post and elsewhere), “OpenAI” refers to OpenAI LP (which now employs most of our staff), and the original entity is referred to as “OpenAI Nonprofit.”
LW Update 2019-03-12 -- Bugfixes, small features
by jimrandomh — 2019-03-12T21:56:40.109Z

We've deployed an updated version of LessWrong, with a bunch of bug fixes and small features added since last time (March 1).

Features and Changes

Auto-linkify URLs in posts and comments

Coauthored posts appear on your user profile page

Drafts that are shared with you appear in the Drafts section of the user profile page

Negative-karma change notifications don't use red color

Posts on the All Posts page have a menu on the right

Bug Fixes

Address some compatibility issues with Opera Mini, Edge, and IE11

Remove broken New Users Notification and New Posts Notification settings

Table of Contents: Empty blocks can't be headings

Remove spurious All Posts View setting from user profile page

Fix a visual glitch during loading (SSR) of the all-posts page

Fix a bug where the RSS synchronization cron job would sometimes stop running

Don't redundantly show the AF icon on alignmentforum.org

Remove a duplicate Load More button on the user profile page

Fix a crash in the Sunshine Sidebar with reports of already-deleted comments

Various fixes to search indexing

Fix a security vulnerability that could reveal unpublished drafts

API Changes

Add localStartTime and localEndTime resolvers to posts, which return the time of a meetup in the time zone of the meetup location (null if no location specified)

Technical

Clean up sequences

Infrastructure for validating database contents against schemas

Optimize some slow mongodb queries

Migration infrastructure for dealing with denormalized fields

robots.txt now blocks crawlers from our test servers (no change on the main site)
You Get About Five Words
by Raemon — 2019-03-12T20:30:18.806Z

Cross posted from the EA Forum.

Epistemic Status: all numbers are made up and/or sketchily sourced. Post errs on the side of simplistic poetry – take seriously but not literally.

If you want to coordinate with one person on a thing about something nuanced, you can spend as much time as you want talking to them – answering questions in realtime, addressing confusions as you notice them. You can trust them to go off and attempt complex tasks without as much oversight, and you can decide to change your collective plans quickly and nimbly.

You probably speak at around 100 words per minute. That's 6,000 words per hour. If you talk for 3 hours a day, every workday for a year, you can communicate 4.3 million words worth of nuance.

You can have a real conversation with up to 4 people.

(Last year the small organization I work at considered hiring a 5th person. It turned out to be very costly and we decided to wait, and I think the reasons were related to this phenomenon)

If you want to coordinate on something nuanced with, say, 10 people, you realistically can ask them to read a couple books worth of words. A book is maybe 50,000 words, so you have maybe 200,000 words worth of nuance.

Alternately, you can monologue at people, scaling a conversation past the point where people realistically can ask questions. Either way, you need to hope that your books or your monologues happen to address the particular confusions your 10 teammates have.

If you want to coordinate with 100 people, you can ask them to read a few books, but chances are they won't. They might all read a few books worth of stuff, but they won't all have read the same books. The information that they can be coordinated around is more like "several blogposts." If you're trying to coordinate nerds, maybe those blogposts add up to one book because nerds like to read.

If you want to coordinate 1,000 people... you realistically get one blogpost, or maybe one blogpost worth of jargon that's hopefully self-explanatory enough to be useful.

If you want to coordinate thousands of people...

You have about five words.

This has ramifications on how complicated a coordinated effort you can attempt.

What if you need all that nuance and to coordinate thousands of people? What would it look like if the world was filled with complicated problems that required lots of people to solve?

I guess it'd look like this one.
[Old] Wayfinding series
by Gordon Seidoh Worley — 2019-03-12T17:54:16.091Z

As I said in the intro to the "mapmaking series" post, I'm reposting some content from my old blog that people found useful. They particularly found things I wrote in this series useful, and of the two series I agree that this is likely the more useful one, the former being reposted mainly because this one occasionally references it. If you are familiar with LW much you probably don't need to read the first post.

As I also said in the first post, this is old content that I no longer entirely endorse, although I think most of it remains useful and surprisingly skillful advice given how little I knew back then. I've been about 7 different people between when I wrote these posts and now, to give you some idea of the distance I'm suggesting I have from it. Nonetheless, I think learning to master the topics I discuss here is pretty essential if you want to attain to at least whatever level of mastery I've achieved at being a human, much else besides not included.

I hope these words treat you well.

finding your way

having now a good sense of maps and mapmaking, we can begin to use our maps to find our way through the territory. this requires more than the skill of map reading, which is implicit to mapmaking. you need the skill of the wayfinding: finding your way through the territory to a destination.

what sort of destination? the kinds of destinations that exist in the territory: possible future worlds states you could find yourself in. for example, hungry? find your way to eating a sandwich. thirsty? find your way to drinking a glass of water. ennui? find you way to having a life's purpose. to put it in a story:

you are trying to go to your friend's house. you have a map that shows you that to get there you need to walk down the street, though a patch of brush, past an angry dog, and then up a narrow lane with your friend's house at the end. the map tells you where to go, but just reading that map is not enough to get you there.

you know how to walk down the street just fine, but when you come to the brush just knowing you must go through it is not enough: you have to know how to clear a path through it. and if you try to pass by the angry dog he'll probably bite you, so you also have to know how to approach the dog to keep him from attacking. only then do you get to go up the narrow lane to reach your friend'

...
[Old] Mapmaking Series
by Gordon Seidoh Worley — 2019-03-12T17:32:04.609Z

A few years ago I had a different blog than my current one and I wrote two series of posts there. I no longer think they are especially good, but thanks to my recent linking of the second series in comments on a discussion here on LW, I have gotten feedback that they were useful to a couple people. So in the spirit of making this content a little more accessible, easily findable, and linkable, I'm reposting them here with minimal editing. I'm grouping them together in two posts to cover the two series of posts I made.

This first series is about LW rationality basics largely covered in the sequences, but I wanted to express those ideas in my own words. It was mostly useful for me to do that work of putting my understanding into words, and I'm not sure how much reading the series in this post is worthwhile to others, but the second series references it, so seems worth having it here for ease of access. The second series is probably more valuable, and I'll link it here once it's up. ETA: It's up.

map and territory

let's start at the beginning. both my beginnings with rationality and the first place you should start to find the winning way.

by now the metaphor seems well worn to me, but i have to remember that there was a time when i didn't know about it. a time when i could get confused about the correspondence between my own thoughts and reality. and even a time when i had the words to separate the two but could still get things confused. so let me try to ease you in to our first attempt to break and reintegrate your thinking: the metaphor of the map and the territory.

a wise king commands a map of perfect detail be made of his kingdom. the map makers begin the project. animals in their own and neighboring kingdoms are slaughtered to make the parchment. forests are cleared to collect all their resin and giant holes are left in the ground from extracting graphite to make the ink. the army of map makers works furiously day and night until at last they tell the king the project is finished. in the morning, they say, the map shall be revealed to him.
when the king wakes up he notices it's unusually dark. looking out the royal window, he sees that the sky has turned a yellowish tan and the light is blocked out. he stares at the sky, examining the lines that seem to have replaced the sun and clouds. slowly realization dawns on him: the sky has been replaced by a map!
the k

...
The tech left behind
by Leafcraft — 2019-03-12T14:47:16.217Z

Hello, I am asking for some insights for a research I am doing. Can you cite examples of technologies that have been forgotten? What I mean by "forgotten" is not things we don't know how to do but used to (I suspect there aren't that many), nor things that are no longer in use but used to (mechanical television), but things that were decently developed (either in theory or in practice) but never "saw the light of day" anyway.

It's my first time posting, so I won't do much policing on the answers, thanks in advance.
New York Restaurants I Love: Pizza
by Zvi — 2019-03-12T12:10:01.002Z

Previously in series: New York Restaurants I Love: Breakfast

Previously in pizza: Restaurant Guide 2: Pizza

If you don’t live in and aren’t visiting New York, and want to know how to find good pizza, refer to the guide above. It tells you what you need to know.

This series is about the particular places I love. I want to help people find them, so they can enjoy, and can help the places stay in business. It does not mean these are the best pizza places in the city. Again, some of these are some of the best places. They are the ones that have the most value to me, based on where I live and my taste in a pie.

I mostly order plain (red) pizza, either regular, Sicilian or where available Grandma Sicilian. Most toppings, in my view, detract rather than add value. I’ll take fresh garlic when available, and can be easily talked into ricotta. Where I can get prosciutto, I’ll often take that, but definitely not other meats. I can’t speak to topping quality, in most cases.

Remember, if you order in New York, use Slice.

1. Joe & Pat’s

I have never been to the original on Staten Island, but since this opened on 10th Street and 1st Avenue last year, it instantly became my clear go-to place for a pie. They aren’t cheap, but they’re worth it.

Their regular pie is quite good. a light blend:

Their vodka pie is also quite good. The real treasure is the Grandma Sicilian. Pictured with half pepperoni, because my friend wanted that yesterday.

That’s what you should usually order. It’s a huge amount of piping hot, super flavorful and perfectly balanced pizza that does everything I am looking for in this style of pizza.

I’ve also tried their pasta. The lasagna was quite good, the pesto less so, as one would expect.

It’s a kid-friendly and nice place to sit down and have an amazing pizza as a family, which isn’t something the area was missing before, and it blows everything else locally out of the water on quality except maybe Motorino, which we’ll get to in a bit.

2. Muzzarella Pizza

Good, fast, cheap. Usually one picks at most two. At Muzzarella on 14th Street and Avenue A, you get all three. The place doesn’t look like much. Until a few months ago it didn’t take credit cards. The ingredients aren’t quite top level the way they are at most other places on this list, but they know exactly how to mix and prepare them to give that classic sauce-cheese goodness mix that one craves, and the ...