trentmkelly/LessWrong-43k huggingface dataset:
On Robin Hanson’s Board Game
by Zvi — 2018-09-08T17:10:00.263Z

Previously: You Play to Win the Game, Prediction Markets: When Do They Work?, Subsidizing Prediction Markets

An Analysis Of (Robin Hanson at Overcoming Bias): My Market Board Game

Robin Hanson’s board game proposal has a lot of interesting things going on. Some of them are related to calibration, updating and the price discovery inherent in prediction markets. Others are far more related to the fact that this is a game. You Play to Win the Game.

Rules Summary

Dollars are represented by poker chips.

Media that contains an unknown outcome, such as that of a murder mystery, is selected, and suspects are picked. Players are given $200 each. At any time, players can exchange $100 for a contract in all possible suspects (one of which will pay $100, the rest of which will pay nothing).

A market is created for each suspect, with steps at 5, 10, 15, 20, 25, 30, 40, 50, 60 and 80 percent. At any time, each step in the market either contains dollars equal to its probability, or it has a contract good for $100 if that suspect is guilty. At any time, any player can exchange one for the other – if there’s a contract, they can buy it for the listed probability. If there’s chips there, you can exchange a contract for the chips. Whoever physically makes the exchange first wins the trade.

At the end of the game, the winning contract pays out, and the player with the most dollars wins the game.

Stages of Play

We can divide playing Robin’s game into four distinct stages.

In stage one, Setup, the source material we’ll be betting on is selected, and the suspects are generated.

In stage two, the Early Game, players react to incremental information and try to improve their equity, while keeping an eye out for control of various suspects.

In stage three, the Late Game, players commit to which suspects they can win with and lock them up, selling off anything that can’t help them win.

In stage four, Resolution, players again scramble to dump now-worthless contracts for whatever they can get and to buy up the last of the winning contracts. Then they see who won.

Setup

Not all mysteries will be good source material. Nor do you obviously want a ‘certified good’ source. That’s because knowing the source material creates a good game, is a huge update.

A proper multiple-suspects who-done-it that keeps everyone in suspense by design keeps the scales well-balanced, ensuring that early resolutions are ...
[Hammertime Final Exam] Accommodate Yourself; Kindness Is An Epistemic Virtue; Privileging the Future
by tcheasdfjkl — 2018-09-08T05:54:18.095Z

[this is my entry for the Hammertime Final Exam. I answered all three prompts but took much longer than five minutes writing each part.]

1. Accommodate Yourself

(related:

Society Is Fixed, Biology Is Mutable

;

Design

;

Radical Acceptance

as acknowledgement of reality. this is one of the first & most valuable lessons I have gained from the rationalist community, though I don’t think I’ve seen it stated in quite these terms.)

People often want to make themselves better - stronger, more hardworking, more able. We compare ourselves to ideals and we find ourselves lacking; we strive to improve ourselves to better fit the roles we want to play in the world.

What if, instead of taking the world as given and striving to adapt ourselves to it, we took ourselves as given and looked for ways to adapt our world to us?

Examples:

“I can’t get around much because my feet hurt - and even taking public transit is bad because I have to stand while I wait for the train - so I have to fix my feet or else I’m doomed” → “My feet hurt, so I’ll look for other ways to get around, like a bike or an electric scooter. When I take public transit, I’ll carry a light portable stool to make waiting for trains painless.”
“I have to stop picking at my nails! Argh, why can’t I do it!” → “Let me see if having a thing to fidget with removes this urge. Oh, it basically does! Good!”
“Argh why can’t I follow this conversation, everyone else seems to be able to do it” → “Hmm, it seems I can’t follow conversations well in loud spaces, let me make plans in quieter ones instead.”
“I should be more hardworking and not procrastinate so much!” → “Hmm, it seems I don’t have a lot of energy and my executive function is often not very good. Let me scale back my plans to match my energy levels, at least for now, and think about how I might make my work fit my brains and/or find an environment that’s easier for me to work in and/or find a different way to support myself and/or look into ADHD medication.”
“I don’t fit into these jeans, I’m so fat, I need to lose weight.” → “Let me find some other jeans, these ones don’t fit me.”

To apply this mindset, think of a situation where a limitation of yours leads to failure or frustration. Now, flip your view of the situation around. You are as you are; it is the world that does not fit. Now, if the world were designed for people like you, how would it be different? What in yo...
Archiving link posts?
by Said Achmiz — 2018-09-08T05:45:53.349Z

Link rot is a huge problem. At the same time, many posts on Less Wrong—including some of the most important posts, which talk about important concepts or otherwise advance our collective knowledge and understanding—are link posts, which means that a non-trivial chunk of our content is hosted elsewhere—across a myriad other websites.

If Less Wrong means to be a repository of the rationality community’s canon, we must take seriously the fact that (as gwern’s research indicates) many or most of those externally-hosted pages will, in a few years, no longer be accessible.

I’ve taken the liberty of putting together a quick-and-dirty solution. This is a page that, when loaded, scrapes the external links (i.e., the link-post targets) from the front page of GreaterWrong, and automatically submits them to archive.is (after checking each link to see whether it’s already been submitted). A cronjob that loads the page daily ensures that as new link-posts are posted, they will automatically be captured and submitted to archive.is.

This solution does not currently have any way to scrape and submit links older than those which are on the front page today (2018-09-08). It is also not especially elegant.

It may be advisable to implement automatic link-post archiving as a feature of Less Wrong itself. (Programmatically submitting URLs to archive.is is extremely simple. You send a POST request to http://archive.is/submit/, with a single field, url, with the URL as its value. The URL of the archived content will then—after some time, as archiving is not instantaneous—be accessible via http://archive.is/timegate/[the complete original URL].)
I am the very model of a self-recursive modeler
by Jacob Falkovich — 2018-09-06T21:45:41.646Z

I am the very model of a self-recursive modeler

My consciousness encompasses itself in many meta layers

To Russel’s paradox, I have the set of all the answers

I cut the hair of all the non-self-barbering hairdressers

I think about me thinking of my thoughts with regularity

In my Cartesian theater, I see myself with clarity

I tell you that I lie without the slightest contradiction

The story of my life is told in meta-metafiction

My inner simulator replicates the total universe

I simulate myself as I am writing out this line of verse

My secrets are unknown to me, I keep them confidential

Referring to myself I’d say I’m quite self-referential

My strange loop’s Escherer than yours, it’s Bacher and it’s Godeler

I am the very model of a self-recursive modeler

Cross-posted in full from Putanumonit
No standard metric for CFAR workshops?
by ioannes — 2018-09-06T18:06:00.021Z

Update: CFAR used standard metrics in its 2015 study, which I didn't know about when drafting this post. It doesn't appear that they tracked these metrics in their most recent impact report.

My outstanding questions are in this comment.

Update #2: CFAR replies to outstanding questions here.

It seems strange that CFAR doesn't use a standardized metric to track the impact of its workshops over time.

From CFAR's mission statement:

CFAR exists to try to make headway in this domain – the domain of understanding how human cognition already works, in practice, such that we can then start the process of making useful changes, such that we will be better positioned to solve the problems that really matter.

A couple of frameworks from psychology could serve as useful metrics for assessing progress towards this mission:

Was there a change in a participant's big 5 personality traits after workshop attendance?
Was there a change in a participant's Raven's Progressive Matrices performance after workshop attendance?

As far as I can tell, CFAR's thesis is that cognitive changes will drive changes in behavior (and correspondingly, impact on the world).

I'd expect big cognitive changes to result in changes in big 5
personality traits. Specifically, I'd expect improved cognition to result in decreased neuroticism & increased conscientiousness.

I'd also expect big cognitive changes to result in improved performance on the Raven's Matrices.

In other words, if CFAR workshops drive big changes in cognition, I'd expect these changes to reflect on well-validated psychological measures. If there's not a before-workshop/after-workshop change in these measures, that would be evidence that CFAR workshops are not causing big cognitive changes in workshop participants.

As far as I know, workshop participants aren't being assessed on measures like this, so it's hard to know what impact the workshops are actually having.

CFAR current metric is "increase in expected impact" or IEI:

In May 2016, we set out to count the number of alumni who have had an increase in expected impact due to their involvement with CFAR by sending out a survey to our alumni...

For each person’s responses, we manually coded whether it seemed like 1) their current path was high-impact, 2) their current path was substantially better than

...
Quick Thoughts on Generation and Evaluation of Hypotheses in a Community
by Ben Pace — 2018-09-06T01:01:49.108Z

At lunch we talked about hypothesis evaluation and generation, and I thought I'd put some of the thoughts I had into a post.

Here’s some points that seem relevant to me when thinking about institution building around this topic.

Evaluation often needs to come a long time after the generation process.

Andrew Wiles spent over six years in his house trying to prove Fermat's Last Theorem, and he didn't try to write up the steps of his research before he proved it. Had he tried to bring a whole academic sub-community along for the ride, I'd guess he wouldn't have completed the proof before he died. It was correct for him to wait until he had a clear reason why the new math was useful (i.e. he had proven the theorem), before he presented it in public writings+lectures, else it would not have been worth it for the community to learn the speculative new math.
Einstein had a similar experience, where there was a big gap between the time when he had sufficient evidence to state and trust that the hypothesis was true, and the time when enough evidence was communicable to his fellow physics community that they believed him.
Sometimes it takes a couple of years in a small group to turn an idea into a clearly testable and falsifiable set of predictions, at which point it can be incorporated into the community.

However, evaluation by a common set of epistemic standards is required before the model that generated the hypothesis should become common knowledge. It is very costly to make something common knowledge, and is only worth the cost when the community has good reason to trust it.
It's also important that small groups attempting to test a new intuition or paradigm are able to cease work after a reasonable effort has been given. When I look around the world I see a great number of organisations living past their project, merely because it is by far the path of least resistance. It's important to be able to try again.

I remember someone saying that in Google X, all projects must come with a list of criteria such that if the criteria are ever met, the project must be cancelled. Often they're quite extreme and don't get reached, but at least there's an actual critera.

Regarding the current LW community, I see a few pieces of infrastructure missing before we're able to get all of this right.

A process that is able to check a new

...
Beauty bias:  "Lost in Math" by Sabine Hossenfelder
by avturchin — 2018-09-05T22:19:20.609Z
Link: https://www.amazon.com/Lost-Math-Beauty-Physics-Astray/dp/0465094252

This is technically just a link post for those who didn't see the book yet. The main idea, as I understand it, is that while some physical theories are beautiful, other are complex and "ugly". Beauty should not be taken as evidence for truth.

It looks like sometimes in AI safety research aesthetics may be taken as an evidence (I will not provide examples, as in each case it may be just my interpretation), and thus possibility of such beauty bias should be taken into account.
AI Governance: A Research Agenda
by habryka — 2018-09-05T18:00:48.003Z
Link: https://www.fhi.ox.ac.uk/wp-content/uploads/AI-Governance_-A-Research-Agenda.pdf

Allan Dafoe, head of the AI Governance team at FHI has written up a research agenda. It's a very large document, and I haven't gotten around to reading it all, but one of the things I would be most excited about is people in the comments quoting the most important snippets that give a big picture overview over the content of the agenda.
Counterfactuals and reflective oracles
by Nisan — 2018-09-05T08:54:06.303Z

Summary: Take the argmax of counterfactual expected utility, using a thin counterfactual which itself maximizes a priori expected utility.

Followup to: An environment for studying counterfactuals, Counterfactuals, thick and thin

Replacement for: Logical counterfactuals and differential privacy

In An environment for studying counterfactuals, I described a family of decision problems that elicit counterfactuals from an agent. In this post I'll redefine that family using reflective oracles, and define an optimal agent. I was aided in this work by discussions with
Alex Appell,
James Brooks,
Abram Demski,
Linda Linsefors, and
Alex Mennen.

The problem

Here's the (not quite) Bayes net for the problem. I changed it a little from the last post, to make the thin counterfactuals more explicit:

E controls exploration and S is a hidden state. The observation O is isomorphic as a random variable to E×S, but the particular isomorphism ^κ used by the problem is unknown to the agent.

The agent is given O and a prior P; it outputs a set C of counterfactual expected utilities, one for each action. The agent's action A is equal to the argmax of these expectations, unless it's overridden by E. The action and hidden state determine the utility U.

The dotted lines mean that P is equal to the marginal over E×O×A×U. The machinery of reflective oracles resolves the circularity.

Now I'll define this distribution in detail, using Python-like pseudocode. I ought to use a probabilistic programming language, but I don't know any of those. Note that if you have a reflective oracle, you can compute expectations of distributions in constant time, and you can compute argmaxes over sets of real numbers, with ties broken in an undetermined way. Real numbers and distributions can be represented by programs.

environment()

The function environment() takes an agent and outputs a sample from a joint distribution over E×O×A×U. The agent is a function that takes the environment and an observation and stochastically returns a set of counterfactual expectations, which is itself encoded as a function from actions to real numbers.

The environment samples

...
thought: the problem with less wrong's epistemic health is that stuff isn't short form
by the gears to ascension — 2018-09-05T08:09:01.147Z

like what if there was no long form content and just a feed of short form content?

like I'm a bit salty that that never got implemented after the conversation I had with oliver where I was like "it'll suck if you don't do this", and well, now it sucks the way I said it would :/
Hypothesis about how social stuff works and arises
by the gears to ascension — 2018-09-04T22:47:38.805Z

EDIT, 2022:

this post is still a reasonable starting point, but I need to post a revised version that emphasizes preventing dominance outside of play. All forms of dominance must be prevented, if society is to heal from our errors. These days I speak of human communication primarily in terms of the network messaging protocols that equate to establishing the state of being in communication, modulated by permissions defined by willingness to demonstrate friendliness via action. I originally wrote this post to counter "status is all you need" type thinking, and in retrospect, I don't think I went anywhere near far enough in eliminating hierarchy and status from my thinking.

With that reasoning error warning in mind, original post continues:

Preface

(I can't be bothered to write a real Serious Post, so I'm just going to write this like a tumblr post. y'all are tryhards with writing and it's boooooring, and also I have a lot of tangentially related stuff to say. Pls critique based on content. If something is unclear, quote it and ask for clarification)

Alright so, this is intended to be an explicit description that, hopefully, could be turned into an actual program, that would generate the same low-level behavior as the way social stuff arises from brains. Any divergence is a mistake, and should be called out and corrected. it is not intended to be a fake framework. it's either actually a description of parts of the causal graph that are above a threshold level of impact, or it's wrong. It's hopefully also a good framework. I'm pretty sure it's wrong in important ways, I'd like to hear what people suggest to improve it.

Recommended knowledge: vague understanding of what's known about how the cortex sheet implements fast inference/how "system 1" works, how human reward works, etc, and/or how ANNs work, how reinforcement learning works, etc.

The hope is that the computational model would generate social stuff we actually see, as high-probability special cases - in semi-technical terms you can ignore if you want, I'm hopeful it's a good causal/generative model, aka that it allows compressing common social patterns with at least somewhat accurate causal graphs.

The thing

So we're making an executable model of part of the brain, so I'm going to write it as a series of changes I'm going to make. (I'm uncomfortable with the structured-ness of this, if anyone has any ideas f...
Birth order effect found in Nobel Laureates in Physics
by Bucky — 2018-09-04T12:17:53.269Z

[Epistemic status: Three different data sets pointing to something similar is at least interesting, make your own mind up as to how interesting!]

Follow-up to: Fight Me, Psychologists, Birth Order Effects are Real and Very Strong, 2012 Survey Results, Historical mathematicians exhibit a birth order effect too

In Eli Tyre’s analysis of birth order in historical mathematicians, he mentioned analysing other STEM subjects for similar effects. In the comments I kinda–sorta preregistered a study into this. Following his comments I dropped the age requirement I mentioned as it no longer seemed necessary.

I found that Nobel Laureates in Physics are more likely to be firstborn than would be expected by chance. This effect (10 percentage points) is smaller than the effect found in the rationalist community or historical mathematicians (22 and 16.7 percentage points respectively) but is significant (p=0.044).

More brothers were found in the study then sisters (125:92 (58%)). After correcting for the correct expected ratio (~52%) this was found to not be significant (p=0.11).

I was unable to find sufficient data on Fields medal, Abel prize and Turing award winners.

My data and analysis is documented here. With Eli's kind permission I used his spreadsheet as a template. I have kept Eli’s data on the same Table – rows 4-153 are his.

Methodology

My methods matched Eli’s closely except for the data sets I looked at, see his post for more information.

Initially I attempted to replicate Eli’s results in other mathematicians by analysing Fields medal and Abel prize winners. Unfortunately I was unable to gather sufficient additional data. This is partly due to crossover in names between these mathematicians and the list from which Eli was working.

It also seems to be the case that less biographical information is available for people born after ~1950. This might be partly due to these people and their siblings being more likely to be still alive so data protection rules prevent e.g. geni from listing their full details (siblings’ details are often set to “private”) but there could be other reasons. For Fields medals awarded before 1986 I found data on 12/30 recipients, after that only 3/30.

I had a brief look at Turing award winners, as this would have seemed a relevant field to compare to the results from the rationalist community that inspired the studies, but came across the same ...
nostalgebraist - bayes: a kinda-sorta masterpost
by Kaj_Sotala — 2018-09-04T11:08:44.170Z
Link:  http://nostalgebraist.tumblr.com/post/161645122124/bayes-a-kinda-sorta-masterpost/

Extended criticism of "Bayesianism" as discussed on LW. An excerpt from the first section of the post:

Like most terms ending in -ism, [Bayesianism] can mean a number of different things. In its most limited sense, “Bayesianism” is a collection of technical/mathematical machinery, analogous to a tool or toolbox. This collection, which I will call “the Bayesian machinery,” uses a particular way of representing knowledge, and if you can represent your knowledge in that way, the machinery tells you how to alter it when presented with new evidence.

The Bayesian machinery is frequently used in statistics and machine learning, and some people in these fields believe it is very frequently the right tool for the job. I’ll call this position “weak Bayesianism.” There is a more extreme and more philosophical position, which I’ll call “strong Bayesianism,” that says that the Bayesian machinery is the single correct way to do not only statistics, but science and inductive inference in general – that it’s the “aspirin in willow bark” that makes science, and perhaps all speculative thought, work insofar as it does work. (I.e., if you’re doing these things right, you’re being Bayesian, whether you realize it or not.)

Strong Bayesianism is what E. T. Jaynes and Eliezer Yudkowsky mean when they say they are Bayesians. It is usually what I am talking about when I say I “don’t like” Bayesianism. I think strong Bayesianism is dead wrong. I think weak Bayesianism may well be true, in that the Bayesian machinery may well be a very powerful set of tools – but I want to understand why, in a way that defines the power of a tool by some metric other than how Bayesian it is.

Contents of the post:

0. What is “Bayesianism”?

1. What is the Bayesian machinery?

1a. Synchronic Bayesian machinery

1b. Diachronic Bayesian machinery

3. What is Bayes’ Theorem?

4. How does the Bayesian machinery relate to the more classical approach to statistics?

5. Why is the Bayesian machinery supposed to be so great?

6. Get to the goddamn point already. What’s wrong with Bayesianism?

7. The problem of ignored hypotheses with known relations

7b. Okay, but why is this a problem?

8. The problem of new ideas

8b. The natural selection analogy

9. Where do priors come from?

10. It’s just regularization, dude

11. Bayesian “Occam factors”