Harley-ml/lesswrong huggingface dataset:
# Link: Does the following seem like a reasonable brief summary of the key disagreements regarding AI risk?

[This is a link to a question asked on the EA Forum by Aryeh Englander](https://forum.effectivealtruism.org/posts/HayWBGerpYFk3GsZR/brief-summary-of-key-disagreements-in-ai-risk). (Please post responses / discussion there.)

Does the following seem like a reasonable brief summary of the key disagreements regarding AI risk?

Among those experts (AI researchers, economists, careful knowledgeable thinkers in general) who appear to be familiar with the arguments:

*   Seems to be broad (but not universal?) agreement that:

*   Superintelligent AI (in some form, perhaps distributed rather than single-agent) is possible and will probably be created one day
*   By default there is at least a decent chance that the AI will not be aligned
*   If it is not aligned or controlled in some way then there is at least a decent chance that it will be incredibly dangerous by default

*   Some core disagreements (starred questions are at least partially social science / economics questions):

*   Just how likely are all of the above?
*   Will we have enough time to see it coming, and will it be obvious enough, that people will react appropriately in time to prevent bad outcomes?

*   Still might be useful to have _some_ people keeping tabs on it (Robin Hanson thinks about 100), but not that many

*   How hard is it to solve?

*   If easy then less time needed to see it coming, or inventors more likely to incorporate solutions by default
*   If really hard then may need a long time in advance

*   How far away is it?
*   Can we work on it profitably now given that we don't know how AGI will work?

*   If current ML scales to AGI then presumably yes, otherwise disagreement

*   Will something less than superhuman AI pose similar extreme risks? If yes: How much less, how far in advance will we see it coming, when will it come, how easy is it to solve?
*   Will we need coordination mechanisms in place to prevent dangerous races to the bottom? If yes, how far in advance will we need them?
*   If it's a low probability of something really catastrophic, how much should we be spending on it now? (Where is the cutoff where we stop worrying about finite versions of Pascal’s Wager?)

*   What about misuse risks, structural risks, or future moral risks?
*   Various combinations of these and related arguments result in anything from "we don't need to worry about this at all yet" to "we should be pouring massive amounts of research into this"
# What spiritual experiences have you had?

The [recent adversarial collaboration on spiritual experiences on Slate Star Codex](https://slatestarcodex.com/2019/12/25/acc-how-much-significance-should-we-ascribe-to-spiritual-experiences/) includes this paragraph:

> It was also discovered that people in the United States, Australia, the United Kingdom, and Scandinavia do not tend to share their spiritual experiences with others. Hood et al. wonder if this is why such spiritual experiences are thought to be uncommon (as fewer people in these societies might have heard reports of others’ spiritual experiences).

This naturally lead me to wonder, what spiritual experiences have LessWrong readers have that they are willing to share, since the readership of LW is (very likely) overwhelmingly from Nordic and Anglophone countries?

_NB: If you're like me, you find the term "spiritual experience" awkward because it suggests these are discrete events somehow separate from the rest of normal experience. The linked post mentioned the alternative term "mystical perception", which to me seems better on the "perception" front but worse on the "mystical" one. So please be a bit generous in what you count as "spiritual experience", noting it reasonably includes not just rare, special, and highly memorable experiences but also regular, ongoing experiences and common, easily-forgettable experiences because they blend together in your memory._
# We need to revisit AI rewriting its source code

It feels like community discussion has largely abandoned the topic of AGI having the self-modifying property, which makes sense because there are a lot of more fundamental things to figure out.

But I think we should revisit the question at least in the context of narrow AI, because the tools are now available to accomplish exactly this on several levels. This thought was driven by reading a blog post, [Writing BPF Code in Rust](https://blog.redsift.com/labs/writing-bpf-code-in-rust/).

BPF stands for [Berkeley Packet Filter](https://en.wikipedia.org/wiki/Berkeley_Packet_Filter), which was originally for network traffic analysis but has since been used for [tracing](https://en.wikipedia.org/wiki/Tracing_(software)) the Linux kernel. The pitch is that this can now be used to let userspace code run in the kernel, which is to say change the way the kernel is working.

The ability to use code to write code is very old; it is arguably the core insight of [LISP](https://en.wikipedia.org/wiki/Lisp_(programming_language)). But this is becoming increasingly common as a practice now, including things like writing OCaml or Haskell for generating correct C code, and increasingly powerful [code generation](https://en.wikipedia.org/wiki/Code_generation_%28compiler%29) tricks in compilers.

It's also possible to change how the compilers work now. The [Julia language](https://julialang.org/) has a package named [Cassette.jl](https://github.com/jrevels/Cassette.jl), which allows dynamically injecting behavior into the Just-in-Time compiler. As it happens both this Julia trick and the BPF trick for the Linux kernel rely on [LLVM](https://llvm.org/).

All of this means at present we can in fact write code that modifies itself, and that modifies the way it is compiled, and that modifies the way the environment runs it (assuming the environment is Linux). The infrastructure seems to be present for large-scale, multi-layer self modification to take place. This seems like it makes questions about self modification testable in a way they weren't before. The BPF and Cassette.jl tricks don't even require software written for them explicitly, they work on previously existing code whose authors had no idea such capability existed. These methods are independent of ideas like [Software 2.0](https://medium.com/@karpathy/software-2-0-a64152b37c35)/[Differentiable Programming](https://medium.com/syncedreview/julia-computing-mit-introduce-differentiable-programming-system-bridging-ai-and-science-738c8a9eb0b9), and combined they make me wonder if the safety problems we are concerned with might actually start appearing at the level of applications first.
# Has there been a "memetic collapse"?

I want to know if there has actually been a "memetic collapse" along the lines described [here](https://www.facebook.com/yudkowsky/posts/10155616782514228) and [here](https://www.lesswrong.com/posts/WQFioaudEH8R7fyhm/local-validity-as-a-key-to-sanity-and-civilization).

Does anyone have evidence or arguments in either direction? Or even ideas for how we would be able to tell?
# How was your decade?

Hi everybody,

With the New Year coming up and next year being 2020, I think it's worth reflecting on the last 10 years. As you probably remember the last Sequences post came out towards the end of the decade. How have things gone for you since then? If you're willing to share, here's some things to consider. **You can answer as much or as little of this as you want**, don't let it scare you off from posting:

(**EDIT: This prompt seems to reliably cause people to write stuff that's uncomfortable to post as a comment. If you find this happens to you, go ahead and just excerpt a part which *is* comfortable to post as a comment and post that instead.**)

## Before & After

How were things for you at the start and end of the decade? Some categories to ponder:

- Philosophy
- Skills
- Career
- Lifestyle

## Between

How did things change over time? What was your progression through the 2010's?

- What made you say "oops" and "duh"? What did you update on?
- How have your preferences changed over time? (e.g. Music, Food)
- Did you break any promises? Did you keep some difficult ones?
- Do you have any new habits? Did you break some?

## Experiences

How did you experience the last 10 years? What proportion of the time was spent on good experiences vs. bad? Any notable highlights? 

## Worth Noting

- When did you first read [The Sequences](https://www.readthesequences.com/), if ever?
- Are you overall satisfied or dissatisfied with how the decade went?
- Is there any advice you'd give to your past self? How do you think they'd react to it?
(Just so we don't have to read it 30 times, [yes, you would buy bitcoin](https://www.lesswrong.com/posts/MajyZJrsf8fAywWgY/a-lesswrong-crypto-autopsy))
- What do you think the zeitgeist of this decade was? What aspects and events do you think you'll remember the 2010's for? What do you think *society at large* will remember the 2010's for?

And of course if there's anything else you'd like to share, feel free. :)
# Stupidity and Dishonesty Explain Each Other Away

The _explaining-away effect_ (or, collider bias; or, Berkson's paradox) is a statistical phenomenon in which statistically independent causes with a common effect become anticorrelated when conditioning on the effect.

In the language of [_d_-separation](https://en.wikipedia.org/wiki/Bayesian_network#d-separation), if you have a [causal graph](https://www.lesswrong.com/posts/hzuSDMx7pd2uxFc5w/causal-diagrams-and-causal-models) X → Z ← Y, then conditioning on Z unblocks the path between X and Y.

Daphne Koller and Nir Friedman give an example of reasoning about disease etiology: if you have a sore throat and cough, and aren't sure whether you have the flu or [mono](https://en.wikipedia.org/wiki/Infectious_mononucleosis), you should be relieved to find out it's "just" a flu, because that decreases the probability that you have mono. You _could_ be inflected with both the influenza and mononucleosis viruses, but if the flu is completely sufficient to explain your symptoms, there's no _additional_ reason to expect mono.[^koller-and-friedman]

[^koller-and-friedman]: Daphne Koller and Nier Friedman, _Probabilistic Graphical Models: Principles and Techniques_, §3.2.1.2 "Reasoning Patterns"

Judea Pearl gives an example of reasoning about a burglar alarm: if your neighbor calls you at your dayjob to tell you that your burglar alarm went off, it could be because of a burglary, or it could have been a false-positive due to a small earthquake. There _could_ have been both an earthquake _and_ a burglary, but if you get news of an earthquake, you'll stop worrying so much that your stuff got stolen, because the earthquake alone was sufficient to explain the alarm.[^pearl]

[^pearl]: Judea Pearl, _Probabilistic Reasoning in Intelligent Systems_, §2.2.4 "Multiple Causes and 'Explaining Away'"

Here's another example: if someone you're arguing with is wrong, it could be either because they're just too stupid to get the right answer, or it could be because they're being dishonest—or some combintation of the two, but more of one means that less of the other is required to explain the observation of the person being wrong. As a causal graph—[^code]

[^code]: Thanks to Daniel Kumor for [example $\LaTeX$ code for causal graphs](https://dkumor.com/posts/technical/2018/08/15/causal-tikz/).

![stupidity → wrongness ← dishonesty](https://i.imgur.com/aAblF44.png)

Notably, the decomposition still works whether you count subconscious motivated reasoning as "stupidity" or "dishonesty". (Needless to say, it's also symmetrical across persons—if _you're_ wrong, it could be because _you're_ stupid or are being dishonest.)
# What are the most exciting developments from non-Europe and/or non-Northern-Hemisphere?

My vague sense of "macrohistory of the world" is fairly Euro-centric, and Northern Hemisphere Centric. And I'm not sure how much of that is to with "well, for good or for ill Europe kinda took over the world and everything is entangled with that" and how much of it is "I grew up in a European-descended country which wrote European-descended history books".

At some point I'd be interested in figuring out how to build a proper sense of how history all fits together, but for now, just wanna get bombarded with cool stuff.

What are the most important bits of history, inventions, and developments from non-Europe and non-Northern history? Ideally, stuff that ended up playing a longterm role in humanity's capabilities, but random stuff like "The Incas independently invented writing" would be good too.
# Good goals for leveling up?

I'm new to LessWrong and am trying to level up and internalize techniques to improve my life. I also want to increase my connection to the community, but that's less important for this question. I'm setting some goals for next year and was thinking about how to encode these meta-goals into SMARTer goals.

My question: what are some good goals for leveling up in a wholistic way? I don't just want to read things. I want to become conversant in them, and I want to put them into practice. I've written down some ideas below (I don't intend to do _all_ of them; they're just ideas at this point). Feel free to critique them and/or ignore them and write your own.

* Spend `X` hours per week interacting with people's Rationality posts on LessWrong and Facebook.
* Spend `X` hours per week reading Rationality blogs, posts, and sequences.
* Write `X` blog / LW post per month.
* Learn `X` new Rationality technique per week and apply it to a problem I have (and post about it).
* Read through `X` Sequence / Book

Again: As a newbie, what is a good set of goals to help me level up quickly and sustainably?

**Edit:** Based on the comments, I should specify that these would not be my _only_ goals for the year. They would just be the ones that have to do with learning and applying Rationality techniques.
# CFAR's 2019 Fundraiser

[CFAR's 2019 fundraiser](https://rationality.org/fundraiser) is live, and runs through January 15! You can donate [here](https://rationality.org/donate).

We recently did an [AMA](https://www.lesswrong.com/posts/96N8BT9tJvybLbn5z/we-run-the-center-for-applied-rationality-ama) here on LessWrong, and released posts about our [progress and plans](https://www.lesswrong.com/posts/vj6CYLuDPw3ieCB4A/cfar-progress-report-and-future-plans-1), and our [finances](https://rationality.org/resources/updates/2019/cfar-financial-overview); expect more posts soon from Dan Keys about metrics, and from Anna Salamon about our mission.

Donations to CFAR are unusually valuable at the moment—our long-term sources of funding are uncertain, which is making it difficult for us to make long-term plans and promises. We would love to have your support!
# Looking for non-AI people to work on AGI risks

I'm worried about AGI safety, and I'm looking for non-AI people to worry with. Let me explain.  
  
A lecture by futurist Anders Sandberg, online reading, and real-life discussions with my local Effective Altruist group, gave me as a non-AI person (33-yo physicist, engineer, climate activist and startup founder) the convictions that:  
  
\- AGI (Artificial General Intelligence, Superintelligence, or the Singularity) is a realistic possibility in the next decades, say between 2030 and 2050  
\- AGI could well become orders of magnitude smarter than humans, fast  
\- If unaligned, AGI could well lead to human extinction  
\- If aligned ('safe'), AGI could still possibly lead to human extinction, for example because someone's goals turned out to be faulty, or because someone removed the safety from the code  
  
I'm active for two climate NGOs, where a lot of people are worrying about human extinction because of the climate crisis. I'm also worrying about this, but at the same time, I think the chance of human extincion due to AGI is much larger. Although the chance is much larger, I don't believe it to be 100%: we could still stop AGI development, for example (I think that makes more sense than fleeing to Mars or working on a human-machine interface). Stopping development is a novel angle for many safe AI researchers, futurists, startup founders, and the like. However, many non-AI people think this is a very sensible solution, at least if all else fails. I agree with them. It is not going to be an easy goal to achieve and I see the penalty, but I think it makes the most sense from the options we have.  
  
Therefore, I'm looking for non-AI people, who are interested to work with me on common sense solutions for existential risks posed by AGI.  
  
Does anyone know where to find them?
# Folding Couch Monitor

I find working on a laptop to be pretty rough ergonomically. The only way for my wrists to be comfortable is if the laptop is flat on my lap, and keeping my wrists happy is [very important](https://www.jefftk.com/p/floor-mouse) to me. This means my neck is angled down at the screen, though, and I can't do this for very long without it getting painful.

When I'm home I usually sit on the couch to use my laptop, and I was thinking about how I might be able to do that with a screen at a pleasant height. I've read about [various approaches](https://mgsloan.com/posts/comfortable-airplane-computing/) that usually involve elevating the laptop screen and using an external keyboard, but I really like being able to use my laptop's touchpad. I decided to set up an external monitor that folds out over the couch when I want to use it, and folds away against the wall when I don't:

[![](https://www.jefftk.com/folding-monitor-in-use.jpg)](https://www.jefftk.com/folding-monitor-in-use-big.jpg)

[![](https://www.jefftk.com/folding-monitor-in-use-front.jpg)](https://www.jefftk.com/folding-monitor-in-use-front-big.jpg)

[![](https://www.jefftk.com/folding-monitor-folded-picture.jpg)](https://www.jefftk.com/folding-monitor-folded-picture-big.jpg)

I started by making some diagrams:

[![](https://www.jefftk.com/folding-monitor-diagram-horizontal.jpg)](https://www.jefftk.com/folding-monitor-diagram-horizontal-big.jpg)

[![](https://www.jefftk.com/folding-monitor-diagram-vertical.jpg)](https://www.jefftk.com/folding-monitor-diagram-vertical-big.jpg)

I cut a section of cabinet-grade plywood to a bit bigger than the monitor, and mounted the monitor in the middle. I was able to re-use some of the mounting hardware from the stand to make something where the monitor would be easily removable. Once I got the monitor on, however, it wouldn't come off again, so something went wrong there.

I made a spacer for the hinge-side, and used a pair of door hinges. Unfortunately, after Julia had painted everything I realized the spacer wasn't thick enough, so for now there's an ugly unpainted wooden shim. It's built like a door, and feels very sturdy.

The monitor can swing out all the way, enough to be used as a TV from a second couch on the far wall:

[![](https://www.jefftk.com/folding-monitor-fully-open-picture.jpg)](https://www.jefftk.com/folding-monitor-fully-open-picture-big.jpg)

You can also have it partly open, if multiple people want to hang out on the closer couch and look at something:

[![](https://www.jefftk.com/jeff-anna-monitor.jpg)](https://www.jefftk.com/jeff-anna-monitor-big.jpg)

To keep it in position when I'm using it as a computer monitor, I have a short length of string between a pair of cup hooks:

[![](https://www.jefftk.com/folding-monitor-string.jpg)](https://www.jefftk.com/folding-monitor-string-big.jpg)

I do look a little silly using it:

[![](https://www.jefftk.com/folding-monitor-jeff-obscured.jpg)](https://www.jefftk.com/folding-monitor-jeff-obscured-big.jpg)

I built this today and have only just started using it, but so far I'm very happy and my neck feels great!
# What will quantum computers be used for?

In the early days of electronic computers, machines like [ENIAC](https://en.wikipedia.org/wiki/ENIAC) were used for niche applications such as calculating ballistic trajectories and simulating nuclear explosions. These applications are very far removed from what computers are predominantly used for today.

It seems we have reached a similar development stage with regard to quantum computing. The applications researchers cite, cryptographic analysis and quantum systems simulation, are once again very far removed from everyday life.

Does anyone have a prediction on what quantum computers will be used for once they become affordable enough for regular people? Or will it forever remain just a research tool? Are quantum computers the key to cracking quantum chemistry and thereby molecular nanotechnology? (this is my guess as to what the actual impact of quantum computing will be)
# Since figuring out human values is hard, what about, say, monkey values?

So, human values are fragile, vague and possibly not even a well defined concept, yet figuring it out seems essential for an aligned AI. It seems reasonable that, faced with a hard problem, one would start instead with a simpler one that has some connection to the original problem. For someone not working in the area of ML or AI alignment, it seems obvious that researching simpler-than-human values might be a way to make progress. But maybe this is one of those false obvious ideas that non-experts tend to push after a cursory learning about a complex research topic.

That said, assuming that the value complexity scales with intelligence, studying less intelligent agents and their version of values maybe something to pursue. Dolphin values. Monkey values. Dog values. Cat values. Fish values. Amoeba values. Sure, we lose the inside view in this case, but the trade-off seems at least being worthy of exploring. Is there any research going in that area?
# Theories That Can Explain Everything

It's generally accepted here that theories are valuable to the extent that they provide testable predictions. Being falsifiable means that incorrect theories can be discarded and replaced with theories that better model reality (see [Making Beliefs Pay Rent](https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences)). Unfortunately, reality doesn't play nice and we will sometimes possess excellent theoretical reasons for believing a theory, but that theory will possess far too many degrees of freedom to make it easily falsifiability.

The prototypical example are the kinds of hypotheses that are produced by evolutionary psychology. Clearly all aspects of humanity have been shaped by evolution and the idea that our behaviour is an exception would be truly astounding. In fact, I'd say that it is something of an [anti-prediction](https://wiki.lesswrong.com/wiki/Antiprediction).

But what use is a theory that doesn't make any solid predictions? Firstly, believing in such a theory will normally have a significant impact on your priors, even if no-one observation would provide strong evidence of its falsehood. But secondly, if the existing viable theories all claim A and you propose viable a theory that would be compatible with A or B, then that would make B viable again. And sometimes that can be a worthy contribution in and of itself. Indeed, you can have a funny situation arise where people nominally reject a theory for not sufficiently constraining expectations, while really opposing it because of how people's expectations would adjust if the theory was true.

See also: [Building Intuitions on Non-Empirical Arguments in Science](https://slatestarcodex.com/2019/11/06/building-intuitions-on-non-empirical-arguments-in-science/)
# LW Tel Aviv: Global Poverty: An eye level view

[Ofir Reich](https://www.facebook.com/ofir.reich), former Data Scientist at Center for Effective Global Action (CEGA), will be speaking on "Global Poverty, an eye level view."  

We meet at Google, 12th floor 98 Yigal Alon Street, Tel Aviv.

Summary:

What do people living in extreme poverty, who make up somewhere between one sixth and half of the world's population (depending on how you count), look like? What houses do they live in? What do they work in? What do they eat? How do they conduct themselves financially, socially and with their families? Why do they have so many children? Is their situation getting better or worse? And how is it actually that some countries are 100 times poorer than others? We'll have experiences and photos, statistics, economic analysis and modern research.
# Excitement vs childishness

I've [heard](https://www.youtube.com/watch?v=m_R5Z4_khNw&t=5540s) Robin Hanson and others make the argument that people are often biased towards the [fast takeoff](https://wiki.lesswrong.com/wiki/AI_takeoff) scenario because it's exciting to think about (or "sexy"). On the other hand, there is a bias towards disbelieving the fast-takeoff scenario because it's childish.

I think most of us can agree that both are indeed biases, i.e., should be assigned zero weight, because both are about attributes that don't correlate with what's true. So we have the excitement-bias and the childishness-bias. The question is, how do they compare?

It feels obvious to me that the childishness bias is far stronger. I see people signaling maturity all the time, and childishness seems to be extremely low status in the relevant circles. It's so low status that it's not uncommon to see people say things about dangers from AI that are accurately summarized as "even though I don't know anything about this topic, I will evaluate its legitimacy based on how many childish sounding arguments I hear because obviously, they are not the real concern and people who defend them have zero credibility." Even among those who provide other arguments, it seems more common than not that they also assume that the fast takeoff scenario is less likely a priori because it's childish. Conversely, I've never heard anyone imply that fast takeoff must be true, or even likely, because it's exciting. It seems that, to have a similar effect, the excitement bias would have to do some heavy lifting in a purely subconscious way.

Given all this, the fact that I've seen far more discussion about the excitement-bias than the childishness-bias seems wrong.

_Disclaimer: I don't think this is a strong argument that fast takeoff is likely, and I also don't think that bias towards excitement is weak in general – just that it's weak among the relevant class of people. Finally, I don't think this bias is common on LessWrong, only just about everywhere else._
# CFAR Participant Handbook now available to all

[Google Drive PDF](https://drive.google.com/file/d/1UZYBtOJ3QZ7FTI_4eKjVzBSNUqC_Uba3/view)

Hey, guys—I wrote this, and CFAR has recently decided to make it publicly available. Much of it involved rewriting the original work of others, such as Anna Salamon, Kenzie Ashkie, Val Smith, Dan Keys, and other influential CFAR founders and staff, but the actual content was filtered through me as single author as part of getting everything into a consistent and coherent shape.

I have mild intentions to update it in the future with a handful of other new chapters that were on the list, but which didn't get written before CFAR let me go. Note that such updates will likely not be current-CFAR-approved, but will still derive directly from my understanding of the curriculum as former Curriculum Director.
# Meta-discussion from "Circling as Cousin to Rationality"

A lot of the discussion on [this post](https://www.lesswrong.com/posts/jLwFCkNKMCFTCX7rL/circling-as-cousin-to-rationality) ended up being about LessWrong norms. I've moved that particular thread over to the comments here, and left a comment there pointing over here.

(Some of those comments, including the initial one, were object-level relevant to this post. I apologize for moving all of them indiscriminately. Our comment-moving-features are a bit janky and it's easier to move an entire thread than individual subthreads. I also apologize for breaking a lot of the comment-permalinks in that thread, and we'll look into fixing those. Meanwhile, you can actually still hover-over the comments in question on LW to see a preview of the comment, and you can also copy the comment-hash from the link and apply it to the new post to get a working link)
# Running and Optimizing

Two years ago I decided I was going to start [running between my house and the subway station](https://www.jefftk.com/p/running) each day on my commute, for a total of about five miles a week. Initially I was very interested in getting faster, and would time myself a couple days a week, trying to beat my previous best:

[![](https://www.jefftk.com/running-times-spring.png)](https://www.jefftk.com/running-times-spring-big.png)

Unfortunately, after six months I was pushing myself too hard, and my knees started hurting. I stopped timing my runs and stopped running so hard, and they got better again. I'm still running, but at a gentler pace.

This experience was a good illustration of how optimizing for a metric can often bring you towards your overall goals for a while, but then continuing to optimize on it can start to bring you away from them again.
# Open & Welcome Thread - January 2020

If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a [shortform post](http://www.lesslong.com/))

And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1).

The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Predictably Predictable Futures Talk: Using Expected Loss & Prediction Innovation for Long Term Benefits

I recently gave a talk at the [3rd Oxford Workshop on Global Priorities Research](
https://globalprioritiesinstitute.org/3rd-oxford-workshop-on-global-priorities-research/)

This covered a few topics:
1. How we can think about and use Expected Loss to understand and optimize judgemental forecasting setups.
2. Some thoughts around how (1) applies to our decisions about long-term forecasting.
3. A few slides on Foretold and how similar systems could be used to help with (1) and (2).

I recently recorded a quick version of this talk on YouTube. As always, I'd be curious to get feedback on these ideas.

## Original Talk Abstract

The last several decades have brought a large number of contributions to statistical and judgemental forecasting endeavors. The next several decades may bring significantly more.

As things get more advanced, not only will our understanding of how to predict distant events improve, but our understanding of the accuracy of these forecasts will improve as well. This should help us to resolve questions regarding when and how much to trust these forecasts.

This talk will present an overview of what we can expect if things go well and how thinking about this can inform our current actions when optimizing for the long term future.

## Related Links:

[Foretold](https://www.foretold.io/)

[Expected Loss Graphs](https://observablehq.com/@oagr/differential-entropy-test/2)

Blog posts on "Amplifying generalist research via forecasting":

https://www.lesswrong.com/posts/cLtdcxu9E4noRSons/part-1-amplifying-generalist-research-via-forecasting-models

https://www.lesswrong.com/posts/FeE9nR7RPZrLtsYzD/part-2-amplifying-generalist-research-via-forecasting
# Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal

> Reviving an old General Semantics proposal: borrowing from scientific notation and using subscripts like 'Gwern~2020~' for denoting sources (like citation, timing, or medium) might be a useful trick for clearer writing, compared to omitting such information or using standard cumbersome circumlocutions.

Moved to [`gwern.net`](https://www.gwern.net/Subscripts)
# Are "superforecasters" a real phenomenon?

In [https://slatestarcodex.com/2016/02/04/book-review-superforecasting/,](https://slatestarcodex.com/2016/02/04/book-review-superforecasting/,) Scott writes:

> …okay, now we’re getting to a part I don’t understand. When I read Tetlock’s paper, all he says is that he took the top sixty forecasters, declared them superforecasters, and then studied them intensively. That’s fine; I’d love to know what puts someone in the top 2% of forecasters. But it’s important not to phrase this as “Philip Tetlock discovered that 2% of people are superforecasters”. This suggests a discontinuity, a natural division into two groups. But unless I’m missing something, there’s no evidence for this. Two percent of forecasters were in the top two percent. Then Tetlock named them “superforecasters”. We can discuss what skills help people make it this high, but we probably shouldn’t think of it as a specific phenomenon.

But in this article [https://www.vox.com/future-perfect/2020/1/7/21051910/predictions-trump-brexit-recession-2019-2020](https://www.vox.com/future-perfect/2020/1/7/21051910/predictions-trump-brexit-recession-2019-2020,), Kelsey Piper and Dylan Matthews write:

> Tetlock and his collaborators have run studies involving tens of thousands of participants and have discovered that prediction follows a power law distribution. That is, most people are pretty bad at it, but a few (Tetlock, in a Gladwellian twist, calls them **[“superforecasters”](https://smile.amazon.com/dp/B00RKO6MS8/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1)**) appear to be systematically better than most at predicting world events.

seeming to disagree. I'm curious who's right.

So there's the question of "is superforecaster a natural category" and I'm operationalizing that into "do the performances of GJP participants follow a power-law distribution, such that the best 2% are significantly better than the rest"?

Does anyone know the answer to that question? (And/or does anyone want to argue with that operationalization?)
# Old Airports

One thing I enjoy about aerial imagery is that you can see traces of how cities used to be. Tracing the spaces left behind by [old train lines](https://www.jefftk.com/p/medford-branch-railroad) is a lot of fun, but recently I've been looking at some airports:

Old [Saint-Pierre Airport](https://en.wikipedia.org/wiki/Saint-Pierre_Airport) ([link](https://goo.gl/maps/Cnfpdw1JoUnDoV4m8)), now with housing:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541282/old-airport-st-pierre-2x_jdjwja.png)

Old [Jijiga Airport](https://en.wikipedia.org/wiki/Jijiga_Airport) ([link](https://goo.gl/maps/WWamBGWa2dUi23Zz7)), also with housing, mostly recognizable as a piece of the street grid that doesn't match:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541281/old-airport-jijiga-2x_fqee0t.png)

Old [Guangzhou Airport](https://en.wikipedia.org/wiki/Guangzhou_Baiyun_International_Airport_(former)) ([link](https://goo.gl/maps/k9Y3qyusHs7m74Kq7)), barely visible:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541279/old-guangzhou-airport-2x_byclqf.png)

Old [Quito Airport](https://en.wikipedia.org/wiki/Old_Mariscal_Sucre_International_Airport) ([link](https://goo.gl/maps/19ddiZ6V3K5eypX69)), now a large park and very visible. I think this the airport we used when we visited Ecuador [in 2012](https://www.jefftk.com/news/quito):

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541281/old-quito-airport-2x_dlstfx.png)

Old [Rio Branco Airport](https://en.wikipedia.org/wiki/Presidente_M%C3%A9dici_International_Airport) ([link](https://goo.gl/maps/4GMpGST4vrqbvMDx6)) is now a road:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541279/old-rio-branco-airport-2x_hcssyg.png)

Old [Austin Airport](https://en.wikipedia.org/wiki/Robert_Mueller_Municipal_Airport) ([link](https://goo.gl/maps/Lj6L8S9sFzHyV53H9)) is very hard to recognize. All the ones above were single-runway, but Mueller airport had two large runways at an angle. Single-runway ones stand out a lot more, but this is still visible as a section of the city that doesn't really fit:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541283/old-austin-airport-2x_ildubl.png)

Old [Lima Airport](https://en.wikipedia.org/wiki/Limatambo_International_Airport) (link) closed sixty years ago and has been completely built over. But the runways survive as a pair of crossing roads:

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/v1578541280/old-lima-airport-2x_qjf1ek.png)

There's a list [on Wikipedia](https://en.wikipedia.org/wiki/List_of_defunct_international_airports) but my favorites are ones where I've been looking at an aerial photo and thought "there *has* to have been an airport there!"

*Comment via:* [*facebook*](https://www.facebook.com/jefftk/posts/10100131176742592)
# Offer of co-authorship

I am offering interested persons to become a co-author on (a revised version of) my paper "[forecasting using incomplete models](https://arxiv.org/abs/1705.04630)". The paper was rejected from the Electronic Journal of Statistics with possibility to resubmit. The main criticism of the reviewers was: not enough citations and discussion of relation to other work. The job of the collaborator would be to revise the paper in order to (i) address this concern (ii) improve the overall presentation and make it more accessible. After that, the paper will be resubmitted to EJS (and/or other publications venues, according to what we will decide together).

Naturally, I will work closely with the collaborator and review the changes ey make, but ey are expected to do the lion share of the work in the revision.

The requirements for a candidate collaborator are:

* Must: The knowledge required to understand the paper in full, including all the technical details. It is fine if ey need to refresh eir memory or look up particular theorems / definitions, but ey need to have enough background for that to be efficient. To see what sort of knowledge is necessary, one is advised to skim the paper. Mostly it involves measure theory, probability theory and functional analysis.

* Advantage: Strong background in statistics, familiarity with the relevant literature

* Advantage: Experience with doing literature surveys on technical subjects

* Advantage: Experience with writing good explanations of technical subjects

* Advantage: Experience with peer-reviewed academic publications

In order to apply, email me at [rot13](https://rot13.com/) of inarffn.xbfbl@vagryyvtrapr.bet. In your email, please share with me (i) your name (ii) your background (iii) the extent to which you are compatible with the above requirements and (iv) your motivation for applying.
# A method for fair bargaining over odds in 2 player bets!

Alice and Bob are talking about the odds of some event E. Alice's odd's of E are 55% and Bob's are 90%. It becomes clear to them that they have different odds, and being good (and competitive) rationalists they decide to make a bet.

Essentially, bet construction can be seen as a bargaining problem, with the gap in odds as surplus value. Alice has positive EV on the "No" position for bets at >55% odds. Bob has neutral or better EV on the "Yes" position for bets at <90% odds.

Naive bet construction strategy: bet with 50/50 odds. Negative EV for Alice, so this bet doesn't work.

Less naive bet construction strategy: Alice and Bob negotiate over odds. The problem here, in my eyes, is that Alice and Bob have an incentive to strategically misrepresent their private odds of E in order to negotiate a better bet. If Alice is honest that her odds are 50%, and Bob lies that his odds are 70%, so they split the difference at 60%, Bob takes most of the surplus value.

If both were honest and bargaining equitably, they'd have split the difference at 72.5% instead. So I'll call 72.5% the "fair" odds for this bet.

A nicer and more rationalist aligned bet construction strategy wouldn't reward dishonesty! So, here it is.

1\. Alice and Bob submit their maximum bets and their odds.

2\. Take the minimum of the two maximum bets. Let's say its $198.

3\. Construct 99 mini bets*; one at 1% odds of E, 2% odds of E... 99% odds of E. Each player automatically places 2$ on each mini bet that is favorable according to their odds ($198/99 = $2).

*99 chosen for simplicity. You could choose a much higher number for the sake of granularity.

So, in this case, Alice accepts the No position on all bets at =>55% odds, and Bob accepts the Yes position on all bets at =<90% odds, so they make 35 $2 bets, the average odds of which are 72.5%, which is the fair odds.

Observe that there is no incentive for either player to have misrepresented their odds. If Alice overrepresented her odds as 60%, she would just deny herself the ability to bet on bets at 56% through 59%, which have positive EV for her.

Note that Alice and Bob only bet $70 -- less than half of the maximum bet. If Bob wanted Alice to bet more money than she was really willing to risk, he might try to convince her that his odds were close to hers, such that a high maximum bet would still lead to a low actual bet. Does this seem like a problem to you? I think this method is still an improvement.

*The mini bets can be abbreviated analytically as one bet at average odds, I just like the mini bets concept for making the intuition clear.

--

I am not sure how exciting this method is to anyone. I like it because misrepresentation of value is a core problem in 2 player bargaining, and I realized betting is a bit of a special case. Other special cases exist too-- anything where players would be willing to accept uncertainty over the size of the trade (and are trading a continuous good such that that's even possible).
# Please Critique Things for the Review!

I’ve spent a [lot of time defending LW authors’ right to have the conversation they want to have](https://www.lesswrong.com/posts/5Ym7DN6h877eyaCnT/meta-tations-on-moderation-towards-public-archipelago), whether that be early stage brainstorming, developing a high context idea, or just randomly wanting to focus on some particular thing. 

LessWrong is not only a place for finished, flawless works. Good intellectual output requires both [Babble](https://www.lesswrong.com/posts/i42Dfoh4HtsCAfXxL/babble) and [Prune](https://www.lesswrong.com/posts/rYJKvagRYeDM8E9Rf/prune), and in my experience the best thinkers often require idiosyncratic environments in order to produce and refine important insights. LessWrong is a **full-stack intellectual pipeline.** 

But the 2018 Review is  supposed to be *late stage* in that pipeline. We’re pruning, not babbling here, and criticism is quite welcome. We’re deliberately offering just as much potential prize money ($2000) to reviewers as to the top-rated authors.

Nominated authors had the opportunity to opt out of the review process, and none of them did. Getting nominated is meant to feel something like “getting invited to the grown-ups table”, where your ideas are subjected to serious evaluation, and that scrutiny is seen as a sign of respect.

In my current expectations, the Review is one of the primary ways that LessWrong ensures high epistemic standards. But how well that plan works is proportional to how much effort critics put into it.

The Review and Voting Phases will continue for another until January 19th. During that time, review-comments will appear on the voting page, so anyone considering how to vote on a given post will have the opportunity to see critiques. The reviews will appear abridged initially, so I’d aim for the first couple sentences to communicate your overall takeaway. 

The Review norms aren’t “literally anything goes” – ridicule, name-calling etc still aren’t appropriate. I’d describe the intended norms for reviews as “professional”. But, posts nominated for the Review should treated as something like “the usual frontpage norms, but with a heavier emphasis on serious evaluation.”

I’m still not sure precisely what the rules/guidelines should be about what is acceptable for the final Best of 2018 Book. In some cases, a post might make some important points, but also make some unjustified claims. (I personally think Local Validity as Key to Sanity and Civilization falls in this category). My current best guess is that it’d be fine if such posts end up in the book, but I’d want to make sure to also include reviews that highlighted any questionable statements.

Happy Critiquing!
# Is it worthwhile to save the cord blood and tissue?

My wife and I are expecting a baby this May. I only recently learned about this, but apparently there is an option to save the cord blood and tissue during the delivery. This seems potentially very useful, so I'm wondering if anyone has done research into this or has found trustworthy, informative resources on this topic.

My current understanding is that this tissue has a lot of stem cells. And the stem cells are useful for all sorts of medical procedures, and are even more likely to be used in the future. But currently we don't have an easy (or cheap?) way to get them.

Broad questions I have:

*   How valuable are the stem cells right now and how valuable are they expected to be in the future?
*   How hard is it to get stem cells for yourself / your child right now vs in the future?
*   Will the collected stem cells be only useful for the baby or the mother too?
*   Can we reasonably expect the cryo companies to last long enough and not go under?
*   Have you had experience donating it?
*   Have you had experience storing it?
# Using Vickrey auctions as a price discovery mechanism

I recommend "[Pricing niche products: Why sell a mechanical keyboard kit for $1,668?](https://kevinlynagh.com/notes/pricing-niche-products/)" for providing a practical case study in price dynamics that helped with my economic intuitions.

The author's friend had created a new custom keyboard kit. Their friend's previous kit had sold out in minutes, so clearly something was amiss with their "estimate costs and premiums and then set a price" approach:

> I’m not a fan of this inside-out approach, for several reasons:
>
> *   factors like “brand premium” are inherently subjective — the temptation to compare to others limits potential upside and differentiation
> *   picking a (new, higher) price may have reputational downsides (because o_f course_ your customers spend all day in mechanical keyboard chat rooms and may gripe about you “selling out the community”)
> *   you will second guess yourself regardless of the outcome; either you sell out again (goto 0) or you sell too few and then must live with the shame of having $20k worth of unsold keyboard in your garage
>
> The most compelling argument against simply picking a price, though, is that it _limits how much you can learn about your market_.

Instead, they run a [Vickrey auction](https://en.wikipedia.org/wiki/Vickrey_auction) (or "second-price sealed-bid auction") and find that the demand curve supports 3x the list price they would have chosen:

> I can’t overstate the benefits of knowing the demand curve.

> In my friend’s case, the auction let them sell far above their initial price _and_ revealed that the market was deep enough to justify a larger production run.

(I discovered this post via [The Prepared](https://theprepared.org/newsletter), a newsletter that I'd strongly recommend.)