Harley-ml/lesswrong huggingface dataset:
# Excusing a Failure to Adjust

Crossposted from [Figuring](https://figuringfiguring.com) [Figuring](https://figuringfiguring.com/2020/06/05/excusing-a-failure-to-adjust/).

Here is a handle for a mistake I make all the time that works really well for me… that is the handle works really well for me. I make this mistake fairly often, but of course I notice it in other people more often. 

Imagine that your credence in a belief is represented by a physical slider. When you learn a new piece of evidence or argument or whatever, there are two kinds of mental motions you can react with. You can adjust the slider up or down, or you can explain why you don’t have to adjust the slider at all. 

Sometimes the right thing to do really is to not adjust the slider. Sometimes people will think that you should adjust the slider when really you shouldn’t for reasons that are difficult to explain. But this is far far rarer than cases where you should adjust the slider, but instead you focus on coming up with reasons to not adjust the slider. On some days, it seems like 90% of human cognition is coming up with reasons to not adjust the slider. 

And of course, for beliefs that you like, you will come up with excuses for refusing to adjust the slider down way more often than you will come up with excuses for refusing to adjust the slider up. And vice versa for beliefs that you dislike.

Probably, you would be better off banning that mental motion altogether. Nearly every new piece of evidence should cause some adjustment, even if it is very small. Most conversations should be about how much to adjust, not about whether to adjust. But banning that mental motion is difficult, and before you can, you must learn to notice the motion, and learn how it feels different from other motions. 

This is the best tool I have found for noticing so far. I ask myself: “Am I trying to explain why I don’t need to adjust the slider?”  Something different might work better for you, but I highly recommend trying at all to notice the difference.
# What is the interpretation of the do() operator?

To reductively explain causality, it has to be explained in non-causal terms, most likely in terms of total propability distributions. Pearl explains causality in terms of causal graphs which are created by conditionalizing the propability distribution on not  event, but do(event). What does this mean? It's easy enough to explain _in causal terms_: You make it so event occurs without changing any of its causal antecedents. But of course that fails to explain causality. How could it be explained without that?
# Objective Dog Ratings: An Introduction & Explanation

Because they’re not all good dogs, Brent.

Obviously, not everyone wants the same thing from dogs, but that’s not necessarily the same as saying that any dog is as good as the next, or even that any dog rating system must be subjective. If we’re going to entertain the idea that some dogs are simply better than other dogs, though, then we have to specify how that is.

What we’re looking for is a dog with a certain amount of wolfishness, a dog which is as close to being a wolf as one can get without sacrificing any of those essential characteristics which define a dog as such. Basically, a dog which a politically progressive, forward-thinking wolf would not be ashamed to know. They must be loyal, intelligent, and hardworking, they must have a sense of dignity, they must like humans, and above all they must be healthy. A dog which is perfect in every other way, but is unhealthy, is a bad dog, because it would not be good to _be_ that dog. 

Unfortunately, I’m incapable of doing anything without taking it at least a little bit seriously, so ratings will be on a three-star scale, from ★ (Mediocre) to ★★★ (Good). It is also possible that some dogs will not get any stars at all. Those are bad dogs, Bront. 

I do not know which dog breed will turn out to be _the _dog breed, the dog of the gods, but I do have my suspicions (some breed of spitz-type, probably), and it’s important to note that this has nothing to do with how much I personally like a dog. Some of my favorite dogs will get no more than two stars, and some may even get just one star. This isn’t “Dogs which are the best at sitting in my lap and being petted,” or “Most Instagrammable dogs.” 

Let us begin.
# Thoughts on Neuralink update?

[Here](https://www.youtube.com/watch?v=DVvmgjBL74w) is the full stream.

I haven't seen LessWrong discuss BCI technology all that much, so I'm curious what some of the people here think about the current SOTA, and whether such devices will eventually be able to live up to their lofty promises.
# microCOVID.org: A tool to estimate COVID risk from common activities

This is a linkpost for a model and web tool (that I and several friends created) to quantitatively estimate the COVID risk to you from your ordinary daily activities:

This website contains three outputs of our work:

1.  **a web** [**calculator**](https://www.microcovid.org/calculator) that you can use to calculate your COVID risk (in units of microCOVIDs, a 1-in-a-million chance of getting COVID).
2.  **a** [**white paper**](https://www.microcovid.org/paper) that explains our estimation method. EAs might be particularly interested in the footnotes throughout, and the detailed [research sources](https://www.microcovid.org/paper/14-research-sources) section.
3.  **a** [**spreadsheet**](https://www.microcovid.org/spreadsheet) to compute your COVID risk in more detail and to track your risk over time. EAs might find this more customizable and powerful than the web calculator.

If you have different beliefs than us and would like to use a version of the model that reflects your beliefs rather than ours, you can make modifications to your copy of the spreadsheet, or fork the repository and make a personal copy of the web calculator. We also hope you will [submit suggestions](https://www.microcovid.org/contact), either by emailing us or by making issues or pull requests directly on Github.

Our group house has been using this model as the basis of a shared agreement/protocol, based on a budget of 3,000 microCOVIDs per year to spend outside the house (about 58 per week). We know of another group house that (last we heard) was operating on \*4\* microCOVIDs per week!

We hope this helps you personally live a better pandemic life with more safety *and* more flexibility.

(also linkposted to the [EA Forum](https://forum.effectivealtruism.org/posts/MACKemu3CJw7hcJcN/a-tool-to-quantitatively-estimate-the-covid-risk-to-you-from ))
# Plans / prepping for possible political violence from upcoming US election?

The US presidential election is now just 2 months away! There's a (IMO) plausible argument that we might have a disputed election that leads to large-scale political violence; for details see

*   [Vox: "Will He Go? A law professor fears a meltdown this November"](https://www.vox.com/policy-and-politics/2020/6/3/21257133/trump-2020-election-meltdown-lawrence-douglas)
*   [The American Interest: "The Danger of Electoral Violence in the United States"](https://www.the-american-interest.com/2020/06/19/the-danger-of-electoral-violence-in-the-united-states/)
*   [Metaculus: If President Trump loses the 2020 election, will he concede?](https://www.metaculus.com/questions/4609/if-president-trump-loses-the-2020-election-will-he-concede/)

I don't think it's likely, but I do think it's likely *enough* to make it worth some contingency planning (for US residents like me).

Like most of us, I'm not in any position to actually *do* anything about this problem, i.e. to reduce the probability of large-scale political violence from a disputed election. I just want to prudently plan for myself, family, friends, and/or community to get safely through whatever happens, if anything happens.

I'm curious what (if anything) people here are doing, or plan to do, or would recommend, to prepare for this contingency. Stock up on food / water / supplies? Take an extended vacation to another country, or to a lower-risk place within the country? (And where would that be?) Move assets into gold / bitcoin / international index funds / whatever? Buy weapons? Or what?

I'm *especially* interested to hear from people who have experience or knowledge of what large-scale political violence is actually like, from first- or second-hand experience, or from being well-informed about history and world politics (which I am not). And for people in the USA, what are your personal plans? But everyone is very welcome to answer / comment / discuss!
# (Humor) AI Alignment Critical Failure Table

The "Friendly AI Critical Failure Table" was originally posted by Eliezer in 2003; when I ran across a mention of it somewhere, I realized that a lot of people probably haven't seen this old classic yet.

(For people not familiar with the concept, "critical failure tables" are found in some tabletop role-playing games. If you are trying to do something and fail really badly (a "critical failure"), you may be required to consult a table that contains a list of catastrophic outcomes that your screw-up might cause, rolling some dice to determine which one of them you did cause.)
# Why is Bayesianism important for rationality?

My impression from the Sequences seems to be that Eliezer considers Bayesianism to be a core element of rationality. Some people have even referred to the community as Bayesian Rationalists. I've always found it curious, like it seemed like more of a technicality most of the time. Why is Bayesianism important or why did Eliezer consider it important?
# SlateStarCodex Online Meetup: Connor Leahy on GPT-N

**Title of talk**: _We Got Our AGI Firealarm, Now What?_

**Summary**: There has been much disagreement over how big of a deal GPT3 is in the larger context of progress towards strong AGI. While most pundits will claim it's not "really" "intelligent" or "conscious" or that it doesn't "understand" words, I want to make the case that GPT3 (or its near term successors) may be our final warning shot before we see real AGI emerge. And as a bonus, I will show you how to build your very own AGI in three easy steps! Why wait for Deepmind, you can build your very own paperclip maximizer!

September 13th at 10:30 PDT, 17:30 UTC, 20:30 IDT

Please sign up here by an hour before the event, and we will send you an invitation. [https://forms.gle/4eeL6DX6jjBGykMJ6](https://forms.gle/4eeL6DX6jjBGykMJ6)
# If there were an interactive software teaching Yudkowskian rationality, what concepts would you want to see it teach?

I've been noticing some complaints (such as [this post by Richard Ngo](https://www.lesswrong.com/posts/FuGfR3jL3sw6r8kB4/ricraz-s-shortform?commentId=75gg3Nzn6KMst8eNt)) lately about the quality of the modern LW community's contribution to the big picture of humanity's knowledge.

Ideally, if it were the case that reading something automatically made you internalize deeply everything it said, then just by having a group of people who have read The Sequences, you'd have a superteam of intellectuals. And while I do think LW is a pretty cool group of smart thinkers, that isn't fully the case- just reading The Sequences isn't enough. To really internalize the lessons that one must learn, one must apply the principles, push against the problem, see where their understanding needs improvement, and where they are good enough.

The simplest form of this is having a high-quality Anki deck that tests users on the principles, both by testing recall of the stated principle itself, and even more importantly, giving them test cases where they can apply the principles (in the same vein as Ankifying medium-difficulty multiplication problems).  I have seen some rationality-themed Anki decks, but many of the cards are poorly formatted (both esthetically and in terms of learnability), and are also poorly curated. Ideally, if there were to be an Anki deck, it would be well formatted, and the cards would be carefully chosen to maximize quality of information.

Another idea that I've been thinking about is making explorables, a la [Nicky Case](https://ncase.me/), that would introduce important rationality concepts. This would have the advantage of providing more flexibility in experience than Anki, but also would sacrifice the benefits of having already implemented SRS.

My question is: if there were to be either an Anki deck or an explorable teaching concepts from The Sequences, targeted primarily as an aide for current LW users, but also as an introduction aimed at the public at large, what concepts from The Sequences would you most want to see covered?
# Study Group for Progress – 50% off for LessWrongers

Recently I [announced the Study Group for Progress](https://www.lesswrong.com/posts/RbK3GqT3uGxdghRfR/announcing-the-study-group-for-progress): a weekly discussion/Q&A on the history, economics and philosophy of progress, with featured guests including Robert Gordon (*Rise & Fall of American Growth),* Margaret Jacob, and Richard Nelson.

LessWrong members can now [get a 50% discount on registration by using this link](https://highergroundeducation.formstack.com/forms/progress_studies_enrollment_adult_student_50_percent_discount). Register by Monday, September 7.
# Sunday September 6, 12pm (PT) — Casual hanging out with the LessWrong community

This Sunday at 12pm (PDT), we will have another one of our weekly online meetups. This time we will simply hangout without much of an introduction. In the future we will be back with our curated talks, but my guess is having a more casual hangout meetup is actually something I would be more excited about this week.

*If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up* [*here*](https://forms.gle/iwFatbhys9muPmQA7)*.*

Gather.town
-----------

We will be meeting in [gather.town](https://gather.town/hwEYAKZHNxIG2Hgo/lesswrong). We will also have a backup Zoom room here, in case Gather Town breaks for anyone, which Charlie Steiner has conveniently volunteered to moderate (thank you!): [https://illinois.zoom.us/j/94331893306?pwd=SitHMW93ZlJpd01oOHdjTUtYVWQydz09](https://illinois.zoom.us/j/94331893306?pwd=SitHMW93ZlJpd01oOHdjTUtYVWQydz09)

Details
-------

**When?** Sunday September 6, 12pm (PT)

**Where?** [https://gather.town/hwEYAKZHNxIG2Hgo/lesswrong](https://gather.town/hwEYAKZHNxIG2Hgo/lesswrong?fbclid=IwAR2Fi3KqKSgQxp5Po_34bVNkNLC9mDk4ESHWGMRYC5tsms8JmsHZrTqprfw)

**Backup Zoom Room:** [https://illinois.zoom.us/j/94331893306?pwd=SitHMW93ZlJpd01oOHdjTUtYVWQydz09](https://illinois.zoom.us/j/94331893306?pwd=SitHMW93ZlJpd01oOHdjTUtYVWQydz09)
# What's the best overview of common Micromorts?

I want to get generally oriented on how various common risks compare against each other. I've seen some of this come up in recent Covid discussion, but I'm interested in a good article that's like "Here's all the most dangerous stuff it's likely that you do, and here's how it breaks down for various sub-activities."

This question triggered by "the first few google results not being that good."
# Open & Welcome Thread - September 2020

If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a [shortform post](http://www.lesslong.com/))

And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lessestwrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1#Getting_Started) section of the [LessWrong FAQ](https://www.lessestwrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lessestwrong.com/tag/open-threads?sortedBy=new).
# When should I be concerned about my Oura measurements indicating COVID-19?

The [Rockefeller Neuroscience Institute](https://wvumedicine.org/rni/) seems to have developed an app that can detect COVID-19 3 days before symptoms based on Oura data but the app isn't publically available. How should a user that doesn't have access to the app interpret his Oura measurements to know when they should self-isolate or get tested?
# How long does it takes to read the sequences?

Sorry if that has been answered before, I searched quickly and could not find the answer.

To be more specific, how long does it takes an average reader to read all of the texts in [https://www.lesswrong.com/rationality](https://www.lesswrong.com/rationality) ?

I already read them all (more specifically the version on https://www.readthesequences.com/ ) but I am asking so that I can give an estimate to the people I recomend it to.
# "Learning to Summarize with Human Feedback" - OpenAI

[https://openai.com/blog/learning-to-summarize-with-human-feedback/](https://openai.com/blog/learning-to-summarize-with-human-feedback/)

Eliezer's Summary/Thoughts: [https://mobile.twitter.com/ESYudkowsky/status/1301954347933208578](https://mobile.twitter.com/ESYudkowsky/status/1301954347933208578)
# Efficacy of Vitamin D in helping with COVID

so a new research found that high dose of Vitamin D significantly improve out comes for COVID patient:

[https://www.sciencedirect.com/science/article/pii/S0960076020302764?via%3Dihub](https://www.sciencedirect.com/science/article/pii/S0960076020302764?via%3Dihub)

another source on facebook claim that:

\- A study in Indonesia found that out of the patients that died from COVID-19, 98.9% of them were deficient in vitamin D, while only 4% of the patients with sufficient vitamin D died.

-A study of patients in New Orleans found that 84.6% of the COVID-19 patients in the ICU were deficient in Vitamin D while only 4% of the patients in the ICU had sufficient levels of Vitamin D.

-A study in the Philippines found that for every standard deviation increase in vitamin D people were 7.94 times more likely to have a mild rather than severe COVID-19 outcome and 19.61 times more likely to have a mild rather than critical outcome.

I couldn't find any mention of this on lesswrong, \[granted I haven't look very hard\], anyone who have done their reseach on this can help me determine the import of Vitamin D in fighting this pandemic?

and if it's true, anyway we can profit from this? any stock or index fund?
# Indignation in response to the 1890 census

In reading about the development of technology, I keep an eye out for changes in society as well. I commented recently that [we don’t seem to celebrate major achievements as much anymore](https://rootsofprogress.org/celebrations-of-progress). But it’s not just technology that Americans used to view differently. It’s growth of all kinds.

The book *Computer: A History of the Information Machine* tells the story of the 1890 census. It was the first census to be computed, not by hand, but with tabulating machines, developed by Herman Hollerith. On August 16, 1890, the grand total was announced: the population of the United States was 62,622,250.

“But”, it says, “this was not what the allegedly fastest-growing growing nation in the world wanted to hear.” It quotes a contemporary account in a periodical, *The Electrical Engineer*, from 1891 (emphasis added):

> The statement by Mr. Porter \[the census director\] that the population of this great republic was only 62,622,250 sent into **spasms of indignation a great many people who had made up their minds that the dignity of the republic could only be supported on a total of 75,000,000.** Hence there was a howl, not of “deep-mouthed welcome,” but of frantic disappointment.

The book continues:

> The press loved the story. In an article headlined “Useless Machines” the Boston Herald roasted Porter and Hollerith; “Slip Shod Work Has Spoiled the Census,” exclaimed the New York Herald; and the other papers soon took up the story.

“Spasms of indignation” because population growth was too *low* for “the dignity of the republic”. Americans were *proud* of being the fastest-growing country. Today, in contrast, people fear *over*population, and the general slowing of world population growth is generally considered to be good news.

*Something* changed in American attitudes in the last 100+ years, not just toward technology or the economy as such, but more fundamentally toward growth itself.
# Updates Thread

If you've [updated your belief](https://wiki.lesswrong.com/wiki/Belief_update) about something you think is worth noting, post it here.

*   It doesn't have to be a full blown "mind change", just an [incremental](https://www.lesswrong.com/posts/627DZcvme7nLDrbZu/update-yourself-incrementally) [update](https://www.youtube.com/watch?v=d6PgCN7ySGQ) to your beliefs.
*   I'm thinking it'd be good to have a low bar for what is "worth noting". Even if it's something trivial, I figure that the act of discussing updates itself is beneficial. For rationality practice, and for fun!
*   That said, I also expect that browsing through updates that other people on LessWrong make will lead to readers making similar updates themselves a decent amount of the time.
*   I've been developing a strong opinion that journaling and self-reflection in general is incredibly useful. Significantly underrated even among those that preach it. This thread is a way to perform such journaling and self-reflection.
# What's the CFAR position on how the workbook can be used?

The [CFAR handbook](https://rationality.org/files/cfar-handbook.pdf) is a great resource for applied rationality. I talked with another rationlist in Berlin about creating a reading group around it. Did CFAR articulate a position around how the handbook can be used and whether such a group would be okay? If so, are there any norm such a reading group should fulfill (e.g. making it clear it's independent of CFAR)?
# Is there a way to view "read" LessWrong posts?

I want to be able to see the LessWrong posts that I've already read/visited, like in a list. I mainly just want to be able to re-save some ones that I really liked but deleted from my bookmarks.

I didn't vote on all the posts I read (it's kind of like a habit I had to build), so I can't do that (although, that would be nice too) (I guess that would be a related question).
# Do mesa-optimizer risk arguments rely on the train-test paradigm?

Going by the [Risks from Learned Optimization sequence](https://www.alignmentforum.org/s/r9tYkB2a8Fp4DN8yB), it's not clear if mesa-optimization is a big threat if the model continues to be updated throughout deployment. I suspect this has been discussed before (links welcome), but I didn't find anything with a quick search.

Lifelong/online/continual learning is popular and could be the norm in future. I'm interested in how that (and other learning paradigms, if relevant) fits into beliefs about mesa-optimization risk.

If you believe the arguments hold up under a lifelong learning paradigm: is that because there could still be enough time between updates for the mesa-optimizer to defect, or some other reason? If you believe the train-test paradigm is likely to stick around, why is that?
# Sunday September 13, 12:00PM (PT) — talks by John Wentworth, Liron and more

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see [here](https://www.lesswrong.com/tag/lesswrong-events) for previous weeks' transcripts).

*   Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
*   We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

*If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up* [*here*](https://forms.gle/iwFatbhys9muPmQA7)*.*

Speakers
--------

*   John Wentworth: **Talk topic "How to detect unknown unknowns"**
    *   Previously curated posts:
        *   [Alignment As A Bottleneck To Usefulness Of GPT-3](https://www.lesswrong.com/posts/BnDF5kejzQLqd5cjH/alignment-as-a-bottleneck-to-usefulness-of-gpt-3)
        *   [Everyday Lessons from High-Dimensional Optimization](https://www.lesswrong.com/posts/pT48swb8LoPowiAzR/everyday-lessons-from-high-dimensional-optimization)
        *   [Transportation as a Constraint](https://www.lesswrong.com/posts/4s2gbwMHSdh2SByyZ/transportation-as-a-constraint)
*   Liron Shapira: **Bloated MVPs: Startups that raised money for logically-impossible ideas**
    *   Previously curated post: [How Specificity Works](https://www.lesswrong.com/posts/pFvZXFWbtvKvGiACJ/how-specificity-works)
*   Ruby Bloom: **Unprecedentedly Accessible Expertise**
    *   Previously curated post: [Conversational Cultures: Combat vs Nurture](https://www.lesswrong.com/posts/ExssKjAaXEEYcnzPd/conversational-cultures-combat-vs-nurture-v2)

Details
-------

**When?** Sunday September 13, 12:00PM (PT)

**Where?** [https://us02web.zoom.us/j/89420485417](https://us02web.zoom.us/j/89420485417)
# Notes on good judgement and how to develop it (80,000 Hours)

This post by 80,000 hours struck me as more than usually relevant to my interests in developing the art of rationality. It doesn't really say anything new, but it does provide a decent summary of a frame that I think is an important subset of epistemic rationality, in the form of "good judgement". 

> More practically, I think of someone with good judgement as someone able to:
> 
> 1.  Focus on the right questions
> 2.  When answering those questions, synthesise many forms of weak evidence using good heuristics, and weigh the evidence appropriately
> 3.  Be resistant to common cognitive biases by having good habits of thinking
> 4.  Come to well-calibrated conclusions
> 
> Owen Cotton-Barratt wrote out [his understanding of good judgement](https://forum.effectivealtruism.org/posts/xMRpu4nAeGeAXy9ns/good-judgement-and-its-components), breaking it into ‘understanding’ and ‘heuristics’. His notion is a bit broader than mine.
> 
> Here are some closely related concepts:
> 
> *   **Keith Stanovich’s work on ‘rationality’**, which seems to be something like someone’s ability to avoid cognitive biases, and is [~0.7 correlated with intelligence](https://sci-hub.tw/https://www.sciencedirect.com/science/article/abs/pii/S0160289616303555) (so, closely related but not exactly the same)
> *   **The cluster of traits (listed later) that make someone a good ‘superforecaster’** in Philip Tetlock’s work (Tetlock also claims that intelligence is only modestly correlated with being a superforecaster)
> 
> Here are some other concepts in the area, but that seem more different:
> 
> *   **Intelligence:** I think of this as more like ‘processing speed’ – your ability to make connections, have insights, and solve well-defined problems. Intelligence is an aid in good judgement – since it lets you make more connections – but the two seem to come apart. We all know people who are incredibly bright but seem to often make dumb decisions. This could be because they’re overconfident or biased, despite being smart.
> *   **Strategic thinking:** Good strategic thinking involves being able to identify top priorities, and develop a good plan for working towards those priorities, and improving the plan over time. Good judgement is a great aid to strategy, but a good strategy can also make judgement less necessary (e.g. by creating a good back-up plan, you can minimise the risks of your judgement being wrong).
> *   **Expertise:** Knowledge of the topic is useful all else equal, but Tetlock’s work (covered more below) shows that many experts don’t have particularly accurate judgement.
> *   **Decision making:** Good decision making depends on all of the above: strategy, intelligence, and judgement.

I do disagree with some of the distinctions being made in the post. As an example, just in the section above, the conception of "Intelligence" as "processing speed" is really flawed, and in-practice intelligence already measures something closer to "good judgement". But overall, the post seems decent as a potential intro into a bunch of rationality stuff.
# Egan's Theorem?

When physicists were figuring out quantum mechanics, one of the major constraints was that it had to reproduce classical mechanics in all of the situations where we already knew that classical mechanics works well - i.e. most of the macroscopic world. Likewise for special and general relativity - they had to reproduce Galilean relativity and Newtonian gravity, respectively, in the parameter ranges where those were known to work. Statistical mechanics had to reproduce the fluid theory of heat; Maxwell's equations had to agree with more specific equations governing static electricity, currents, magnetic fields and light under various conditions.  
  
Even if the entire universe undergoes some kind of phase change tomorrow and the macroscopic physical laws change entirely, it would still be true that the old laws *did* work before the phase change. Any new theory and any new theory would still have to be consistent with the old laws working, where and when they actually did work.  
  
This is [Egan's Law](https://wiki.lesswrong.com/wiki/Egan's_law): it all adds up to normality. When new theory/data comes along, the old theories are still just as true as they always were. New models must reproduce the old in all the places where the old models worked; otherwise the new models are incorrect, at least in the places where the old models work and the new models disagree with them.  
  
It really seems like this should be not just a *Law*, but a *Theorem*.

I imagine Egan's Theorem would go something like this. We find a certain type of pattern in some data. The pattern is highly unlikely to arise by chance, or allows significant compression of the data, or something along those lines. Then the theorem would say that, in any model of the data, either:

*   The model has some property (corresponding to the pattern), or
*   The model is "wrong" or "incomplete" in some sense - e.g. we can construct a strictly better model, or show that the model consistently fails to predict the pattern, or something like that.

The meat of such a theorem would be finding classes of patterns which imply model-properties less trivial than just "the model must predict the pattern" - i.e. patterns which imply properties we actually care about. Structural properties like e.g. (approximate) conditional independencies seem particularly relevant, as well as properties involving abstractions/embedded submodels (in which case the theorem should tell how to find the abstraction/embedding).  
  
Does anyone know of theorems like that? Maybe this is equivalent to some standard property in statistics and I'm just overthinking it?
# Are there non-AI projects focused on defeating Moloch globally?

_[Meditations on Moloch](https://slatestarcodex.com/2014/07/30/meditations-on-moloch/)_ lays out a rather pessimistic view of the future, and then offers a super-intelligent AI "gardener" as the solution. A lot of the rationalist community is focused on AI, which makes sense in that light (and of course because of the existential risk of unaligned AI), but I don't know of any projects focused on non-AI solutions to countering or defeating Moloch. Some projects exist to counter specific local coordination problems, but apparently none to counter the global gardening problem in the original post? Am I missing such a project? Is there a reason that AI is the only plausible solution? Is this low-hanging fruit waiting to be picked?

_edited to add some clarifications:_

*   By defeating Moloch "globally" I mean in the sense of the global race to the bottom - preventing humanity from "reaching the sea" in the metaphor from the original Meditations on Moloch (which itself is borrowed from the Apocrypha Discordia). This doesn't mean solving all local coordination problems forever, just preventing us from reaching the absolute worst case that Bostrom conjures of our own destruction, the "Disneyland with no children".
*   Yes, I've read _Inadequate Equilibria_.
# If Starship works, how much would it cost to create a system of rotable space mirrors that reduces temperatures on earth by 1° C?

There are many proposed geoengineering solutions that could reduce temperatures on earth. Unfortunately, a lot of them have a mix of side effects and lock-in effects where the changes in temperature come years after deployment which creates risk of unintended consequences.

If we would have a constellation of space mirrors that can be rotated as we desire to let in less or more sunlight, how much would it cost to bring up enough to reduce temperatures on earth by an average of  1° C? Let's say that Elon's promise of being able to lunch a Starship that brings up 100,000kg for 1,000,000$ works out, what would it cost to produce and deploy those mirrors?
# Spoiler-Free Review: Orwell

Orwell is another game in the discrete-choices-over-time genre. In this case, you are an investigator, and choose which ‘datachunks’ to upload into the system. From there, others will take action.

Like other games in the genre, if you are going to play, play it blind.

I’d rank the game as lower Tier 3 – it’s good at its job, but not essential. It mostly does what it sets out to do, creating an experience and an atmosphere. It has some big frustrations along the way.

Orwell has three problems that prevent it from doing better. It’s also short.

You should play Orwell if and only if the concept of Orwell seems like something you want to experience.

Problem one, which is not in any way a spoiler, is that a lot of the game effectively involves finding the datachunks, or links on pages that lead to new pages that in turn contain datachunks. Several times I got frustratingly stuck trying to figure out where the game wanted me to click. Similarly, there is a star by things that are new, which leads to furious “make the star go away” actions to allow for better searching.

Problem two, which is a _minor_ spoiler, is that the game often gives you less choices than it looks like it can, or than it easily could. Events mostly seem to proceed in order, so you don’t really have the option to withhold most datachunks. Several times I wanted to not upload something, but the game would simply not proceed if I didn’t do it. This leads to the problem of, if I don’t upload this, I could spend a _long_ time not knowing if that’s the only way to advance the game while looking for some other way to advance it that might or might not exist. I would have appreciated a lot more flexibility. Mostly all the system gives you are some binary choices where two chunks conflict and you have to decide which one to go with.

Problem three requires spoiling the experience to talk about, so that would be a distinct post.
# Rationality for Kids?

UPDATE 11NOV:

I came up with a game to use as an icebreaker. And I'd love ideas for future variations. It's a combination of Credence Calibration, 20 Questions, and Taboo. The children are trying to determine which of three possible states exist on the card which I have face down (for my first iteration, the possibilities will be "Cat", "Rat", and "Dog"). Every kid gets 30 poker chips to allocate to each of the three possibilities. Kids will then take turns asking a yes or no question, but before each Q, I roll a six sided die. If it comes up six, all chips placed on a wrong answer are turned in, otherwise, they ask their question, I answer with something on a scale of "Never" to "Always", and they are permitted to reallocate their chips. But there is a catch: they are not permitted to use certain words (i.e. cat, dog, rat, meow, bark, pet, etc.) in their questions.  
The point is to find tests which can serve as evidence between the possibilities and recognize how confidence should change according to evidence.

Would be interested in other possible states for future iterations

END UPDATE

  

So I really appreciate the lessons I've learned from "Rationality", but I wish I had learned them earlier in life. We are now homeschooling my kids, and I want to volunteer to teach my kids plus others who are interested lessons about thinking rationally.

Does anyone have recommendations on how to put together a curriculum which gets at the core ideas of rationality, but is oriented towards young kids? Some criteria:

Children will likely range from 7-11, meaning they should be simple concepts and require very little prior knowledge and only the simplest math.

Lessons should be interactive.

Lessons should include TRUE experiments (not just doing fun stuff with chemicals).

Lessons should be fun and appealing enough that parents will want to sign their kids up.

Any other suggestions on the course (wording that will be appealing without sounding too "nerdy" or alarming to the conservative types who usually homeschool) are welcome.

  

UPDATE: the Inflection Point Curriculum appears to be the middle school version of what I am looking to do: [https://drive.google.com/file/d/1tcUJXRlZXeKjAWeU9Y37FcPKv3lj6PsX/view?usp=sharing](https://drive.google.com/file/d/1tcUJXRlZXeKjAWeU9Y37FcPKv3lj6PsX/view?usp=sharing)

I currently envision the course as a combination of game type exercises like Credence Calibration, Zendo, and Meta-Forms, and experiments like adjusting the air composition of a room and investigating bernoulli effects using things like paper and shower curtains. Other ideas: investigating citrus batteries, water absorption by celery, and the light spectrum of various sources as split by a prism.
# Sunday September 20, 12:00PM (PT) — talks by Eric Rogstad, Daniel Kokotajlo and more

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see [here](https://www.lesswrong.com/tag/lesswrong-events) for previous weeks' transcripts).

*   For the first hour, we will have a series of lightning talks each lasting about 5 minutes followed by discussion. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
*   From 1PM to 2PM, we'll have a hangout in breakout rooms. If you are not interested in the talks, feel free to just show up for this part (or the other way around).
*   We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

*If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up* [*here*](https://forms.gle/iwFatbhys9muPmQA7)*.*

Speakers
--------

*   [Eric Rogstad](https://www.lesswrong.com/users/esrogs): **The AI Does Not Care about Your Atoms Either**
*   [Daniel Kokotajlo](https://www.lesswrong.com/users/daniel-kokotajlo): **Why GWP is a bad metric for thinking about timelines and takeoff speeds**

Details
-------

**When?** Sunday September 20, 12:00PM (PT)

**Where?** [https://us02web.zoom.us/j/84108675070](https://www.google.com/url?q=https://us02web.zoom.us/j/84108675070&sa=D&source=calendar&usd=2&usg=AOvVaw3JRMiGUZOh6ucvtAQVMFmj)
# Objective Dog Ratings: The Shiba Inu

  

![](https://wmbsaltworks.files.wordpress.com/2020/05/shiba_inu_at_the_entrance_of_a_house_-japan_2010_a.jpg?w=621&h=466)

  

The Shiba Inu
=============

Shibas are a spitz dog, which, if you remember from the [original Dog Ratings post](https://www.lesswrong.com/s/SyNM8S9FbJjWyf3D6/p/uAYz5jMHPt4xgXwr5), is my projected winner for the All-Dog All-Stars Contest, but that doesn't mean that _this_ spitz will take the trophy home.

Let's take a look:

First, it must be noted that shibas have the curly tail which is most distinctive of domesticated animals, which is a Good Thing. Props to you, noble shibas.

Behaviorally, shibas are very clean animals. They clean themselves, they're very easy to housebreak, and...well, they _will_ track mud into the house and not wipe their feet off first, but they're still dogs. You can only expect so much from them. They're clever, but [the jury is out on just how much so](https://thesmartcanine.com/are-shiba-inus-smart/).

Shibas do not bark very much, instead preferring to emit terrible, nearly goat-like noise called a ["shiba scream,"](https://web.archive.org/web/20070810225914/http://www.northeastshibarescue.com/traits.html) which has been described as ["bloodcurdling"](https://myfirstshiba.com/what-is-a-shiba-inu-scream/) and like the ["screams of the damned."](https://www.youtube.com/watch?v=BhgFe6b8Nek) The name sounds like an anime attack move, and I wholeheartedly approve.

I could not find much in the way of [terrible diseases](https://drferox.tumblr.com/post/158340268583/hi-dr-ferox-off-the-back-of-your-post-about), which is a little surprising given their history. Shibas were extensively crossbred with other dogs in the 19th century and, between crossbreeding, distemper outbreaks, and WWII food shortages, only three "bloodlines" or "variations" of shiba still remained in the post-war era. It's hard for me to get details on this, so I can only assume that the gene pools were relatively large and healthy or the breeders did a good job, because, as I say, they aren't a tragedy of inbreeding.

This is, admittedly, a little bit conjecture, but I imagine that their healthiness has something to do with [the story of their preservation](https://web.archive.org/web/20180516180104/https://www.jkc.or.jp/modules/worlddogs/entry.php?entryID=105&categoryID=5) as a breed. Shibas were bred to do a job (hunting and flushing out small game and, on occasion, boar), and hunters were among those who tried to preserve them.

Anyway, the point is, they are pretty healthy dogs, and that's pretty important here at Objective Dog Ratings. Shibas aren't exceptionally intelligent, but there's something awful and awesome about their unique vocal talents, so I'm going to award them a very tentative, by-the-tip-of-their-nose, three stars. A forward-thinking, politically progressive wolf would not be ashamed to know a shiba, though they might well be a little unsettled.

**Rating: ★★★** _out of_ **★★★ (Good)**

Original post (with more pictures) [here](https://wmbsaltworks.wordpress.com/2020/05/02/objective-dog-ratings-shiba-inu/#more-470).
# Draft report on AI timelines

Hi all, I've been working on some AI forecasting research and have prepared a draft report on timelines to [transformative AI](https://www.openphilanthropy.org/blog/some-background-our-views-regarding-advanced-artificial-intelligence#Sec1). I would love feedback from this community, so I've made the report viewable in a Google Drive folder [here](https://drive.google.com/drive/u/1/folders/15ArhEPZSTYU8f012bs6ehPS6-xmhtBPP).

With that said, most of my focus so far has been on the high-level structure of the framework, so the particular quantitative estimates are very much in flux and many input parameters aren't pinned down well -- I wrote the bulk of this report before July and have received feedback since then that I haven't fully incorporated yet. I'd prefer if people didn't share it widely in a low-bandwidth way (e.g., just posting key graphics on Facebook or Twitter) since the conclusions don't reflect Open Phil's "institutional view" yet, and there may well be some errors in the report.

The report includes a quantitative model written in Python. [Ought](https://ought.org/) has worked with me to integrate their forecasting platform [Elicit](https://elicit.ought.org/) into the model so that you can see other people's forecasts for various parameters. If you have questions or feedback about the Elicit integration, feel free to reach out to elicit@ought.org[.](elicit@ought.org.)

Looking forward to hearing people's thoughts!
# EA Relationship Status

A friend [observed](https://www.facebook.com/groups/eaparents/permalink/2854556011433232) that fewer people in the effective altruism movement are married than you might expect. I was curious: what are marriage rates like within the EA community? The 2018 [EA Survey](https://www.rethinkprojects.org/easurvey) asked about relationship status, and we can look at how that varies by age:

[![](https://www.jefftk.com/ea-relationship-status-by-age.png)](https://www.jefftk.com/ea-relationship-status-by-age-big.png)

I'm using "ever married" for people who are currently married or have ever been married, including people who are now divorced, widowed, or separated. Since some of these buckets might be pretty small, let's add sample size information:

[![](https://www.jefftk.com/ea-relationship-status-by-age-with-sample-size.png)](https://www.jefftk.com/ea-relationship-status-by-age-with-sample-size-big.png)

The anonymized survey data doesn't have 35-44 data, and the 65+ group looks suspiciously like it has the 35-44 group lumped in with it. I've [filed a bug](https://github.com/rethinkpriorities/ea-data/issues/27) and if they fix it I'll update the post. For now, probably ignore the 65+ group.

For comparison, here's 2018 ACS data ([via](https://usa.ipums.org/)) for US marriage rates, on the same scale:

[![](https://www.jefftk.com/usa-marriage-rate-by-age.png)](https://www.jefftk.com/usa-marriage-rate-by-age-big.png)

And, at least in the US, people with more education are more likely to be married, after age 28:

[![](https://www.jefftk.com/usa-marriage-rate-by-age-and-education.png)](https://www.jefftk.com/usa-marriage-rate-by-age-and-education-big.png)

Now, EAs are not all American, and even then they're different from Americans as a whole in many ways other than being interested in effective altruism. On the other hand, when I look at what fraction of my (Swarthmore) college friends are married, a group similar to EAs in many ways, it's not far from the graph for the US at large. (And it's off in the positive direction). I see similar numbers by school for [people who were 30-34 in 2014](https://www.nytimes.com/interactive/2018/03/29/upshot/college-marriage-class-differences.html).

There does seem to be something real here. Some guesses as to why:

*   EAs often highly prioritize their careers.
*   EAs are generally less interested in having children.
*   EAs are often poly.
*   EAs often live in group houses.
*   EAs are more willing to be weird; less likely to do something because it is the standard thing to do.

Other ideas?
# Where is human level on text prediction? (GPTs task)

I look at graphs like these (From the GPT-3 paper), and I wonder where human-level is:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/585ac09b045f68460300c1c1b8121214fb18d82b2d7586a2.png)

Gwern seems to [have the answer here](https://www.gwern.net/newsletter/2020/05#gpt-3): 

> GPT-2-1.5b had a cross-entropy validation loss of ~3.3 (based on the perplexity of ~10 in [Figure 4](https://www.gwern.net/images/ai/2019-radford-figure4-gpt2validationloss.png), and $\log_2(10)=3.32$). GPT-3 halved that loss to ~1.73 judging from [Brown et al 2020](https://www.gwern.net/images/ai/2020-brown-figure31-gpt3scaling.png) and using the scaling formula ($2.57 \cdot (3.64 \cdot 10^3)^{-0.048}$). For a hypothetical GPT-4, if the scaling curve continues for another 3 orders or so of compute (100–1000×) before crossing over and hitting harder diminishing returns, the cross-entropy loss will drop, using to ~1.24 ($2.57 \cdot (3.64 \cdot 10^6)^{-0.048}$).
> 
> If GPT-3 gained so much meta-learning and world knowledge by dropping its absolute loss ~50% when starting from GPT-2’s near-human level, what capabilities would another ~30% improvement over GPT-3 gain? What would a drop to ≤1, perhaps using wider context windows or recurrency, gain?

So, am I right in thinking that if someone took random internet text and fed it to me word by word and asked me to predict the next word, I'd do about as well as GPT-2 and significantly worse than GPT-3? If so, this actually lengthens my timelines a bit.  
(Thanks to Alexander Lyzhov for answering this question in conversation)