Harley-ml/lesswrong huggingface dataset:
# Blog Post Day III

  
_You are invited to join us_ [_online_](https://discord.com/invite/AHTePn) _on Saturday the 13th of June, to write a blog post, e.g. that one you've been thinking about writing but never got around to. This is the third event of this type;_ _[the first one went well](https://www.lesswrong.com/posts/NpZsCERfyKT7cJN9a/blog-post-day-retrospective), and [so did the second](https://www.lesswrong.com/posts/Snujc8H8vqfLpbNdp/blog-post-day-ii-retrospective)._

We'll set an official goal of having the post put up by midnight. Also, we'll meet up in a [special-purpose discord channel](https://discord.gg/j5TzpVF) to chat, encourage each other, swap half-finished drafts, etc. If like me you are intending to write the thing one day eventually, well, here's a reason to make that day this day.
# Introduction to Existential Risks from Artificial Intelligence, for an EA audience

(This is an event post for LW-Israel. The main post is [here](https://www.lesswrong.com/posts/Kfiqd3bAgaiN8QF2i/introduction-to-existential-risks-from-artificial), posted separately because event posts are geographical.)

Among the causes that Effective Altruists support, existential risk stands out as different from ordinary charitable concerns, and existential risk from Artificial Intelligence even more so.

In this talk, which assumes no prior knowledge in artificial intelligence, Vanessa Kosoy will explain the problem in question, and how researchers in the field are trying to solve it. Vanessa is a Research Associate with the Machine Intelligence Research Institute (MIRI) studying the mathematical formalization of general intelligence and value alignment.

The talk is brought to you by Effective Altruism Israel and LessWrong Israel.

We start on Tuesday June 23, at 19:00 Israel Daylight Time, 16:00 GMT.  
See the time in your timezone [here](https://www.thetimezoneconverter.com/?t=19%3A00&tz=Jerusalem).

Please sign up [here](https://huji.zoom.us/meeting/register/tJMkf-qrrT8sEtfvY2MShfu8kYNTg8TRgQWB) and we'll send you the URL for the Zoom meeting.
# GPT-3: A Summary

> With massive size comes massive generalization ability: GPT-3 is competitive in many benchmarks w_ithout even tuning on the target task._ \[...\] Perhaps the most impressive part, though, is that even at such a massive scale, the model still scales smoothly in performance instead of plateauing, implying that still-larger models would perform e_ven better._
# Open & Welcome Thread - June 2020

If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a [shortform post](http://www.lesslong.com/))

And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1).

The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Is a near-term, self-sustaining Mars colony impossible?

Some argue that Elon Musk's plan to build a self-sustaining colony on Mars is effectively impossible for the foreseeable future without discontinuous tech progress. (See e.g. Jess Riedel's claim [here](https://twitter.com/Jess_Riedel/status/1268308554399637504), which Robin Hanson [agrees with](https://twitter.com/robinhanson/status/1268310885652725761).)  
  
Is there a simple explanation for why this might be so? I'd be especially interested in Fermi estimates of key bottleneck resources.
# The Stopped Clock Problem

When a low-probability, high-impact event occurs, and the world “got it wrong”, it is tempting to look for the people who did successfully predict it in advance in order to discover their secret, or at least see what else they’ve predicted. Unfortunately, [as Wei Dai discovered recently](https://www.lesswrong.com/posts/3jYwvwrNXyD3koHTM/open-and-welcome-thread-june-2020?commentId=fWW6orw8pj3NCX9mz), this tends to backfire.

It may feel a bit counterintuitive, but this is actually fairly predictable: the math backs it up on some reasonable assumptions. First, let’s assume that the topic required unusual levels of clarity of thought not to be sucked into the prevailing (wrong) consensus: say a mere 0.001% of people accomplished this. These people are worth finding, and listening to.

But we must also note that a good chunk of the population are just pessimists. Let’s say, very conservatively, that 0.01% of people predicted the same disaster just because they always predict the most obvious possible disaster. Suddenly the odds are pretty good that anybody you find who successfully predicted the disaster is a crank. The mere fact that they correctly predicted the disaster becomes evidence only of extreme reasoning, but is insufficient to tell whether that reasoning was extremely good, or extremely bad. And on balance, most of the time, it’s extremely bad.

Unfortunately, the problem here is not just that the good predictors are buried in a mountain of random others; it’s that the good predictors are buried in a mountain of extremely poor predictors. The result is that the mean prediction of that group is going to be noticeably _worse_ than the prevailing consensus on most questions, not better.

* * *

Obviously the 0.001% and 0.01% numbers above are made up; I spent some time looking for real statistics and couldn't find anything useful; [this article](https://247wallst.com/special-report/2013/08/19/industries-making-the-most-money-on-doomsday-preppers/) claims roughly 1% of Americans are "preppers", which might be a good indication, except it provides no source and could equally well just be [the lizardman constant](https://slatestarcodex.com/2013/04/12/noisy-poll-results-and-reptilian-muslim-climatologists-from-mars/). Regardless, my point relies mainly on the second group being an order of magnitude or more larger than the first, which seems (to me) fairly intuitively likely to be true. If anybody has real statistics to prove or disprove this, they would be much appreciated.
# Russian x-risks newsletter spring 2020

Coronavirus is rampant in Russia, but the lockdown is easing for political and economic reasons. It was not very strict, in fact: many people were on streets, and infections per day grew 100 times during lockdown in April. The reported infection level peaked at the beginning of May. It declined since in Moscow in 2-3 times, but grew in the regions, so total daily cases are almost not changing in May. There is obvious underreporting of the number of actual cases, but the peak in Moscow seems real: my friends are reporting fewer cases in their facebook.

Voting to extend Putin's rule indefinitely via the new Constitution is scheduled for July 1, 2020, and this could be a political reason to report fewer cases.

A fakenews was officially [refuted](https://pravo.ru/news/221273/) that coronavirus leaked after the explosion in Novosibirsk Vector biolab in fall 2019 and this information was prohibited for dissemination in social networks.

Vector is [working](https://www.naukogradpress.ru/nauka/2020/05/17/vektor-planiruet-zaregistrirovat-vaktsinu-ot-koronavirusa-v-sentyabre/) on a vaccine and it is promised soon, maybe in fall 2020, but many people I asked are sceptical of its effectiveness and safety and prefer to wait for FDA approved vaccine.

Russia [limited](https://www.rbc.ru/economics/17/03/2020/5e70e89f9a79472c2ce2cb46?from=from_main) export of some grains.

Moscow is [designated](https://rg.ru/2020/04/28/tehnologii-dok.html) as a special area to test AI technology by providing easier access to collected personal data.

Russia [declared](https://www.gazeta.ru/army/news/2020/06/02/14498491.shtml) that it may use nuclear weapons first if its existence is threatened by conventional weapons attack.

The Russian economy is hurt both by coronavirus lockdown and low prices of oil and low demand for natural gas.

Russia joined one more war by [sending](https://www.rferl.org/a/russia-jets-libya-syria/30651821.html) fighter jets and mercenaries in Libya to participate in the civil war but suffered defeat near Tripoli from Turkish drones.
# How to validate research ideas?

Let's say you have an idea that you think might be interesting to investigate, possibly a new aspect in AI safety, maybe some new algorithm.

If you're an experienced researcher, you probably have plenty of intuition to think through it, consider the possible outcomes and decide whether it's worth investigating.

If you have a decent academic network, you can probably bring it up even in casual conversations with people who are as good or better than you in the field to get a sense of their intuitions.

  

What if you have none of those things? Is there an online forum for such discussions, just like I'd go to reddit when looking for e.g. game recommendations? Maybe \*this\* is the forum?

And are there any good recommendations how to try and work it out by yourself, without referring to the wisdom of online crowds?
# Most reliable news sources?

I've longed assumed that the vast majority of what the news discusses is irrelevant in the long run, and not that well reported at that. But the world seems to be moving faster these days, and I have more of a sense that I want to know what happened this week, because it might impact what I _do_ next week.

For the first time in my life, I have some inclination to follow current events as they happen. And I find that I don't really know how to do that.

What are the most reliable / least-politicized news sources?

In particular, I want a resource that I can refer to that will tell me what happened in the past 3 days, in as factual and unbiased a way as possible. I expect that I might have to do further research, to get context for the events. But to start, I want a place where I can go that will tell me _what happened_, with a minimum of narrativizing, political outrage, etc.
# An Illustrated Proof of the No Free Lunch Theorem

This is a proof of one of the many No Free Lunch Theorems in statistical learning. Rafael Harth already did a very good writeup [here](https://www.lesswrong.com/posts/JAD4u5dRwFApC3qYF/understanding-machine-learning-ii) of the same proof. I had a vision for doing some very nice diagrams which could help bridge the many summations and inequalities in this proof. I think I maybe only executed up to ~20% of that vision.

I have a good amount of experience making graphics and visuals, and this was still challenging. I think, until I find a better method, that I'll probably not make more of these. I felt a combination of:

- difficulty using my tool (Inkscape) to get the "real" result in my head
- frustration that the similarities between the images couldn't be abstracted out in some nice tool

For these sorts of graphs, I feel like [Mermaid](https://mermaid-js.github.io/mermaid/#/) is maybe half-way there. But I feel like there's a lot of value to finding the specific visuals that *you as the author* think are good to help convey the intuition, so I'm a little hesitant to just use a compositional tool...
# What does “torture vs. dust specks” imply for insect suffering?

There is a non-negligible probability that insects suffer. Many insects have nociception. Furthermore, the argument-by-analogy approach to determining whether an animal suffer works—an insect’s response to stimuli that would cause pain in other animals is similar. It is estimated that there are 10^19 insects living at any point in time. If one believes that dust specks on a sufficiently large number of people (this doesn’t have to be 3^^^3 after all) is worse than torture of one person, does that mean insect suffering should swamp many other considerations when prioritizing social problems to address? (This is _not_ meant to be a “proves-too-much” argument against siding with dust specks; it’s a genuine question.)
# Are there good ways to find expert reviews of popular science books?

As the title says. Non-expert reviews only evaluate how good the writing was, or how impressed the reviewer was by the ideas; but I typically want something that evaluates the actual scientific claims.

My one trick has been to look at Google Scholar for papers that cite the book and search for "review" within those, which sometimes finds reviews published at academic journals, but this doesn't seem very effective overall.
# What past highly-upvoted posts are overrated today?

I've noticed the quality of LW posts varies greatly, even among top posts. Some of this is subjective or comes from what I want out of LW, but I suspect some posts are not nearly as useful or fun to readers in 2020 as their karma makes them appear.

Answers should be links to particular posts with more than ~75 karma that the answerer has a relatively unknown strong criticism of, or a general pattern (e.g. posts from LW 1.0 often have a higher score than they deserve because \[reason\]). Please give a reason why, and don't automatically downvote a post someone else links.
# Can Covid-19 spread by surface transmission?

I've seen a lot of claims lately that coronavirus doesn't spread on surfaces, and therefore we can stop worrying about packages (and probably takeout food, even uncooked?). 

My understanding of the claim is something like "we just haven't actually seen any evidence of surface transmission, despite the entire world paying attention, therefore it probably doesn't happen."

It's a bit trickier to prove a negative, but can anyone who either strongly believes that covid *can't* spread via surfaces, or who believes that it *can* (at least sometimes), write up the reasons for their beliefs?
# Keep: Move checked items to bottom

I've been using Keep as my main note-taking app for years, and our house recently switched after Wunderlist was [discontinued](https://www.facebook.com/jefftk/posts/10100131493647512). It turns out they added a list feature, where checked items are sorted below unchecked ones:

[![](https://www.jefftk.com/grocery-list-keep-unchecked-at-top.png)](https://www.jefftk.com/grocery-list-keep-unchecked-at-top-big.png)

This is on by default for new users, but I've been using Keep long enough (disclosure: I work [for Google](https://www.jefftk.com/p/rejoining-google)) that I seem to have been grandfathered into the old default:

[![](https://www.jefftk.com/grocery-list-keep-unchecked-mixed-in.png)](https://www.jefftk.com/grocery-list-keep-unchecked-mixed-in-big.png)

I didn't know about this new display option, and would grumble to myself about how my housemates wouldn't delete things from the shared list when they were done with them, and how we were building up so much clutter. When I asked them, though, I learned that it wasn't a problem for them because they were all on the new layout.

Turns out there's a setting:

[![](https://www.jefftk.com/keep-settings-screen-move-checked-items.png)](https://www.jefftk.com/keep-settings-screen-move-checked-items-big.png)

I'm much happier with it now, and having old items stick around at the bottom feels useful (what do we usually get...) instead of feeling like clutter.
# Does equanimity prevent negative utility?

Even though I've only just begun dabbling in meditation, I believe that it is an incredibly powerful tool. I do have one worry though relating to what exactly counts as suffering. If I scald myself, this triggers my pain nerves to fire and it is possible that this produces negative utility, even if my conscious mind is able to maintain complete equanimity?

And while it might produce less negative utility in most cases, it is possible that someone who can achieve equanimity might voluntarily choose to put themselves in situations where they experience far more suffering than they ever would without this ability with the end result being net negative for them. A large part of what worries me is that I am massively uncertain about this issue and I honestly don't have any idea of how we could settle it conclusively.

Some factors that might make us more likely to believe this is the case:

*   If we believe that insects feel pain that is axiologically relevant, then this would demonstrate that higher-level processing isn't a necessary component
*   Multiagent models of the brain make it more plausible that one might experience pain, whilst another isn't
*   Panpsychism could be taken to suggest that it is possible for individual components of the brain to suffer independently
# Is there something like a "meta-encyclopedia"?

With Wikipedia showing increasing signs of ideological uniformity, I thought I should try to diversify my consumption of reference material. A potentially very useful tool for this purpose would be a "meta-encyclopedia", i.e. a place where users can find a list of all existing encyclopedia entries on a given topic. Questions:

1\. Does something like this exist?

2\. If not, what are the closest substitutes?

My current method is to simply do a Google search for the topic of interest followed by the word 'encyclopedia', e.g. 'comparative advantage encyclopedia'. This method is already quite useful, with a much lower false-positive rate than I would have expected, though I feel there's still considerable room for improvement.
# 2020 LessWrong Demographics Survey

Edit: The survey is now closed. You can see the results [here](https://www.lesswrong.com/posts/yn4Aw6jejS3SHKuzu/2020-lesswrong-demographics-survey-results)
============================================================================================================================================================

DISCLAIMER: This is not a return of the old annual survey. I am not a mod nor an owner of this website. This means that the raw data from this survey gets released publicly (but your responses will be anonymous). Feel free to not answer any question you feel uncomfortable about. The survey will also be a lot shorter than the old annual surveys (32 questions). It will be closed on the 13th of July, I will show the results in a separate post titled "[2020 LessWrong Demographics Survey Results](https://www.lesswrong.com/posts/yn4Aw6jejS3SHKuzu/2020-lesswrong-demographics-survey-results)"

You can start taking the survey (<10 minutes) by clicking on the following link:

[https://forms.gle/XHaXpXVNmnXLEAFW6](https://forms.gle/XHaXpXVNmnXLEAFW6)
# Turns Out Interruptions Are Bad, Who Knew?

I’ve been known to accuse people who say open offices are “fine with a few mitigations” of not paying attention to the cost of their mitigations. I believed they shrunk their thoughts down to the point that not much was lost from an interruption, at the cost of only being able to think the thoughts that fit in that interval. Any thought that would take too long to process could not be conceived of.

I’ve also been known to accuse people who advocate for deep, uninterrupted work without the distractions of social media of “not understanding how valuable social media is to me”. And  besides, my workflow works best with frequent breaks (that I choose the timing of) because I “background process”.

\*Cough\*

I maintained this illusion until, inspired by a [stupidly expensive device that only does one thing](https://getfreewrite.com/products/freewrite-traveler), I taped my old phone to [a bluetooth keyboard](https://www.amazon.com/gp/product/B07MM1JZ6G/ref=as_li_qf_asin_il_tl?ie=UTF8&tag=acesoundergla-20&creative=9325&linkCode=as2&creativeASIN=B07MM1JZ6G&linkId=d741702457d019e31efc7acfe35ce295)\* and began to write in offline mode. It was immediately a magical experience. It was so \*quiet\*. I could go on my porch and write and it was *quiet*. My thoughts got much larger because I wasn’t subconsciously afraid I’d interrupt them. I began to feel angry at my laptop. Why did it insist on hurting me so much? Why couldn’t it be pure like the offline phone/keyboard experience? Why couldn’t I just create things?

\[\* I only found two bluetooth keyboards with an inlay for phones/tablets. The other one lacks a built in battery, and shipped with a broken key\]

Locally, this lasted for about 10 minutes before the social media cravings kicked in. But that was enough. I deeply resented work for taking me away from my magic writing device and making everything so *noisy*. 

Since I started, my desire for using the quiet device has waxed and waned. At first I thought this was reflective of some deep pathology, but after two weeks it looks a lot more like “sometimes the benefits of quiet outweighs the benefits of being able to look stuff up, sometimes they don’t’”.  I’ve also changed how I interact on a connected device- I’m more likely to close Signal, less likely to open Twitter. This is less due to a utilitarian calculation of the costs and benefits of Twitter, and more that once I’m in a good state, I can notice how switching to Twitter is almost physically painful.

The problem is that I wasn’t wrong that social media was genuinely very valuable to me, and that was before we were all locked inside. But I definitely was wrong that getting those benefits were costless, in a way very analogous to mistakes I accused others of making. I’m glad I have the information now, but I haven’t figured out what to do with it yet.
# What gripes do you have with Mustachianism?

I'm referring to [Mr. Money Mustache's](https://www.mrmoneymustache.com/) blog and the philosophy that he preaches there.

I tend to agree with what he says, and perhaps border on being a fanboy. However, I never really [stress](https://www.marketplace.org/2017/09/19/ray-dalio-bridgewater-associates-economy/) [tested](https://youtu.be/HXbsVbFAczg?t=401) these beliefs, and discussing gripes that others have with him seems like a good way of doing that.
# An online prediction market with reputation points

This is an advert for [an online prediction market in reputation points I’m making](https://freeideafutures.herokuapp.com).

It’s still in development, but currently you can sign up with Facebook and trade in two markets I’ve set up. The current scores are provisional -  I’ll reset everyone’s point balance at some point if this is successful, so don’t worry about making bad trades. 

I largely followed [a proposal by Paul Christiano](https://sideways-view.com/2019/10/27/prediction-markets-for-internet-points/), but I haven't implemented all his ideas yet. 

I'd be grateful if people could make some trades so that I can check everything's working. I’d also like to hear people’s thoughts on this. In particular:

* Would you and people you know use this? If not, why not?
* How can I improve it? What features would you want?
# For moderately well-resourced people in the US, how quickly will things go from "uncomfortable" to "too late to exit"?

Expanding from a comment thread: [https://www.lesswrong.com/posts/3jYwvwrNXyD3koHTM/open-and-welcome-thread-june-2020?commentId=FcZwHEWEt6EAP54qq](https://www.lesswrong.com/posts/3jYwvwrNXyD3koHTM/open-and-welcome-thread-june-2020?commentId=FcZwHEWEt6EAP54qq)

Growing up, I had a neighbor who'd escaped the Cultural Revolution as a child, going from very wealthy in China (she had bound feet, and had literally never seen food being cooked) to working- and then middle-class in the US. I have a number of friends with relatives who escaped or survived (or didn't) the Nazi Holocaust. In both of these cases, there were many months where someone could get out pretty easily (but at significant cost), but it wasn't clearly necessary. And then many weeks where the rich could get out (at a very high cost), and then it was too late. The US Civil War took 4 months from first declaration of secession to the shooting war starting at Fort Sumter. Travel was much harder back then, so I'm not sure how possible escape was was for most.

As things get ... weirder in the US, politically and economically, there is a (very low IMO) chance that things will collapse badly enough that it's worth significant disruption (leaving friends and family, possibly abandoning hard-to-move assets, certainly incurring expenses) to move out.

I expect that my perception of the chance over time, conditional on it actually happening, to be roughly logistic in shape, and with some intersection on a graph (whose shape I'm unsure of) of "cost/difficulty to get out". And complicated by contagion - it's not clear the US can fully collapse without taking with it most places I would want to go.

So: what are YOUR triggers and preparations for escaping if Very Bad things happen in the US?
# [Link] "Will He Go?" book review (Scott Aaronson)

[https://www.scottaaronson.com/blog/?p=4845](https://www.scottaaronson.com/blog/?p=4845)

From the beginning:

> [Will He Go?](https://www.amazon.com/Will-He-Go-Election-Meltdown/dp/1538751887), by legal scholar Lawrence Douglas, is, at 120 pages, a slim volume focused on a single question: _what happens if the 2020 US election delivers a narrow or disputed result favoring Biden, and Trump refuses to concede?_ This question will, of course, either be answered or rendered irrelevant in half a year. And yet, in my estimation, there’s at least a 15% probability that Will He Go? will enter the ranks of the most important and prescient books ever written. You should read it right now (or at least read [this Vox interview](https://www.vox.com/policy-and-politics/2020/6/3/21257133/trump-2020-election-meltdown-lawrence-douglas)), if you want to think through the contours of a civilizational Singularity that seems at least as plausible to me as the AI Singularity, but whose fixed date of November 3, 2020 we’re now hurtling toward.

> In one of the defining memes of the past few years, a sign in a bookstore reads “Dear customers: post-apocalyptic fiction has been moved to the Current Affairs section.” I was reminded of that as Douglas dryly lays out his horror scenario: imagine, hypothetically, that a President of the United States gets elected on a platform of racism and lies, with welcomed assistance from a foreign adversary. Suppose that his every outrage only endears him further to his millions of followers. Suppose that, as this president’s deepest (and perhaps only) principle, he never backs down, never apologizes, never acknowledges any inconvenient fact, and never accepts the legitimacy of any contest that he loses—and this is perfectly rational for him, as he’s been richly rewarded for this strategy his entire life. Suppose that, during the final presidential debate, he pointedly refuses to promise to respect the election outcome if he loses—a first in American history. And suppose that, after eking out a narrow win in the Electoral College, he then turns around and disputes the election anyway (!)—claiming, ludicrously, that he would’ve won the popular vote too, if not for millions of fraudulent voters. Suppose that, for their own sordid reasons, Republican majorities in the Senate and Supreme Court enable this president’s chaotic rule, block his impeachment, and acquiesce to his daily cruelties and lies.

> Then what happens in the _next_ election?
# Thomas Kwa's Bounty List

Recently I noticed three things:

* About once a week, I have questions no one I know personally can answer; maybe Stack Exchange or LW could, but I don't have time to write them up properly. Other times they're too difficult to be answered on SE/LW without some incentive.

* There are theoretically life improvements that I can make, but reading about lifehacks all day to find them would be net-negative.

* Often I want something to exist, but don't have the time or desire to create it myself.

So, inspired by [this](https://codegolf.meta.stackexchange.com/questions/5243/list-of-bounties-with-no-deadline/), and [this](https://acesounderglass.com/2020/04/18/talk-to-me-for-an-hour-2/), I've decided to create a list of bounties on things I want. To be eligible for a bounty, reply to an answer.
# Testing Hanson's hypothesis about the uselessness of health care.

Robin Hanson has hypothesized that much health care is useless, and that if we radically cut health care (just the part that looks most useless, or even at random), there would be no effect on health outcomes.

People have been staying away from clinics and hospitals in droves. So, are general outcomes generally worse, excluding Covid-19 itself?

Like all such natural experiments, there are confounders, like reduction in other infections diseases improving outcomes; and it is hard to isolate the effects of Covid-19. But overall, this seems like an excellent opportunity to test his hypothesis.
# List of public predictions of what GPT-X can or can't do?

Watching [this video](https://www.youtube.com/watch?v=7qPDwsCLbZc), (Edit: around 25:50) I was pleased to see a specific prediction made by the person claiming that GPT-3 couldn't do reasoning: He claimed that GPT-3's ability to _unscramble_ a word was not reasoning but mere pattern-matching and that he would be very surprised if GPT-3 could _scramble_ an arbitrary word. He said that if it could do that, that would be evidence that it really was reasoning.

Anyone know of other prominent (controversial, important) public predictions of what GPT-3 can or can't do? What about GPT-4? Or GPT-2? It would be interesting to compile a list and see if any patterns jump out, e.g. people consistently over or under-estimating it.
# The Economic Consequences of Noise Traders 

Credits to quanticle for bringing this paper  to my attention.

From the abstract:

The claim that financial markets are efficient is backed by an implicit argument that misinformed "noise traders" can have little influence on asset prices in equilibrium. If noise traders' beliefs are sufficiently different from those of rational agents to significantly affect prices, then noise traders will buy high and sell low. They will then lose money relative to rational investors and eventually be eliminated from the market. We present a simple overlapping-generations model of the stock market in which noise traders with erroneous and stochastic beliefs (a) significantly affect prices and (b) earn higher returns than do rational investors. Noise traders earn high returns because they bear a large amount of the market risk which the presence of noise traders creates in the assets that they hold: their presence raises expected returns because sophisticated investors dislike bearing the risk that noise traders may be irrationally pessimistic and push asset prices down in the future. The model we present has many properties that correspond to the "Keynesian" view of financial markets. (i) Stock prices are more volatile than can be justified on the basis of news about underlying fundamentals. (ii) A rational investor concerned about the short run may be better off guessing the guesses of others than choosing an appropriate portfolio. (iii) Asset prices diverge frequently but not permanently from average values, giving rise to patterns of mean reversion in stock and bond prices similar to those found directly by Fama and French (1987) for the stock market and to the failures of the expectations hypothesis of the term structure. (iv) Since investors in
assets bear not only fundamental but also noise trader risk, the average prices of assets will be below fundamental values; one striking example of substantial divergence between market and fundamental values is the persistent discount on closed-end mutual funds, and a second example is Mehra and Prescott's (1986) finding that American equities sell for much less than the consumption capital asset pricing model would predict. (v) The more the market is dominated by short-term traders as opposed to long-term investors, the poorer is its performance as a social capital allocation mechanism. (vi) Dividend policy and capital structure can matter for the value of the firm even abstracting from tax considerations. And (vii) making assets illiquid and thus no longer subject to the whims of the market -- as is done when a firm goes private -- may enhance their value.
# What are the high-level approaches to AI alignment?

I'm writing a post comparing some high-level approaches to AI alignment in terms of their [false positive risk](https://www.lesswrong.com/posts/JYdGCrD55FhS4iHvY/robustness-to-fundamental-uncertainty-in-agi-alignment-1). Trouble is, there's no standard agreement on what various high-level approaches to AI alignment there are today, either in terms of what constitutes these high-level approaches or where to draw the line in categorizing various specific approaches.

So, I'll open it up as a question to get some feedback before I get too far along. What do you consider to be the high-level approaches to AI alignment?

(I'll supply my own partial answer below.)
# What is meant by Simulcra Levels?

Simulcra levels are a concept that have seen a lot of play on Less Wrong. I suspect that different people are using these concepts in slightly different ways, so I thought it might make sense to ask a question to provide a central location for recording these theories and helping people disambiguate. For this reason, I don't think that there should be a single answer, but rather multiple answers. If you propose a theory, I'd suggest ideally providing a descriptive title or label.
# Set image dimensions using markdown

When embedding an image using the markdown editor, is it possible to specify the image dimensions? It seems that both of these do not work:

Inline HTML:
```
<img src="https://i.imgur.com/25Magmb.png" width="123" height="123">
```

Some markdown variant I found on stackoverflow:
```
![alt text](https://i.imgur.com/25Magmb.png =123x123)
```
# Bathing Machines and the Lindy Effect

From Wikipedia:

> The **Lindy effect** is a theory that the future [life expectancy](https://en.wikipedia.org/wiki/Life_expectancy) of some non-perishable things like a technology or an idea is proportional to their current age, so that every additional period of survival implies a longer remaining life expectancy.

There are a some shallow criticisms you could make of it as a predictive tool. How do you define the thing? What's the proportion of age to life expectancy? How often do we update it?

But these are all problems of refining a loose claim into an unambiguous prediction. How would we evaluate whether this heuristic is a useful predictive tool?

Let's imagine that we're updating the Lindy effect predictions every year.

[Bathing machines](https://en.wikipedia.org/wiki/Bathing_machine) were roofed and walled wooden carts rolled to the beach to allow swimmers to wash and change in privacy. The earliest evidence of them is an engraving from 1735, and they were almost entirely out of use by the early 1920s. Their technological lifespan is around 185 years.

![](https://upload.wikimedia.org/wikipedia/commons/thumb/7/7b/MermaidsAtBrighton.jpg/220px-MermaidsAtBrighton.jpg)

Let's be generous to the Lindy effect, and say that it needs to give a lifespan estimate that's within 25% of the true value to count as correct. In this case, it needed to predict that bathing machines would become obsolete sometime between 1874-1966.

People often use the Lindy Effect to predict that at any given time, we're halfway through the lifespan of anything that currently exists. Using that value, it generated correct predictions between 1805 and 1851, about 25% of the lifespan of the bathing machine. So across the lifetime of the bathing machine, the Lindy Effect was usually wrong.

To take this a step further, I wrote a Python script that finds the most successful Lindy Effect proportion. The code is in the comments. Here are my results.

*   **The most successful historical Lindy Effect predictions, using the 25% confidence interval, estimate that a thing will be around for 125% of its age in any given year.**
*   **It will be right only 40% of the time throughout the lifespan of the thing.**
*   **It will fail the year that the thing becomes obsolete.**

Estimating that the bathing machine would be around for 125% of its present age would therefore have worked from 1846-1920.

What are the takeaways? Even interpreted very generously, the Lindy Effect is usually wrong. It's most successful when it estimates a thing has "one more phase of life" left; analogous to entering retirement. It's only accurate during the last 40% of the thing's actual lifespan.
# $1,000 Bounty for Pro-BLM Policy Analysis

I want to know what Americans with no particular relevant expertise should do in order to achieve the policy goals of the Black Lives Matter movement in the United States. I'm offering a $1,000 bounty (payable directly to you or to a charity of your choice) to whoever provides the analysis on this question that I find most informative in answering this question.

For any given proposed recommended action, I want to know \*which\* policy goals it will help achieve, \*how\* it will help achieve them, and \*why\* it should be expected to do so—bearing in mind that most political interventions do not achieve their intended outcomes, so the burden is on the proponents of any given intervention to explain why it is worthwhile.

The analysis to beat is Chloe Cockburn's from the Open Philanthropy Project ([part 1](https://docs.google.com/document/d/1GGgEZ8ebFd6--C4wLeJV9XrX1OPPg40NL6F1QDo53Bs/edit); [part 2](https://docs.google.com/document/d/1qE4FBuiKxb3qna_vDgRU6C1xz6Znthy5a6H5QIRf2lo/edit)). If nobody offers an analysis that I find more informative than that one, the bounty will not be awarded. (Note, though, that I'm not expecting anything nearly as long or comprehensive as that document! If you offer one single recommendation, and successfully answer the above questions about it, that's better than a long list of things without a clear bottom line or clear justifications.)

The deadline is Sunday, July 5. I will judge the submissions in my sole discretion. I may additionally award smaller bounties to non-winners who nonetheless end up influencing my thoughts on what should be done, or on how to judge the submissions. Please don't hesitate to submit something just because you don't think it's worth $1,000.

Submissions may be posted publicly in the comments of this post, or sent to me via private message. I'm also cross-posting this to [Facebook](https://www.facebook.com/taymonbeal/posts/10213882533057819) and [Tumblr](https://taymonbeal.tumblr.com/post/621223508749484033/1000-bounty-for-pro-blm-policy-analysis).
# Image GPT

My hot take:

Not too surprising to me, considering what GPT-3 could do. However there were some people (and some small probability mass remaining in myself) saying that even GPT-3 wasn't doing any sort of reasoning, didn't have any sort of substantial understanding of the world, etc. Well, this is another nail in the coffin of that idea, in my opinion. Whatever this architecture is doing on the inside, it seems to be pretty capable and general.

I don't think this architecture will scale to AGI by itself. But the dramatic success of this architecture is evidence that there are other architectures, not too far away in search space, that exhibit similar computational efficiency and scales-with-more-compute properties, that are useful for more different kinds of tasks.
# What are some good public contribution opportunities? (100$ bounty)

I haven't gotten into MSFP or the like, but I like helping with the world's plot anyway. I've won 25$ in the Oracle contest, so I've told Stuart to put it towards further public contribution opportunities such as contests. He suggested that I do a small contest myself, for the 25$ and another 25$ from him.

So, selfish as I am, in one month, the 50$ go to the highest-karma answer, and another 50$ from me go to the answer that actually gets me to contribute. (In case of ambiguity on the latter, I try to award the cause of my best contribution. AF karma would be a good measure, except maybe the opportunities aren't all on AF...)

Examples of where I'm likely to contribute can be found via: https://www.alignmentforum.org/users/gurkenglas

Note how 25$ of prize money turn out worth more than 25$ in your pocket :)

Edit: Contest is over, contestant wins 50 dollars by walkover.
# Likelihood of hyperexistential catastrophe from a bug?

I've been reading a fair bit about "worse than death" scenarios from AGI (e.g. posts like [this](https://arbital.com/p/hyperexistential_separation/)), and the intensities and probabilities of them. I've generally been under the impression that the worst-case scenarios have extremely low probabilities (i.e. would require some form of negative miracle to occur) and can be considered a form of Pascal's mugging.

Recently, however, I came across [this](https://openai.com/blog/fine-tuning-gpt-2/) post on OpenAI's blog. The blog post notes the following:

> **Bugs can optimize for bad behavior**

> One of our code refactors introduced a bug which flipped the sign of the reward. Flipping the reward would usually produce incoherent text, but the same bug also flipped the sign of the KL penalty. The result was a model which optimized for negative sentiment while preserving natural language. Since our instructions told humans to give very low ratings to continuations with sexually explicit text, the model quickly learned to output only content of this form. This bug was remarkable since the result was not gibberish but maximally bad output.

This seems to be the exact type of issue that could cause a hyperexistential catastrophe. With this in mind, can we really consider the probability of this sort of scenario to be very small (as was previously believed)? Do we have a reason to believe that this is still highly unlikely to happen with an AGI? If not, would that suggest that current alignment work is net-negative in expectation?
# [Personal Experiment] One Year without Junk Media: Six-Month Update

The [mediums](https://www.lesswrong.com/posts/koa48Qio5LP48xMM2/mediums-overpower-messages) of the media I consume affect me more than the content so six months ago I committed to [one year without junk media](https://www.lesswrong.com/posts/wQnJ4ZBEbwE9BwCa3/personal-experiment-one-year-without-junk-media).

Here is a list of the biggest changes I've experienced over the last six months.

# Sleep

Junk media gives me insomnia. My insomnia always goes away within 1 week of abstinence. The problem has not returned during 6 months of abstinence.

Even without insomnia, I tend to go to sleep around 2-3 am and wake up around 11 am. I predicted (incorrectly) that long-term abstinence would bring my circadian rhythm closer in line with the day night cycle. This has not happened.

# Education

Before this experiment it took lots of willpower to study things like physics. I have a longer attention span now. I relax by training skills and learning things.

I had to stop studying [quantum field theory](https://www.lesswrong.com/posts/rK3Des3ri5KPiiGXC/intuitive-lagrangian-mechanics) because it has become *too addictive*. This vice appears limited to theoretical physics and junk media. No other subjects have triggered such intensity of addictive behavior.

Theoretical physics aside, I like getting smarter by default.

# Self-Care

I have a garden. I walk more. I write a lot. I cook good food by default. Though self-employed, my productivity at work remains unchanged.

# Socialization

Abstaining from junk media seems to increase my socialization from "almost none" to "a little bit" but it's hard to tell due to the confounding effects of COVID-19. Socializing makes me happier.

# Cleaning

When I begin abstaining from junk media I usually get a small bump to my tidiness. I predicted (incorrectly) this effect would become more pronounced over time. Instead, my tidiness returns to baseline levels as I acclimate.

# Stress

The common thread is lower overall stress. I have lower overall sympathetic nervous system activation. This could be the root cause of my lengthened attention span and reduced insomnia.
# If AI is based on GPT, how to ensure its safety?

Imagine that an advance robot is built, which is uses GPT-7 as its brain. It takes all previous states of the world and predicts the next step. If a previous state of the world includes a command, like "bring me a cup of coffee", it predicts that it should bring coffee and also predicts all needed movements of robot's limbs. GPT-7 is trained on a large massive of human and other robots data, it has 100 trillions parameters and completely opaque. Its creators have hired you to make the robot safer, but do not allow to destroy it.
# Online Curated LessWrong Talks

This Sunday at 11am (PDT), we're running a session of four "lightning talks" by curated LessWrong authors.

*   Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
*   We want this to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

[**Please RVSP here for the event here**](https://docs.google.com/forms/d/e/1FAIpQLSfBN-4_FIPddLV955nWY21uIumUNrNjYNaDJjiFyPUDg0hZUg/viewform), so we know how many people are coming.

Speakers
--------

*   **Abram Demski**
    *   Curated posts: [An Orthodox Case Against Utility Functions,](https://www.lesswrong.com/posts/A8iGaZ3uHNNGgJeaD/an-orthodox-case-against-utility-functions) [Toward a New Technical Explanation of Technical Explanation)](https://www.lesswrong.com/posts/tKwJQbo6SfWF2ifKh/toward-a-new-technical-explanation-of-technical-explanation), ...
*   **alkjash**
    *   Curated posts: [Babble](https://www.lesswrong.com/posts/i42Dfoh4HtsCAfXxL/babble), [Hammers and Nails](https://www.lesswrong.com/posts/QzBuuNEqJGQFeWM4f/hammers-and-nails), [Hammertime Final Exam](https://www.lesswrong.com/posts/Q7MsMshzbzhEs729s/hammertime-final-exam)
*   **orthonormal **
    *   Curated posts: [The Loudest Alarm Is Probably False](https://www.lesswrong.com/posts/B2CfMNfay2P8f2yyc/the-loudest-alarm-is-probably-false)
*   **eukaryote**
    *   Curated posts: [The funnel of human experience](https://www.lesswrong.com/posts/SwBEJapZNzWFifLN6/the-funnel-of-human-experience), [Spaghetti Towers](https://www.lesswrong.com/posts/NQgWL7tvAPgN2LTLn/spaghetti-towers), [Naked mole-rats: A case study in biological weirdness](https://www.lesswrong.com/posts/fDKZZtTMTcGqvHnXd/naked-mole-rats-a-case-study-in-biological-weirdness)

Details
-------

**When?** Sunday June 21, at 11am, Pacific Daylight Time

**Where?** We'll share a call link on this post (most likely using Zoom). 

[**Please RVSP here for the event here**](https://docs.google.com/forms/d/e/1FAIpQLSfBN-4_FIPddLV955nWY21uIumUNrNjYNaDJjiFyPUDg0hZUg/viewform), so we know how many people are coming.
# SlateStarCodex deleted because NYT wants to dox Scott

[NYT Is Threatening My Safety By Revealing My Real Name, So I Am Deleting The Blog](https://slatestarcodex.com/2020/06/22/nyt-is-threatening-my-safety-by-revealing-my-real-name-so-i-am-deleting-the-blog/)

PS: One [suggestion](https://www.lesswrong.com/posts/zhRYqMee72k4Kydzw/creating-better-infrastructure-for-controversial-discourse) I have is to allow anonymous posts on Lesswrong that show the author’s anonymized karma. This is far from a good or complete solution, but I imagine it would at least come in handy in situations like this.