Harley-ml/lesswrong huggingface dataset:
# Open problem: how can we quantify player alignment in 2x2 normal-form games?

In my experience, [constant-sum games](http://www.cs.umd.edu/~hajiagha/474GT13/Lecture09102013.pdf) are considered to provide "maximally unaligned" incentives, and [common-payoff games](http://www.cs.umd.edu/~hajiagha/474GT13/Lecture09102013.pdf) are considered to provide "maximally aligned" incentives. How do we quantitatively interpolate between these two extremes? That is, given an arbitrary $2\times2$ payoff table representing a two-player [normal-form game](https://en.wikipedia.org/wiki/Normal-form_game) (like Prisoner's Dilemma), what extra information do we need in order to produce a real number quantifying agent alignment? 

If this question is ill-posed, why is it ill-posed? And if it's not, we should probably understand how to quantify such a basic aspect of multi-agent interactions, if we want to reason about complicated multi-agent situations whose outcomes determine the value of humanity's future. (I started considering this question with Jacob Stavrianos over the last few months, while supervising his [SERI project](https://www.lesswrong.com/posts/buaGz3aiqCotzjKie/game-theoretic-alignment-in-terms-of-attainable-utility).)

Thoughts:

*   Assume the alignment function has range $[0,1]$ or $[-1,1]$.
    *   Constant-sum games should have minimal alignment value, and common-payoff games should have maximal alignment value.
*   The function probably has to consider a strategy profile (since different parts of a normal-form game can have different incentives; see e.g. [equilibrium selection](https://en.wikipedia.org/wiki/Equilibrium_selection)).
*   The function should probably be a function of player A's alignment *with* player B; for example, in a prisoner's dilemma, player A might always cooperate and player B might always defect. Then it seems reasonable to consider whether A is *aligned with* B (in some sense), while B is not aligned with A ([they pursue their own payoff without regard for A's payoff](https://www.lesswrong.com/posts/8LEPDY36jBYpijrSw/what-counts-as-defection)).
    *   So the function need not be symmetric over players.
*   The function should be invariant to applying a separate positive affine transformation to each player's payoffs; it shouldn't matter whether you add 3 to player 1's payoffs, or multiply the payoffs by a half.
*   ~~The function may or may not rely only on the players' orderings over outcome lotteries, ignoring the cardinal payoff values. I haven't thought much about this point, but it seems important.~~ EDIT: I no longer think this point is important, but rather confused.

If I were interested in thinking about this more right now, I would:

*   Do some thought experiments to pin down the intuitive concept. Consider simple games where my "alignment" concept returns a clear verdict, and use these to derive functional constraints (like symmetry in players, or the range of the function, or the extreme cases).
*   See if I can get enough functional constraints to pin down a reasonable family of candidate solutions, or at least pin down the type signature.
# AI-Based Code Generation Using GPT-J-6B

Above is a link to an interesting post about synthetic code generation with a transformer model trained on *The Pile*, which contains a large chuck of GitHub and StackOverflow. Due to *CommonCrawl*'s deficiency in this area, the much smaller GPT-J-6B outperforms OpenAI’s largest publicly available GPT-3 models. The performance is impressive enough that one wonders how capable a 100+ billion parameter model trained on *The Pile* will be, let alone what an AlphaGo-level engineering effort towards the end of synthetic code generation would achieve. 

As the *The Pile* was created to provide a dataset for 100 billion paramater+ models, we may not have to wait long. The examples in the post are clearly trivial, but I personally take this to be something of a fire alarm. I was not previously aware of how poorly-optimized GPT-3 was for code generation, and I have updated toward surprising gains in this area in the next few years.

I no longer consider agents with superhuman performance in competitive programming to be a ridiculous thing to pursue. 

It is useful to remind myself of how shocked I would be to see such things in 2012. In 2012 I would have taken this as a sign that AGI was near. 

Scenario-based planning postulates that one should predict symptoms emblematic of a given scenario and then robotically assume you are in said scenario once a sufficient number of these symptoms come to pass. I am unsure whether there is wisdom in this approach, but I find it a discomfiting line of thought.
# How to make errands fun

Despite being a believer in outsourcing all tasks that cost less than my [aspirational hourly rate](https://nav.al/hourly-rate), I regularly have to do such tasks when they're impractical to outsource. This is almost always, as the most outsourceable tasks are those that repeat often, but cannot be automated. Doing legwork to do paperwork always happens for unique paperwork.

I'm going through a continuous phase where I just can't accept that I actually have to go to the bank and wait in a queue and listen to terrible music. So I want to offer myself (and others who may read this) some nudges towards making life more bearable when running non-repetitive but boring errands.

Make it physical training!
==========================

People make neat compartments for things that are physical training, and things that are not physical training. But way back before you could pay to come into an air-conditioned room and move metal around, there were no such compartments. You moved. That was it. So maybe you jump on your bike and ride to the bank. Or you see a nice tree on your way back, and you climb it. Or you jump a couple of fences and shorten the distance. Or you just take a long walk.

Make it rest from physical training!
====================================

Having compartments for physical movement isn't all bad because they allow you to go all in one one particular skill or muscle group. Nobody gets a good deadlift from just walking to the bank. So maybe, instead of having compartments for rest days and non-rest days, you just have errand days where you can't make it to the gym and you go: "Oh, I guess today is rest day."

Make it errand-related learning!
================================

Is there some learning experience that you could benefit from? Bureaucracy is actually a great training tool for figuring out people and their motivations. A bank teller isn't just a REST API that you have to use in person: it's a unique human, with a unique set of motivations, desires, quirks, mannerisms, and more. If you become interested in seeing the person, and not the interface, an entire world opens up. See [Everything is fertile](http://nickcammarata.com/writing/everything-is-fertile).

Make it non-errand-related learning!
====================================

Did you know that they make audiobooks?

Make it an opportunity to socialize!
====================================

Go with a friend! "Hey, wanna keep me company while I do X and then go get ice cream?" Or you go with several friends, and arrive to the bank with your entourage. Just don't make the errand the entirety of the meeting: reward your friends' patience and go do something more fun afterwards.

Make it stop!
=============

Like, just suffer. Maybe if you experience enough spiritual pain, you will finally find a way not to do the errand any more.
# Who wants to run a B2B SaaS startup?

A friend of mine has been bootstrapping a business-to-business software-as-a-service startup that's seeing serious growth. It needs someone who can put dedicated effort into scaling it, but my friend is near the end of their career and looking to retire. What do people do in this situation?

More details: they were running a traditional labor-limited small business and they automated some of the work. This automation was a huge improvement and they realized it could be useful to other companies. In early 2019 they had a web app ready and started taking external customers. In late 2020 they started to see serious growth, which has continued. They let me share some numbers:

[![](https://www.jefftk.com/b2b-saas-daily-revenue-30d-average.png)](https://www.jefftk.com/b2b-saas-daily-revenue-30d-average-big.png)

This is a run rate of ~$340k/y, up from ~$100k/y a quarter ago, ~$52k a quarter before that, ~$12k/y a quarter before that, and ~$7k a year ago. That's growth of ~50x in a year, or 7.8% a week, which is extremely good. Expenses are servers ($40/m), plus the time they have been putting into it (~20hr/wk).

They have ~50 customers, and churn is essentially zero: they have only had one customer leave, and that was because that customer lost the contract they were using the service for. Their customers are really happy: the service is very reasonably priced given the large amount of work it saves. My friend estimates that they are at under 1% of their potential customers with the current offering, has ideas for several other things they could offer, and doesn't know of any competitors.

It looks to me like it's uncommon for someone to sell a business at this stage: typically a founder would at least stay through the hyper-growth phase. Unfortunately, my friend is not able to give the company the attention it deserves, and it could grow much more effectively under management that wants to be running a fast-growing startup.

I don't know what makes sense here. Possibly someone who is thinking of founding a startup might instead be interested in buying this one? I don't know if there's some thing where you buy a company with its own stock, vesting over some number of years? This would be a very good situation compared to most tech startups: they have product-market fit, customers love it, they're profitable, and they're seeing excellent growth. Alternatively, perhaps there are companies that buy businesses like these and run them? Does anyone have experience with something like this?

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100220609788052)*
# The aducanumab approval

> As the world knows, the FDA approved Biogen’s anti-amyloid antibody today, surely the first marketed drug whose Phase III trial was stopped for futility. I think this is one of the worst FDA decisions I have ever seen, because – like the advisory committee that reviewed the application, and [like the FDA’s own](https://blogs.sciencemag.org/pipeline/archives/2020/11/06/aducanumab-at-the-fda) statisticians – [I don’t believe](https://blogs.sciencemag.org/pipeline/archives/2019/12/06/they-dont-know) that Biogen really demonstrated efficacy. No problem apparently. The agency seems to have approved it based on its demonstrated ability to clear beta-amyloid, and is asking Biogen to run a confirmatory trial to show efficacy.
> 
> \[...\]
> 
> So the FDA has, for expediency’s sake, bought into the amyloid hypothesis although *every single attempt* to translate that into a beneficial clinical effect has failed. I really, really don’t like the precedent that this sets: what doesn’t get approved, now? I suppose only things that definitely cause harm, because otherwise why not just ask for the same deal that Biogen got, and go out and prove efficacy while you turn a profit? I know that this is just the libertarian turn that many people have [wished for](https://blogs.sciencemag.org/pipeline/archives/2018/05/25/federal-right-to-try), but I’m still not sanguine about it.
# You can now apply to EA Funds anytime! (LTFF & EAIF only)

The [Long-Term Future Fund](https://funds.effectivealtruism.org/funds/far-future) (LTFF) and the [EA Infrastructure Fund](https://funds.effectivealtruism.org/funds/ea-community) (EAIF) are looking for grant applications:

*   **You can now apply for a grant anytime**. We have removed the previous round-based system, and now aim to evaluate most grants within 21 days of submission (and all grants within 42 days), regardless of when they have been submitted. If you indicate that your application is time-sensitive, we will aim to get back to you more quickly (potentially within just a few days). [**Apply now**](https://funds.effectivealtruism.org/apply-for-funding)**.**
*   **You can now suggest that we give money to other people, or let us know about ideas for how we could spend our money.** We’re interested in both high-level ideas and concrete, shovel-ready grant opportunities. We will read all suggestions, but we expect to follow up on only a small number. It’s hard to find great grants, so we really appreciate your suggestions! [**Suggest a grant**](https://docs.google.com/forms/d/e/1FAIpQLSejotmg5sUus676-ywhbQD_192kAteygLbzRudUmxiJKM7byw/viewform)**. **
*   We fund student scholarships, career exploration, local groups, entrepreneurial projects, academic teaching buy-outs, top-up funding for poorly paid academics, and many other things. We can make anonymous grants without public reporting. We will consider grants as low as $1,000 or as high as $500,000 (or more in some cases). As a reminder, [**EA Funds is more flexible than you might think**](https://forum.effectivealtruism.org/posts/caayRw2pgNLtqt9Pz/ea-funds-is-more-flexible-than-you-might-think)**.**
*   The LTFF is managed by Asya Bergal (chairperson), Adam Gleave, Evan Hubinger (newly appointed), and Oliver Habryka. For the coming months, they will be joined by Luisa Rodriguez as a guest manager. See its [recent payout report](https://forum.effectivealtruism.org/posts/diZWNmLRgcbuwmYn4/long-term-future-fund-may-2021-grant-recommendations).
*   The EAIF is managed by myself (interim/acting chairperson), Buck Shlegeris, Max Daniel, and Michelle Hutchinson. For the coming months, Linh Chi Nguyen and Michael Aird will join as guest managers. See its [recent payout report](https://forum.effectivealtruism.org/posts/zAEC8BuLYdKmH54t7/ea-infrastructure-fund-may-2021-grant-recommendations) and [AMA](https://forum.effectivealtruism.org/posts/KesWktndWZfGcBbHZ/ea-infrastructure-fund-ask-us-anything).
*   The [Animal Welfare Fund](https://funds.effectivealtruism.org/funds/animal-welfare) will continue on a round-based system. For recent updates, see [Request For Proposals: EA Animal Welfare Fund](https://forum.effectivealtruism.org/posts/yDhijyrbu3fYPLbn2/request-for-proposals-ea-animal-welfare-fund) and [Animal Welfare Fund: Ask us anything!](https://forum.effectivealtruism.org/posts/pmr9tR2GYoxDMqbor/animal-welfare-fund-ask-us-anything)

[**Apply here**](https://funds.effectivealtruism.org/apply-for-funding). We look forward to hearing from you!
# Non-poisonous cake: anthropic updates are normal

I am on a quest to show that [anthropics probability are normal](https://www.lesswrong.com/posts/BYy62ib5tAkn9rsKn/sia-is-basically-just-bayesian-updating-on-existence), at least in the absence of exact duplicates.

So consider this simple example: a coin is tossed. This coin is either fair, is $3/4$ biased to heads, or $3/4$ biased to tails; the three options are equally likely. After being tossed, the coin is covered, and you eat a cake. Then you uncover the coin, and see that it was tails.

You can now update your probabilities on what type of coin it was. It goes to a posterior of $1/6$ on the coin being heads-biased, $1/3$ on it being fair, and $1/2$ on it being tails-biased[^calc]. Your estimated probability of it being tails on the next toss is $(1/6)(1/4)+(1/3)(1/2)+(1/2)(3/4)= 7/12$.

[^calc]: The probability of tails given the heads-biased coin is $1/4$; given the fair coin it is $1/2=2/4$, and given tails-biased it is $3/4$. So the odds are $1:2:3$; multiplying these by the (equal) prior probabilities doesn't change these odds. To get probabilities, divide the odds by $6$, the sum of the odds, and get $1/6$, $2/6=1/3$ and $3/6=12$.

Now you are told that, had the coin come up heads, there would have been poison in the cake and you would have died before seeing the coin.

![](https://www.dropbox.com/s/ced2he400eejnc5/Poison_cake.png?raw=1)

This fact makes the problem into an anthropic problem: you would never have been alive to see the coin, had it come up heads. But I can't see how that would have changed your probability update. If we got ethics board approval, we could actually run this experiment. And for the survivors in the tail worlds, we could toss the coin a second time (without cake or poison), just to see what it came up as. In the long run, we would indeed get roughly $7/12$ tails frequency. So the update was correct, and the poison makes no difference.

Again, it seems that, if we ignore identical copies, anthropics is just normal probability theory. Now, if we knew about the poison, then we could deduce that the coin was tails from our survival. But that information gives us exactly the same update as seeing the coin was actually tails. So "I survived the cake" is exactly the same type of information as "the coin was tails".

## Incubators

If we had more power in this hypothetical thought experiment, we could flip the coin and [create you](https://www.lesswrong.com/posts/svhbnSdxW3XmFXXTK/anthropic-decision-theory-i-sleeping-beauty-and-selflessness#The_Sleeping_Beauty_problem__and_the_incubator_variant) if it comes up tails. Then, after getting over your surprise, you could bet on the next flip of the coin - and the odds on that will be the same as in the poison cake and in the non-anthropic-case. Thus updates are the same if:

1. Standard coin toss, you see tails.
2. Poison cake situation, you survive the cake.
3. You're created on tails flip and notice you exist.
# Are bread crusts healthier?

When you think about it for a second, almost definitely not. They're exactly the same stuff as bread, just more burnt, and so more likely to have the nutritious stuff broken down, and contain carcinogens. Indeed a quick internet search "busts the myth" that crusts are healthier.

So when my baby wasn't very hungry, and I was deciding which bits of the roll to give him for his last few bites of supper, why did I catch myself thinking "let's give him the crust, it's healthier"?

I guess the virtue theory of diet runs deeper in my subconscious than I realized. "If you are virtuous, and eat unpleasant foods, then they are good for you and you will grow strong and healthy." Usually when we have biases we can run rings around ourselves justifying them. It's good to occasionally catch ourself so red handedly in the process of magical thinking that we just have to accept we're occasionally irrational and work on rectifying that.
# Assume long serving politicians are rationally maximizing their careers

Imagine you had no idea how to ride a bull, and you encountered a bull rider. The rider is swinging his free arm and body wildly, at apparent random. Surely hugging the bulls back and moving less is better. You would immediately think the man is crazy, perhaps he’s having a seizure.

Now imagine your friend says “he’s been on that bull for five minutes, a town record”. Know you should reconsider. Surely someone has tried your strategy and lasted less. Flailing must help him stay on the bulls back, strange as it seems.

Politicians are the same way. Every political system has a steady stream of talented challengers, who incumbents must defeat. The Nancy Pelosis, Boris Johnson’s and Ayatollah Khameinis of the world may look like their flailing, but usually the know exactly what they’re doing.
# Why did no LessWrong discourse on gain of function research develop in 2013/2014?

While I wasn't at 80% of a lab leak when Eliezer asseted it a [month ago](https://www.facebook.com/yudkowsky/posts/10159653334879228?__cft__[0]=AZWPer58c5bHvoZdDYZaBmccmDblXHGqf88oJrB_q5sh5_cU9dUY-8aD9gXKV-R5BhgC5cIE1OsBLIeUp2ni7XwjaYBvbDHRlXn8_3JIQkpWoOYurWdwRmpU-66br6PyuPU&__tn__=%2CO%2CP-R), I'm now at 90%. It will take a while till it filters through society but I feel like we can already look at what we ourselves got wrong.  

In 2014, in the LessWrong survey more people considered bioengineered pandemics a global catastrophic risk then AI. At the time there was a public debate about gain of function research. On [editoral](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4271556/) described the risks of gain of function research as:

> Insurers and risk analysts define risk as the product of probability times consequence. Data on the probability of a laboratory-associated infection in U.S. BSL3 labs using select agents show that 4 infections have been observed over <2,044 laboratory-years of observation, indicating at least a 0.2% chance of a laboratory-acquired infection ([5](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4271556/#B5)) per BSL3 laboratory-year. An alternative data source is from the intramural BSL3 labs at the National Institutes of Allergy and Infectious Diseases (NIAID), which report in a slightly different way: 3 accidental infections in 634,500 person-hours of work between 1982 and 2003, or about 1 accidental infection for every 100 full-time person-years (2,000 h) of work ([6](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4271556/#B6)).
> 
> A simulation model of an accidental infection of a laboratory worker with a transmissible influenza virus strain estimated about a 10 to 20% risk that such an infection would escape control and spread widely ([7](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4271556/#B7)). Alternative estimates from simple models range from about 5% to 60%. Multiplying the probability of an accidental laboratory-acquired infection per lab-year (0.2%) or full-time worker-year (1%) by the probability that the infection leads to global spread (5% to 60%) provides an estimate that work with a novel, transmissible form of influenza virus carries a risk of between 0.01% and 0.1% per laboratory-year of creating a pandemic, using the select agent data, or between 0.05% and 0.6% per full-time worker-year using the NIAID data.

Even at the lower bar of 0.05% per full-time worker-year it seems crazy that society continued playing Russian Roulette. We could have seen the issue and protested. EA's could have created organizations to fight against gain-of-function research. Why didn't we speak every Petrov day about the necessity to stop gain of function research? Organizations like OpenPhil should go through the 5 Why's and model why they messed this up and didn't fund the cause. What needs to change so that we as rationalists and EA's are able to organize to fight against tractable risks that our society takes without good reason?
# Up-to-date advice about what to do upon getting COVID?

LessWrong has hosted a [few](https://www.lesswrong.com/posts/F3q7eL7pdQqhWFTYh/what-should-we-do-once-infected-with-covid-19) [posts](https://www.lesswrong.com/posts/B9qzPZDcPwnX6uEpe/coronavirus-justified-practical-advice-summary) [that](https://www.lesswrong.com/posts/CewTrCRfyCgW4NJQM/rob-bensinger-s-covid-19-overview#3__Things_to_do_if_you_get_sick_) at least in part tell people what they should do upon getting COVID. That being said, they don't seem to have been updated since last March, and presumably more must be known now. The topic is newly relevant now that the [delta variant](https://en.wikipedia.org/wiki/SARS-CoV-2_Delta_variant) is spreading in many locations, with vaccination rates being low in most countries.
# 10 Deadly Viruses And Bacteria Created In Labs - 6 SARS 2.0

The post is written in January 23, 2019 about 10 Deadly Viruses created in labs:

> 6 SARS 2.0
> ----------
> 
> [Severe acute respiratory syndrome](https://listverse.com/2016/06/29/10-people-who-were-the-patient-zero-of-a-deadly-epidemic/) (SARS) is a lethal virus. More than 700 people were killed during a SARS epidemic that infected 8,000 people in 29 countries between 2002 and 2003. Now, scientists have made it deadlier.
> 
> The new mutant SARS virus was created by a group of researchers led by Dr. Ralph Baric of the University of North Carolina. They call it SARS 2.0. The researchers developed the virus by adding some protein to the naturally occurring SARS. SARS 2.0 is immune to vaccines and treatments used to cure the naturally occurring SARS virus.[^\[5\]^](https://motherboard.vice.com/en_us/article/wnxqnm/ethical-questions-arise-after-scientists-brew-super-powerful-sars-20-virus)
> 
> The team said that the research was necessary because the natural SARS virus could mutate and become immune to our vaccines. By creating a deadlier and mutated virus, we could develop stronger vaccines that will save us from a more lethal SARS epidemic. That is, if the natural SARS ever mutates.
> 
> **However, other scientists are concerned because the SARS 2.0 that is supposed to save us from a deadly SARS epidemic could start that epidemic if it ever** [**escapes**](https://listverse.com/2017/08/12/top-10-remarkable-escapes-across-the-berlin-wall/) **from the lab.**

It seems like listverse outperformed most of the EA ThinkTanks on the question of thinking about pandemics pre-COVID.
# Irrational Modesty

At an online meetup I attended, I talked to a fellow who was working on some software that he thought would be useful for aggregating information for Effective Altruists. He said he would like to work on it full time this summer rather than get an internship - but that this was not financially feasible. 

His idea seemed not-obviously stupid and I suggested he apply for a LTFF or EAIF grant. He said he did not want to as he was worried the money could go to better use.

 I asked him if he thought the grant-makers were in a better position than him to decide if his project would be an efficient use of their capital. He said he did. I then asked him if he thought his idea was so likely to be a waste of time that even reading his proposal would be net negative in expectation; he said, "No". 

He is now working on his proposal.

Here we have an example of someone who is likely much smarter than me but was unable to think clearly due to irrational modesty. I am worried this may be true of some *extremely* capable people, and how tragic it would be if this is the case. 

The vast majority lack the requisite ability to do alignment research. I have a good model of my talents and capabilities, and I am certain I do not. In truth, it is not even close. Those who are capable, confident in their abilities, and motivated to work on this problem do not need a peptalk. But the possibility that there is a class of highly-talented would-be-motivated people who lack confidence in their abilities still haunts me.

I am reminded of this humorous exchange:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/92be32ff836a8d733a6de2a2bf8736f4009068a462c9a325.png)

Though quentin's blindness to his own intelligence is humorous, it *is* a mistake and a rather large one. Irrational modesty is every bit as bad as overconfidence, possibly more so. 

In the event  anyone reading this has objective, reliable external metrics of extremely-high ability yet despite this feels unworthy of exploring the possibility that they can contribute directly to research, my advice is to act as if these external metrics correctly assess your ability until you have thoroughly proven to yourself  otherwise.

There is no virtue in clutching Kryptonite. I advise you to drop it and see how far you can fly.
# Four Components of Audacity 

For a long time I've wondered how to measure nonconformity. To measure nonconformity I needed to define "nonconformity". But no matter how I defined "nonconformity" my definitions felt so subjective they could apply to anybody, from a certain point of view. If everybody is nonconformist then nobody is nonconformist because the word "nonconformist" isn't meaningful.

Today I realized that the opposite of conformity is audacity.

> # **audacity**
>
> *noun, plural* **au·dac·i·ties.**
>
> 1. boldness or daring, especially with confident or arrogant disregard for personal safety, conventional thought, or other restrictions.
>
> 2. effrontery or insolence; shameless boldness:
> *His questioner's audacity shocked the lecturer.*
>
> 3. Usually **audacities** . [audacious](https://www.dictionary.com/browse/audacious) or particularly bold or daring acts or statements.
>
> ―[Dictionary.com](https://www.dictionary.com/browse/audacity)

Audacity is bold, daring, shameless and impertinent. Cultivate these qualities and you will cultivate nonconformity.

My favorite technique of **boldness** is to simply tell the truth. One trick is to never prefix statements with ["I believe"](https://www.lesswrong.com/posts/CqyJzDZWvGhhFJ7dY/belief-in-belief). Don't say "I believe $x$". If $x$ is true then just say "$x$". (If $x$ is untrue then don't say $x$ and don't believe $x$.) The unqualified statement is bolder. [Crocker's rules](https://wiki.lesswrong.com/wiki/Crocker%27s_rules) encode boldness into a social norm.

**Daring** comes from [doing things that scare you](https://www.lesswrong.com/posts/qHpazCw3ryvBojGSa/my-fear-heuristic).

**Shamelessness** comes from not caring what other people think on short time horizons.

> The most impressive people I know care a lot about what people think, even people whose opinions they really shouldn’t value (a surprising numbers of them do something like keeping a folder of screenshots of tweets from haters). But what makes them unusual is that they generally care about other people’s opinions on a very long time horizon—as long as the history books get it right, they take some pride in letting the newspapers get it wrong. 
>
> ―[*The Strength of Being Misunderstood*](https://blog.samaltman.com/the-strength-of-being-misunderstood) by Sam Altman

**Impertinence** comes from treating superiors as equals. I don't know how to cultivate impertinence because [I'm status-blind to begin with](https://www.lesswrong.com/posts/jvpPknEbbsjx44wty/status-regulating-emotions). Impertinence is the opposite of submission; it is dangerous to be impertinent when your livelihood is on the line.
# Idea: Give the Nobel peace prize to institution designers

The Nobel committee has a problem. They keep giving the peace prize to people to war criminals.

In their most recent debacle, they have the 2019 prize to Abey Ahmed of Ethiopia for making peace with Eritrea. It seemed sensible, a popular young presidents buried the hatchet with their former territory to focus on development. Ahmed seemed like a good man, a great candidate.

Unfortunately, being a good man is easily faked. We now know Ahmed’s real motivation for peace was expanding his coalition relative to internal rivals that might stop his rise to dictatorship. Shortly after the peace deal Ahmed canceled federal elections to destroy subnational rivals. The Tigrayans resisted, and now Ethiopian and Eritrean troops together are busily destroying Tigray.

The problem is that politicians can and do pretend to be “good” to get power. If you are a potential ally, then whatever you define as good can easily be impersonated; committed democrat, liberal, communist, developmental state. After all, any politician that can’t pretend to care never reaches the national stage. The Nobel Peace prize is one more feather in their cap, which only requires a bit more pretending. But in weak democracies the charade only lasts as long as the leader is vulnerable. Once the leader gets enough mercenaries, expectation and appointees, eliminating rivals is worth more than Norwegian admiration.

What are the Norwegian bigwigs to do? They want to give prizes to high-status people that further peace. Sure, they can give grants to NGO’s and activists, but these political lightweights are lower-status.

How about constitutional designers? The real problem of politics is not finding someone who isn’t a sociopath, it’s designing a system that makes the sociopaths serve the people. Thomas Jefferson sold pardons after he lost re-election, but for decades before that he worked hard for the people to build his career.

The lawyers and politicians who write them are lower status than presidents, but they are higher status than NGO leaders or street activists. And they are not in power, so they can’t commit atrocities later.

More importantly, the institutional design, coalition building and negotiating that produces peace and democracy don’t get enough publicity. It’s a long hard slog, slow boring of hard beams. The us didn’t protect democracy by picking good people, they distributed violence capacity between 13 governors and restricted the president from appointing West Point students.

I live in DC with a large Ethiopian community, who are acrimoniously split over Abey Ahmed. The Amharans argue that Ahmed is “good” and the Tigrayan leaders are “bad”(from crimes in the 90s) so the violence is “good”. I try to explain that when Ahmed has the only remaining army, democracy will  hinge on his patronage networks need. 

Maybe giving a Nobel peace prize to a Hamilton instead of a Washington will teach people that peace needs thought-out institutions not “good” people.
# Burst work or steady work?

Some people work at a steady pace, below their Epic Maximum Work Capacity. Others try to keep up their Maximum Capacity, sometimes(?) burning out, but getting more done in the short bursts.

Over a time of, say, 6-18 months, which practice will lead to more work in the long run? Are there data or basic facts that lead us to one or the other? Are they both situationally useful, if so *in which situations*? And what about people's individual neurochemistry? (dopamine, "spoons", discipline, motivation, etc.)

Which is more true, more often? "Slow is smooth, smooth is fast", or "work in focused bursts"? [Blue Origin](https://www.inc.com/lisa-calhoun/digging-into-how-jeff-bezos-landed-customer-1.html) or [SpaceX](http://www.parabolicarc.com/2016/09/25/60-80-work-weeks-good-idea/)?
# How do you deal with people cargo culting COVID-19 defense?

In Berlin there are rules for public transportation where all passengers have to wear FFP-2 masks. Nobody, seems to care whether people wear the masks in a way that seals. The security personal in public transportation only wears surgical masks. Whenever, I see security personal  in public transportation they do nothing besides standing around. Security personal that stands around doing nothing might make people feel safer, but the idea of having security personal with less masking makes them unable to credible enforce mask wearing and endangers passengers. People on public transportation talk the same amount they did pre-COVID-19, seemingly not knowing that talking widely increases spread. 

In the [podcast](https://clearerthinkingpodcast.com/?ep=029) between Spencer Greenberg and Jacob Falkovich, Jacob makes the point that it seems like people don't understand that COVID-19 is caused by a virus and that they way to prevent getting it is to show one's virtue by engaging in socially approved rituals against COVID-19. 

It becomes increasingly hard for me to take people seriously and the security people with less masking then the passenger were a huge strike. How do you deal emotionally with it? Do you become cynic?
# Two non-obvious lessons from microcovid.org

At the 2021 Summer Solstice, Elizabeth Van Nostrand made a brief speech thanking the organizers of microcovid.org, which I found very heartwarming and meaningful.

I wish I had thought in advance about taking the opportunity to make a few public remarks about microcovid.org: things I wish the community knew, that weren't obvious.

Here's what I would've said, in terms of my lessons from this project:

1. Build your *own* oxygen mask. Next, share it with others.
2. Connect and collaborate with non-rationalists.


# 1) Build your *own* oxygen mask. Next, share it with others.

We didn't *start out* trying to create a resource for our whole community, let alone a website with many thousands of users. All we wanted was to save our own asses. We looked at the precipitous "autonomy crunch" we were facing, and said "oh shit, our house is going to explode if we don't fix this."

So, we built a spreadsheet — *for ourselves* first. Other group houses asked about it, and the momentum snowballed inexorably from there. Each broadening of project scope was compelled by a commensurate rise in demand, and corresponding deeply felt motivation.

I think many people who have altruistic or worldsaving ambitions could stand to have more focus on first making their own lives not suck. Fixing huge problems in your own life — and then later making an extra effort to share and export them — is one important path to altruistic impact.

# 2) Connect and collaborate with non-rationalists
To my knowledge, I'm the only project member out of the top dozen or so top contributors who self-identifies as a rationalist.

The "core idea" is an extremely, extremely rationalist idea. But the implementation took writers, copyeditors, web developers, backend developers, UX designers, a medical doctor whose patients were among our first users, and many more. These folks had to understand the core idea and know how to use it, but did not have to be skilled enough at quantitative risk thinking to have designed it in the first place.

The final product had a vastly more scalable reach because many people, who had very little identity-level commitment to epistemics, looked at it and said things like "I need to access this on my phone, I won't ever use a spreadsheet" or "This has too much jargon, move all these details to the appendix."


Thank you again everyone for the gratitude and recognition; and for using the system to make your own lives suck a little less!
# How do the side effects of Novavax compare to other COVID-19 vaccines?

I'm curious whether Novavax has the same side-effects of the mRNA vaccines where normal side-effects often involve the person being feeling too sick to do anything and having fever the next day. Unfortunately, most sources I find while googling aren't very detailed when they talk about side effects. If Novavax would have side-effects that are more similar to a normal flu vaccine then to mRNA vaccines that would be good to know.
# I Will Organize Meetups in Norfolk, Virginia [Hampton Roads Metro Area]

Hi!

I recently moved to Norfolk, Virginia. I used to organize the [in-person meetups prior to COVID when I lived in Houston, Texas](https://www.lesswrong.com/groups/pwitck4YZQsnXYp9q) and still organize meetups there virtually. 

Now that I'm living in a new place, and am vaccinated, I want to attend in-person meetups again, but have searched and don't see any in the Norfolk, Virginia area so I'm starting a meetup group here!

Norfolk is part of the broader [Hampton Roads](https://en.wikipedia.org/wiki/Hampton_Roads) community of cities. I don't know the area too well yet, but it seems like Norfolk is a relatively central location within the metro area (correct me if I'm wrong, please) and is one of the larger cities so it seems like a good spot to run meetups. I'm happy to travel to other cities in the area to host meetups if most potential attendees are concentrated outside of Norfolk (let me know!).

The meetups I organize are open to all, and unless mentioned otherwise are focused on building a social group and community in the area. If there are enough group members and meetup attendees, and enough interest in specific ideas then I'm happy to organize smaller more focused on something specific / productive kinds of meetups and events. Each social meetup will include 1 or 2 suggested readings so that we can have a common shared topic to start from, but meandering as far as our interests take us is welcome! Note that even if you are only interested in smaller, specific / focused meetups, it is strategically beneficial to build up a social rationalist community in your area, because that's where such groups and events typically draw from. Thus, if you don't want to attend social meetups here, please still reach out to me so I can connect you with anyone else who may share your same interest in the area, if they are available and willing.

Please comment on this post and/or message me here on LessWrong if you live in the area (or visit from time to time or often) and want to attend meetups. Let's build a community, have fun, and improve our art of rationality!

Cheers,  
Willa
# Announcing the Replacing Guilt audiobook

*tldr; Complete narration of the Replacing Guilt series is available here:* [*anchor.fm/guilt*](https://anchor.fm/guilt)

I discovered Nate Soares' [Replacing Guilt series](http://mindingourway.com/guilt/) in late 2019 via the [Bayesian Conspiracy podcast](https://www.thebayesianconspiracy.com/2019/08/92-replacing-guilt/) and found it immensely valuable. Over ~40 posts, Nate channels the guilt-based motivation that is common in rationalists / EAs into more productive emotional drives.  

I read the series on my Kindle, thanks to [lifelonglearner's ePUB version](https://www.lesswrong.com/posts/rFrefJgPvYkdtG7Pt), but was disappointed that no audio version existed. To make it more accessible, I reached out to Nate and got permission to produce the official audiobook.

The rationalist community has a fantastic tradition of volunteer narration — Slate Star Codex , AI to Zombies, and, of course, HPMOR. These free resources have added immense value to my life, so narrating Replacing Guilt feels like my tiny way of reinvesting in our rationalist commons. 

For a 2-minute summary of the series and why you might be interested in it, check out [this post](http://mindingourway.com/replacing-guilt/) (or listen to the [audio version](https://anchor.fm/guilt/episodes/01-Replacing-guilt-eaatlb)).

You can find the individual episodes at [anchor.fm/guilt](https://anchor.fm/guilt). The complete audiobook can be [streamed](https://anchor.fm/guilt/episodes/Replacing-Guilt-full-audiobook-e13ct4d/a-a5vrdtu) in any podcast app, or you can download the [mp3 file here](https://drive.google.com/file/d/1nyOFEZZ_QtdnT7ujsB8Uy9Es_A4DKQGa/view?usp=sharing).
# What will be the aftermath of the US intelligence lab leak report?

I wrote a long post laying out the [evidence for the lab leak theory](https://www.lesswrong.com/posts/wQLXNjMKXdXXdK8kL/fauci-s-emails-and-the-lab-leak-hypothesis). It's very likely that when the US intelligence community reports on 25. August on their data about the orgins of the COVID-19 they will conclude that it was a lab leak. This then will lead to the discovery that the virology community mislead the public and Fauci will be forced to resign. Once he's resigned a senate committee will have a lot of fun asking him how the funding decision for Baric & Shi was made in 2015 while the gain of function ban was in place and why the safety framework for gain of function research that was instituted after the ban was lifted in 2019 was never applied to the grant even so it was paid till 2019. They will also enjoy subpoenaing Twitter and Facebook for the documents about how those started censoring the lab leak hypothesis. 

These events make it quite likely that we have a huge crisis of faith in the medical establishment. Milton Freedman used to say: “Only a crisis - actual or perceived - produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes the politically inevitable.”

There are likely two months to plan what to do once things blow up and it might be valuable to plan for the scenario. Is this the time where structural change to the medical system can be made? What solutions should be proposed to deal with malicious and incompetent scientists? 

Unfortunately, there's a good chance this will happen while we have a new SARS-CoV-2 variant that would be better fought by giving a third vaccination dose. It might be pretty important to have Novavax around as an alternative to the current vaccines to give people alternatives to the current solutions. 

Lastly, this scenario might be worth trading on. This isn't financial advice, but if you want to trade it might be worthwhile about coming up with your own probability for this event happening and thinking through what it will do to the markets.
# Deep limitations? Examining expert disagreement over deep learning

A recent publication by Carla Zoe Cremer, who's working at the Future of Humanity Institute:

> We conducted 25 expert interviews resulting in the identification of 40 limitations of the deep learning approach and 5 origins of expert disagreement. These origins are open scientific questions that partially explain different interpretations by experts and thereby elucidate central issues in AI research. They are: abstraction, generalisation, explanatory models, emergence of planning and intervention. We explore both optimistic and pessimistic arguments that are related to each of the fve key questions. We explore common beliefs that underpin optimistic and pessimistic argumentation. Our data provide a basis upon which to construct a research agenda that addresses key deep learning limitations.
> 
> 1.  Abstraction: Do current artificial neural networks (ANNs) form abstract representations effectively?
> 2.  Generalisation: Should ANNs’ ability to generalise inspire optimism about deep learning?
> 3.  Explanatory, causal models: Is it necessary, possible and feasible to construct compressed, causal, explanatory models of the environment as described in [Lake et al. (2017)](https://arxiv.org/abs/1604.00289) using deep learning?
> 4.  Emergence of planning: Will sufficiently complex environments be sufficient in enabling deep learning algorithms to develop the capacity for hierarchical, long-term reasoning and planning?
> 5.  Intervention: Will deep learning support and require learning by intervening in a complex, real environment?

Personally, I'm fairly confident that the first three won't be major problems for deep learning. I'm much more uncertain about the fourth and fifth, since they correspond to types of data that seem quite difficult to obtain. (I'm happy to agree with the fourth in principle, but in practice the "sufficient complexity" might be well beyond the sorts of training environments AI researchers currently think about.)
# Sam Altman and Ezra Klein on the AI Revolution

On _The New York Times_'s Ezra Klein Show podcast, Klein interviews OpenAI CEO Sam Altman on the future of AI [(archived with transcript)](https://archive.is/BT2Lr). (Note the nod to _Slate Star Codex_ in the reading recommendations at the end.)

My favorite ("favorite") part of the transcript was this:

> EZRA KLEIN: Do you believe in 30 years we're going to have self-intelligent systems going off and colonizing the universe?  
> SAM ALTMAN: Look, timelines are really hard. I believe that will happen someday. I think it doesn't really matter if it's 10 or 30 or 100 years. The fact that this is going to happen, that we're going to help engineer, or merge with or something, our own descendants that are going to be capable of things that we literally cannot imagine. That somehow seems way more important than the tax rate or most other things.

"Help engineer, or merge with, or something." Indeed! I just wish there were some way to get Altman to more seriously consider that ... [the _specific details_ of the "help engineer [...] or something" _actually matter_](https://www.lesswrong.com/posts/GNnHHmm8EzePmKzPk/value-is-fragile)? Details that Altman himself may be unusually well-positioned to affect?