Harley-ml/lesswrong huggingface dataset:
# How to compute the probability you are flipping a trick coin

Suppose you flip a coin $n$ times and get $n$ heads in a row. What is the probability the next flip will land heads?

Suppose the coin is either a fair coin with one heads or a trick coin with two heads. Let $X$ denote our training data of $n$ heads. We want to find $P(\text{fair}|X)$ and $P(\text{trick}|X)$. Let $P(\text{trick})=\epsilon$. It follows that $P(\text{fair})=1-\epsilon$.

We use Bayes theorem $P(A|B)=\frac{P(B|A)P(A)}{P(B)}$.

$$\begin{eqnarray*}
P(\text{fair}|X)&=&\frac{P(X|\text{fair})P(\text{fair})}{P(X)}\\
&=&\frac{P(X|\text{fair})P(\text{fair})}{P(\text{fair})P(X|\text{fair})+P(\text{trick})P(X|\text{trick})}\\
&=&\frac{2^{-n}(1-\epsilon)}{(1-\epsilon)2^{-n}+\epsilon1}\\
&=&\frac{(1-\epsilon)}{(1-\epsilon)+2^n\epsilon}
\end{eqnarray*}$$

$$\begin{eqnarray*}
P(\text{trick}|X)&=&\frac{P(X|\text{trick})P(\text{trick})}{P(X)}\\
&=&\frac{P(X|\text{trick})P(\text{trick})}{P(\text{trick})P(X|\text{trick})+P(\text{fair})P(X|\text{fair})}\\
&=&\frac{1\epsilon}{\epsilon1+(1-\epsilon)2^{-n}}\\
&=&\frac{\epsilon}{\epsilon+(1-\epsilon)2^{-n}}\\
&=&\frac{2^n\epsilon}{(1-\epsilon)+2^n\epsilon}
\end{eqnarray*}$$

I have flipped perhaps a hundred coins. One was double-headed trick coin. On the one hand, I flip trick coins with anomalously high frequency. On the other hand, double-headed trick coins are more likely than regular coins to get flipped than fair coins. I estimate the flip frequency of double-headed trick coins to be one in ten thousand.

$$\epsilon=\frac{\text{trick coins}}{\text{total coins}}=\frac1{10,000}=10^{-4}$$

What does it look like when we graph our probabilities with $\epsilon=10^{-4}$?

![](https://s3-us-west-2.amazonaws.com/www.lsusr.com/lesswrong/coin-graph.png)

For the first 5 heads you can remain confident you are flipping a regular coin. Around 10 heads the exponential takes off. You quickly become confident you are not flipping a regular coin. At 20 heads in a row you can be confident you are not flipping a regular coin.

# The Inflection Point

The inflection point occurs when the probabilities are equal.

$$\begin{eqnarray*}
P(\text{fair}|X)&=&P(\text{trick}|X)\\
\frac{(1-\epsilon)}{(1-\epsilon)+2^n\epsilon}&=&\frac{2^n\epsilon}{(1-\epsilon)+2^n\epsilon}\\
1-\epsilon&=&2^n\epsilon\\
\frac1\epsilon-1&=&2^n\\
\frac1\epsilon&=&2^n+1\\
\epsilon&=&\frac1{2^n+1}\\
\end{eqnarray*}$$

![](https://s3-us-west-2.amazonaws.com/www.lsusr.com/lesswrong/coin-graph-2.png)

A linear increase in your data has predictive power equal to an exponential increase in the strength of your prior.
# Suppose $1 billion is given to AI Safety. How should it be spent?

What are the current bottlenecks in AI Safety progress, and if they were solved with money, what would be the next bottlenecks? 

Do current researchers need more money? Do we need to catalyze the creation of more researchers? Do we need to alter public opinion? Something else?

If you had $1 billion to spend on AI Safety, what would you do with it all, from start to finish?
# Thinking About Generators

Every so often I think about what we would do in an extended power outage. Generally this comes up when a crisis is in the news, so not a good time to buy things. During the [February Texas Storm](https://en.wikipedia.org/wiki/2021_Texas_power_crisis) I set myself a reminder for three months later to think about generators. So: here I am.

We currently have two sources of standard 120V power in a blackout:

*   Battery: [1 kWh of lead-acid](https://www.jefftk.com/p/battery-manual).
*   Solar: [2 kW of solar](https://www.jefftk.com/p/home-solar-resiliency)

(2 kW is the capacity of the [SPS](https://www.sma-sunny.com/us/how-to-explain-secure-power-supply-to-homeowners/); the panels themselves are rated for a total of 5 kW.)

They do have downsides, though. The battery has a limited capacity, and the solar requires the sun to be shining, the panels to be clear, and the sun to be in the right part of the sky. They complement each other somewhat, using the battery to time-shift, but there are many blackout situations (ex: winter storm) where once the battery is depleted the solar wouldn't give us anything.

A fossil fuel generator is potentially a good fit here. While it doesn't have the long-term properties of solar, fuel is cheap enough that you can have a capacity far larger than you would want to spend on batteries.

For example, a generator can turn a five gallon can of gas into ~15 kWh. That's already much more than my battery, and each additional can of gas is far cheaper than a large battery.

On the other hand, gas is a pain. It turns to varnish in a few months and (especially if it has some ethanol) it pulls in water from the air, so even if you're using fuel stabilizer you want to rotate it. It's not safe or legal to store in your basement, and the temperature fluctuations of storing it outdoors aren't great for it either. Storing gas in your car's gas tank could be good, since it will get automatically rotated, but most cars now have anti-siphon filters.

Propane seems more attractive: it doesn't go bad, you can store it outdoors if you keep it out of direct sun, and while it's less efficient than gasoline a 20lb propane tank still gets you ~10 kWh.

You can get inverter generators that can take both gasoline and propane (ex: [$535](https://smile.amazon.com/Champion-Power-Equipment-100900-Ultralight/dp/B08J9NZ9D7/)), which seems like a good combination? Thoughts?

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100216930865642)*
# How concerned are you about LW reputation management?

I've been considering why I sometimes vote, but don't comment on posts, even though I'd like to. Sometimes, I delete substantial comments that I've already written.

I think the reason is something like reputation management. Ideally, we'd all feel safe to debate and consider ideas here, without fearing looking stupid. After all, that's how we learn.

Yet I want to be able to signal "I've put a lot of work into this post, believe it really offers some value, and am prepared to stand behind it" in some cases. I fear that commenting a lot, or posting imperfect posts, puts me at risk of losing some of that credibility. The more I comment or post (or at least the more above average I do so), the more I risk accumulating a track record of incorrect statements. Also, even *participating* in discussions that aren't my area of writerly focus makes me perceive a credibility risk, analogous to how some scientists carefully avoid speaking about topics beyond their specialty.

A side issue is that public comments can sometimes activate my "called out in public" social sense. For that reason, I have started to use PMs and zoom calls more, both to de-confuse myself with someone else's post, and to have informal discussions off the public record. Those discussions often seem *much* more alive and productive than the ones I have through the comment section here.

While it seems beneficial to the community to have people willing to publicly list their mistakes and risk a free and open dialog, I wonder how much this reputation management concern is felt by others?

Also, if it's not just me, what would be the best way to solve this problem? I can think of a few:

1.  Deciding that this is a feature, not a bug, and continuing to treat reputation as a fragile and valuable thing.
2.  LW developers create an official and convenient anonymizing option, where posters can choose not to have their profile name attached to a comment/post but can reveal it later if they choose.
3.  Actively taking steps to overcome this phobia, perhaps by creating a "ways I've been wrong" page, or making an effort to treat the comment section of posts more as a casual and informal space.
4.  Creating two accounts, one public and one pseudonymous.
5.  Using private messages, zoom calls, and chat for more informal conversation.
6.  Looking into guidance aimed at academics for managing their online presence.
# Houston Rationalists Virtual Meetup May 19

Hey y'all,

Our next meetup is Wednesday May 19th from 5:30pm to 8:30pm central time!

We'll meet virtually via Jitsi, and I'll post the meetup room link in the comments here and on our other platforms 5-10 minutes prior to 5:30pm.

I have two suggested readings for this meetup:

(1) [Why Is It Hard To Acknowledge Preferences](https://astralcodexten.substack.com/p/why-is-it-hard-to-acknowledge-preferences), by Scott Alexander

(2) [Ars Longa, Vita Brevis](https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/), also by Scott Alexander

If you have any readings and/or topics suggestions, please comment here or message me.

It's been a little while, I hope all of y'all have been well! I'm looking forward to catching up, hanging out, and having some good conversations :) 

Cheers,

Willa
# SpaceX should provide IPFS proxies with Starlink

If everything goes according to plan for SpaceX, SpaceX will start permanent bases on moon and mars in the next decade. One challenge for moon and mars bases will be that a good chunk of the software we have on earth stops working when there's no internet connection.

Package managers, whether they are Ubuntu's Snap or programming tools like npm download their packages from the internet and fail when there are minutes of delay for traffic between earth and mars. The same goes for App stores and operating system updates.

If those services would use IPFS instead of TCP/IP they would still work on mars if there's an IPFS bridge between earth and mask that transfers IPFS packages that are requested on mars but currently unavailable from earth to mars.

While it's possible to write custom software for Mars that works without TCP/IP it would benefit a Mars base a lot when as much of normal software as possible that can use IPFS to transfer data uses IPFS.

One reason why IPFS currently isn't used more is that every node that normally requests data from the IPFS network has to broadcast it's IP address openly together with the data it wants to access. In that configuration an attacker can gather knowledge about which software is run within a company network which is undesirable.

From the privacy perspective it would be advantageous if a user only has to reveal to his ISP which package they want to access via IPFS. If the user only would have to reveal to his ISP the content they want to access they would have better privacy then they currently have with TCP/IP.

This setup would be advantageous for the ISP as well because content that's downloaded by many users doesn't have to be requested multiple times from its original source which saves both the ISP and the original data source bandwidth costs.

Legacy ISPs profit a bit from this setup by being able to bill Netflix, Youtube and Amazon prime for hosting servers on their premises to do a similar job, but this predatory setup is bad for the open internet. It means that only companies that can afford servers at every ISP can get comparable performance for their video service while if this job would be done by the ISP, the ISP would pay for the servers on their network.

IPFS to TCP/IP being superior for a lot of content is a reason for SpaceX to provide IPFS proxies to their Starlink customers. Providing IPFS proxies will be an additional selling point to distinguish SpaceX from legacy ISPs. Overall the ability to incentivize software companies to save bandwidth by using IPFS for transferring data in a way that also works on Mars is more important to SpaceX's overall goals.
# This Sunday, 12PM PT: Scott Garrabrant on "Finite Factored Sets"

This Sunday at 12pm (PT), we're running another LW weekly online meetup.  
MIRI's Scott Garrabrant will be giving a talk introducing Finite Factored Sets, a new proposed successor to Judea-Pearl-style causal reasoning. Scott writes that Finite Factored Sets is "a framework which I find roughly as technically interesting as logical induction", with wide applications to embedded agency, inference, abstraction, causality, and temporal reasoning.

In the days after the talk, Scott will be posting a sequence on Finite Factored Sets to LessWrong. For now, if you want to read more, Scott has [written a short post on background motivations](https://www.lesswrong.com/posts/gEKHX8WKrXGM4roRC/saving-time).

The talk will be given in Zoom. Afterwards there'll be Q&A and some open-ended discussion.

[https://us02web.zoom.us/j/7151633248?pwd=a2ZkQlpwNU9IbWF5c3U4ejlvdFRvUT09](https://us02web.zoom.us/j/7151633248?pwd=a2ZkQlpwNU9IbWF5c3U4ejlvdFRvUT09)
# How refined is your art of note-taking?

Over the last few months of studying "effective scholarship," I've concluded that a key skill is note-taking. In the near future, I plan to write fairly extensively on the subject. For now, I'm curious to hear how the LessWrong community takes notes. Some particular questions:  
  
1) How consistently do you take notes when you're reading up on a new skill or subject?

2) Do you regularly refer back to old notes?

3) Do you approach note-taking differently for different subjects or purposes?

4) Have you adopted a specific note-taking method and used it consistently for more than a few months?

5) What role does note-taking play for you? Is it a way to focus your attention? To make extracts from the text for easier reference later? To comprehend the material better through the act of making notes?
# Sabien on "work-life" balance

From Duncan Sabien:

> This morning, a friend of mine referred to his "work-life balance" and then grimaced at himself, and noted that he doesn't really like that term.
> 
> (As far as I can tell, he doesn't like it because his work is important to him, and is part of being alive, and his non-work life isn't some *fundamentally* different kind of thing.)
> 
> This led me to the mental distinction between one's *directly* valuable life experiences, and one's indirectly/instrumentally valuable ones.
> 
> Like, there are many fewer layers involved when you, say, snuggle up to someone you love, or pop a delicious food in your mouth, or bounce on a trampoline.  There's a very short, direct path between the action and the reward; it's just straightforwardly close to your values and the things which bring you joy and fulfillment.
> 
> Whereas if you're good at your work and you think that your job is important, there's an intervening layer or three—I'm doing X because it unblocks Y, and that will lead to Z, and Z is good for the world in ways I care about, and also it earns me $ and I can spend $ on stuff...
> 
> I think it's less about "work-life balance" and more about the ratio of direct vs. indirect value.  "How many of the things that I'm doing pay off directly, versus how many of them are knocking over dominos that eventually *lead* to the payoffs I'm seeking?"
> 
> (And how close is that ratio to one which I will actually find sustainable and enjoyable and healthy.)
> 
> This, to me, is a distinction that cuts closer to the true joints of reality than the arbitrary categories of "work" and "everything else."  As a single easy example, this also applies to the balance, in one's relationships, between straightforwardly valuable interactions with people you enjoy, and meta interactions/maintenance/laying the groundwork for the future.
# The Reebok effect


> The principle here is that if you’re trying to make the result sound as impressive as possible, an unintended consequence is that you’re revealing the upper limit.

Here is the full quote by Phil Price (from Andrew Gelman's blog):

> Many years ago I saw an ad for a running shoe (maybe it was Reebok?) that said something like “At the New York Marathon, three of the five fastest runners were wearing our shoes.” I’m sure I’m not the first or last person to have realized that there’s more information there than it seems at first. For one thing, you can be sure that one of those three runners finished fifth: otherwise the ad would have said “three of the four fastest.” Also, it seems almost certain that the two fastest runners were not wearing the shoes, and indeed it probably wasn’t 1-3 or 2-3 either: “The two fastest” and “two of the three fastest” both seem better than “three of the top five.” The principle here is that if you’re trying to make the result sound as impressive as possible, an unintended consequence is that you’re revealing the upper limit.
# Books on Infant and Early Childhood Development?

Recently, a friend of mine recommended "BabySparks", an app providing a set of development milestones and targeted growth activities for babies and toddlers.

This *feels* like the sort of thing I should be able to pull from a few books, rather than a subscription-based app. I'm also not particularly interested in Yet Another App, nor am I particularly keen on handing my infant's personal data to Yet Another Social Media company.

I'm especially interested in understanding the varying stages of development, and in activities to perform with aforementioned infant during their first year.

So: Any non-app resources that LessWrongers would care to recommend?
# Abstraction Talk

I gave a talk a couple weeks on abstraction. It moves very fast, but it covers more material in one place than any individual post I've written, and includes a few things which haven't showed up in posts yet (especially near the end).

[Slides are here](https://docs.google.com/presentation/d/1sHzrxMjSaXwrC7pgGklE5fLPrIgF_0Tsz4PiB8qK0VA/). The slides are hard to see sometimes in the video; I recommend keeping them open to the side of the video.
# List of good AI safety project ideas?

Can we compile a list of good project ideas related to AI safety that people can work on? There are occasions at work when I have the opportunity to propose interesting project ideas for potential funding, and it would be really useful if there was somewhere I could look for projects that people here would really like someone to work on, even if they themselves don't have the time or resources to do so. I also keep meeting people who are searching for useful alignment-related projects they can work on for school, work, or as personal projects, and I think a list of project ideas might be helpful for them as well.

I'm particularly interested in project ideas that are currently not being worked on (to your knowledge) but where it would be great if someone would take up that project. Or alternatively, project ideas that are currently being worked on but where there are variations on those ideas that nobody has yet attempted but someone should.

Occasionally someone will post an idea or set of ideas on the Alignment Forum, for example Ajeya Cotra's ["sandwiching" idea](https://www.alignmentforum.org/posts/PZtsoaoSLpKjjbMqM/the-case-for-aligning-narrowly-superhuman-models#Potential_near_future_projects___sandwiching_) or the recent [list of ideas from Stuart Armstrong and Owain Evans](https://www.alignmentforum.org/posts/f69LK7CndhSNA7oPn/ai-safety-research-project-ideas). I also sometimes come across ideas mentioned towards the end of a paper or buried somewhere in a research agenda. But I think having a larger list somewhere could be really useful.

(Note: I am *not* looking for lists of open problems, challenges, or very general research directions. I'm looking for suggestions that at least point towards a concrete project idea, and where an individual or small team might be able to produce useful results given current technology and with sufficient time and resources.)

Please post ideas or links / references to published ideas in the comments, if you know of any. Ideas mentioned as part of a larger post or paper would count, but please point to the section where the idea is mentioned.

If I get enough links or references maybe I'll try to compile a list that others can use.
# What's your visual experience when reading technical material?

In my never-ending quest to understand the best way to read a textbook, I'm back to where I started - exploring the importance of visual imagination. When you read technical material, such as science or math, or posts on rationality, to what degree do you visualize the text? Here are some possibilities, but no need to confine yourself.

1.  Is your visual imagination "permanently off," so that you experience the concepts in a symbolic or diagrammatic manner?
2.  Can you activate your visual imagination when you choose, but you're not in the habit?
3.  Is your visual imagination for STEM material a consistent aspect of how you read?

Beyond this, do you feel that you'd get a lot out of being more able to visualize text, or having a more active visual imagination? Have you noticed any change in your ability to visualize over time?
# Is anyone else frustrated with 'un-informative' post titles?

The latest post that finally impelled me to ask this question:

- [A problem and three ideas - LessWrong](https://www.lesswrong.com/posts/QqwZ7cwEA2cxFEAun/a-problem-and-three-ideas)

This isn't specific to that post and, having noticed that it is a cross-post, I can appreciate that this is (at least somewhat) tricky to improve, but a lot of the titles of posts are *very* un-informative. Even qualifying that the above post is specific to AI or AI alignment still doesn't seem like a significant improvement, or an improvement that's 'good enough'.

I notice that I often completely ignore posts with sufficiently vague titles. Upon (very shallow) introspection, I feel like the titles are pretty *literally* clickbait, i.e. 'click me to discover what this is about'. That seems like behavior that warrants 'punishing' (to some degree anyways).

One potential solution (mitigation) would be to include (one of) the tags in the post title, or maybe at least in the RSS feed for posts, à la Stack Overflow, e.g.:

- [google chrome - navigator.clipboard is undefined - Stack Overflow](https://stackoverflow.com/questions/51805395/navigator-clipboard-is-undefined)

In the above, the 'actual' title of the question is just "navigator.clipboard is undefined" and it seems like the 'first' tag is automatically included in the page title.
# Wrist Update

It's been about a year now [since my wrists got really bad](https://www.jefftk.com/p/wrist-issues). Unfortunately, it's no longer limited to just my wrists: I'm having similar issues with my hands. Last time I wrote about how my doctor thought that this was likely due to ganglion cysts, but since then we've done more imaging and that looks like it probably isn't the problem. After talking to a rheumatologist, it looks like \[1\] it's probably a kind of inflammatory arthritis, which is a chronic autoimmune disease.

So far, the only things that have helped have been minimizing use and icing ([flexible wrist packs](https://smile.amazon.com/Arctic-Flex-Wrist-Ice-Pack/dp/B07VGY2G4Y/) are nice!). I've tried two nonsteroidal anti-inflammatory drugs (NSAIDs), Ibuprofen and Naproxen Sodium, without effect. I'm about to start on the immunosuppressant methotrexate, which I have mixed feelings about, but positive on balance.

I'm still doing lots of dictation, but I haven't needed to continue with voice control. When using the computer/phone, as long as I dictate words, I've been able to still use my hands for navigation. The biggest problem is when I have substantial tasks that aren't just reading and writing, such as spreadsheet work or [green-field coding](https://www.jefftk.com/p/secular-solstice-2020). Luckily this is only a very small portion of my regular work, and I've been doing fine there, though I'm not sure what dictation will look like once I'm back in the office. I'm back to playing music, which I'm really happy about, but no [jammer](https://www.jefftk.com/p/rearranging-axis-49-keys) or fast melody.

  
\[1\] My understanding is that the diagnosis is primarily based on the appearance of my hand and wrist bones under x-ray. My joint spacing is narrow and my bone density has decreased, especially in my wrist. Comparing x-rays from 2020 and 2018, it is most apparent in 2020 but 2018 also showed the issues. They ran a lot of blood tests, which were all normal, including testing negative for rheumatoid factor, cyclic citrullinated peptide antibodies, sedimentation rate, and c-reactive protein. No visible swelling, but substantial stiffness in the morning—I initially didn't report this is a symptom, because I'd assumed this is just the way hands are in the morning. Since I also have some persistent dry scaly patches of skin, they thought most likely psoriatic arthritis. Possibly something I've had since a kid, since some kinds of juvenile arthritis can cause tendonitis-like symptoms.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100218415620182)*
# [Event] Weekly Alignment Research Coffee Time

Just like every Monday now, researchers in AI Alignment are invited for a coffee time, to talk about their research and what they're into.

Here is the [link](https://app.gather.town/app/aPVfK3G76UukgiHx/lesswrong-campus). 

And here is the [everytimezone time](https://everytimezone.com/s/a0d91f01) (what matters is the hour, I don't change the day anymore).

Note that the link to the walled garden now only works for AF members. Anyone who wants to come but isn't an AF member needs to go by me. I'll broadly apply the following criteria for admission:

*   If working in a AI Alignment lab or funded for independent research, automatic admission
*   If recommended by AF member, automatic admission
*   Otherwise, to my discretion

I prefer to not allow people who might have been interesting but who I'm not sure will not derail the conversation, because this is supposed to be the place where AI Alignment researchers can talk about their current research without having to explain everything.

See you then!
# What is the most effective way to donate to AGI XRisk mitigation?

There are now many organizations in field of Existential Risk from Artificial General Intelligence. I wonder which can make the most effective use of small donations.

My priority is on mathematical or engineering research aimed at XRisk from superhuman AGI.

My donations go to MIRI, and for now it looks best to me, but I will appreciate thoughtful assessment.

*   [Machine Intelligence Research Institute](https://intelligence.org/) pioneered AGI XRisk mitigation (alongside FHI, below) and does foundational research. Their approach aims to avoid a  rush to implementation of an AGI which includes unknown  failure modes.
*   [Alignment Research Center](http://alignmentresearchcenter.org/): Paul Christiano's new organization. He has done impressive research and has worked with both academia and MIRI.
*   [Center for Human-Compatible Artificial Intelligence](https://humancompatible.ai/) at Berkeley: If you're looking to sponsor academia rather than an independent organization, this one does research that combines mainstream AI methods with serious consideration of XRisk.
*   [Future of Humanity Institute](https://www.fhi.ox.ac.uk/) at Oxford is a powerhouse in multiple relevant areas, including AGI XRisk Research.
*   [The Centre for the Study of Existential Risk](https://www.cser.ac.uk/) at Cambridge. Looks promising. I haven't seen much on AGI XRisk from them.
*   [Leverhulme Center for the Future of Intelligence](http://lcfi.ac.uk/), also at Cambridge and linked to CSER
*   Smaller organizations with scope that goes beyond AGI XRisk. I don't know much about them otherwise.
    *   [Global Catastrophic Risk Institute](http://www.gcrinstitute.org/)
    *   [Center on Long-Term Risk](https://longtermrisk.org/)
*   Donating money to a grant-disbursing organization makes sense if you believe that they have better ability to determine effectiveness than you. Alternatively, you might be guided by their decisions as you make your own donations.
    *   [Open Philanthropy Project](https://www.openphilanthropy.org/)
    *   [Long-Term Future Fund](https://app.effectivealtruism.org/funds/far-future)
    *   [Survival and Flourishing Fund](http://survivalandflourishing.fund/)/[Survival and Flourishing. org](http://survivalandflourishing.org/)
    *   Solenum Foundation ([see here](https://www.edge.org/conversation/jaan_tallinn-existential-risk)). Recently, [Jaan Tallinn discussed  a new initiative](https://www.youtube.com/watch?v=qQCB0geKRUE)  for an assessment pipeline that will aggregate expert opinion on the most effective  organizations.
    *   [Berkeley Existential Risk Initiative](https://existence.org/)
*   [Future of Life Institute](https://futureoflife.org/): It's not clear if they still actively contribute to AI XRisk research, but did disburse grants a few years ago.

Are there others?
# What is the Risk of Long Covid after Vaccination?

I received my 2nd shot of the Pfizer vaccine in the middle of April, but I still have some anxieties about some higher risk activities like indoor dining. Mostly, I'm worried about the possibility of long Covid.

[This article](https://www.nytimes.com/2021/05/25/health/cdc-coronavirus-infections-vaccine.html), although it presents only one anecdote, does show that long Covid is at least possible after being fully vaccinated. As immunity evasive variants grow as a share of new infections, we might even expect this kind of occurrence to become more common as time goes on. A [redditor claims](https://www.reddit.com/r/COVID19positive/comments/noiunr/multiple_friends_have_covid_all_are_fully/) that four vaccinated people he knows still ended up symptomatic. He could be lying, but maybe these variants are getting good at their job. 

I wish I could present better sources for my concerns but I'm afraid scrappy anecdotes and self reported data is really all we have to work with on this front right now. Or so I thought, until I ran into [this](https://www.medrxiv.org/content/10.1101/2021.05.24.21257738v1.full-text#ref-20) preprint from the folks from JoinZOE and the Covid Symptom Study app. If you take a look at tables 8 and 9, you can see that the authors found no evidence for reduced incidence of symptoms lasting >28 days after vaccination in the younger age group (given that infection did occur, of course). The effect was significant in the older age group. I should add that this was self reported data and not a trial.

What are we to make of this? Do vaccines not protect us against long Covid in the way that we hoped they might? Of course, if we don't get infected, this isn't an issue. But if we are infected, then are we really at equal risk of developing long Covid as someone who was infected without vaccination? What other sources out there discuss this?
# Russian x-risks newsletter spring 21

Despite vaccination with Russian Sputnik, I’ve got covid. It was relatively mild, but with elevated levels of D-dimer, which means higher risk of thrombosis. I am fine now.

Notwithstanding availability of vaccines, only 10 per cent of Russian population has vaccinated and the reasons is strong disbelieve in what government say. Some people wait more classical Chumakov vaccine which is built around inactivated viruses of covid, not mRNA or vectors, but its production is limited. 

Lockdowns measures are also almost absent in Russia and life looks normal. In May, the covid incidence started to grow again in Moscow, may be fuelled by new strains. 

Meanwhile, Russia accumulated largest in decades army on the west border with Ukraine in spring, and even delivery agricultural machinery was delayed because of urgent demand for railroad tracks from military. The was speculation about possible annexing of the part of Ukraine, but nothing has happened. Ukraine now got access to very effective Turkish drones and it may use them to reclaim lost land in Donbas, so Russian deployment may be a countermeasure.

Reforestation of Siberia may be not a solution for global warming, as trees have lower albedo than grass. To support grasslands in Siberia we need… mammoth, according to the director of [Pleistocene park](https://elidourado.com/blog/dawn-of-geoengineering/?fbclid=IwAR05ya-PeGi2cuRSeLw8hyyCjpezrg8mfZtly95php3FRFR_cGPZYTVJ2DM).

Russian scientists [expect]( https://lenta.ru/news/2021/03/06/gripp/) that the next pandemic will be from bird flu H5N8, which is slowly adapting to humans in last 10 years.

Also, the systems of dams on major Russian rivers is old and were built in Soviet time. A collapse of any such system would be very large local catastrophe; meanwhile strong floods are [expected](https://iz.ru/1171739/2021-05-31/monitoring-situatcii-s-razlivom-eniseia-prodolzhaetsia-v-krasnoiarske) in Enisey river in June  Such catastrophe may shift public perception of hydroenergy in the world which is currently regarded as safe and renewable source.

Russian “success” in hypersonic missiles [prompted](https://www.popularmechanics.com/military/weapons/a35217288/army-hypersonic-missile-darpa-operational-fires/?utm_medium=social-media&utm_source=facebook&utm_campaign=socialflowFBPOP&fbclid=IwAR1ShjjbwVWGSpcyuzIDA-zkWh6CVQhOb4kpaC21ixR7chFLtHoL6snaBb4) Pentagon to develop its own even quicker missiles; such missiles give less time for a counterstrike decision and increase the chances of accidental nuclear war. 

The most plausible explanation of the recent UFO reports near US fleet is Russian or Chinese drones which pretend to look like alien spacecraft and spoof radars with fake images about speed and positions, thus prompting them to turn on war mode and provide valuable information, according to [Drive]( https://www.thedrive.com/the-war-zone/40054/adversary-drones-are-spying-on-the-u-s-and-the-pentagon-acts-like-theyre-ufos ). However, Russian drones’ development are lagging even from Turkish, so “UFOs” are less probable to be Russian drones than Chinese.
# What question would you like to collaborate on?

In [Why don't long running conversations happen on LessWrong?](https://www.lesswrong.com/posts/Jr5RrJaNxTdeBubni/why-don-t-long-running-conversations-happen-on-lesswrong) adamzerner writes:

>  Here is how things currently work:
> 
> *   Someone writes a post.
> *   It lingers around the front page for a few days. During this time, conversations emerge in the comments section.
> *   After a few days, the post no longer persists on the front page and conversations largely fizzle out.

I'd like to try and have a longer [polymath project](https://polymathprojects.org/general-polymath-rules/) style collaboration focused on answering a question together. Instead of each person working to give their individual answers to the question, we'd come up with an answer together through an extended discussion.  
  
When/if we've reached some sort of milestone by either answering the question, or making some interesting progress towards answering the question, we show the results of our collaboration to lesswrong in the form of a new post.

  
As a starting point, here are my proposed rules (up for negotiation):

1.  **Each top level answer to this post should be its own question.** Please state the question at the top of your comment. Then you can give background information or try to convince us that your question would be a good one to pick for this exercise. The question can be one already asked on lesswrong.
2.  Each person can submit a maximum of 3 questions.
3.  We have 5 days to submit questions and receive feedback or requests for clarification.
4.  Please only submit questions that you yourself are willing to work on.
5.  Once the 4 days are up, all the top level (non-meta) questions will be compiled into a new post to be voted on. (Hopefully that will help with the problem of the earliest submitted questions getting more upvotes).
6.  We work on whichever question gets the most votes.
7.  We can sort out the logistics of the collaboration (where should discussion take place? should we use a wiki or google doc? how long should we commit to working?etc) once we've sorted out what question we're working on and who is willing to contribute.
8.  If you'd like to comment on this idea itself without submitting a question for the group's consideration, use a "comment" instead of an "answer."

I have no idea how many people would be interested in doing this. But I'd guess it only takes 2 people for this thing to work. And at least one person is interested (me). So worst case scenario the person with the top voted question will have at least one other person to work with.  
  
I'm willing to commit to working for at least a week in my free time (which I have a lot of at this point). Hopefully at least myself and the person who posted the question will be willing to put in a decent amount of effort. But I'd like to leave the collaboration open to very casual contributions.
# Attributions, Karma and better discoverability for wiki/tag features

LessWrong has an associated wiki, tightly integrated with the tagging system. Today, we have just shipped a set of new features to make the LessWrong wiki better:

**1\. Voting and karma for wiki edits**

You can now vote on wiki-edits on the tag-history page, in the Recent Discussion section, and on the All-Posts page: 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/83f28b6bbccce754dde118b5553a636fa2dc23fbcafb558e.png)

This now properly provides karma incentives for improving the wiki. 

**2\. Wiki/tag edits appear on the** [**All Posts page**](/allPosts)**, as do comments on wiki/tag discussion pages**

The All-Posts page now also shows wiki edits on the daily page! This now allows you to much more easily keep track of the wiki-editing activity on the site: 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/39077ca1afe00918d14b6eb42447b498375367205058dc74.png)

You can click on them to expand and see the full diff, as well as vote on any edits.

**3\. Wiki/tag pages have a table of contents, like post pages.**

Wiki and tag pages now have a ToC like post pages. This should make it much easier to navigate long wiki pages, like the Rationality tag: 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/43a5aee27598e6c4c50cf3fae6267735359b877cecdb996f.png)

**4\. Attributions and contributors on tag pages**

I am particularly excited about this one. Just below the ToC on tag pages you can see a list of all contributors to a given tag/wiki page: 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/9818c821e82d57bc7fa52a64932e3c7049391eb688fb7200.png)

The number on the left is the total karma these authors have received for their contributions to this tag page, plus their small-vote strength (this also determines the order of the list of contributors).

But more importantly, when you hover over the author, you get to see which parts of the current tag page where written by them!

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/2cfeefe25feb8c3d07a796c0dad1d2895ae6967e11db0421.png)

The goal is to make it more engaging to produce timeless content, make contributing to the wiki more motivating, and to make it easier to decide which wiki pages are worth reading.

(Looking for some of that wiki-edit karma? Check out the [FAQ](https://www.lesswrong.com/posts/E6CF8JCQAWqqhg7ZA/tagging-faq) and the [Wiki-Tag Dashboard](https://www.lesswrong.com/tags/dashboard)).
# Beijing Academy of Artificial Intelligence announces 1,75 trillion parameters model, Wu Dao 2.0

> BAAI researchers demonstrated Wu Dao's abilities to perform natural language processing, text generation, image recognition, and image generation tasks during the lab's annual conference on Tuesday. The model can not only write essays, poems and couplets in traditional Chinese, it can both generate alt text based off of a static image and generate nearly photorealistic images based on natural language descriptions. Wu Dao also showed off its ability to power virtual idols (with a little help from Microsoft-spinoff XiaoIce) and predict the 3D structures of proteins like [AlphaFold](https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-discovery).

How big of a deal is that? Seems huge. Bigger than switch transformers and 10x bigger than GPT-3.
# Assessing Interest in Group Trip to Secure Panamanian Residency [Imminent Rules Change]

[I’ve previously outlined](https://www.lesswrong.com/posts/jHnFBHrwiNb5xvLBM/second-citizenships-residencies-and-or-temporary-relocation) many of the best options for securing a second residency, and [some reasons you may want to do so](https://www.lesswrong.com/posts/jHnFBHrwiNb5xvLBM/second-citizenships-residencies-and-or-temporary-relocation#Why_Second_Citizenships_May_Be_Useful).

[Panama had one of the most unique, accessible, and attractive programs for doing so](https://www.lesswrong.com/posts/jHnFBHrwiNb5xvLBM/second-citizenships-residencies-and-or-temporary-relocation#Panama_). Unfortunately, [this program’s rules will be changing on August 20th](https://www.mondaq.com/work-visas/1072126/changes-to-the-friendly-nations-visa), becoming much more restrictive, to the extent that I likely would not have mentioned the program under the new rules.

I’m interested in putting together a trip for EAs and Rationalists interested in securing Panamanian permanent residency, and in 5 years potentially citizenship, under the existing rules. I believe that there are some significant benefits to securing second citizenship, and to doing so under the existing Panamanian rules, together.[^1]

Currently, I’d like to assess interest for this trip before taking further action. I previously estimated the citizenship process to cost $2k-4.2k all-in[^2] (not including flights or lodging). Other information, such as time investment needed and the feasibility of success on this tight timeline, I will work to further determine if there’s sufficient interest.[^3]

If I put together a trip, I expect to engage a lawyer to lead us through the process, interface with that lawyer to provide clear step by step requirements to each person, coordinate so that there are some cost savings, and help secure flights and lodging.

If you have any interest in this potential trip to secure Panamanian residency, [please fill out this brief linked Google Form](https://forms.gle/1j3zdBTy7x8iJNC8A).

## Footnotes

[^1]:
     In particular, the cost per person will be lower if pursuing citizenship with someone else.

[^2]:
     I will not be highly surprised if costs have risen due to an influx of interest prior to the rules changing.

[^3]:
     I feel 90% confident that, on average, time investment per person will be less than 10 hours (at home) of pre-work, 8 hours in Panama actively doing things, and 10 days in Panama total (including e.g. waiting time during which work could be done remotely). There is significant possibility the time requirements are far below these (e.g. 3 hours of pre-work, 2 hours actively doing things in Panama, 3 days in-country).
# Good Macro, Bad Micro

So this is a concept that I noticed while reviewing a preventative maintenance manual for the product I support at work. To give a little context, this is a manual that goes out to 500+ power plants around the world, to be used by operations and engineering teams to ensure proper functionality of their installed system and minimize downtime. This is what I would call a “good macro” skill. Being able to think about complex systems and how to create a document that applies to all of them despite their variations. Now, for the irony in all of this, the “bad micro” skill. I don’t have a maintenance schedule for my car. I have a 1988 Suzuki Samurai that has been sitting in my mom’s garage for the past 5 years and I haven’t touched it. My daily driver is a newer car for which I don’t even have a maintenance record. I couldn’t tell you what the oil change interval dates are, and I haven’t rotated the tires ever.

My point here is, it’s easy to be good at macro stuff, writing manuals for general systems. When it comes to micro stuff, my own possessions, I just don’t do it. Incentive could be a factor; I write a manual at work that gets me recognition and praise that in the long term makes it easier for me to advance my career (and pay grade). Maintaining my car, on the other hand, has no obvious benefit other than avoiding issues in the future. There is no praise or recognition for keeping record and following oil change intervals, tire pressure checks, rotating tires, etc. You could argue that neglected maintenance on my car could result in more expensive repairs, decreasing value of the car or in the most extreme cases death (tire blowout resulting in a severe crash). Neglecting the maintenance manual (macro) on the other hand would result in possibly a worse performance review, less recognition and in the worse case getting fired. What’s worse, dying or getting fired? Unfortunately nowadays some people act like losing their job is worse than losing their life. You have an OSHA card but you don’t wear a seatbelt. You speed on your way to work at the crash safety institute (you don’t want to lose your job right!)

I’m not trying to ostracize the bad micro, good macro people. I don’t want everyone to become a safety freak, or live life cautiously at all times. This is too new to me to have any practical advice on how to approach this, I just wanted to externalize it and bring it to discussion.
# SIA is basically just Bayesian updating on existence

I love the [sleeping Beauty problem](https://en.wikipedia.org/wiki/Sleeping_Beauty_problem), but I think it leads people astray, by making them think that anthropic reasoning is about complicated situations with multiple duplicates. That leads to people saying erroneous things, like "SIA implies there are a lot of observers".

But anthropic reasoning isn't that complicated; [SIA](https://www.lesswrong.com/tag/self-indication-assumption), especially, is mostly just Bayesian updating.

Specifically, if there are **no exact duplicates of yourself** in your universe, SIA is just Bayesian updating on the fact that you exist; this is the same update that an outside observer would make, if informed of your existence. So, if theory $T_i$ has prior probability $p_i$ and gives you a $q_i$ probability of existing, then SIA updates $T_i$'s probability to $q_i p_i$ (and then renormalises everything):

$$P_{\textrm{SIA}}(T_i) = P(T_i | \textrm{existence}) = \frac{P(T_i)\ P(\textrm{existence} | T_i)}{\sum_j P(T_j)\ P(\textrm{existence} | T_j)}.$$

This result is easy to see - since SIA is [independent of reference class](https://en.wikipedia.org/wiki/Anthropic_Bias_(book)#Self-indication_assumption), just restrict the reference class to exact copies of you. If there is only on one such copy in the universe, then the update rule follows.

Even if there are multiple exact copies of you, you can still mostly see SIA as Bayesian updating over your *future* observations. See this footnote[^FNC] for more details.

[^FNC]: Suppose that there are $N$ exact copies of you, and that they are going to make $n$ independently random observations. Then as soon as $n$ is much bigger than $\log_2(N)$, you can expect that each copy will make a different observation; so, ultimately, you expect there to be only one exact future copy of you.

    So if you Bayesianly update for each possible future copy (weighted by the probability of that future observation), you will get SIA. This is the trick that [full non-indexical conditioning](https://arxiv.org/abs/math/0608592) uses.

    This can be seen as a partial solution to the [Boltzmann brain problem](https://en.wikipedia.org/wiki/Boltzmann_brain): Boltzmann brains won't diverge, because they won't have future experiences. Personally, I prefer to address the issue by mixing in a bit of decision theory; my decisions are only relevant if I'm not a Boltzmann brain, so I'll start with "I exist and am not a Boltzmann brain" as an initial assumption.

# *Indirect* effect on population size

So, what does this mean for the number of observers in the universe? Well, SIA can have an indirect effect on population size. If, for instance, theory $T_0$ posits that life is likely to happen, then our existence is more likely, so $T_0$ gets a relative boost by SIA compared with most other theories.

So, SIA's boosting of other observers' existence is only an indirect effect of it boosting our existence. The more independent our existence is of them, or the more independent we suspect it might be, the less impact SIA has on them.