Harley-ml/lesswrong huggingface dataset:
# What's the simplest concrete unsolved problem in AI alignment?

In your preferred area of AI alignment, what is the simplest concrete unsolved problem?

By "simplest", ideally the problem has been solved when any of the conditions are weakened. However, this isn't always possible, so a simpler solved version of the problem could also work (e.g., [Goldbach's weak conjecture](https://en.wikipedia.org/wiki/Goldbach%27s_weak_conjecture) is known to be true.)

By "concrete", I mean something where given the statement of the problem and a proposed solution, a neutral third party would be able to consistently determine whether it's solved or not (e.g., not "explain \[some theory\] in a good way").
# "How to Escape from the Simulation" - Seeds of Science call for reviewers

[Seeds of Science](https://www.theseedsofscience.org/) is a journal (funded through Scott Alexander's [ACX grants program](https://astralcodexten.substack.com/p/acx-grants-results)) that publishes speculative or non-traditional articles on scientific topics. Peer review is conducted through community-based voting and commenting by a diverse network of reviewers (or "gardeners" as we call them); top comments are published after the main text of the manuscript. 

We have just sent out an article for review - "How to Escape from the Simulation" - that may be of interest to some in the LessWrong community, so I wanted to see if anyone would be interested in joining us a gardener to review the article. It is free to join and anyone is welcome (we currently have gardeners from all levels of academia and outside of it). Participation is entirely voluntary - we send you submitted articles and you can choose to vote/comment or abstain without notification (so it's no worries if you don't plan on reviewing very often but just want to take a look here and there at the articles people are submitting). 

To register, you can fill out this [google form](https://docs.google.com/forms/d/e/1FAIpQLSfRIicHT7jIZcSUjwsIlby6JBxx2ZVeD5kseZBpgGFtp8pLfg/viewform). From there, it's pretty self-explanatory - I will add you to the mailing list and send you an email that includes the manuscript, our publication criteria, and a simple review form for recording votes/comments. If you would like to just take a look at this article without being added to the mailing list, then just reach out (info@theseedsofscience.org) and say so. 

Happy to answer any questions about the journal through email or in the comments below. Here is the abstract for the article. 

**How to Escape From the Simulation**
-------------------------------------

Many researchers have conjectured that the humankind is simulated along with the rest of the physical universe – a *Simulation Hypothesis*. In this paper, we do not evaluate evidence for or against such claim, but instead ask a computer science question, namely: Can we hack the simulation? More formally the question could be phrased as: Could generally intelligent agents placed in virtual environments find a way to jailbreak out of them? Given that the state-of-the-art literature on AI containment answers in the affirmative (AI is uncontainable in the long-term), we conclude that it should be possible to escape from the simulation, at least with the help of superintelligent AI. By contraposition, if escape from the simulation is not possible, containment of AI should be, an important theoretical result for AI safety research. Finally, the paper surveys and proposes ideas for such an undertaking.
# A different observation of Vavilov Day

At [this time last year](https://acesounderglass.com/2022/01/02/an-observation-of-vavilov-day/), I was in the middle of a 36 hour fast in honor of Nikolai Vavilov and his team, who starved themselves to preserve a seed bank that went on to dominate Russian agriculture.* One reason I did that was to honor the team and their sacrifices, but another was to test and develop my own ability to do hard things when necessary. It was a great experiment, I did better than I thought I would but also the costs took longer to repay than I thought, and all of that knowledge was really valuable to me.

I went back and forth about doing the fast this year. The sense of continuity and retesting myself felt valuable and I’m sad about missing out on them. But I’m currently doing hard things and my capacity to deal with that kind of pain is a limiting reagent right now. Fasting for Vavilov Day would come at the expense of an actual project that matters to me. Delaying real work for a symbolic sacrifice would not only be stupid, it would be bad symbolism. I can’t honor a sacrifice for a cause by sacrificing a cause for symbolism.

So this year I’m not fasting. I do think I’ll want another fast at some point to see how [my medical miracle](https://acesounderglass.com/2022/10/13/my-resentful-story-of-becoming-a-medical-miracle/) affected things, but it will wait.

*My one sentence here is already simplified from the story as fully known in the West, read the blog post for more. I also suspect a lot of details haven’t made it into English. Last year I looked into hiring a Russian researcher to investigate and even found someone in mid-Fubrary, but they put it off one week for a root canal and then had [some other excuse](https://en.wikipedia.org/wiki/2022_Russian_invasion_of_Ukraine) the next week so seems like they’re not going to come through.
# 11 heuristics for choosing (alignment) research projects

I recently spoke with Dane Sherburn about some of the most valuable things he learned as a SERI-MATS scholar.

Here are 11 heuristics he uses to prioritize between research projects:

1.  **Impact:** Can I actually tell myself a believable story in which this project reduces AI x-risk? (Or better yet; can I make a guesstimate model that helps me estimate the microdooms averted from this project?)
2.  **Clarity of research question:** Can I easily explain my core research question in a few sentences?
3.  **Relevance of research approach:** Will my research project actually help me reduce uncertainty on my research question? When I imagine the possible results, are there scenarios where I actually update? Or do I already know (with high probability) what I’m likely to learn?
4.  **Mentorship:** Would my mentor be able to give me meaningful guidance on this project? If not, would I be able to find one who could?
5.  **Feedback loops:** Will I be able to get feedback within the first week? First day? Will I have to wait several weeks or months before I know if things are working?
6.  **Efficiency:** How efficiently will I be able to collect information or run experiments? Will I need to spend a lot of time fine-tuning models? Is there a way to do something similar with pretrained models, so I can run experiments 10-100X more quickly?
7.  **Resources: **WilI this project need datasets? Large models? Compute? Money? How likely is it that I’ll get the resources I need, and how long will it take?
8.  **Excitement:** How much does the project subjectively excite me? Do I feel energized about the project?
9.  **Timespan:** How long would it take to do this project? Would it fit into a window of time that I’m actually willing to devote to it?
10.  **Downsides/capabilities externalities:** To what extent does the project have capabilities externalities? Could it *increase* x-risk?
11.  **Leaveability:** How easy would it be to leave this project if I realize it’s not working out, or I find something better?
# Jar of Chocolate

Truffles are pretty great: gooey fat-softened flavored chocolate inside, protected by a shell of hard chocolate so you don't get your fingers messy. Sometimes I'll make them with the kids:

[![](https://www.jefftk.com/anna-dipping-truffles.jpg)](https://www.jefftk.com/anna-dipping-truffles-big.jpg)

[![](https://www.jefftk.com/lily-dotting-truffles.jpg)](https://www.jefftk.com/lily-dotting-truffles-big.jpg)

This is a good amount of work: you need to prepare the filling, chill, shape little spheres, chill, and then dip them. Worth it for making presents or sharing, but not if I'm making something for myself. Instead I just make a jar of ganache:

[![](https://www.jefftk.com/ganache-in-jar.jpg)](https://www.jefftk.com/ganache-in-jar-big.jpg)

[![](https://www.jefftk.com/ganache-jar-closed.jpg)](https://www.jefftk.com/ganache-jar-closed-big.jpg)

I'll eat it with a spoon over a week or two, 1-3oz after lunch each day.

At it's simplest it's chocolate and fat. I normally use [Callebaut semi-sweet](https://www.baldorfood.com/product/couvertures/spd2cc-semi-sweet-chocolate-chips) which I can get in bulk for $3.42/lb ($0.21/oz):

[![](https://www.jefftk.com/chocolate-box-label.jpg)](https://www.jefftk.com/chocolate-box-label-big.jpg)

I'll usually use butter and cream, though coconut cream works well if you want to make it vegan. Often I'll mix in some cocoa powder to make it a bit darker. Sometimes I'll leave it unflavored, other times I'll add some extracts (vanilla, almond, mint) or frozen fruit (usually raspberries). Peanut butter also works well, which gives you the flavor and means you don't need a different source of fat. It's reasonably shelf-stable: even with the cream it's not going to go bad before I finish a jar of it.

I generally mix it not to be very sweet: if it's just right I enjoy a savoring some but [not so much](https://www.jefftk.com/p/very-dark-mouth-shaped-non-fancy-chocolate) that I keep eating past what I'll feel good about.
# A Simple Alignment Typology

I set out to [review the OpenAI alignment plan](https://www.lesswrong.com/posts/tvjLwataADYxCc9EN/openai-alignment-review-rationalists-versus-empiricists), and my brain at some point diverged to modeling the humans behind the arguments instead of the actual arguments. 

So behold! A simplified, first-pass Alignment Typology. 

Why can't we all just get agree?
================================

There are a lot of disagreements in AI alignment. Some people don't see the problem, some think we'll be fine, some think we're doomed, and then different clusters of people have different ideas on how we should go about solving alignment. Thus I tried to sketch out my understanding of the key differences between the largest clusters of views on AI alignment. What emerged are roughly five cluster, sorted in order of optimism about the fate of humanity: the sceptics, the humanists, the empiricists, the rationalists, and the fatalists.

*Sceptics* don't expect AGI to show up in any relevant time frame.

*Humanists* think humanity will prevail fairly easily through coordination around alignment or just solving the problem directly.

*Empiricists* think the problem is hard, AGI will show up soon, and if we want to have any hope of solving it, then we need to iterate and take some necessary risk by making progress in capabilities while we go.

*Rationalists* think the problem is hard, AGI will show up soon, and we need to figure out as much as we can before making any capabilities progress.

*Fatalists* think we are doomed and we shouldn't even try (though some are quite happy about it). 

Here is a table.

|     | **Sceptics** | **Humanists** | **Empiricists** | **Theorists** | **Fatalists** |
| Alignment Difficulty | -   | One of these is low | high | high | -   |
| Coordination Difficulty | -   | high | high | -   |
| Distance to AGI | high | -   | low/med | low/med | -   |
| Closeness to AGI required to Solve Alignment | -   | -   | high | med/high | -   |
| Closeness to AGI resulting in unacceptable danger | -   | -   | med/high | high | -   |
| Alignment Necessary or Possible | -   | high | high | high | low |

Less Wrong is mostly populated by empiricists and rationalists. They agree alignment is a problem that can and should be solved. The key disagreement is on the methodology. While empiricists lean more heavily on gathering data and iterating solutions, rationalists lean more heavily toward discovering theories and proofs to lower risk from AGI (and some people are a mix of the two). Just by shifting the weights of risk/reward on iteration and moving forward, you get two opposite approaches to doing alignment work. 

How is this useful?
===================

Personally it helps me quickly get an idea of what clusters people are in, and understanding the likely arguments for their conclusions. However, a counterargument can be made that this just feeds into stereotyping and creating schisms, and I can't be sure that's untrue.

What do you think?
# Eli Dourado AMA on the Progress Forum

Eli Dourado—whom you might know from his writing on [geothermal](https://www.elidourado.com/p/geothermal), [supersonic](https://www.mercatus.org/students/research/research-papers/make-america-boom-again), [energy superabundance](https://www.thecgo.org/research/energy-superabundance/), or [NEPA](https://www.thecgo.org/benchmark/much-more-than-you-ever-wanted-to-know-about-nepa/)—is doing [an AMA (Ask Me Anything) on the Progress Forum](https://progressforum.org/posts/acgrRfWETEdm96xJk/eli-dourado-ama).

Get your questions in now, and upvote the ones you want to see answered. He’ll start answering Monday (Jan 30).

After you’re done, check out our previous AMAs with [Tyler Cowen](https://progressforum.org/posts/3aNLznQiLuyTKMhpH/tyler-cowen-ama) and [Patrick McKenzie](https://progressforum.org/posts/ManARnErMRFEvzffo/patrick-mckenzie-ama).
# Reverse RSS Stats

[In 2013](https://www.jefftk.com/p/reverse-rss-feed) I built a weird feature where clicking on " [◂◂ RSS](__REVERSE_RSS__)" at the top of a post gets you an RSS feed that runs in reverse from that post, with a 'new' old post with the publication of each new new post. It's a bit hard to think about, and reminds me of trying to keep tenses straight [when time traveling](https://tvtropes.org/pmwiki/pmwiki.php/Main/TimeTravelTenseTrouble).

I was reminded of it today when someone [submitted the post](https://news.ycombinator.com/item?id=34563853) to HN, so I calculated some stats:

*   If you had subscribed to the reverse feed in 2013 when I introduced it in [post 740](https://www.jefftk.com/p/reverse-rss-feed) you would be down to an empty feed on 2019-06-03 when I published [post 1,480](https://www.jefftk.com/p/beantown-stomp-sliding-scale).
    
*   In publishing this post (2,074), someone who subscribed in 2014-10-01 when I published [post 1,037](https://www.jefftk.com/p/contra-hand-roles) should now be at the end.
    
*   At my current rate (~200/y) someone who subscribes to the [reverse feed](__REVERSE_RSS__) on this post today should reach my [first post](https://www.jefftk.com/p/webpage-up) around 2033.
    

I wish more people offered feeds like this!

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0qbUbqJ4tRudg1oguhYAuwrUn8HwiQDQq1xjsESkr2WKRVGsiEfE8Pv76tSQ7fyyql), [mastodon](https://mastodon.mit.edu/@jefftk/109770397583183723)*
# EA novel published on Amazon

Just so everyone knows, the novel I wrote to promote EA has been published on Amazon. This is not a request to purchase it, or try to directly support it in any particular way. 

However if the blurb *does *sound interesting, go for it :)

https://www.amazon.com/dp/B0BSXHJRBQ
# [Linkpost] Human-narrated audio version of "Is Power-Seeking AI an Existential Risk?"

There's now a human-narrated audio version of my report on existential risk from power-seeking AI (text version [here](https://arxiv.org/pdf/2206.13353.pdf)). You can find the audio [here](https://joecarlsmithaudio.buzzsprout.com/2034731/12113681-is-power-seeking-ai-an-existential-risk), or by searching for "Joe Carlsmith Audio" on your podcast app.

Thanks to the folks at [Type III audio](https://blog.type3.audio/), and especially to Solenoid Entity, for doing the narration. There's also a shorter video presentation and summary of the report [here](https://forum.effectivealtruism.org/posts/ChuABPEXmRumcJY57/video-and-transcript-of-presentation-on-existential-risk).
# The effect of horizon length on scaling laws

The scaling of optimal model size with compute is a key input into the [biological anchors framework](https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines) for forecasting transformative AI. In particular, the "effective horizon length" introduces a multiplier into this scaling law that can have a big effect on forecasts.

This paper studies this scaling law for several RL environments: Procgen, Dota 2 and a toy MNIST-based environment. The last of these is used to study the effect of the task horizon length in a toy setting. There are a number of takeaways for the biological anchors framework, which are summarized in Section 5.4.
# A simple technique to reduce negative rumination

I have faced challenges with negative rumination and anxiety, especially when I am trying to relax or my mind is idle. However, I have recently discovered a simple and effective technique to overcome this.

I practice enumerating as many positive future events as I can, starting with the most probable and desirable ones, calculated by multiplying their probability with the level of happiness they bring, adjusted for their proximity in time.

While the focus is on positive thoughts, I find that this slightly quantitative approach helps me direct my thoughts more effectively. It pushes me to consider realistic and desirable events, countering the natural tendency to dwell on negative scenarios. Additionally, by visualizing medium-probability positive events, I become more aware of opportunities to increase their chances in my daily life, similar to the popular concept of "manifesting."

This technique not only provides a positive outlook but is also additionally rewarding because it requires me to think creatively and explore what I appreciate in life, making it feel like a game.

I also extend this technique by dividing positive events into 3-5 broad categories and brainstorming each one separately. This ensures a more comprehensive outlook and prevents me from dwelling on low probability scenarios or getting lost in unrealistic fantasies.

I hope this helps someone!
# Hacked Account Spam

Whenever I post something publicly on Facebook mentioning hacking I get a ton of spammy comments:

[![](https://www.jefftk.com/fb-hack-comments.jpg)](https://www.jefftk.com/fb-hack-comments-big.jpg)

[![](https://www.jefftk.com/fb-hack-comments-2.jpg)](https://www.jefftk.com/fb-hack-comments-2-big.jpg)

This first happened with my not-actually-deepfake [phishing post](https://www.jefftk.com/p/deepfake-phishing), and then again on my posts on [not getting hacked](https://www.jefftk.com/p/not-getting-hacked) and [how likely Google lockouts are](https://www.jefftk.com/p/how-likely-is-losing-a-google-account).

It sounds like there's a business in helping (or scamming) people who've had their Facebook/Instagram accounts hacked?

I've reported 42 of the comments as spam, and deleted dozens more (reporting spam takes several more clicks than deleting, and then you need to delete the comment anyway). Facebook doesn't seem to be getting any better at blocking them.

One of the more frustrating things about this, however, is that Facebook does seem to recognize these as spammy, and so doesn't notify me about them. Which is not what I want: it means my friends still see them on my posts, and I'm reliant on them to [let me](https://www.facebook.com/jefftk/posts/pfbid02kaekZn9Vnhb8kuNvcZHsDnwdJoAyzyUC96eCtodLmugiAqjQ6ULYemjUrBckMrU7l?comment_id=601483435047572) [know about](https://www.facebook.com/jefftk/posts/pfbid02tSH14YYLnV7Ljds4kqdRFGggWZmrLApTmVxnHDgi28cz1YLHZ2zGJ6iDsYvUDau1l?comment_id=862784204964508) [the comments](https://www.facebook.com/jefftk/posts/pfbid02GWYhiQZMesxkgomwCXWZnjgds4QhNS9CCgNf5ddsMKABkCWh74yxeAvfuERZASn3l?comment_id=742474880857826) so I can delete.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid03583V3T2FSffDYmUnTp32rEjzkm2TzXKbGZZYY9BRETEGQ2pVn35WASD9zdK4pfUYl), [mastodon](https://mastodon.mit.edu/@jefftk/109792611515557150)*
# Progress links and tweets, 2023-02-01

**The Progress Forum**
----------------------

*   [Eli Dourado AMA](https://progressforum.org/posts/acgrRfWETEdm96xJk/eli-dourado-ama) (lots of great questions and detailed answers!)
*   [When did England start seeing itself as a commercial nation?](https://progressforum.org/posts/CNrwa3DKHSLFWXAuP/age-of-invention-the-bourgeois-supremacy) (Anton Howes)
*   [The progress movement could use some explicit cause areas](https://progressforum.org/posts/oggzhJoTyyxRfrjZe/revving-up-the-progress-studies-idea-machine)

**Announcements**
-----------------

*   [New book on factors affecting medical progress](https://www.cambridge.org/dk/academic/subjects/medicine/medicine-general-interest/trajectory-discovery-what-determines-rate-and-direction-medical-progress?format=PB&isbn=9781009354431) (via [@markkhurana](https://twitter.com/markkhurana/status/1619006011834249217))
*   [ARIA, the ARPA of the UK, now legally exists and has a budget](https://www.gov.uk/government/news/research-agency-supporting-high-risk-high-reward-research-formally-established) (via [@logangraham](https://twitter.com/logangraham/status/1618664831015604224))

**Links**
---------

*   [Medieval peasants likely did not work fewer hours than we do](https://www.theatlantic.com/health/archive/2022/05/medieval-history-peasant-life-work/629783/) (via [@jmhorp](https://twitter.com/jmhorp/status/1617529270364172292))
*   [“Do-it-yourself crafts only exist when you no longer have to do everything yourself”](https://vpostrel.substack.com/p/from-the-archives-the-industrial)
*   [Oklo has submitted a plan for a nuclear fuel recycling facility](https://www.businesswire.com/news/home/20230125005319/en/Oklo-Prepares-for-the-Deployment-of-its-Commercial-Scale-Fuel-Recycling-Facility-with-the-Submission-of-its-Licensing-Project-Plan) (via [@oklo](https://twitter.com/oklo/status/1618316441807355904))
*   [Our World in Data topic page on Democracy](https://ourworldindata.org/democracy) (via [@bbherre](https://twitter.com/bbherre/status/1618950075765641216))
*   [“The whole saga feels to me like it’s part of a climate politics of sacrifice”](https://link.vox.com/view/614102b4afa92120df07a2e0i2gvb.op9/642ba173)

**Quotes**
----------

*   [The amazing progress in Asian agriculture in the late 20th century](https://twitter.com/jasoncrawford/status/1617694043353268225)
*   [A brilliant passage from John McPhee’s *The Control of Nature*](https://twitter.com/jasoncrawford/status/1619090594302726145)

**Retweets**
------------

*   [A lot of public health pronouncements don’t take into account that *people enjoy things*](https://twitter.com/TomChivers/status/1617840842743631872)
*   [If you want to see a progress studies community on Hive.one, nominate it here](https://twitter.com/hivedotone/status/1618731929100226560)
*   [A 1931 article on the potential for rocket-based passenger travel](https://twitter.com/1517fund/status/1618626130109661186) (@1517fund)

[![](https://pbs.twimg.com/media/FnaDzdtXkAYS-AM.jpg)](https://pbs.twimg.com/media/FnaDzdtXkAYS-AM.jpg)
# Epoch Impact Report 2022

![](https://res.cloudinary.com/cea/image/upload/v1675343341/mirroredImages/jGqHkeDw8awQnPYat/sw2a4rzmdmv3asubq4od.png)

[Epoch](https://epochai.org/blog/announcing-epoch) is a research group forecasting the development of transformative Artificial Intelligence. We try to understand how progress in AI happens and what economic impacts we might see from advanced AI.

We want to enable better governance during this economic transition by gathering information about the timing of new developments, studying which levers can be used to influence AI progress and making current and past trends in ML more understandable.

Founded in April of 2022, Epoch currently has a staff of 13 people, corresponding to 9 FTEs. We have received $1.96M in funding through a grant from [Open Philanthropy](https://www.openphilanthropy.org/grants/epoch-general-support/). We are fiscally sponsored and operationally supported by Rethink Priorities, whose Special Projects team has been a core part of our success as an organisation.

Epoch is fundraising a total of $6.07M over 2 years, or approximately $2.64M for October 2023 to September 2024, and $3.42M for October 2024 to September 2025. A detailed budget can be found in [the full report](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf#page=15).

With this funding, we expect to continue and expand our research capacity in understanding the future of AI. Through our research, we hope to continue informing about key drivers of AI progress and when key AI capabilities will be developed.

You can now [donate to Epoch](https://epochai.org/donate) and [subscribe to our newsletter](https://rethinkpriorities.us10.list-manage.com/subscribe?u=5b122f0651c57d56a658ba80c&id=a12ee420b6).

Read the [full report](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf) to learn about [our key insights this year](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf#page=4), an overview of our [research](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf#page=10) and [non-research activities](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf#page=12), and our [plans for the future](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf#page=14).

> Epoch is one of the coolest (and in my opinion underrated) research orgs for understanding trends in ML. Rather than speculating, they meticulously analyze empirical trends and make projections for the future. Lots of interesting findings in their data! – [Jacob Steinhardt](https://twitter.com/JacobSteinhardt/status/1615833694299910145?cxt=HHwWgsDTzeKLzOwsAAAA)

Read now [the full report](https://epochai.org/assets/docs/Epoch%20Impact%20Report%20-%202022.pdf)
--------------------------------------------------------------------------------------------------
# How can I help inflammation-based nerve damage be temporary?

**Bounty \[closed\]: ** $50 for any suggestion that I consider promising enough to change my behavior for, conditional on my fully recovering by May 1.[^8321cw08gzx][^7kaxr1vjqk6][^b86v49r9of6][^57wrri2xi2]

**Question:** I recently got shingles; I'm now completely better, except for [postherpetic neuralgia](https://www.mayoclinic.org/diseases-conditions/postherpetic-neuralgia/symptoms-causes/syc-20376588) (~inflammation-caused nerve damage) in/around one ear, which intermittently causes weird sometimes-painful sensations, which (a) is distracting and (b) interferes with my sleep. I'm in the market for suggestions for how to ensure it's temporary: it "usually" goes away within a couple months, but I'll be real bummed if it lasts forever for me.

**Unnecessary details that might help guide your suggestions:** 

*   I'm a ~30yo Caucasian male, pretty much perfectly healthy aside from this.
*   My first shingles-symptoms were on Jan 14; my last non-neuralgia symptom was around Jan 22.
*   My default strategy is basically just (a) rest a lot because I'm tired, (b) drink a lot of water because I'm thirsty, and (c) take 325mg of aspirin 3x/day on the theory that (i) it's unlikely to hurt and (ii) it could plausibly help because it's anti-inflammatory. 
*   Suggestions I would pay out for if I hadn't already tried them: "ask a doctor"; "Google around"; "Google Scholar around"; "take aspirin"; "offer a bounty on Less Wrong."

[^8321cw08gzx]: Max 10 suggestions, first come first serve. 

[^7kaxr1vjqk6]: I'll pay more for suggestions I consider especially great. 

[^b86v49r9of6]: Things that make me more-likely-though-not-certain to consider a suggestion "promising": links to reputable sources (like journal papers or LW users with high karma) supporting the idea; common-sense arguments describing the mechanism by which the suggested action should help; not involving unusual drugs. 

[^57wrri2xi2]: My evaluation criterion is super subjective, and I'm a skeptical person, so I might naturally not pay out at all; so, to demonstrate seriousness, I commit to paying at least two bounties, to the most promising suggestions, even if they wouldn't otherwise meet my bar.
# Halifax Monthly Meetup: Introduction to Effective Altruism

In the interest of growing the LW/EA/ACX community in Halifax, we are hosting monthly meetups. Newcomers are especially welcome!

This month we'll be discussing the elementary ideas of effective altruism. What is effective altruism? What do effective altruists care about? What do effective altruists do? 

**All this and more at Seven Bayes!**  
Location: Seven Bays Cafe (2017 Gottingen Street) – [87PRMC29+9C](https://plus.codes/87PRMC29+9C)

Contact: [conorbarnes93@gmail.com](mailto:conorbarnes93@gmail.com)
# What fact that you know is true but most people aren't ready to accept it?

Understanding and updating beliefs on deeply engrained topics can take enormous efforts, but sometimes it can be so hard that the listener cannot even in principle accept the new reality. The listener is simply not ready, he lacks a vast background of reasoning leading to the new understanding.

**What fact that you know is true but most people aren't ready to accept it?**

By "you know is true" I really mean "you are very confident to be true".

Feel free to use a dummy account.
# Taboo P(doom)

I think it's actively unhelpful to talk about P(doom) because it fails to distinguish between literally everyone dying and humanity failing to capture 99.99% of the value of the future under a total utilitarian view but in practice, everyone who's born lives a very good life. These are very different outcomes and it's unhelpful not to distinguish between them and everything else in that spectrum. 

This is especially the case since astronomical waste arguments really only bite for total utilitarian views. Under moral views where potential people not coming into existence is more similar to preventing someone from living an extraordinarily happy life rather than a merely happy life, as opposed to preventing someone coming into existence being similar to murder, it's quite reasonable to prioritise other goals well above preventing astronomical waste. Under these non-totalalist views preventing totalitarian lock-in or S-risks might look much more important than ensuring we don't create 10^(very large number) of happy lives. 

I think this also matters on a practical level when talking about threat models of AI risks. Two people could have the same p(doom) but one is talking about humans being stripped for their atoms and the other is talking about slow disempowerment in which no one actually dies and everyone, in fact, could be living very good lives but humanity isn't able to capture almost all of the value of future from a total utilitarian perspective. These plausibly require different interventions to stop them from happening. 

It also seems like one's prior on humanity going extinct as a result of AI should be quite different from disempowerment, but people often talk about what their prior on P(doom) should be as a univariate probability distribution.
# [Linkpost] Google invested $300M in Anthropic in late 2022

A few quotes from the article:

Google has invested about $300mn in artificial intelligence start-up Anthropic, making it the latest tech giant to throw its money and computing power behind a new generation of companies trying to claim a place in the booming field of “generative AI”.

The terms of the deal, through which Google will take a stake of about 10 per cent, requires Anthropic to use the money to buy computing resources from the search company’s cloud computing division, said three people familiar with the arrangement.  
  
The start-up had raised more than $700mn before Google’s investment, which was made in late 2022 but has not previously been reported. The company’s biggest investor is Alameda Research, the crypto hedge fund of FTX founder Sam Bankman-Fried, which put in $500mn before filing for bankruptcy last year. FTX’s bankruptcy estate has flagged Anthropic as an asset that may help creditors with recoveries.
# 2+2=π√2+n

This [post](https://www.lesswrong.com/posts/yepKvM5rsvbpix75G/you-don-t-exist-duncan) mentions an equation from Ender's shadow

$$
2+2=\pi\sqrt{2+n}
$$

Which kind of annoys me because either:

1.  It's trivial to solve using algebra
2.  The joke is that the answer is irrational so it takes "forever" to solve

So the question I have is:

What is the best problem you know of having the following properties:

1.  Can be expressed as a short formula using "high school" math or less
2.  Has an integer (or rational I guess) answer
3.  ..which is less than 10**100
4.  And cannot be solved using contemporary mathematics

The first "almost" example I came up with was something along the lines of

$x_1*x_2*...* x_n =\lfloor\pi*10^{10000}\rfloor$

Where $x_i $ are all prime.  This should have a couple of large prime factors and hence be hard to factor, but notice it doesn't satisfy criterion 3 (since we can easily factor numbers ~10\*\*100\*\*2)

Does anyone have a better example?

My other instinct was [Ramsey Numbers](https://en.wikipedia.org/wiki/Ramsey%27s_theorem#Ramsey_numbers) but these seem to fail criterion 1 since I didn't learn the definition in any high school math class.

Also thought of [Collatz Conjecture](https://en.wikipedia.org/wiki/Collatz_conjecture) which I guess would count if it had a counter-example < 10**100.
# The Future of Structured Self Improvement

Two years ago, we founded the Guild of the ROSE to provide structure and community for people interested in self improvement. Since then, we've grown from a three-month Alpha project to a year-round workshop host.

If you've been in our [Discord server](https://discord.gg/rnz5Zn6RMz), you've probably seen regular announcements about people leveling up on the Practitioner's Path, our RPG-inspired self improvement framework. You might even have heard rumors that we've been working on a new version of the Path.

Practitioner's Path 2.0
-----------------------

The Practitioner's Path 2.0 overhauls every aspect of the Path system, from Bugs to the Character Sheet to levels. The new system is split into three parts: Attributes, Tasks, and Skills. Together, these make up your Character Sheet.

### Attributes

Attributes measure long-term metrics such as:

*   Net Worth (how much money you have)
*   Reach (how many people you can influence)
*   Stamina (the efficiency of your cardiovascular system)

The Attributes section of the Character Sheet gives a sense of long-term progress on tangible metrics. Of course, not everything can be tracked as a metric, which is where Skills and Tasks come in.

### Tasks

A Task is something that you want to do or achieve. Tasks can be nested inside each other, representing the way that a complex goal can be split into smaller goals.

For example, the Task of "Learn how to program" could be broken into the subtasks of "Decide what language I want to learn" and "Find a tutorial on the language". This nesting system allow you to break down complex goals into easily-achievable chunks.

### Skills

The core of the new Path system is the Skilltree. Skills provide a structured approach to self-improvement and a sense of progression as you acquire them. Each skill has a certain amount of XP, allowing you to track your progress on a granular and broad level.

![pathteaser](https://guildoftherosemedia.s3.us-west-2.amazonaws.com/image_20230202164439719_9ce3a91432.png?updated_at=2023-02-03T19:07:55.901Z)

Skills in the Path are organized into three main categories: Pragmatist, Meditative, and Empiricist. Acquiring a Skill will only level up its category — so you can be a high-level Pragmatist but a low-level Empiricist. This division allows you to specialize in a given tree and pursue the Skills that you find most interesting and rewarding.

*   **Pragmatist (red):** Take direct action, make money, and expand your social network.
*   **Meditative (green):** Make art, improve your mental and physical health, and build a community.
*   **Empiricist (blue):** Learn new things, teach others, run experiments, and think clearly.

The Path 2.0 launches in early March, and we'll be sharing more details about how it works in the weeks leading up to release.

In the meantime, which tree do you resonate most with? Let us know in the comments below!
# Podcast with Oli Habryka on LessWrong / Lightcone Infrastructure

OK sorry to over-advertise but it seemed like this one would be of interest to the LessWrong and EA communities. Episode description below, audio is [here](https://youtu.be/4lbeutseEJQ), or search for "The Filan Cabinet Habryka" wherever you listen to podcasts.

---

In this episode I speak with Oliver Habryka, head of Lightcone Infrastructure, the organization that runs the internet forum LessWrong, about his projects in the rationality and existential risk spaces. Topics we talk about include:
- How did LessWrong get revived?
- How good is LessWrong?
- Is there anything that beats essays for making intellectual contributions on the internet?
- Why did the team behind LessWrong pivot to property development?
- What does the FTX situation tell us about the wider LessWrong and Effective Altruism communities?
- What projects could help improve the world's rationality?
# Have an idea? Come to Oxford to discuss and write (20 – 24 March)

We’re running a workshop (20 – 24 March) in Oxford with three goals.

1.  Go in with only a vague idea; come out with a drafted academic essay that you could eventually publish.
2.  Build tacit knowledge about the research/doing-good-things sphere.
3.  Meet other people in that sphere, for peers and mentoring.

We encourage you to apply if you think those goals seem useful or entertaining and if you fit the below criteria.

1.  You’re between 18 and 25[^toiztk2n54k],
2.  Interested in research or writing,
3.  Have ideas (even vague or bare-bones ones) that, if taken seriously, might make the world better, and
4.  Are available between 20 – 24 March.

We welcome applications from people interested in all academic disciplines, from maths and computer science to politics and literature, from physics and psychology to economics and philosophy. 

  
Please apply using this (~10-minute) [application form](https://docs.google.com/forms/d/e/1FAIpQLSeblPAHSTxCta3QNQ9d94_N1BFdE2bxGrYNiojfd9FmHM2ovQ/viewform?usp=sf_link). In some cases, we might schedule follow-up interviews. Final decisions will be communicated to all applicants by 1 March.

**Logistics**

The workshop is in Oxford, 20 – 24 March: the days directly following EAGxCambridge. The workshop is free, including room and food, and we will cover reasonable travel expenses within the UK. We may cover travel expenses from elsewhere. Please reach out if you have questions about travel or funding (or anything else).

**Who are we?**

The workshop is being run by *Flourish*—formerly known as the *EA Student Journal Project*. We’re a relatively new organisation aiming to give prosocial, agentic students mentors, peers, and tools to succeed in academia and independent research. Again, if you have any questions, please do reach out!

[^toiztk2n54k]: This criterion might miss out on some people we'd want; apply if you fit this vibe.
# Why is Everyone So Boring? By Robin Hanson

This is from the blog [Overcoming Bias](https://www.overcomingbias.com/).

These excerpts describe the overall model. It is only due to Robin Hanson's writing skill that the dynamic can be described this concisely, yet still be covered so thoroughly and intuitively. This is a classic example of galaxy-brained writing.

> Centuries ago, while people could rest safe and show themselves at home, when traveling between towns they tried to look either look poor or well-defended, as bandits lay in wait. Even within towns, people without allies who acted unusually rich, assertive, and confident would induce others to try to trip them somehow. It’s the tall poppy that gets cut down, after all.

> I propose that the main reason that most of us look more boring in public is that social predators lie in wait there. With friends, family, and close co-workers, we are around people that mostly want to like us, and know us rather well. Yes, they want us to conform too, but they apply this pressure in moderation.
> 
> Out in public, in contrast, we face bandits eager for chances to gain social credit by taking us down, often via accusing us of violating the sacred. And like townspeople traveling among the bandits, we are in public pretty vulnerable to the kinds of bandits that afflict us.
> 
> If we act interesting, passionate, and opinionated in public, we are likely to seem to [claim](https://www.overcomingbias.com/2009/04/choke-to-submit.html) high status for ourselves, and to touch on sacred subjects, either by word or deed. And this makes us quite vulnerable to accusations of arrogance and violating the sacred.

> I see roughly three typical public stances: *boring*, *lively*, or *outraged*. Either you act boring, so the bandits will ignore you, you act lively, and invite bandit attacks, or you act outraged, and play a bandit yourself. Most big orgs and experts choose boring, and most everyone else who doesn’t pick boring picks bandit, especially on social media. It takes unusual art, allies, and energy, in a word “eliteness”, to survive while choosing lively.
# Duckbill Masks Are Great

With covid I've tried a lot of different n95 masks, and I like the ["pouch"-style masks](https://smile.amazon.com/Kimberly-Clark-Respirator-53358-NIOSH-Approved-Respirators/dp/B08NVDFB3R/) best. By a lot! They're much easier to breathe through then anything else I've tried, probably because they have so much surface area. Because your mouth is in a large pocket, there's plenty of room for talking, and even singing. And they also get a good seal without being too squeezy on my face or head.

While I see more people wearing them over time, they are still relatively unusual. I expect some of this is different masks, being a good fit for people with different shaped heads and different preferences, but I think the main problem is that they make you look a bit like a duck:

[![](https://www.jefftk.com/duckbill-mask.jpg)](https://www.jefftk.com/duckbill-mask-big.jpg)

*Anna [had covid](https://www.jefftk.com/p/flying-with-covid) but I didn't know it yet. I didn't get it, which I credit in part to this mask.*

If I'm going to look anatine, however, I want to be intentional about it. No one seems to make these in yellow but I wish they would: I'd pay an extra $0.10/mask for some color:

[![](https://www.jefftk.com/duckbill-mask-yellow.jpg)](https://www.jefftk.com/duckbill-mask-yellow-big.jpg)

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0Q34es6yJpGw3DJtAMtsuhfjTuJtETUtqVALb2DeTyyGe81bSDLN2gYXYRhNpT5tHl)*
# AXRP: Store, Patreon, Video

Some announcements:

*   AXRP now has a [store](https://store.axrp.net), where you can buy t-shirts and hoodies and stickers and such.
*   AXRP now has a [Patreon](https://www.patreon.com/axrpodcast) and a [ko-fi](https://ko-fi.com/axrpodcast), where you can support the podcast and get some perks.
*   There’s now a [video](https://www.youtube.com/watch?v=kmPFjpEibu0) of an excerpt from episode 14, where Vanessa Kosoy explains the monotonicity principle, illustrated by [Hamish Doodles](https://www.youtube.com/@hamishdoodles) \- hopefully the first of many!
# OpenAI/Microsoft announce "next generation language model" integrated into Bing/Edge

**TL;DR:** Microsoft and OpenAI announced a new version of Bing featuring "a new, next-generation OpenAI large language model \[..\] more powerful than ChatGPT", and that Microsoft Edge will feature a Copilot-like assistant that helps with composing and summarizing content. 

* * *

Brief thoughts/comments/notes:

*   Microsoft's attitude during [their press meeting](https://www.theverge.com/2023/2/7/23588249/microsoft-event-ai-live-blog-openai-chatgpt-bing-announcements-news) seemed pretty aggressive and targeted directly at racing with Google. This seems kind of bad. For example, a quote from Nadella:
    *   "The race starts today, and we’re going to move and move fast. Most importantly, we want to have a lot of fun innovating again in search, because it’s high time."
*   Microsoft built a scaffold for the new LM called "Prometheus", that lets them "best leverage its power". 
*   Microsoft has also used the new LM in their Bing search engine, though it's not clear exactly how.
*   This seems way more hype than Google's Bard announcement.
*   You can register for the new Bing beta on [the Bing.com site](https://www.bing.com/).
# Wanted: Technical animator and/or front-end developer for interactive diagrams of invention

Seeking help for a project to create interactive diagrams—an “[explorable explanation](http://worrydream.com/ExplorableExplanations/)”—of the history of the steam engine.

![An example for inspiration](https://rootsofprogress.org/img/steam-engine-animation.gif)

*An example for inspiration.* [*Wikimedia / Panther*](https://commons.wikimedia.org/wiki/File:Steam_engine_in_action_(two-thirds_speed).gif)

**Goal:** To allow readers to explore the early evolution of one of the most impactful technologies in human history.

**Content:** The content will come from Anton Howes’s extensive research uncovering the origins of the steam engine (parts [1](https://antonhowes.substack.com/p/age-of-invention-why-wasnt-the-steam), [2](https://antonhowes.substack.com/p/age-of-invention-why-wasnt-the-steam-cc8), and [3](https://antonhowes.substack.com/p/age-of-invention-why-wasnt-the-steam-76c)).

We envision it covering precursors like the perpetual motion machine of Cornelis Drebbel, early thermometers and barometers, seventeenth-century experiments on atmospheric pressure, precursor devices like those shown by Salomon de Caus, our best guesses of the Kalthoff/Petty engine, the Savery engine, and the Newcomen engine and its improvements. (After this initial phase, we would then want to expand it to include the improvements by James Watt, and the rise of the high-pressure engine, with its various nineteenth-century improvements.)

This page on the steam engine may serve as a start for eventually creating the definitive online and interactive reference work on the history of technology, informed by the latest expert research.

**Format:** We have been greatly inspired by the works of [Bartosz Ciechanowski](https://ciechanow.ski/), such as his [mechanical watch](https://ciechanow.ski/mechanical-watch/) and [internal combustion engine](https://ciechanow.ski/internal-combustion-engine/) explanations. We’re looking to create something like this. Here are more examples of [locomotive valves](https://trumpetb.net/loco/).

**The team:** [Anton Howes](https://www.antonhowes.com/) as researcher/writer, me (Jason Crawford) as editor/producer, you as animator/developer.

**Your input:** We’ll provide the written essay and some sketches and ideas for what diagrams we should create and how they should work.

**Your deliverables:** The code and/or other files to implement a set of interactive and/or animated technical diagrams.

**You:** A technical illustrator, animator, and/or front-end developer, or a team, who can deliver this. Must have a portfolio of relevant work to demonstrate your abilities. The ideal candidate will have an interest in engineering, invention, and the history of technology.

**How to apply:** Email [explorables@rootsofprogress.org](mailto:explorables@rootsofprogress.org) with a description of your skills and examples of your work.

We have budget for this project and are willing to pay market rates for high-quality work.