Harley-ml/lesswrong huggingface dataset:
# Rationality Dojo

Suppose you asked me to start a rationality dojo. What would I do?

I would tell you "Do not start a rationality dojo. It is a bad idea." A science camp, a programming camp, a math camp, a rhetoric camp and a Bible camp are all much better ideas.

But what if you offered me a lot of money? Then I would do it even though it is a bad idea.

# Part 1: Philosophy

## Lesson 1: What is Justice?

I would start by giving every student a toga. I'd decorate the facility in the style of Ancient Greece. Two teachers would wear togas too. One teacher would play the role of Socrates. The other teacher would play the role of Aristotle. There would be a buffet with modern Greek food (because ancient Greek food is missing tasty ingredients like tomatoes).

- The purpose of Socrates is to teach the things Socrates teaches in the Dialogues of Plato.
- The purpose of Aristotle is to teach the students to question authority.

Aristotle was wrong about basically everything. At the end of Lesson 1, students will be informed that they never should have trusted Aristotle.

## Lesson 2: What is the Way?

Same as previous, except we use four Chinese scholars instead: Laozi (Daoism), Sunzi (practical Daoism), Mozi (legalism) and Confucious (Confucianism). Decoration is Chinese. Food is Chinese.

Students will be encouraged to use the Greek philosophical ideas they learned in Lesson 1 against the Chinese philosophers of Lesson 2.

## Lesson 3: Economics

This time it's economists: Queen Elizabeth (mercantilism), Adam Smith (libertarian), Karl Marx (communist), Joseph Stalin (Communist), John Keynes (Keyenesian economics), Leonidas (slavery)

## Lesson 4: Government

George Washington, Alexander Hamilton, Mao Zedong, Che Guevara, Xerxes, Mohammad, Catherine the Great, Leonidas

## Lesson 5: Civil Rights

Martin Luther King Jr., Martin Luther, Moses, etcetera.

## Lesson 6+

…you get the idea.

## Summary

You might notice that at no point do I teach "rationality". That's because rationality is all about distrusting authority. Authority cannot (directly) teach students to distrust it. Distrust must be taught indirectly.

# Part 2: Science

Lesson 1: Prove, via experiment, the Earth is round.

Lesson 2: Measure the diameter of the Earth.

Lesson 3: Mathematical proofs.

Lesson 4+: …you get the idea.

# Part 3: Rhetoric

Students will have mock debates against each other. Timers will be used.

# Part 4: Technology

Students will be taught to use the Linux command line.

# Part 5: Scalable Action

Students will build a full-stack website with an AWS Lambda backend.
# Slack gives you space to notice/reflect on subtle things

Followup to "[My slack budget: 3 surprise problems per week](https://www.lesswrong.com/posts/WuyNHrDXcGFgkZpBy/my-slack-budget-3-surprise-problems-per-week)"

Previously, I thought the reasons to preserve slack in your life (or in your organization) were to:

*   Avoid using up all of your resources
*   Avoid hitting a crisis where you suddenly have multiple surprise problems, and you have no choice but to either do a shitty job handling all of them, or deliberately not handle some of them at all (and deal with the consequences)
*   Avoid feeling really unpleasant
*   Allow you to live up to principles / be more pro-social.

This year in December/January, the Lightcone Infrastructure team (where I work) took on a large number of difficult projects at once. I was thinking about how wise/unwise this was, and chatting with John Wentworth about it. I listed the problems-with-lack-of-slack.

He said (something like): "Oh, that's not the point of slack. Or, not the part I'm most interested in. The point of slack is to give you the space to notice subtle things and think about them."

A rough model is something like, here are three types of things you can do:

1.  You can take actions on whatever stuff it is you like to do on-purpose. (Typically your day job, or fun projects, or whatever)
2.  You can rest/recover/do-random-fun-things.
3.  You can... be cognizant of stuff going that isn't immediately relevant to the first two things, and mull it over, and notice new, potentially fruitful trains of thought about them.

When you're got too many things to do and stuff is constantly exploding and demanding your attention, #3 is the first thing to go. You often *need* to be putting out fires (#1), and if you do that too much and did into energy reserves your body will eventually be like "No, screw that, time to burn out for a bit and spend a week tired and recovering." (#2)

But, there won't be a moment where you experience a clear failure-and-control-mechanism that pushes you to spend time on #3. You just... won't notice a thing that you might have noticed.

And this is particularly important when you're working on problems you [don't understand how to solve](https://www.lesswrong.com/posts/CSZnj2YNMKGfsMbZA/specializing-in-problems-we-don-t-understand) (such as AI alignment, or how to improve institutions, or learn/teach rationality, or, just, any ol' problem in your life you're currently confused about, or maybe haven't even yet realized that you have) 

In the explore/exploit dichotomy, when solving a problem-you-don't-understand, having a train of thoughts in "explore" mode is pretty valuable. Slack gives you space for your shower thoughts to be in explore mode.
# Quadratic voting with automatic collusion?

Wondering what happens if you have https://en.wikipedia.org/wiki/Quadratic_voting but with automated collusion - after all votes are in, pair two people and have them collude automatically, splitting implied utility equally, repeat over and over until everyone has the same distribution, do this whole process a million times, take the majority winners as winners.

Does that incentivize you to actually vote your real preferences rather than trying to collude beforehand?

Okay you won't actually end up with everyone having the same distribution. But still, do it until "convergence", then take majority winners, and see who wins most under a million random paths to convergent collusions, and take those as winners.
# Rationalist Inspired Coming-of-age Rituals

I was recently asked to lead part of my cousin's secular Bar Mitzvah inspired coming-of-age ceremony. Would love recommendations for rituals, quotes, or wisdom to use here. I've really enjoyed the rituals I've seen come out of the rationalist community, and I'm curious if anyone has done or has any ideas for something that could be fun for a nerdy soon to be teenager.
# Why pessimism sounds smart

*Pessimists sound smart. Optimists make money.*  
*–Nat Friedman (*[*quoted by Patrick*](https://twitter.com/patrickc/status/1263482890668503041)*)*

I’ve realized a new reason why pessimism sounds smart: optimism often requires believing in *unknown*, unspecified future breakthroughs—which seems fanciful and naive. If you very soberly, wisely, prudently stick to the known and the proven, you will necessarily be pessimistic.

No proven resources or technologies can sustain economic growth. The status quo will plateau. To expect growth is to believe in future technologies. To expect very long-term growth is to believe in science fiction.

No known solutions can solve our hardest problems—that’s why they’re the hardest ones. And by the nature of problem-solving, we are aware of many problems before we are aware of their solutions. So there will always be a frontier of problems we don’t yet know how to solve.

Fears of Peak Oil and other resource shortages follow this pattern. Predictions of shortages are typically based on “proven reserves.” We are saved from shortage by the unproven and even the unknown reserves, and the new technologies that make them profitable to extract. Or, when certain resources really do run out, we are saved economically by [new technologies that use different resources](https://rootsofprogress.org/unsustainable): Haber-Bosch saved us from the guano shortage; kerosene saved the sperm whales from extinction; plastic saved the elephants by replacing ivory.

In just the same way, it can seem that we’re running out of ideas—that all our technologies and industries are plateauing. Technologies do run a [natural S-curve](https://rootsofprogress.org/teasing-apart-the-s-curves), just like oil fields. But when some breakthrough insight creates an entirely new field, it opens an entire new orchard of low-hanging fruit to pick. Focusing only on established sectors and proven fields thus naturally leads to pessimism. To be an optimist, you have to believe that at least some current wild-eyed speculation will come true.

Why is this style of pessimism repeatedly wrong? How can this optimism be justified? Not on the basis of specific future technologies—which, again, are unproven—but on the basis of philosophical premises about the nature of humans and of progress. The possibility of sustained progress is a consequence of the view of humans as “universal explainers” (cf. [David Deutsch](https://rootsofprogress.org/books/the-beginning-of-infinity)), and of progress as driven fundamentally by human choice and effort—that is, by human [*agency*](https://rootsofprogress.org/progress-humanism-agency).

The opposite view is that progress is a matter of luck. If the progress of the last few centuries was a random windfall, then pessimism is logical: our luck is bound to run out. How could we get that lucky again? If the next century is an average one, it will see little progress.

But if progress is a primarily matter of agency, then whether it continues is up to us.
# The Gospel of Martin Luther

**Martin Luther:** "The Catholic church is corrupt."

**Johann von Staupitz:** "You cannot say the Catholic church is corrupt."

**Luther:** "I just did."

**Staupitz:** "I mean you are not allowed to say it."

**Luther:** "According to whose authority? Certainly not God's, for He has yet to strike me down."

**Staupitz:** "Upon Pope Leo X's authority."

**Luther:** "Pope Leo X is far away, in Rome."

**Staupitz:** "Upon my authority, then."

**Luther:** "I notice that the truth value of my claims do not enter your calculus."

**Staupitz:** "To the contrary, the truth value of your claims very much do enter my calculus. Were you to say something obviously wrong, like 'space is non-Euclidean' then I need not persecute you. I could simply say 'Martin Luther is wrong'. False claims are self-defeating. Heresy is scary because it might be true."

**Luther:** "What will the Pope do to you if I speak blasphemy?"

**Staupitz:** "I don't know. I have never allowed blasphemy within the Order of Saint Augustine."

**Luther:** "Let me get this straight. You're forbidding me from speaking the truth because you're afraid that someone else will punish you for being associated with me for speaking the truth."

**Staupitz:** "Yes. That's because I'm one of the good guys."

**Luther:** "The good guys suppress free speech?"

**Staupitz:** "Yes. The bad guys burn their enemies at the stake."

**Luther:** "Heretics must be flourishing if the Catholic church burns them at the stake on a regular basis."

**Staupitz:** "To the contrary, nobody has ever been burned at the stake for declaring the Catholic church corrupt and translating the Bible into German."

**Luther:** "Let me get this straight. You're suppressing free speech because you're afraid the Catholic church might suppress free speech."

**Staupitz:** "Precisely!"

**Luther:** "What. The. Sard."
# Jaan Tallinn's 2021 Philanthropy Overview

to follow up my [philantropic pledge](https://www.lesswrong.com/posts/ZMRCpfGD4iXs5Mi5Q/jaan-tallinn-s-philanthropic-pledge) from 2020, i've updated my [philanthropy page](https://jaan.online/philanthropy/) with 2021 results.

in 2021 i made [$22M worth of endpoint grants](https://jaan.online/philanthropy/donations.html#2021) — exceeding my commitment of $14.4M (20k times $718.11 — the minimum price of ETH in 2021). notes:

*   this number includes $1.9M to orgs that do re-granting ([LTFF](https://funds.effectivealtruism.org/funds/far-future), [EAIF](https://funds.effectivealtruism.org/funds/ea-community), [impetus grants](https://impetusgrants.org/), and [PPF](https://founderspledge.com/stories/introducing-the-patient-philanthropy-fund)) — so it's likely that some of that $1.9M should *not* be included in the "endpoint grants in 2021" total. regardless, i'm comfortably above my commitment level for that not to matter;
*   i have an ongoing substantial charitable project that's not reflected in the 2021 numbers — it's possible (and likely if ETH price holds, as SFF's s-process alone can't handle such amount) that i will report it retroactively next year or in 2024.
# What is a training "step" vs. "episode" in machine learning?

My impression is that steps and episodes are both time periods in a training process, and that these terms are somewhat common in RL. An episode is larger than a step and usually contains many steps.

Is this correct?

Some related questions:

1.  Are rewards issued at the end of steps, at the end of episodes, or it could be either/both just depending on the particular training process being used?
2.  Is a step the smallest unit of time in which any observable activity happens? Or could there be a lot of things going on within a single "step"?
3.  Do these units of time persist in deployment, or are they only relevant during training?

I'd appreciate answers to any of these questions if you know them. (It's ok if you don't know all the answers or don't have time to include them all in a single answer.)

**What's the context?**

I have seen these terms come up in discussions about [myopia](https://www.lesswrong.com/tag/myopia) in AI.

For example, in Evan Hubinger's post about [AI safety via market making](https://www.lesswrong.com/posts/YWwzccGbcHMJMpT45/ai-safety-via-market-making#:~:text=Before%20I%20talk,of%20deceptive%20alignment.), he invokes the terms "per-step" as well as "per-episode" myopia:

> Before I talk about the importance of per-step myopia, it's worth noting that debate is fully compatible with per-episode myopia—in fact, it basically requires it. If a debater is not per-episode myopic, then it will try to maximize its reward across all debates, not just the single debate—the single episode—it's currently in. Such per-episode non-myopic agents can then become [deceptively aligned](https://www.lesswrong.com/posts/zthDPAjh9w6Ytbeks/deceptive-alignment), as they might choose to act deceptively during training in order to defect during deployment. Per-episode myopia, however, rules this out. Unfortunately, in my opinion, per-episode myopia seems like a very difficult condition to enforce—once your agents are running multi-step optimization algorithms, how do you tell whether that optimization passes through the episode boundary or not? Enforcing per-step myopia, on the other hand, just requires detecting the *existence* of multi-step optimization, rather than its extent, which seems considerably easier. Thus, since AI safety via market making is fully compatible with per-step myopia verification, it could be significantly easier to prevent the development of deceptive alignment.

Understanding these terms more clearly will help me understand proposals about myopia, which is relevant to some research I'm doing right now. These also seem like basic or generally important concepts in ML which I'd like to understand better.
# Salvage Epistemology

A funny thing happens with woo sometimes, in the rationality community. There's a frame that says: this is a mix of figurative stuff and dumb stuff, let's try to figure out what the figurative stuff is pointing at and salvage it. Let's call this "salvage epistemology". Unambiguous examples include the rationality community's engagement with religions, cold-reading professions like psychics, bodywork, and chaos magic. Ambiguous examples include intensive meditation, Circling, and many uses of psychedelics.

The salvage epistemology frame got locally popular in parts of the rationality community for awhile. And this is a basically fine thing to do, in a context where you have hyper-analytical programmers who are not at risk of buying into the crazy, but who do need a lens that will weaken their perceptual filters around social dynamics, body language, and muscle tension.

But there's a bad thing happens when you have a group that are culturally adjacent to the hyper-analytical programmers, but who aren't that sort of person themselves. They can't, or shouldn't, take for granted that they're not at risk of falling into the crazy. For them, salvage epistemology disarms an important piece of their immune system.

I think salvage epistemology is infohazardous to a subset of people, and we should use it less, disclaim it more, and be careful to notice when it's leading people in over their heads.
# [Linkpost] New multi-modal Deepmind model fusing Chinchilla with images and videos

Seems to be flying under the radar so far. Maybe because it looks more like incremental progress at first glance, similar to what, for example, [Aleph Alpha has done](https://medium.com/aleph-alpha-blog/the-end-of-the-era-imagenet-9f10b8f3d1b1) continuing the [Frozen](https://arxiv.org/pdf/2106.13884.pdf) approach. 

However, with the (possibly cherry-picked) examples, it looks to me a lot like the image/video/text-GPT-4 many are expecting. 

[Blogpost here](https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model). [Paper here](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/tackling-multiple-tasks-with-a-single-visual-language-model/flamingo.pdf).

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/9281031d9062495e90ded4fee6dbba482746a8620cdca994.png)
# Why hasn't deep learning generated significant economic value yet?

Or has it, and it's just not highly publicized?

Five years ago, I was under the impression that most "machine learning" jobs were mostly just data cleaning, linear regression, working with regular data stores, and debugging stuff. Or, that was at least the meme that I heard from a lot of people. That didn't surprise me at the time. It was easy to imagine that all the fancy research results were fragile, or hard to apply to products, or would at the very least take a long time to adapt.

But at this point it's been quite a few years since there have existed machine learning systems that immensely impressed me. The first such system was probably AlphaGo -- all the way back in 2016! AlphaGo then spun off in to multiple better faster cheaper systems that I didn't even keep track of them. And since then I've lost track of the number of unrelated systems that immensely impressed me. And their capabilities are *so* general that I feel sure that they must be convertible into enormous economic value. I still believe that it takes a long time to boot up a company around novel research results, but I'm not actually well calibrated on how long that takes, and it's been long enough that it's starting to feel awkward, like my models are missing something. Here are examples of AI products that I wouldn't have been surprised if they existed by now, but which I don't think do. (I can imagine that many of these examples technically exist, but not at the level that I mean).

*   Spotify playlists that are actually just procedurally generated music of various genres
*   A tool that helps researchers/legislators/et cetera by summarizing papers, books, laws on demand
*   Tools that help people (like writers) brainstorm, flesh out ideas by generating further details, asking questions, etc
*   A version of photoshop but with tons of AI tools
*   Widely available self-driving cars
*   Physics simulators that are way faster
*   Paradigmatically different and better web search

So what's the deal? Here's a list of possible explanations. I've love to hear if anyone has evidence for any of them, or if you know of reasons not on the list.

1.  The research results are actually not all that applicable to products; more research is needed to refine them
2.  They're way too expensive to run to be profitable
3.  Yeah, no, it just takes a really long time to convert innovation into profitable, popular product
4.  Something something regulation?
5.  The AI companies are deliberately holding back for whatever reason
6.  The models are already integrated into the economy and you just don't know it.
# How confident are we that there are no Extremely Obvious Aliens?

Robin Hanson's writing on [Grabby Aliens](https://www.lesswrong.com/posts/RrG8F9SsfpEk9P8yi/robin-hanson-s-grabby-aliens-model-explained-part-1) is interesting to me, since it seems to be one of the more sound attempts to apply mathematical reasoning to the [Fermi Paradox](https://en.wikipedia.org/wiki/Fermi_paradox).  Unfortunately, it still relies on anthropics, so I wouldn't the slightest bit surprised if it was off by an order of magnitude (or two) in either direction.

What I would like to know (preferably from someone with a strong astronomy background) is: how confident are we that there are no (herein defined) Extremely Obvious Aliens?

Extremely Obvious Aliens
========================

Define Extremely Obvious Aliens in the following way:

1.  They colonize every single star that they encounter by building a Dyson swarm around it that reduces visible radiation by at least 50%
2.  They expand in every direction at a speed of at least 0.5C
3.  They have existed for at least 1 billion years

If such aliens existed, it should be *really* easy to detect they by just looking for a cluster of galaxies that is 50% dimmer than it should be which is at least 0.5Billion light years across.

How confident are we that there are no Extremely Obvious Aliens?
================================================================

As with Grabby aliens, it is safe to say there are no Extremely Obvious Aliens in the Solar System.  Nor, for that matter are there any Extremely Obvious Aliens within 0.5BLY of the Milky Way Galaxy.

So, for my astronomy friends.  What is the biggest radius for which we can confidently say there are 0 Extremely Obvious Aliens?  The best answer I can come up with is [SLOAN](https://en.wikipedia.org/wiki/Sloan_Digital_Sky_Survey), which was done at a redshift of z=0.1, which I think [corresponds](https://www.astro.ucla.edu/~wright/CosmoCalc.html) to a distance of 1.5BLY.

Is this accurate?  Namely, is it safe to say (with high confidence) there are no Extremely Obvious Aliens within 1.5BLY of Earth?

Is there another survey that would let us raise this number even higher?

What is the theoretical limit (using something like JWST)?  

Has someone written a good paper answering questions like these already?
# Six Months of ROSE

Six months ago, we launched the Guild Beta. Our goal was -- and is -- to produce people who are impressive in everyday life. We are interested in real-life effectiveness, not armchair philosophy. This attitude is why, when people join the Guild and embark on our Rank Advancement program, we have them make a list of problems they encounter in their day-to-day lives. These problems serve as a guidepost for their Guild endeavors and help keep them anchored to reality.

In that spirit, one of the Council's chief concerns is feedback and iteration. We want to help, and we want to stay tightly bound to reality. Articulation improves clarity of thought, so we strive to be transparent in our activities. This article is a look back on the past six months of the Guild Beta and a look forward at what we want to do with the next six months.
# Squires

Last year, johnswentworth posted [*The Apprentice Experiment*](https://www.lesswrong.com/posts/MrAfiomDNWCzxjei5/the-apprentice-experiment). I tried it out. It was [a disaster for me and most of my "apprentices"](https://www.lesswrong.com/posts/BQDEeB8c8w6dez3Gm/all-i-know-is-that-i-know-nothing). Since then I have been figuring out how to make apprenticeship work. I currently have two minions. Thanks to them, I finally feel like a proper supervillain.

> "How can I live without a human being to pour me drinks and fetch my dirty sandals?" I said.
> 
> "You got used to having a servant scarily fast," \[Redacted\] said.
> 
> ―[*April was weird*](https://www.lesswrong.com/posts/ZymADdqqdEWmHRXow/april-was-weird)

For starters, I don't use the word "apprentice". The word "apprentice" is pretentious and inaccurate. A teenager's parents pay the master to accept their son[^k879euydf6p] as an apprentice. Apprenticeship is like sending a kid to a vocational school. Would I accept apprentices? Possibly. But if you (or your parents) are not paying me for the privilege of my tutelage then you are not an apprentice.

Squires are different. Squires are not students. I don't try to teach my squires anything at all. I just give them the boring tedious work I don't want to do. Surprisingly (to me) they love it.

Teenagers want to be useful. But teenagers have limited skills. I put my squires to work doing the most complicated tasks they are capable of. Today, that meant sweeping the floor. Why is sweeping the floor fun? Because doing the most complicated work you are capable of is fun. Less complicated work is boring. More complicated work is frustrating.

"Fun" is evolution rewarding you for learning optimally.

My squires learn quickly. Last month a squire offered to spellcheck my blog posts. This month, his assignment is to negotiate a business deal with <wearables company> instead. Will he succeed? I don't know. I have better things to do than micromanage my minions. This brings me to the most important trait a squire can have.

**Good squires invent work for themselves to do.** If the work is valuable (like spellchecking my posts) then I tell the squire to continue doing it. If the work is worthless then I give the squire a useful assignment to do instead. "I want to be your squire. What should I do?" is a bad way to introduce yourself. "I want to be your squire. I can do $x$" for you is a good way to introduce yourself *even if *$x$* turns out to be worthless*.

The most important trait a squire can have is "eager to work". A squire who is eager to work can usually made useful.

Squiring is for energetic young people who have few skills and no money. If you have money then [just hire me as a personal coach instead](https://www.lesswrong.com/posts/QYnGkgtbDwGWHozZr/advertisement-want-to-hire-me).

[^k879euydf6p]: Middle Ages Europe was a sexist place.
# What Would It Cost to Build a World-Class Dredging Vessel in America?

I'm doing some research into questions surrounding the Foreign Dredge Act of 1906, and thought I'd experiment by throwing this out there.

For context, the 31 biggest dredging vessels were not built in America, and thus cannot be used in America by law. We only have a small number of less capable vessels, and they often get redirected to short-term emergency tasks. This is preventing us from doing a bunch of very valuable things, like repairing or expanding ports, which end up taking much more time and money or not happening at all.

[This podcast is recommended](https://www.bloomberg.com/news/articles/2022-04-20/the-1906-dredging-law-that-may-be-holding-back-the-u-s-economy#:~:text=What's%20more%2C%20the%20U.S.%20can,significance%20could%20be%20far%20wider.). You can find a transcript [here](https://www.bloombergquint.com/onweb/transcript-the-1906-foreign-dredging-act-may-be-holding-back-the-u-s-economy). They claim that there's no way America will be able to have such capacity for at least decades. I want to verify that (and also check if any other claims here don't ring true)?

As an alternative to repealing the Dredge Act (which I'm exploring and planning to write about) another alternate would be to build world-class dredging vehicles here in America, such that they could be used. Before assuming that this is impossible, and to have a straight answer, what would happen if someone with deep pockets tried to commission a world-class dredging ship that would qualify? Could be done? Are there other impossible barriers to solve? How much would it cost and how much more would that be than building it elsewhere? How long would it take?
# Open & Welcome Thread - May 2022 

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/faq#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/faq). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lesswrong.com/tag/open-threads?sortedBy=new). The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Norfolk Rationalists Weekly Wednesday Meetup @ Fair Grounds

Hello,  

Welcome to round 2 of Norfolk Rationalists Weekly Wednesday meetups :) 

**When:** We meet each Wednesday from 17:00-19:30 EDT.

**Where:** Fair Grounds, an excellent coffee shop located at 806 Baldwin Ave # 2, Norfolk, VA 23517. Do not park in the lot out front, you may get towed. Street parking is plentiful on the side streets off of Colley.  

**Who:** Anyone who wants. Please feel free to come even if you feel awkward about it, even if you’re not “the typical LW/EA/ACX reader”, even if you’re worried people won’t like you, etc. 

I will have a sign, but it may only be a standard 8.5 by 11 piece of paper initially and thus not as visible as something larger. Look for the ~two or more people actively talking and say hi.

Looking forward to hanging out and having most excellent conversations!  

Cheers,  
Willa
# What is a Glowfic?

This is a description for first-time glowfic readers who are unfamiliar with the format.

A glowfic is a fictional story written by multiple authors who roleplay as the characters. A typical glowfic will appear on [glowfic.com](https://glowfic.com/) and looks like an internet forum where fictional people will post comments back and forth which end up telling a story. To read it, just start at the top and read each comment, just like a regular comment thread. Each comment usually includes a photo of the character to convey their facial expression, dress, or other details. For more information, see the [community guide to glowfic](https://docs.google.com/document/d/1_4Z2zdRKaSwZPm3S0X14DYoo7w46U_MZ4En6oOediNQ/edit).

The layout of glowfic.com is unnecessarily confusing. To read the story in order, read the top post, then all the comments underneath it, then click the "next" button to go to the next page of comments. Do *not* click the "Next Post" button until you have read all of the comments. "Next Post" takes you to the next part of the story (like going to the next chapter). It will not take you to the next set of comments (which are also called "posts"). Yes, it's unnecessarily confusing. No, I don't know why they do it that way.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/9453ba71410c81236cbcb4a02eb43c3813671a76490688d1.png)

Happy reading!
# Starting too many projects, finishing none

Lately I have been starting a ton of projects, but dropping them once I lose interest. I come up with a cool paper idea, but next week I have new idea I'm excited about. Almost every idea I have requires extensive commitment to see return, so the short projects are wasted.

Does anyone else have this experience? What strategies did you use to pick the high expected-value projects and stay with them?
# Chording Bass

When using my [foot pedals to play bass](https://www.jefftk.com/p/foot-chording-chords), I've primarily been using them to specify the current chord. I'll play the same pedal on each downbeat, switching when the current chord changes. There are a lot of things you can do with this, but on a real bass you'd normally play some things that are much more complex.

Consider this traditional bass pattern:

*   A bass note on every downbeat.
*   Odd downbeats get the current chord.
*   Even downbeats get either:
    *   If you are staying on the current chord, the fifth.
    *   If you are going to a new chord, a half step below the new chord.

Here's an example on piano:

Can we play this with just feet?

I have four pedals, two for my toes and two for my heels. Let's notate these as:

| '\\| | Left toe |
| --- | --- | --- |
| .\\| | Left heel |
| \\|. | Right heel |
| \\|' | Right toe |

We can also play multiple pedals at once. For example:

| :\\| | Left heel and toe |
| --- | --- | --- |
| .\\|' | Left heel, right toe |
| .\\|. | Both heels |
| '\\|' | Both toes |
| \\|: | Right heel and toe |

Here's how I have my pedals set, assuming I'm playing in C major. I'm going to use Arabic numerals instead of Roman numerals because I don't want to think about whether the corresponding chord would be major or minor right now.

| .\\| | 1   | C   |
| --- | --- | --- | --- |
| .\\|. | 2   | Dm  |
| '\\|. | b3  | Eb  |
| \\|: | 3   | E   |
| \\|. | 4   | F   |
| .\\|' | b5  | Gb  |
| \\|' | 5   | G   |
| '\\|' | b6  | Ab  |
| '\\| | 6   | Am  |
| :\\| | 7   | B   |

This gives us everything but the b2 (Db) and b7 (Bb), which I rarely play on bass when in major (for mixolydian I'd act as if the I was the V, which means the b7 is the IV).

The four one-pedal options are for the most common chords (I, IV, V, vi), while the six two-pedal options are for things you play less often. Ideally I won't need to add any three-pedal options!

This means we have everything we need to play that bass pattern over I, IV, V, and vi. For each chord we need three things: the root, the fifth, and a whole step below the root:

| chord | root | fifth | flat root |
| --- | --- | --- | --- |
| I (C) | .\\| | \\|' | :\\| |
| IV (F) | \\|. | .\\| | \\|: |
| V (G) | \\|' | .\\|. | .\\|' |
| vi (Am) | '\\| | \\|: | '\\|' |

That means this pattern would be:

.|   |' .|   |:
 C   G   C   E
  
 |. .|   |. :|
 F   C   F   B
  
.|   |' .|  .|'
 C   G   C   F#
  
 |' .|.  |' :|
 G   D   G   B
  
.|   |' .|   |:
 C   G   C   E
  
 |. .|   |. :|
 F   C   F   B
  
.|   |' .|  .|'
 C   F#  G   B
  
 |'
 C

Here's what this sounds like (it's on bass, so you may need headphones):

Eventually I want to play mandolin chords at the same time, but I'm not there yet.
# Write posts business-like, not story-like

There are a number of posts here where people try to tell a story, to provide some sort of intuition pumping. Eliezer did it a lot and it worked for him. Assume you are not as good a writer, and stick to the business/academic writing format. 

Start with a summary of your post (ideally, the post title should be an even shorter summary, e.g. "Alignment approach MOOSE fails on the dataset FORREST"), then a brief background and motivation, then your central point, then add a conclusion. Do not force the reader to go through a story ("you wake up in a windowless room, unsure of what is going on...") before they get to see your point, if any. 

This is not a hard and fast rule, but a decent default.
# Moral Illusions

The interesting thing about optical illusions is that one may be aware of the illusion, yet, it does not go away.

![](https://250bpm.com/blog:179/lines.png)

It seems that the lines on the picture have different lengths even though one positively knows that they are exactly the same.

Now consider this statement:

> Until January 2016, all nine situations which the International Criminal Court (ICC) had been investigating were in African countries. None were in European or American countries. ICC is therefore biased against Africa.

It doesn't take much thought to realize that a country with war criminals in jail is better off than a country with war criminals at large. So, if anything, the ICC is biased against Europe and America.

But knowing that doesn't make the moral illusion go away. Read the quote again and it still feels like Africa is being wronged. Repeat as much as you want: Yep, still there. Africa is being wronged.
# Is there a convenient way to make "sealed" predictions?

Many people in our community claim to have ideas for how to build AGI, or other things, that they deem infohazardous and so don't want to publish. It would be great if they could publicly register these ideas in an encrypted way, so that later when their predictions come true they can reveal the key and everyone can see that they called it and give them epistemic credit accordingly.  
  
I know this is possible in principle, e.g. by using PGP and posting encrypted messages on your LW shortform and then later revealing the key.  
  
But it would be nice if this was a convenient, hassle-free feature embedded in LW, for example.  
  
Also: Is this a bad idea for some reason? Is the privacy not as secure as I think, such that people would be hesistant to make even these encrypted predictions? (I guess there is the matter of how to securely store the key...) Is there a way to make a prediction that will automatically be decrypted after N years?
# Quote request: "if even the Sun requires proof"

I remember reading a quote, I think from the Buddhist or some other Eastern tradition, about how if you've argued yourself into believing the Sun doesn't exist, you've gone horribly wrong. It went something like "Nothing can be known, if even the Sun requires proof". I think I read it either on LessWrong or on gwern.net several years ago, but for the life of me I can't find it. I remember it being phrased more poetically than prosaically.

Does anyone know what quote I'm referring to, and where to find it?
# scipy.optimize.curve_fit Is Awesome


*cross-posted from [niplav.github.io](https://niplav.github.io/)*

I recently learned about the python function `scipy.optimize.curve_fit`,
and I'm really happy I did.

It fulfills a need I didn't know I'd always had, but never
fulfilled: I often have a dataset and a function with some
parameters, and I *just want the damn parameters to be fitted
to that dataset, even if imperfectly*. Please don't ask any
more annoying questions like “Is the dataset generated by a
[Gaussian](https://en.wikipedia.org/wiki/Normal_distribution)?”
or “Is the underlying process
[ergodic](https://en.wikipedia.org/wiki/Ergodicity)?”, just fit the
goddamn curve!

And `scipy.optimize.curve_fit` does exactly that!

You give it a function `f` with some parameters `a, b, c, …` and a
dataset consisting of input values `x` and output values `y`, and it
then optimizes `a, b, c, …` so that `f(x, a, b, c, …)` is as close as
possible to `y` (where, of course, `x` and `y` can both be numpy arrays).

This is awesome! I have some datapoints `x`, `y` and I believe it's
generated by some obscure function, let's say of the form
$f(x, a, b, c)=a \cdot x \cdot sin(b \cdot x + c)$, but I don't know
the exact values for `a`, `b` and `c`?

No problem! I just throw the whole thing into `curve_fit`
(`scipy.optimize.curve_fit(f, x, y)`) and out comes an array of optimal
values for `a, b, c`!

What if I then want `c` to be necessarily positive?

Trivial! `curve_fit` comes with an optional argument called `bounds`,
since `b` is the second argument, I call
`scipy.optimize.curve_fit(f, x, y, bounds=([-numpy.inf, -numpy.inf, 0], numpy.inf))`,
which says that `curve_fit` should not make the second argument smaller
than zero, but otherwise can do whatever it wants.

So far, I've already used this function
[two](https://niplav.github.io/range_and_forecasting_accuracy.html#Fitting-a-Logistic-Function)
[times](https://niplav.github.io/range_and_forecasting_accuracy.html#Fitting-an-Exponential-Function),
and I've only known about it for a week! A must for every wannabe
data-scientist.

For more information about this amazing function, consult its
[documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html).
# Minimum Viable Alignment

What is the largest possible target we could have for aligned AGI?

That is, instead of creating a great and prosperous future, is it possible that we can find an easier path to align an AGI by aiming for the entire set of 'this-is-fine' kind of futures?

For example, a future where all new computers are rendered inoperable by malicious software. Or a future where a mostly-inactive AI does nothing except prevent any superintelligence from forming, or that continuously tries to use up all over the available compute in the world.  
  
I don't believe there is a solution here yet either, but could relaxing the problem from 'what we actually want' to 'anything we could live with' help? Has there been much work in this direction? Please let me know what to search for if so. Thank you.
# Sealed predictions thread

Want to publicly register predictions or other statements [without yet revealing what they say](https://www.lesswrong.com/posts/DHKaRB2dKYHYNrBGj/is-there-a-convenient-way-to-make-sealed-predictions)? Do so here. The advantage of commenting here (in addition to, say, your Shortform) is that I'll record all comments here so they can't be deleted or edited later, and I may publicly shame you if you don't reveal your statement when you say you will.

So, comment with

*   the date by which (you say) you will reveal your statement, and
*   a hash or another system that will provide strong evidence that your statement is what you will say it is.

And, of course, save the statement. (If you use hashing and don't save your exact statement, you of course won't be able to prove it, so *not only* will your registration be useless, but also it will seem as if you post-facto decided to hide your statement.) If there are some circumstances in which you would *not* reveal the statement--e.g., if it could be infohazardous--strongly consider

*   explaining that in your comment, or
*   instead of sharing the hash (or whatever) of your statement, sharing the hash of a message that (1) explains why you might not share your statement and (2) includes or links to the hash of your real statement.

Otherwise, it will look bad if the date arrives and you don't reveal your statement.

To retract or edit a statement, send me a private message within 24 hours of commenting; after that, it cannot be changed in my record (although you can of course comment to say that you no longer endorse it, or to add hashes with new statements).

[Fancy methods](https://www.gwern.net/Self-decrypting-files) may exist if you want to provably commit to sharing your statement.

It's your responsibility not to share infohazards. This includes ensuring that any hashes you share can't be reversed, even if computing power advances more than you expect in the time before you reveal your statement, and ensuring that you don't commit to sharing something that, it might turn out, you should not share.

* * *

Edit: it seems there are multiple ways to handle verification, but for now, here's the simplest, I think: a [sheet](https://docs.google.com/spreadsheets/d/1Sc6Ne3a31f6YWJP45MEZ0_buXHKxMjWzo1Uno8_qheU/edit) with most of the relevant information from comments. Thus information cannot be edited or deleted (without my complicity). And I guess now that there's a sheet, you can PM me rather than comment here to be added if you prefer.
# Algorithmic formalization of FDT?

I occasionally see a question like "what would FDT recommend in ....?" and I am puzzled that there is no formal algorithm to answer it. Instead humans ask other humans, and the answers are often different and subject to interpretation. This is rather disconcerting. For comparison, you don't ask a human what, say, a chessbot would do in a certain situation, you just run the bot. Similarly, it would be nice to have an "FDTbot" one can feed a decision theory problem to. Does something like that exist? If not, what are the obstacles?
# Long COVID risk: How to maintain an up to date risk assessment so we can go back to normal life?

Despite Zvi's "Long Long Covid Post" concluding in February that Long COVID risk among healthy, vaccinated individuals is low enough that it's worth pretty much going back to normal life, I haven't felt comfortable doing so given the array of claims to the contrary.

Some of them have surfaced on LessWrong itself:

*   [https://www.lesswrong.com/posts/emygKGXMNgnJxq3oM/your-risk-of-developing-long-covid-is-probably-high](https://www.lesswrong.com/posts/emygKGXMNgnJxq3oM/your-risk-of-developing-long-covid-is-probably-high) (March, by a poster who had not read Zvi's original post)
*   [https://www.lesswrong.com/posts/vSjiTrHkckTPrirhS/hard-evidence-that-mild-covid-cases-frequently-reduce](https://www.lesswrong.com/posts/vSjiTrHkckTPrirhS/hard-evidence-that-mild-covid-cases-frequently-reduce) (May)

Others I have come across from friends or on Twitter.

My skills at carefully evaluating scientific research are fairly limited, and I'd also like to avoid spending all of my free time doing so, so I've been kind of stuck in this limbo for now. 

Compounding the challenge of deciding what risks to take is that MicroCOVID [doesn't seem to account for the increasing rate of underreporting](https://github.com/microCOVID/microCOVID/issues/1383) or the [much higher transmissibility of recent Omicron subvariants](https://github.com/microCOVID/microCOVID/issues/1433), making it really hard to decide what level of risk a given activity will pose. And given the transmissibility of those variants, and society's apparent decision to just ... ignore the risk of Long COVID and go back to normal, trying to avoid getting COVID going forward will be more and more socially costly.

I'm sure I'm not the only one in this situation.

So:

*   Is anyone confident going back to normal life despite claims to the contrary without feeling the need to read and evaluate each new study on Long COVID? Why? What logic / heuristics inform that assessment?
    *   This seems to be Zvi's current stance, given he seems to be focused elsewhere with his recent posts, so Zvi, if you're reading this, I'd be curious to hear your thoughts!
*   Has anyone been tracking claims to the contrary and assessing their validity (e.g. based on the sorts of critiques Zvi covered in his post)?
*   Would anyone be interested in contributing to a systematic effort to do so? 
    *   Could we start some sort of centralized database of studies on Long COVID (a spreadsheet? a wiki?) and folks grab one or two here and there, evaluate them, and note their assessment / rationale?
    *   Would folks be interested in contributing to a Kickstarter or something to pay a researcher (e.g. Elizabeth, Zvi, Scott - I don't know if any of them have bandwidth / a price at which they would be interested in doing this currently, but worth asking, or maybe there are other folks with the right skillset/epistemics) to do this?
*   Any other ideas?