Harley-ml/lesswrong huggingface dataset:
# Game that might improve research productivity

To figure out how to make progress on problems that require cognitive effort, it seems useful to think of times in the past when you’ve made lots of cognitive progress. For me, high school math camp was the most cognitively productive time.

It seems plausible that the main reason I was productive at math camp was because of the competitive atmosphere. In light of this, I want to experiment with systems that harness people’s competitiveness to drive learning and insight, since maybe this can increase productivity substantially. “Insight Hunt” is a way this could be done. 

I suspect the below idea is somewhat far from the ideal and that it could change a fair bit in response to user feedback. If anyone tries an Insight Hunt, please leave your feedback and results (including the name of the winner if you want increased motivation) in the comments! Also, feel free to suggest alternative games/systems which leverage competitiveness for cognitive productivity.

Lastly, I suspect that Insight Hunt works best when the stuff the players want to learn is more “shallow” i.e. knowledge/facts that require few inferential steps from what most people know to understand. This is because if one person wants to learn linear algebra (not shallow), and another person knows linear algebra, the game will devolve into a lecture (I suspect). Examples of shallow knowledge include alignment theory (given some pre-requisites, like basic ML; this is the primary use-case I'm excited about) and psych study results. Non-examples include theoretical physics and math.  
 

**Insight Hunt**

*   There are three players; one player is the “judge” and two are the “seekers”
*   Who plays as the judge rotates such that everyone is the judge equally often
*   Every round:
    *   The seekers spend 8 minutes trying to find “insights” (i.e. true and useful sentences) that score highly according to the judge, who scores the insights based on how useful and interesting they found the sentence
        *   Before the round, it’s in the judge’s interest (who wants to learn) to tell the seekers what kind of stuff they want to know, or e.g. what papers/posts/ideas they want summarized to them
            *   Examples include “what is deep double descent?” or “what are the main reasons sleeping 8 hours might be important?” or “what’s John Wentworth’s main problem with ELK?”
        *   Finding insights might look like quickly digesting a paper, DMing someone for a quick insight, or using an on-demand tutor service
    *   The seekers each get 1 minute to explain each insight; after 1 minute is up, the judge awards anywhere from 1 to 10 points
        *   Optional: scores are displayed on a [scoreboard](https://www.amazon.com/scoreboard/s?k=scoreboard)
        *   Optional: a prize (e.g. $1000 or being announced as the winner publicly) is given to the person with the highest score in the end
# What are the top 1-10 posts / sequences / articles / etc. that you've found most useful for yourself for becoming "less wrong"?

Posts can be from LessWrong, the EA Forum, or any other blog or forum. Articles from other sources are fine as well, as are videos or podcast episodes. Books don't count for the purpose of this post, but if there is a specific chapter in a book that you found particularly useful then that counts.
# How many generals does Russia have left?

I’ve come across a large number of articles talking about the unprecedented number of generals (7) killed so far during the current war (see https://www.washingtonpost.com/world/2022/03/26/ukraine-russan-generals-dead/ for instance), but I can’t easily find information online about how many generals Russia actually has. Without a sense of how common and how crucial generals are in the Russian military, it’s hard to make any particular update on the situation. The way that the news has been worded suggests that seven is a high percentage in this context, but considering the level of positive propaganda from the media, I’m not sure I should put any trust into the framing here. If anyone could share further information about the number/strategic importance of Russian generals, it would be greatly appreciated.
# Mental nonsense: my anti-insomnia trick

My dad used to suffer from insomnia, holding imaginary meetings in his head late into the night. I'm the same way.

One stress-relief technique from [Johns Hopkins](https://www.hopkinsmedicine.org/health/wellness-and-prevention/sleepless-nights-try-stress-relief-techniques) is a breathing + mantra exercise. I find their mantra recommendation cringey:

> Breathing in I am calm, breathing out I am coping.

Mantras, guided imagery, and [autogenic training](https://www.goodtherapy.org/learn-about-therapy/types/autogenic-training) are generally hard for me, because I feel pressure to "stay focused on the mantra."

![Tense Woman Lying in Bed Along Sleeping Boyfriend by Stockwood on Envato  Elements](https://elements-video-cover-images-0.imgix.net/files/3be126ce-3f98-4ac8-bcea-9d114b9de9f8/inline_image_preview.jpg?auto=compress%2Cformat&fit=min&h=394&w=700&s=7ab5e99dc0d6cc0a5f747195ed00ca97)

*BREATHING IN I AM CALM, BREATHING OUT I AM COPING*

**My weird trick** is to replace the mantra with free-form inner-monologue nonsense syllables. Here's an example:

> Nimbla doobla deeble dee... simba dimba lima nooble doo...

I let the pacing and "mental voice" vary, but mainly aim for a sleepy-sounding tone.

At the same time, I let myself visualize nonsense images. My visual imagination isn't very vivid, and I don't try to create any specific images. It's nothing more than a patchy and low-res mental screensaver. Not even this exciting:

![abstract art GIF by Alex Apostolides](https://media2.giphy.com/media/3ohzdSYttKXTTqTWIE/giphy.gif?cid=ecf05e47fbzxjvuxhcb8jkuo02hwdlr0gtzc0xdmmttkyyb2&rid=giphy.gif&ct=g)

This puts me right to sleep.

If you have refinements of anti-insomnia tricks that aren't on bog-standard lists like [this one](https://www.mayoclinic.org/diseases-conditions/insomnia/diagnosis-treatment/drc-20355173), go ahead and add them in the comments!
# Retreat for rationality meetup organizers (apply to attend!)

**TL;DR** I'm running a retreat for meetup organizers in the Bay Area from July 21–24. More info [here](https://bit.ly/meetups-retreat); you can apply using [this form](https://forms.gle/t9x3HgtyBSYrNVas8).

ETA: The application form will be open through the end of the day on April 30th, and final decisions will go out by May 7th.

* * *

### Why a retreat?

I think we would all benefit from having an international community of rationality and ACX meetup organizers who can go to one another for advice and support. The goal of this retreat is to jumpstart that community. You'll spend time together and help each other solve problems, hopefully forming connections that will allow you to reach out for help even after you've gone back home to your respective cities.

In addition to getting to know each other, you'll be taking part in activities and discussions with the goal of improving your meetup group. I hope that you'll come out of the retreat with a clearer vision for your local community, and concrete plans for how to work towards that vision.

### Who is this retreat for?

You're invited if you:

*   Currently organize ACX/LW/rationality meetups or other rationalist community events, anywhere in the world
*   Would like to start an ACX/LW/rationality community in a city where none exists, or where the community has been small or dormant for a long time
*   Formerly organized ACX/LW/rationality meetups and are interested in sharing your knowledge and experience

If you're unsure whether you 'qualify', please err on the side of applying anyway — if we don't think it makes sense for you to come, we'll let you know!

### Food, lodging, and travel are covered

We don't want money to be an obstacle preventing anyone from attending, so your travel to and from the Bay will be fully reimbursed, and your food and lodging will be covered during the retreat itself. If you would like to stay longer in the Bay, we by default expect that you will cover your own expenses for that time, but if that's a problem for you, please don’t hesitate to reach out to me.

### [Apply here](https://forms.gle/t9x3HgtyBSYrNVas8)

or check out [this FAQ doc](https://bit.ly/meetups-retreat) for more info.

* * *

Let me know if you have any questions! I can be reached in various ways, but probably the most reliable is emailing meetupsmingyuan@gmail.com. I'm looking forward to your applications!
# What would make you confident that AGI has been achieved?

Consensus seems to be that [there is no fire alarm for artificial intelligence](https://intelligence.org/2017/10/13/fire-alarm/). We may see the smoke and think nothing of it. But at what point do we acknowledge that the fire has entered the room? To be less euphemistic, what would have to happen to convince you that a human-level AGI has been created?

I ask this because it isn’t obvious to me that there is *any* level of evidence which would convince many people, at least not until the AGI is beyond human levels. Even then, it may not be clear to many that superintelligence has actually been achieved. For instance, I can easily imagine the following hypothetical scenario:

* * *

A future GPT-N which scores a perfect 50% on a digital Turing Test (meaning nobody can detect if a sample output is written by humans or GPT-N), is announced by OpenAI. Let’s imagine they do the responsible thing and don’t publicly release the API. My intuition is that most people will not enter panic mode at that point, but will either:

1.  Assume that this is merely some sort of publicity stunt, with the test being artificially rigged in some way.
2.  Say something like “yes, it passed the Turing test, but that doesn’t *really* count because \[insert reason x\], and even if it did, that doesn’t mean it will be generalizable to domains outside of \[domain y that GPT-N is believed to lie inside of\].”
3.  Claim that being a good conversationalist does not fully capture what it means to be intelligent, and thereby dismiss the news as being yet another step in the long road towards “true” AGI.

The next week, OpenAI announces that the same model has solved a massive open problem in mathematics, something that a number of human mathematicians had previously claimed wouldn’t be solved this century. I predict a large majority of people (though probably few in the rationalist community) would not view this as being indicative of AGI, either.

The next week, GPT-N+1 escapes, and takes over the world. Nobody has an opinion on this, because they’re all dead.

* * *

This thought experiment leads me to ask: at what point would you be convinced that human-level AGI has been achieved? What about superhuman AGI? Additionally, at what point would you expect the average (non-rationalist) AI researcher to accept that they’ve created an AGI?
# How to Make Your Article Change People's Minds or Actions? (Spoiler: Do User Testing Like a Startup Would)

Are you writing an article that’s supposed to change people’s minds and/or actions? For example, an article about career advice, productivity, or getting people to apply for a job? Then this post is for you!

TL;DR: For important articles I recommend doing real user testing (even with just one person!). Try it and you’ll never go back.

I specifically recommend:
=========================

(1) Watch them read it, don’t just ask for Google Doc comments
--------------------------------------------------------------

Open a zoom, share screen, and ask them to share their thoughts live as they read.

(2) Show it to your target audience, don’t just ask others what your target audience would think
------------------------------------------------------------------------------------------------

If your article is supposed to change developers’ career plans, ask a dev who might change their career plan. Don’t ask a biologist what a dev might think.

(3) Check if they changed their mind, not just if they like your writing
------------------------------------------------------------------------

If they end with “I’ve got to do this now!”: That's a good sign. 

If they say “sounds interesting, I’ll think about it”, that’s not as good.

I personally like to first ask the other person for their opinion (for example, “what’s your current career plan?”), then let them read the article, and then ask again.

The general intuition:
======================

If you want to adopt my frame of mind, ask yourself: Would your idea of how to improve \[the article’s persuasiveness\] also improve \[how engaging a startup’s app\] is?

Here are some exercises, do you think these would work?

*   Let GPT-3 look at the app (or article) and tell you how engaging the app (or how persuasive the article) is
*   Try to guess in advance all the reasons why your target audience won’t agree with the article, or won’t be engaged with the app

Hint: Is the test \[checking if a real user from your target audience changes their mind\] or \[using some proxy to guess whether a real user would change their mind\] ?

How to start?
=============

Easiest: post in [Bountied Rationality](https://www.facebook.com/groups/bountiedrationality), or in another group that has your target audience.

A draft post you can copy & paste: “Hey, I’ll pay $10 for someone in \[describe your target audience here\] to review a post of mine for the EA/lesswrong forum. I want to do it live, over a video call. The call is capped at 30 minutes (and expected to be less). Comment or message me if you’re interested. Tagging or referring relevant people is also appreciated.”

Thanks to Matt Brooks and Daniel Reeves for [helping](https://www.facebook.com/groups/bountiedrationality/posts/3165691307008244/) make this article better <3
# Meta wants to use AI to write Wikipedia articles; I am Nervous™

Meta is experimenting with using AI to write Wikipedia articles: [https://ai.facebook.com/research/publications/generating-full-length-wikipedia-biographies-the-impact-of-gender-bias-on-the-retrieval-based-generation-of-women-biographies](https://ai.facebook.com/research/publications/generating-full-length-wikipedia-biographies-the-impact-of-gender-bias-on-the-retrieval-based-generation-of-women-biographies) 

I personally have a very bad feeling about this. I’m most afraid of this making it easier to spam the encyclopedia with fake information that looks plausible on the surface, and therefore doesn't get fully fact-checked.  This could also create perverse incentives where SEO companies put out false information online to bias Meta's algorithm, and thereby sneak their way in to the encyclopedia. The decision to make this open source seems incredibly foolish to me as well, considering how easily a service like this could be misused. *(Edit: It has been pointed out to me that if they made it closed-source that wouldn't be great either, since then we would have no idea what it was doing under the hood. Either way I wouldn't be happy, so I'm not sure their choice counts as a point against them.)*

Am I overreacting? Is this actually a good thing? Is this actually way worse than I think it is? Who knows!

What are your thoughts on this?
# Abandoned prototype video game teaching elementary circuit theory

About a decade and several hobbies ago, I was inspired to make an educational video game that would teach the intuitions behind elementary circuit theory (batteries, switches, diodes, capacitors, inductors, etc.).

After an inordinate amount of time,[^t3ldh562n2] I wound up with this prototype browser game:

[**Click here to play the game**](https://sjbyrnes.com/1235/)

Its biggest limitations (i.e., the biggest reasons that I'm calling it a “prototype”) are (1) I only got around to making five “levels” (i.e., puzzles to solve), (2) it's not very user-friendly.[^6diok7kwnh8]

Still, I gave it to my 7-year-old kid last night, and he was really enjoying it and clearly getting something out of it. That made me feel sad that it’s been sitting on my hard drive for a decade, and (on current trends) always will be.

So, **if anyone wants to take this thing and develop it further, let me know, and maybe I’ll send you the code repository and give you permission to do with it as you please**. (For example, if you commercialize it, you can keep all the profits.)

Meanwhile, enjoy. If you have nothing better to do with your time than to try to solve the puzzles, then please use [spoiler-hiding](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq#How_do_I_insert_spoiler_protections_) as appropriate in the comment section. 

[^t3ldh562n2]: This was 99.99% a solo project, but thanks to Michael Prettyman for some help and advice on the art assets. All remaining ugliness is fully my responsibility! 

[^6diok7kwnh8]: In a well-designed game, the user naturally and effortlessly learns the game interface, because it's presented bit by bit, with good affordances and/or tutorials, and user-tested, etc. I didn't get around to doing any of that, sorry. I think adults would be able to get the interface relatively quickly, especially if they happen to already know some elementary circuit theory (ironically). Younger kids might need a bit of help.
# Ways to organize a search for a missing person in LA from abroad?

TL/DR: My father-in-law is missing, asking for help/advice in coordinating search from abroad

This is not the thing I hoped I would post here after years of lurking, but situation is rather extreme and I try to get all the help I could get.

My father-in-law, Igor, has gone missing in LA after the stroke; because of his medical condition he is confused and unable to use his phone. He doesn’t have any friends or relatives in the US, his English is not very good (he is Ukrainian and only moved to LA last year), and he doesn’t have a place to go to – his landlord has evicted him from the apartment he rented when Igor was admitted to the hospital.

My wife and I live in Georgia (the country, not the state), and we try to organize a search for him from abroad; we can’t come to the US.

So the question is – what would be the recommended course of action to coordinate the search for a missing person from abroad? We have filed a missing person report to the police, we post to Facebook groups and made a [simple website with basic information](https://findigor.help/) – what could be our other options?
# [Link] Training Compute-Optimal Large Language Models

New LM scaling paper from DeepMind ([abs](https://arxiv.org/abs/2203.15556), [pdf](https://arxiv.org/pdf/2203.15556.pdf)).  

Abstract (my emphasis):

> We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. **We find that current large language models are significantly undertrained, a consequence of the recent focus on scaling language models whilst keeping the amount of training data constant**. By training over 400 language models ranging from 70 million to over 16 billion parameters on 5 to 500 billion tokens, **we** find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled. **We test this hypothesis by training a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4× more more data**. **Chinchilla uniformly and significantly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B) on a large range of downstream evaluation tasks.** This also means that Chinchilla uses substantially less compute for fine-tuning and inference, greatly facilitating downstream usage. **As a highlight, Chinchilla reaches a state-of-the-art average accuracy of 67.5% on the MMLU benchmark, greater than a 7% improvement over Gopher.**

Brief comments on my blog [here](https://nostalgebraist.tumblr.com/post/680262678831415296/an-exciting-new-paper-on-neural-language-model).

Presumably has implications for Bio Anchors?
# The Halting Problem and the Impossible Photocopier

*Epistemic status: the halting problem may actually still hold for quantum computers, I have no idea. I'm mostly interested in pointing out the sneakiness of assumptions.*

The Halting Problem is the following: imagine some class of algorithms $\mathbb A$. Can we have an algorithm $F$ which takes as inputs a description of another algorithm $A$ and an input $I$ for that algorithm, and outputs either $true$, if $A(I)$ halts or $false$, if $A(I)$ doesn't.

The original solution (by Alan Turing): no by contradiction. Imagine we have a such an algorithm. Now imagine a reversing algorithm $R$, which halts if given $false$ as an input, and runs forever if given $true$ as an input. Imagine also a copier algorithm $C$, which duplicates the input (so $C(x) = (x,x)$, and always halts. 

Now imagine we string these algorithms together, $R(F(C(x))) = G(x)$. What happens if we feed $G$ into $G$? Well $C(G) = (G, G)$, then $F(G, G)$ refers to whether $G$ halts when given the input $G$. So does $G$ halt when fed $G$? Think about this for a moment.

Imagine $G(G)$ halts. Then $F(G, G) = true$, which is passed to $R$, which runs forever.

Imagine $G(G)$ doesn't halt. Then $F(G, G) = false$, which is passed to $R$, which halts.

So we have a paradox.

This works for any class of algorithm you can imagine, it's a fantastically general proof which says nothing about the nature of the algorithms at hand.

* * *

Or does it!?

Note the hidden assumption. That $R$ and $C$ exist. What if they don't?

For a string of bits, or a Turing machine (as the problem was originally posed) the existence of $C$ is trivial. But for a string of *qu*bits, it isn't. The [no-cloning theorem](https://en.wikipedia.org/wiki/No-cloning_theorem) states that "it is impossible to create an independent and identical copy of an arbitrary unknown quantum state". So in fact the standard proof of the halting problem does not apply to quantum computers.

This is possibly the sneakiest assumption I've ever seen snuck into an argument! A proof which seems general to any algorithm class falls down because you can't have a quantum photocopier!

*I will repeat that I am not actually making a claim about quantum computers here. I have no idea if the Gödel's Incompleteness-based proof holds. I am merely pointing out the sneakiness of this assumption.*
# If AGI were coming in a year, what should we do?

Suppose AGI was very likely to arrive first around a year from now, with multiple projects close, but one a few months ahead of the others, and suppose that the AI safety community agreed that this was the case. What should our community do?

How would you answer differently for AGI in 4 months from now? 2 years from now? 5 years from now? 10 years from now? 20 years from now?

Some potential subquestions:

1.  What technical research should we try to get the project closest to AGI to rely on? How would we get them to use it? Or, if we could build AGI first, what would we do to reduce risks?
2.  What infrastructure, defences or other technology should we try to have ready or use?
3.  How should we reach out to governments, and what should we try to convince them to do?
4.  What research should we work on?
5.  What else?

My motivations for this question are to get people to generate options for very short timelines and to get an idea of our progress so far.
# Using Prediction Markets to Guide Government Policy

Using Prediction Markets to Guide Government Policy

If you use prediction markets to determine government policy then rational (and irrational) actors will manipulate the prediction markets in order to manipulate government policy. If you use prediction markets to predict government policy then rational (and irrational) actors will manipulate governments in order to create alpha in the prediction markets.

In [*Your Enemies Can Use Your Prediction Markets Against You*](https://www.lesswrong.com/posts/RzDehSs7KQcBpa2bf/your-enemies-can-use-your-prediction-markets-against-you) I explained how Mars could use a prediction market created by Earth to incentivize space pirates to overthrow the pro-Earth government of Ceres.

Suppose you're the governor of Ceres and you want to protect Ceres from space pirates. One way to protect Ceres from space pirates is to buy a laser defense array, but lasers are expensive and weapons are negative sum. Is there cheaper alternative?

Yes. You can manipulate market prices.

Mars is not acting against you directly. They are buying insurance in a market that pays anyone who can turn the Ceres government into a pro-Mars regime. Space pirates can profit by buying shares in the prediction market that pay money if Ceres shifts to a pro-Mars stance and then invading Ceres.

The prediction market has limited liquidity. If you use taxpayer dollars to buy those shares first then space pirates won't be incentivized to invade Ceres. (This strategy works (albeit more expensively) if you buy the shares second too, because space pirates could just sell their shares to you at a profit without having to bother actually assaulting Ceres.)

You win because you don't have to fight the pirates. The pirates win because they don't have to fight you. Ceres taxpayers lose because they have to pay the Dogecoin which goes (indirectly) into the pockets of space pirates—but that might be better than being raided, pillaged and plundered. Is there a way to buy up all the shares while earning lots of money instead of losing lots of money?

Yes. You can manipulate government policy.

Step 1: Buy all the shares that pay money if the Ceres government turns pro-Mars because otherwise space pirates would invade Ceres.

Step 2: Now that you're bought all those shares, you can profit by pivoting your foreign policy to be pro-Mars.

Taxpayers win. All they had to do was sell their government to the highest bidder.
# Using prediction markets to generate LessWrong posts

I have a secret to reveal: I am not the author of my recent LessWrong posts. I know almost nothing about any of the topics I have recently written about.

Instead, for the last two months, I have been using [Manifold Markets](https://manifold.markets/home) to write all my LessWrong posts. Here's my process.

To begin a post, I simply create 256 conditional prediction markets about whether my post will be curated, conditional on which [ASCII](https://en.wikipedia.org/wiki/ASCII) character comes first in the post (though I'm considering switching to [Unicode](https://en.wikipedia.org/wiki/Unicode)). Then, I go with whatever character yields the highest conditional probability of curation, and create another 256 conditional prediction markets for the next character in the post, plus a market for whether my post will be curated if the text terminates at that character. I repeat this algorithm until my post is complete.

Here's an example screenshot for a post that is currently in my drafts:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/7d8e98f8432239b68b8f0303c35eceffa7d19daf050d3efe.png)

Looks interesting!

Even this post was generated using this method.
# [Link] sona ike lili

Toki Pona is a language constructed by Sonja Lang, as foretold by linguistic determinism. It is famous for only having about 120 or so words, and yet still being able to express a wide variety of concepts. Today, I am announcing [*sona ike lili*](https://sonaikelili.substack.com) (lit. little bad knowledge), a blog where I will translate famous LessWrong articles into toki pona. Currently, there are two articles: [Scope Insensitivity](https://sonaikelili.substack.com/p/mute-li-ante-lili-e-pilin-jan) and [Taboo Your Words](https://sonaikelili.substack.com/p/o-kepeken-ala-e-nimi-sina).
# April 2022 Welcome & Open Thread

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/faq#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/faq). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lesswrong.com/tag/open-threads?sortedBy=new). The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# How can a layman contribute to AI Alignment efforts, given shorter timeline/doomier scenarios?

Versions of this question have probably been asked before, but MIRI at least is getting more pessimistic and I’ve seen Eliezer express multiple times that he doesn’t know what actually useful advice to give people who aren’t in the field yet.

I don’t want the world to end, but I am only a decently intelligent person of medicore competence. I could try to read and grok alignment research until I can have productive thoughts, but I do not anticipate that helping (though I will probably start doing more reading anyways; I’m tentatively planning to try reading Jaynes). Should I go into some kind of advocacy? I don’t know how I would do productive work there either, really. 

I would guess there are others in a roughly similar situation to mine. Does anyone have ideas?
# Inflection AI: New startup related to language models

Inflection AI ([website](https://www.inflection.ai/about.html), [announcement](https://greylock.com/portfolio-news/a-new-paradigm-in-human-machine-interaction/)) is a new startup founded by Mustafa Suleyman, Reid Hoffman, and Karén Simonyan. It appears they'll be doing something with language models. They're also looking to hire a [safety team](https://jobs.ashbyhq.com/inflection/d9ea657d-a27a-4b40-b84f-36980f9777eb); I bet they'll need to do RLHF as well as toxicity filtering, etc.
# Retrospective: Practical Social Networking

Practical Social Networking was the third Guild of the ROSE course. Taught by Councilor David Youssef, the course was an introduction to the art of growing and nurturing a healthy social web. While some people got a lot out of it, we also struggled with engagement and there were some missed opportunities.
# What words should be shorter in the rational dictionary?

I read some linguistics books a while back, and more recently I've been reading (against my better judgement) [Mad Investor Chaos and the Woman of Asmodeus](https://glowfic.com/posts/4582). So, I've been thinking about word length and concept importance a lot. 

**Question 1:**  Should we be picking one or two syllable words for phrases/concepts central to thinking better? Is the hit to interpretability (which LessWrong is not really optimised for as is) worth it? What other significant costs am I missing?

**Question 2:** Which phrases/concepts?
# Unfinished Projects Thread

Post about your unfinished projects, incomplete inventions, or not-so-crazy ideas here.

This is an easy-entry version of the old [Crazy Ideas Threads](https://www.lesswrong.com/search?terms=crazy+ideas). 

> in the *short run* giving LessWrong members and **lurkers** a stronger incentive than usual to write well-received stuff has the potential to be great for the site. 

I am posting this in the spirit of the [Good Heath Week](https://www.lesswrong.com/posts/meYCJHJQDDGh5nSN2/good-heart-week-extending-the-experiment), especially as a place for lurkers to offer their thoughts and maybe grow them.

Please post one project/idea per thread.
# Approach to Screen Time

Screens can be really engrossing, and learning moderation is pretty hard. This is one of a small number of places (along with staying up as late as they want and [eating as many sweets](https://www.jefftk.com/p/how-much-is-a-sweet) as they'd like) where an approach of letting our kids play with them as much as they want doesn't seem like it would be something that they were happy with in retrospect. The main issue I see here is in crowding out other things: when not on screens the kids do a lot of social, physical, and/or imaginative things that I think are pretty good for them. Not all screen time is equal from my perspective, either. If they're, say, getting better at reading I'm happy for them to do a lot more of that than if they're playing a video game fun alone.

I don't feel like we have the perfect approach here, and we've gone through a few different systems, but this is what we're currently using:

*   [Podcasts](https://www.lilywise.com/podcasts) and audio books: unlimited. I have a bunch of story tapes that my grandmother recorded for us, and Lily especially enjoys them. Let me know if you want a link.
    
*   [Messaging](https://www.jefftk.com/p/kids-ncurses-messenger), [blogging](https://www.jefftk.com/p/helping-the-kids-post): unlimited, though they don't do these very much.
    
*   Learning games: unrestricted before breakfast on weekends and between dinner and bedtime snack. If they think something (a program or a portion of a program) should count as a learning game they need to bring it to us and demonstrate that it is. Once a quarter all precedents are cleared and they need to bring games back to us, because otherwise you have "learning games" where they have mastered all of the learning portions and are just playing the games. This is primarily games like [ABC Mouse](https://www.abcmouse.com/) and [CodeSpark](https://codespark.com/), though Lily recently proposed that karaoke count (which I said it was fine as long as she wasn't doing songs she knew all the lyrics to).
    
*   TV and movies: movie day once a week (usually on a day with bad weather) with our [nanny](https://www.jefftk.com/p/childcare-iii) for about an hour and a half; occasional movies together as a family.
    
*   Anything else: fifteen minutes a day; unlimited during long car trips and airplane rides. If they are playing a game together (End it isn't just one of them watching the other one play) then they can pool their time and play for a half an hour.
    

I asked the kids what they thought of the rules:

*   Anna: they're good.
*   Lily: they're ok.

I asked what they'd do if they were a parent:

*   Lily: If I was a parent I would change the fifteen minutes to ten minutes. Screen time is kind of bad for kids. I also like having an hour and a half for movies, but I think maybe it's a bit much?
# What are some ways in which we can die with more dignity?

Eliezer's recent [kidding not kidding](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy) death with dignity post suggests that our chances of survival are so low that we should just focus on going out with some semblance of dignity (ie. "at least we made an attempt that wasn't truly pathetic).

Or at least that's what it seems to be claiming initially. If you read it to the end it becomes clear that Eliezer is proposing a frame for thinking about how to act in scenarios with a low probability of success[^0oln9j7hxes]. In particular, he seems to be criticizing the tendency of people to say "well we need to assume X because otherwise we would be doomed anyway" as the tendency is to power on multiple assumptions and end up focusing on and over specific-case. In contrast, Eliezer suggests that it is better to position yourself sucks that you would be able to leverage positive moral violations in general as then you aren't just betting on one particular scenario.

I often find that reframing a problem is quite conducive to producing solutions, so I'm asking how we could actually die with more dignity[^jzk36gyv7cl].

[^0oln9j7hxes]: I find it fascinating how post-rationalist this is. 

[^jzk36gyv7cl]: People could just post their thoughts on the original post, but this question serves as a nudge to actually attempt to generate solutions rather than to just grimly reflecting on it.
# A simple guide to life

![Jason's Simple Guide to Life](https://jasoncrawford.org/img/guide-to-life.png)

I first made a version of this chart [seven years ago today](https://twitter.com/jasoncrawford/status/584199198720921600). It’s worth a re-up.

The meaning of this chart is:

*   **Everything you do should be justified either by being inherently enjoyable, or by being important for some other purpose.** Absolutely minimize activities that satisfy neither of these criteria: things that are neither fun nor important. (This seems obvious, but think of how often it’s violated: online flame wars, doomscrolling and general overconsumption of news, long sob stories about trivial inconveniences, endless stewing over long-ago wrongs, etc.)
*   **Spend the vast majority of your time on things that are** ***both*** **enjoyable** ***and*** **important,** such as (hopefully) career and family. Some time on chores, taxes, etc. is unavoidable. Some time on games and diversions is fine. But both should be small relative to the big, meaningful, deeply rewarding things.

(And just to anticipate one reaction: if you *enjoy* arguments on the Internet, then they can go under “fun and games”.)

It’s not a *complete* guide to life, but it’s important and something I apply often.
# Should we push for banning making hiring decisions based on AI?

There are fast takeoff scenarios but there are also scenarios where AI slowly take more and more power in society till they have enough society that other stakeholders lack the power to move against AI agents. If we want to prepare for such a world it might be important to reduce the amount of power of AI within organizations and increase the amount of power that humans have within existing organizations.

Amazon gets criticized for [firing people based on algorithmic decisions](https://www.theguardian.com/commentisfree/2021/jul/05/amazon-worker-fired-app-dystopia) without a human making the decision. Would it be valuable to pick this fight to argue for human decision making when it comes to allocating power (the nature of what hiring decisions are about)?
# A Word to the Wise is Sufficient because the Wise Know So Many Words

# Collect Ontologies

An **ontology** is a way of bucketing reality. For example, Russians distinguish голубой from синий whereas Anglos bucket both into ["blue"](https://www.lesswrong.com/posts/DJJPWZAtwcuEj5WQc/blue-is-arbitrary-1). An ontology is an implicit set of priors about the world. If you never bucket people according to skin color then you will be bad at predicting who prefers chopsticks over forks. If you always bucket people according to skin color then you will miss out on human universals.

It takes an untrained neural network a long time to distinguish poisonous snakes from nonpoisonous snakes. It will take you much less time if you treat coral snakes and milk snakes as separate species, even though they look similar to each other.

Different ontologies are appropriate for different contexts. Zero-sum models of the world work great when planning adversarial strategies. Zero-sum models of the world are counterproductive when attempting to put together a family dinner. The more different ontologies you know, the the faster you can process new data.

The more ontologies you know the more information it takes you to distinguish between them. In practice, it doesn't matter because the entropy required to distinguish ontologies is so tiny[^1].

[^1]: More precisely, [distinguishing between $n$ ontologies takes little training data entropy relative to the entropy required to operate within an ontology with $n$ tunable parameters](https://www.lesswrong.com/posts/ZWcE8c6KyrcTZ7qHb/hypothesis-space-entropy).

# The (Cheap) Price

How many ontologies can a person learn? Words aren't ontologies but I think the number of words a person knows provides a reasonable Fermi estimate of the number of ontologies bouncing around our heads. A normal person knows perhaps 32,768 words. If we treat each word as a distinct binary ontology of equal prior probability then it takes 15 bits of entropy to figure out which ontology to use.

The entropy of English text is estimated at 2.3 bits per letter. It takes approximately 6 letters to figure out what word a person is trying to say. Technically, this is just a roundabout way of saying the average English word is about 6 letters long but my point is even with tens of thousands of ontologies to draw from, it takes a tiny bit of entropy (on the order of one word) to find the needle in the haystack. A word to the wise is literally sufficient.

That's assuming all prior probabilities are equally probable. If we allow for unequal prior probability distributions then the median entropy cost is less than a single word. The more ontologies you learn, the smarter you become, because each ontology contains a heavy Bayesian prior you can apply to new situations you encounter. The extreme informatic efficiency of pre-learned ontologies is how you get good at [small data](https://www.lesswrong.com/posts/xkWEJPwEArWFKH5jM/small-data).
# Bayeswatch 6.5: Therapy

"What do you mean I need therapy?" said Vi.

"Our records indicate you may have suffered psychological trauma on one of your recent missions," said Eliza.

"Which mission?" said Vi.

"You know," said Eliza. She leaned close to whisper, "Bangui."

"I can neither confirm nor deny whether I have ever been to Bangui," said Vi.

"You're too young to deal with the psychological impact of the psyop weapon the Bangui cloners deployed," said Eliza.

"I can neither confirm nor deny who was too young for Bangui. If you want me to go to therapy for psychological trauma then tell me, in writing, where I received the trauma and why. Otherwise, how can I prevent it in the future?" said Vi.

"This isn't healthy. You should be on mandatory medical leave," said Eliza.

"Are you allowed to do that?" said Vi.

"Not without a reason," said Eliza.

"Is that all?" said Vi.

"There is one more thing. Miriam recommended you be promoted. All that remains is the psychiatric sign-off. I need to certify you're mentally stable enough for the job," said Eliza.

"Does that mean?" said Vi.

"Congratulations, Agent," said Eliza.
# Best non-textbooks on every subject

The best way to learn a subject is undoubtedly by reading a textbook on it. But I find textbooks a drudgery, and tend to give up after a couple of chapters.

On the other hand I don't need a deep broad formal knowledge in every subject. I often just want to know enough that I know what it's about, the broad questions in the topic, and how to learn more when I need to.

On the other hand popular books are easy to read, but often teach you about the subject, without actually teaching any of the subject itself. They're full of anecdotes about the founders of the field, and metaphors for what some of the fields are like, but at the end you may end up more misguided than you went in.

There are however the rare popular books that aim to actually give the reader useful knowledge, rather than the illusion of knowledge. For example Godel, Escher, Bach on logic and formal systems, Quantum Computing since Democritus on computer science and Who We Are and How We Got Here on ancient DNA.  
  
These examples vary hugely in how involved they are, their style, and how readable they are, but they all share one thing in common: none of them talk down to the reader - they all assume the reader is an intelligent person whose perfectly capable of understanding the topic, but might just be missing a lot of background knowledge.  
  
What other books do you know of like that?  
  
Ideally all answers should give the title of a single book, optionally with a brief description, and a set of bullet points describing what they liked and didn't like about the book.

I'm more interested in physical sciences than social sciences, since it's common in the social sciences to introduce a thesis in book form, so it's easy to find good quality non-textbooks. Meanwhile in the physical sciences most original research is done in research papers, and most pedagogical work in textbooks, leaving much poorer pickings for non-textbooks.
# Best informative videos on the Internet

Inspired by Yair Halberstadt's [similar question](https://www.lesswrong.com/posts/GiEdQ79EDMjggtBKf/best-non-textbooks-on-every-subject) about non-textbooks, I'm asking this question to see if people have any videos that they would want to recommend to others.

The criterion is that the videos must be informative or educational. I don't want a playlist of your favorite songs, but one of your favorite math talks would be fine.

Other than that, the answers can take any form: the videos can be recordings of talks, interviews, panel discussions or educational YouTube videos; they can be linked wholesale in a public playlist or listed individually, you can post a single video and give reasons why people should watch it, you can post an entire channel if you think all of their videos are worth watching, *et cetera*.

As an example of the kind of content I'm looking for, I'll recommend two YouTube channels that I think are quite underrated: [Richard Borcherds](https://www.youtube.com/channel/UCIyDqfi_cbkp-RU20aBF-MQ) and [Strategy Stuff](https://www.youtube.com/c/StrategyStuff).
# Google's new 540 billion parameter language model 

Google just announced a very large language model that achieves SOTA across a very large set of tasks, mere days after DeepMind [announced](https://arxiv.org/abs/2203.15556) *Chinchilla,* and [their discovery](https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models) that data-scaling might be more valuable than we thought.

Here's [the blog post](https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html), and here's [the paper](https://storage.googleapis.com/pathways-language-model/PaLM-paper.pdf). I'll repeat the abstract here, with a highlight in bold,

> Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model (PaLM). 
> 
> We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned stateof-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. **A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model.** PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies.