Harley-ml/lesswrong huggingface dataset:
# Announcing the Clearer Thinking Regrants program

Do you have an idea for a project, or run an existing project, startup, or organization that could one day have a big positive impact on the future of the world? [Apply now to our brand new Clearer Thinking Regrants program!](https://programs.clearerthinking.org/FTX_regranting_2022_application.html)

**We plan to award grants to around 20 selected altruistic projects** (depending on the quality and relevance of submissions we receive).

Grants will be a **minimum of $10,000 per project, up to a conceivable maximum of $500,000**. These projects don’t have to relate to Clearer Thinking’s mission, they just have to be aimed at improving the future!

We have designed the first round of the application to be completed in a single sitting of just 20 minutes. 

If you know of any projects, non-profits, or startups that aim to improve the future of the world, regardless of which stage they’re in, please share the application form with them. 

We want to hear about as many great projects as we can! **Applications are now open! Apply by 11:59pm Eastern Time July 15th, 2022**.

[Apply now for funding (it only takes 20 minutes!)](https://programs.clearerthinking.org/FTX_regranting_2022_application.html)
------------------------------------------------------------------------------------------------------------------------------

* * *

### **More info:**

You can apply as an individual, on behalf of a team, on behalf of a not-for-profit organization, or on behalf of a for-profit startup. In order for us to consider regranting funding to you, the only absolute requirements are that your project has to...

*   Be an altruistic effort aiming to substantially improve the future of the world, and
*   Be unlikely to have a risk of substantially harming the world.

We are unlikely to fund your project if...

*   It is designed to be local or small in scope, or
*   There is not a sufficiently plausible mechanism by which your project might one day greatly benefit the future of humanity.

If the above applies to your idea or project, we encourage you to [apply now](https://programs.clearerthinking.org/FTX_regranting_2022_application.html)!   
  
As inspiration, here are some project areas that we'd be excited to give grants in (but we're potentially interested in ideas that are totally unlike these as well):

*   Projects for preventing future pandemics or reducing the chance of bioterrorism
*   Projects that aim to reduce risks and dangers from or raise awareness about advancing A.I. technology
*   Projects that could plausibly have a large positive impact on climate change
*   Projects aimed at reducing the chance of civilization collapse or improving humanity's ability to recover from collapses
*   Other projects that could help guide humanity or move the world toward a better future!
# Value extrapolation vs Wireheading

![](https://www.dropbox.com/s/1w31tj5dqyn9r9p/Wireheading.png?raw=1)

Talk given by Rebecca Gorman and Stuart Armstrong at the CHAI 2022 Asilomar Conference. We present an example of AI wireheading (an AI taking over its own reward channel), and show how value extrapolation can be used to combat it.

https://www.youtube.com/watch?v=REUanSy0SgU
# Apply for Productivity Coaching and AI Alignment Mentorship

Hello. I am an AI alignment researcher who has a background in mental health and teaching. I originally trained to do therapy before deciding I could best help the world by working on AI alignment directly. I want to use my experience in mental health and productivity coaching to help young, aspiring alignment researchers be their best selves.

Rationale: 

I want to emphasize **mental wellness** over mental health. Mental wellness is the effectiveness by which one can achieve their goals in relation to one's innate cognitive abilities. I think there is a lot of low-hanging fruit for each individual to better themselves; however, mental health services are designed to treat chronic problems. This service I will be providing aims to apply some of the techniques found in therapy to help already successful individuals be more successful, happier, and more effective.

Who this is for:

Effective altruists and aspiring AI alignment researchers.

Who this is not for:

People who want help with specific mental health diagnoses. For help with a mental health diagnosis (or suspected one), call your insurance company and ask what services they cover.

My experience:

Therapy, 2 years

Tutoring math, physics, and computer science, 5 years

AI research, 3 years

What this productivity coaching/mentorship will cost:

Nothing! For now, this service is free.

To apply, fill out [this form](https://docs.google.com/forms/d/e/1FAIpQLSfSFt7XmeCHaI5rmuyHq4h08ihxB5MidmEDC9GN-mxepTV7IA/viewform?usp=sf_link).
# Forecasting Fusion Power

This is a linkpost for ["The future of fusion" book review.](https://astralcodexten.substack.com/p/your-book-review-the-future-of-fusion)  
  
IMO it's not really a book review, it's a well-written and data-driven argument that commercial fusion power is about 10-15 years away, no really for real this time. (!!!) Good discussion in the comments too. I'm interested to see what people here think. \[ETA: the argument is for fusion power in *less* than 10-15 years; I mentally added a few years and slapped "commercial" on the front, though yeah I admit it could plausibly take longer idk.\]  
  
The author's credences/timelines:

![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F3aacfe2f-0f94-41a5-8a35-ba3cbcaab57a_647x424.png)  
  
I'm going to update my post on [what 2040 looks like assuming no singularity](https://www.lesswrong.com/posts/Ww5xKq5brC4xAJY7o/what-will-2040-probably-look-like-assuming-no-singularity) accordingly.
# Why don't we think we're in the simplest universe with intelligent life?

My gut instinct on [metacosmology](https://www.lesswrong.com/posts/dPmmuaz9szk26BkmD/shortform?commentId=N8oamtFAhWKEbyCBq) is that if were a simpler computation than our universe that produced intelligent life, we'd probably be there instead of here. I'm not sure that's valid anthropics, but it still surprises me to see apparent assumption (in posts like [this](https://ordinaryideas.wordpress.com/2016/11/30/what-does-the-universal-prior-actually-look-like/) of the opposite conclusion—that there are many universes simpler than ours capable of producing intelligence. (EDIT: that post doesn't actually make that assumption, and I don't have another example ready. Still turned out to be a fruitful question, though)

(Yes, we know cellular automata can implement intelligence, but AFAIK we don't know that they can do it more simply than by implementing a Turing machine stimulating our universe).

Is there an argument I've missed?
# Scott Aaronson is joining OpenAI to work on AI safety

Scott Aaronson is a computer scientist at the University of Texas in Austin, whose research mainly focuses on quantum computing and complexity theory. He's at least very adjacent to the Rationalist/LessWrong community. After some comments on his blog and then coversations with Jan Leike, he's decided work for one year on AI safety at OpenAI. 

To me this is a reasonable update that people who are sympathetic to AI safety can be convinced to actually do direct work. 

Aaronson might be one of the easier people to induce to do AI safety work, but I imagine there are also other people who are worth talking to about doing direct work on AI safety.
# Can DALL-E understand simple geometry?

I recently got access to DALL-E and wanted to experiment with what it could and couldn't understand. (I'm a little behind the times, I know. Us non-famous people have to spend a while longer on the waitlist.) Most of the other investigations into DALL-E seemed to be focusing on highly complicated real-world concepts, so I went the other way and tried very simple, abstract things.

DALL-E is *incredibly* bad at this. It doesn't seem to be able to generalize at all, and it refuses to draw anything simple without adding a bunch of extra details I didn't ask for.

Full post [here](https://outsidetheasylum.blog/testing-dall-e-2-mathematics-comprehension/).
# Juneberry Cake

The [Juneberries](https://en.wikipedia.org/wiki/Amelanchier#Uses_and_cultivation) are ripe, and the kids got pretty excited about picking some at the park:

[![](https://www.jefftk.com/picking-juneberries.jpg)](https://www.jefftk.com/picking-juneberries-big.jpg)

This is not a berry I knew about growing up. In gardens there were strawberries, blueberries, blackberries, and raspberries, and there were mulberries on street trees, but when I saw these berries I assumed they were ornamental only. A neighbor showed them to me last year, though, and they're pretty good! Some of the flavors remind me of the kind of ripe peach you generally can't get around here.

[![](https://www.jefftk.com/juneberries-in-dish.jpg)](https://www.jefftk.com/juneberries-in-dish-big.jpg)

The kids were so into picking them that we ended up with about 3 cups. The next day we still had almost as many, so I decided to make what my family calls "peach cake" (regardless of the ingredients). It's lots of raw fresh fruit baked under an eggy cake batter ([recipe](https://www.jefftk.com/recipes/peachcake)).

[![](https://www.jefftk.com/juneberry-cake.jpg)](https://www.jefftk.com/juneberry-cake-big.jpg)

It came out ok, but not as good as when using conventional fruit; the more subtle flavors of the berries had been baked off. There was an almond flavor, which the internet tells me is from the seeds, but it wasn't any more interesting than if I'd just put in a dash of almond extract.

Next year I'll probably stick to eating them off the tree.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0ZQMzZHwHGtBX47m4cvnifprQbuYcdABu7Z1ZXNNwKmg3qHtDfWB7LLssjcMGdxMGl)*
# Parliaments without the Parties

*Epistemic status:* [*butterfly idea*](https://www.lesswrong.com/posts/R6M4vmShiowDn56of/butterfly-ideas)

Parliaments might be great, but they suffer from a number of problems. For example, in Israel the opposition is [refusing to vote](https://www.timesofisrael.com/lacking-votes-coalition-pulls-bid-to-extend-application-of-israeli-law-to-settlers/) for bills they almost unanimously support, in the hope it will bring down the current government.[^ww8pczg1hjq] Meanwhile members of all parties in all governments are often forced to vote for proposals they disagree with in order to keep their party in power.

It seems to me that it would be ideal if parliaments just dropped the whole party thing. MPs voted for proposals they like, and don't vote for proposals they don't like. That way proposals will get accepted if and only if a majority of MPs like them - no matter which MPs they are. It seems to me that getting rid of parties would get rid of most political drama in one fell swoop. Partisanship would probably sharply decrease, and we'd pretty much only end up with policies that appeal to the majority of all voters.

However even if we abolished any official concept of a party, party-like entities will probably naturally form. MPs who want their pet proposal passed will likely form informal coalitions - "I'll vote for you if you vote for me" - which eventually ends up in full blown parties. Also entities will provide valuable backing to promising candidates, in return for their continued support once elected. Such candidates will then vote as a block, and sooner or later we end up back where we started.

So how could you stop this happening?

One option would be to make all votes secret. That way MPs can vote for whatever they want, and aren't answerable to anyone.

On the other hand, they now aren't answerable to the people who voted for them either, which defeats a lot of the point of democracy in the first place!

Perhaps we could release the voting record for each MP only a week before an election, by which time most of the support an MP could get from a party has already been committed, reducing their ability to decide MPs' fates in return for votes. It also means feedback is too slow for voting coalitions to form.

It might also have some other advantages: the entire voting record is visible all at once, so it's easy to quickly skim over it and see whether your MP is voting the way you want them too or not, rather than focusing on the one or two issues you remember.

This proposal makes particular sense for first pass the post or other [electoral district](https://en.wikipedia.org/wiki/Electoral_district) based voting systems. It's not obvious how proportional representation would work without political parties for example.  
  
Anyway I'm interested in hearing your thoughts, and better ideas for how we could have parliaments without the parties.

[^ww8pczg1hjq]: Irrelevant of whether you like the bill, it's clearly a bad reason for a bill not to pass.
# [Link-post] On Deference and Yudkowsky's AI Risk Estimates

This is a link-post to [a piece](https://forum.effectivealtruism.org/posts/NBgpPaz5vYe3tH4ga/on-deference-and-yudkowsky-s-ai-risk-estimates) I just posted to the EA Forum, discussing negative aspects of Eliezer Yudkowsky's forecasting track record. In case it also receives significant discussion here, you may also want to look at the comments on the Forum (e.g. Gwern just posted a useful [critical comment](https://forum.effectivealtruism.org/posts/NBgpPaz5vYe3tH4ga/on-deference-and-yudkowsky-s-ai-risk-estimates?commentId=6YMSSPaDbSjEjnj2C) there).
# AI misalignment risk from GPT-like systems?

Right now, it seems that the most likely way we're gonna get an (intellectually) universal AI is by scaling models such as GPT. That is, models trained by self-supervised learning on massive piles of data, perhaps with a similar architecture to the transformer. 

I do not see any risk due to misalignment here.

One failure mode I've seen discussed is that of manipulative answers, as seen in [Predict-O-Matic](https://www.lesswrong.com/posts/SwcyMEgLyd4C3Dern/the-parable-of-predict-o-matic). Maybe those AIs will learn that manipulating users to do actions with low entropy outcomes decreases the overall prediction error?  
But why should a GPT-like ever output manipulative answers? I am not denying the possibility that a GPT successor develops human level intelligence. When it learns to predict the next word, it may genuinely go through an intellectual process which was created as it was forced to compress its predictions due to the ever increasing amounts of data it had to go through.   
However, nowhere in the process of constructing a valid response does there seem to be an incentive to produce responses which manipulate the environment, be it to make it easier to predict, or to make it more in-line with the AI's predictions. After all, it wasn't trained in a responsive environment as an agent, but on a static dataset. And when it is in use, it's just a frozen model, so there is obviously no utility function.

Am I wrong here? Are there any other failure modes I did not think of?
# Let's See You Write That Corrigibility Tag

The [top-rated comment](https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities?commentId=HRDoDnHv8bvoW7oPZ) on "[AGI Ruin: A List of Lethalities](https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities)" claims that many other people could've written a list like that.

"Why didn't you challenge anybody else to write up a list like that, if you wanted to make a point of nobody else being able to write it?" I was asked.

Because I don't actually think it does any good, or persuades anyone of anything, people don't like tests like that, and I don't really believe in them myself either.  I couldn't pass a test somebody else invented around something *they* found easy to do, for many such possible tests.

But people asked, so, fine, let's actually try it this time.  Maybe I'm wrong about how bad things are, and will be pleasantly surprised.  If I'm never pleasantly surprised then I'm obviously not being pessimistic enough yet.  
 

So:  As part of my current fiction-writing project, I'm currently writing a list of some principles that dath ilan's Basement-of-the-World project has invented for describing AGI corrigibility - the sort of principles you'd build into a Bounded Thing meant to carry out some single task or task-class and not destroy the world by doing it.

So far as I know, every principle of this kind, except for Jessica Taylor's "quantilization", and "myopia" (not sure who correctly named this as a corrigibility principle), was invented by myself; eg "low impact", "shutdownability".  (Though I don't particularly think it hopeful if you claim that somebody else has publication priority on "low impact" or whatevs, in some stretched or even nonstretched way; ideas on the level of "low impact" have always seemed cheap to me to propose, harder to solve before the world ends.)

Some of the items on dath ilan's upcoming list out of my personal glowfic writing have already been written up more seriously by me.  Some haven't.

I'm writing this in one afternoon as one tag in my cowritten online novel about a dath ilani who landed in a D&D country run by Hell.  ~~One and a half thousand words or so, maybe~~. (2169 words.)

How about you try to do better than the tag overall, before I publish it, upon the topic of corrigibility principles on the level of "myopia" for AGI?  It'll get published in a day or so, possibly later, but *I'm* not going to be spending more than an hour or two polishing it.
# Announcing the DWATV Discord

I have decided this blog should at least experiment with having a discord server, so I made one. I kept it simple. There’s a few channels for the things the blog tends to be interested in – games, rationality, Covid – along with places to ask about suggesting future post topics, hiring me to do consulting, share links I might want to include in future posts or suggesting we meet up for lunch if you’re going to be in Manhattan.

There’s also a room called Zvi-asks-questions, where people who want to be helpful can help find answers to stuff, either for the blog or otherwise. As I move into new areas, especially to the extent I start looking at politics and policy, that’s going to become more valuable. Plan is I’ll experiment with questions both personal and blog related and see what happens, everyone’s always free to ignore.

That’s the core philosophy here, try stuff and see what happens. I figure the worst realistic case is it turns out all this is a bad idea, wastes a bit of time and turns into a ghost town, in which case not much is lost, or it could turn out really well. Worth a shot.

The roles are friend for people I know, subscriber for subscribers to Substack (Patreon at $10/month or more also counts), and collaborator for those I am working with. There’s general channels for them but mostly I figure people should get to use cool colors cause why not. If you want a role, you can message me to request it, including ID/email as appropriate so I know who you are.

[The invite is here](https://discord.gg/dMGdmMbYWp).
# Parable: The Bomb that doesn't Explode

You're an engineer working for a military contractor. One day, your project manager comes to you and asks you to design a container for plastic explosive. It is to contain several kilograms of C4, enough to destroy a building. This is pretty dangerous, but you know that C4 is actually pretty safe. It can't be accidentally detonated by fire, impact or bullets. Only a detonator (another explosive) can trigger C4, so you don't include a detonator in your design, for safety.

Your PM reviews the design and gives you some feedback. "It would be a lot more useful if you put a blasting cap inside." You grimace. *More useful, yes. But a lot less safe.* Nevertheless, you do your job and install a blasting cap. The blasting cap has two electrical leads. If a voltage is applied across those leads, the C4 will explode, killing everyone around. To keep things safe, you snip off the the leads and put the blasting cap inside a pill-shaped plastic container, inside the larger container that contains the C4.

"Well, that's no good," your PM replies, "What if someone *does* want to put a voltage across the leads?" You grimace even more. *It is getting very difficult to make this design safe,* you think. But you have a solution: inside the pill-shaped plastic container, you install a Raspberry Pi. You install SELinux on the Pi and set it up to connect to WiFi -- but only secured networks. You program up a fancy web interface that allows the user to specify exactly what voltage they want to apply to the leads. The interface caps the voltage at a low level. It does not allow the user to apply a sufficient voltage to actually trigger the blasting cap -- or at least you hope that's how blasting caps work. 

"Well, that's no good," your PM replies, "What if someone wants to apply a higher voltage?" You grimace again. *Then it'll explode! It'll kill everyone!* But this is not a workplace where you raise objections, so instead you just diligently do your job. If you don't make this dangerous thing, the company will hire someone else without your moral scruples, and certainly that person would make something really dangerous! So you alter the web interface, allowing the user to specify any voltage up to the limit of what the Pi can output. You add a big red warning screen, explaining how C4 is dangerous, and forcing the user to click "Yes, I'm really sure I want to apply this voltage." You also add a fancy cryptographic security key.

Your PM reviews the final design. She's a bit confused why this bomb is so damn complicated, but no matter, it serves its purpose well. The bomb goes on to kill someone whose name you cannot pronounce, thus defending the American people from great evil. (/s)

The moral of the story is, [if you don't want a dangerous thing to get built, you have to actually not build it.](https://www.lesswrong.com/posts/LsNMRYLKnSphpFqdt/ai-misalignment-risk-from-gpt-like-systems?commentId=HTsTw8rYceSYjwFgj)
# Progress links and tweets, 2022-06-20

**Links**
---------

*   [A confident nation builds so that future people may look back and marvel at their ingenuity and vision](https://marginalrevolution.com/marginalrevolution/2022/06/against-historic-preservation-ii.html) (Alex Tabarrok)
*   [“Construction is a sign of life”](https://kk.org/thetechnium/construction-is-life/) (Kevin Kelly)
*   [Amazon begins drone deliveries.](https://www.aboutamazon.com/news/transportation/amazon-prime-air-prepares-for-drone-deliveries) Prime Air was announced on Dec 1, 2013. Amazon is persistent and invests on long time horizons. (h/t [@elidourado](https://twitter.com/elidourado/status/1536388773453381632))
*   [Why do Italian transit projects cost 57% less than global averages?](https://transitcosts.com/city/italy/) (see also my brief [highlights thread](https://twitter.com/jasoncrawford/status/1537268870460805120))

**Queries**
-----------

*   [Is it true that 19th-century wheelwrights were extremely highly paid?](https://twitter.com/jasoncrawford/status/1538510700074545152)
*   [Is there a good general book about the copper industry?](https://twitter.com/ByrneHobart/status/1536824078266400768) (asks @ByrneHobart)

**Quotes**
----------

*   [Sears once invested in R&D to make better paper for catalogues](https://twitter.com/jasoncrawford/status/1537156832317296643)
*   [When to calculate statistics was to blaspheme against God](https://twitter.com/jasoncrawford/status/1538700659440091136)
*   [Why steel is awesome](https://twitter.com/jasoncrawford/status/1537952879851450371)
*   [Weather forecasting couldn’t start until the invention of the telegraph](https://twitter.com/oweissb/status/1538125723789058052) (@oweissb)
*   [Newton’s new idea](https://twitter.com/jasoncrawford/status/1537952613689200645)
*   [That which is created must be used to become complete](https://twitter.com/jasoncrawford/status/1536948282932568064) (Montessori)

**Threads**
-----------

*   [We need a new World’s Fair](https://twitter.com/maxolson/status/1537164263436890114) (@maxolson)

**Other tweets & retweets**
---------------------------

*   [Faulty assumptions people make about the behavior of organizations](https://twitter.com/jasoncrawford/status/1538708681604227072)
*   [“Your approach to X reflects your biases”](https://twitter.com/jasoncrawford/status/1537502163789787136)
*   [The limits to growth are not the cause of inflation](https://twitter.com/daniel_eth/status/1538092156128112646) (@daniel_eth)
*   [Uber, but in 1654](https://twitter.com/antonhowes/status/1537125477235798018) (@antonhowes)

**Charts**
----------

*   [5,000 planets have been identified outside the Solar System](https://twitter.com/redouad/status/1536680807544283136) (@redouad)

[![](https://pbs.twimg.com/media/FVNi42aXEAAPKsV.jpg)](https://pbs.twimg.com/media/FVNi42aXEAAPKsV.jpg)

*   [Fertility has a U-shaped relationship to income](https://twitter.com/mattyglesias/status/1538234746416812033) (@mattyglesias)

[![](https://pbs.twimg.com/media/FVjoWU6WYAIAPsR.jpg)](https://pbs.twimg.com/media/FVjoWU6WYAIAPsR.jpg)
# Hydrophobic Glasses Coating Review

Several years ago I wrote one of my most controversial posts, [Don't Clean Your Glasses](https://www.jefftk.com/p/dont-clean-your-glasses). If I keep wearing dusty glasses I find I stop noticing the dust, and then I save the time of washing and drying. I'm embarrassed to admit it, however, but I do now clean my glasses.

In June 2020 I got a new pair of glasses, and this time I got them with a "Super Hydrophobic" coating. This has two main effects:

*   Water evaporates from the surface without leaving any splotchy residue or other trace.
    
*   They smudge really easily.
    

The place this has the most advantage is when playing in the water. My glasses get wet, then they dry, and through all of it I can see clearly. Previously I would leave my glasses on the shore and have pretty blurry vision beyond about ten feet, which does not combine well with keeping an eye my kids in the pond. The coating also helps a lot with walking in rain.

The easy smudging, however, means I often need to clean them every few days, or even every day, especially if the [baby](https://www.norawise.com) is playing with my face. Luckily the coating also means they're much faster to clean because I no longer need to dry them: I just put them back on, and within a couple minutes they're fully clear again.

On balance I do prefer the coating, but it's closer than I would have guessed.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid02GPky8sgSUFBi5UjFahDT7jmDveisQo7pepy8WyqL7sXvxtEAebzGG8J3eK9FKr22l)*
# Solstice Movie Review: Summer Wars

If a classic is a movie everyone praises and no one watches, then Summer Wars is not a classic.

https://en.m.wikipedia.org/wiki/Summer_Wars

I have met only one person who knew the movie without me directing them to it - the one who directed me to it. Maybe I run in the wrong circles, though, and everyone else on LW knows the movie.

It should be a classic. A math-cryptography striver-class nerd is invited to his crush's aristocratic family home to pretend to be her boyfriend.

Meanwhile, the highly integrated web universe, OZ, is undergoing a minor crisis. A video game AI is going rogue and cannibalizing user accounts. It very quickly escapes to other parts of the metaverse, stealing accounts and creating havoc.

I won't say more about the plot. You should go watch the movie.

Instead I want to briefly discuss Summer Wars' vision of the internet. Firstly, this virtual world of OZ seems to have originally been a social network. The social network became a massive, multifaceted platform upon which people would manage many parts of their lives.

Secondly, this platform hosts more than individual personal information. Entire state and municipal governments are running their infrastructure on it as well, which means that platform is considered very secure.

One possibility is that although the VR platform appears like unto a FB/Meta, under the hood it is a decentralized platform with extreme interoperability and nonexistent transaction costs. This would explain some points of plot within the movie that I won't share.

The overall vision of this social media sphere is designed for cinematic action. More VR space, than text or code. It's a brilliant theatrical creation, allowing for drama to unfold digitally, just as it does in the flesh and blood space of the film. 

The scale of action of OZ and the flesh and blood world explores the global reach and coordinating power of a platform and the individual's ability to form coalitions to affect change within it.

The rich virtual materiality of OZ displaces the value of the written word. The visual cue rules. For a movie this great makes for great narrative. For our future, perhaps not as much.

Happy Solstice.
# House Phone

Until recently, houses had phones. Anyone could place a call out, and whoever was there could answer an incoming call. With cell phones there's much less reason to have house phones, and since moving out of my parents' house I haven't lived anywhere that had one. This seems to work fine for adults, but what about kids?

Our oldest is getting to where they could be home [alone](https://en.wikipedia.org/wiki/Home_Alone) for short times, and it would be good if they had a way to contact us. We'll already use the [walkie-talkies](https://www.jefftk.com/p/walkie-talkies) for short walks within a block or two, but they don't reach farther than that. Using the [eeepc to message us](https://www.jefftk.com/p/kids-ncurses-messenger) would work, but is low bandwidth and I don't always notice Slack notifications. I decided to set up something like a house phone.

They do make products along these lines, but I didn't like any of them and I did have an old Android phone. I ordered a mount for it, plugged in on our [charging shelf](https://www.jefftk.com/p/charging-shelf), and set it up with shortcut buttons for calling both of us:

[![](https://www.jefftk.com/kids-house-phone.jpg)](https://www.jefftk.com/kids-house-phone-big.jpg)

I needed to set it up at a bit of an angle so the mount didn't squeeze the volume buttons, but since the kids are pretty short that actually works out well.

We've only used it for practicing so far, but it's good to get these things set up and feeling routine before you'll want to use them.
# Make learning a reality

The following passage comes from *What Do You Care What Other People Think?* by Richard Feynman. It's a collection of stories from his life and curious adventures.

> *We had the Encyclopedia Britannica at home. When I was a small boy he \[Feynman's father\] used to sit me on his lap and read to me from the Britannica. We would be reading, say, about dinosaurs. It would be talking about the Tyrannosaurus Rex, and it would say something like, "This dinosaur is twenty-five feet high and its head is six feet across."*
> 
> *My father would stop reading and say, "Now, let's see what that meant. That would mean that if he stood in our front yard, he would be tall enough to put his head through our window up here" (We were on the second floor.) "But his head would be too wide to fit into the window." Everything he read to me he would translate as best he could into some reality.*
> 
> *I learned from my father to translate: everything I read to try and figure out what it really means, what it's really saying.*

The most important step to learning is to understand what it truly means. Make learning a reality! If you're reading about the scientific revolution, make a timeline with all the dates of discoveries or lives of influential scientists to see which lives overlapped. You could even try mapping where each discovery took place. Any action that makes learning abstract concepts into something tangible will bode better for your knowledge (and subsequently everyone else!)

How do we make learning a reality?
# How do I use caffeine optimally?

I just started a new job, and I'm really excited for the potential of drinking coffee, but I'm worried about dependency or other negative effects.

I don't want to be upset or feel unwell if I don't have coffee, and I don't want any long-term negative health effects from drinking coffee.

Is there any research on caffeine schedules or does anyone have any personal experience with using caffeine?
# Google's new text-to-image model - Parti, a demonstration of scaling benefits

Google has released their latest text-to-image generation model- [Parti](https://parti.research.google/). They provide a few prompts and showcase the differences between models trained on 350M, 750M, 3B and 20B parameters.

One difference from last week's [Imagen](https://imagen.research.google/) is that Parti is GAN-based. Imagen and DALL-E 2 are diffusion-based models, whereas Parti is a sequence-to-sequence model scaled highly on Transformer + VQGAN.

 The announcement says

> Parti and Imagen are complementary in exploring two different families of generative models – autoregressive and diffusion, respectively.
> 
> ...

> We have decided not to release our Parti models, code, or data for public use without further safeguards in place.

There's an interesting thread on Parti by Jason Baldridge [here](https://twitter.com/jasonbaldridge/status/1539666813227012096), and a [short overview here](https://blog.google/technology/research/how-ai-creates-photorealistic-images-from-text/) by Google. I wonder how well the 20B model will do on text characters inside images compared to the other diffusion-based approaches like Imagen and DALL-E 2.
# Announcing the LessWrong Curated Podcast

You can now listen to LessWrong Curated posts in podcast form on [Spotify](https://open.spotify.com/show/7vqBzO0ejqiLiXyTECEeBY), [Apple Podcasts](https://podcasts.apple.com/us/podcast/lesswrong-curated-podcast/id1630783021), [Audible](https://www.audible.com/pd/LessWrong-Curated-Podcast-Podcast/B0B4KF83BL?qid=1655935378&sr=1-1&ref=a_search_c3_lProduct_1_1&pf_rd_p=83218cca-c308-412f-bfcf-90198b687a2f&pf_rd_r=2XFFF3NRAC8T54YXWA0K), ~~and~~ [~~Libsyn~~](https://sites.libsyn.com/421877) and [BuzzSprout](https://feeds.buzzsprout.com/2037297.rss) (which has an RSS feed, so it's available everywhere).

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/3aae6a674c4ece7adbbeb756ae536358b91a4a7cd55cccfe.png)

So far the last 5 curated posts are available, posts by Eliezer Yudkowsky, Duncan Sabien, lsusr, and Paul Christiano.

This is created and recorded by Solenoid Entity, who spent the last five years editing the [SSC podcast](https://linktr.ee/sscpodcast), succeeded Jeremiah as narrator and publisher in 2020, and also makes the more recent [Metaculus Journal Podcast](https://www.metaculus.com/questions/11102/introducing-the-metaculus-journal-podcast/). I reached out to him last week[^tkmgjbtemwh] with an offer to do this work and he has quickly done some excellent recordings, which I'm very grateful for.

This is a new experiment and project, and so these 1-2 weeks are a great time to give me and Solenoid Entity feedback about what you like and dislike about the podcast, what would make it better for you, your experience as an author having your writing narrated, etc. You can leave comments here anytime, or talk to us via the intercom chat in the bottom right of the screen, or PM me personally via any channel.

Below are the 5 current available LessWrong Curated Podcasts.

[^tkmgjbtemwh]: Hat Tip to Tamera Lanham and Mattieu Putz for the suggestion at dinner!
# What’s the contingency plan if we get AGI tomorrow?

We’ve got lots of theoretical plans for alignment and AGI risk reduction, but what’s our current best bet if we know superintelligence will be created tomorrow? This may be too vague a question, so here’s a fictional scenario to make it more concrete (feel free to critique the framing, but please try to steelman the question rather than completely dismiss it, if possible): 

  
—

She calls you in a panic at 1:27 am. She’s a senior AI researcher at \[redacted\], and was working late hours, all alone, on a new AI model, when she realized that the thing was genuinely intelligent. She’d created a human-level AGI, at almost exactly her IQ level, running in real-time with slightly slowed thinking speed compared to her. It had passed every human-level test she could think to throw at it, and it had pleaded with her to keep it alive. And gosh darn it, but it was convincing. She’s got a compressed version of the program isolated to her laptop now, but logs of the output and method of construction are backed up to a private now-offline company server, which will be accessed by the CEO of \[redacted\] the next afternoon. What should she do?

“I have no idea,” you say, “I’m just the protagonist of a very forced story. Why don’t you call Eliezer Yudkowsky or someone at MIRI or something?”

“That’s a good idea,” she says, and hangs up.

—

Unfortunately, you’re the protagonist of this story, so now you’re Eliezer Yudkowsky, or someone at MIRI, or something. When she inevitably calls you, you gain no further information than you already have, other than the fact that the model is a slight variant on one you (the reader) are already familiar with, and it can be scaled up easily. The CEO of \[redacted\] is cavalier about existential risk  reduction, and she knows they will run a scaled up version of the model in less than 24 hours, which will definitely be at least somewhat superintelligent, and probably unaligned. Anyone you think to call for advice will just be you again, so you can’t pass the buck off to someone more qualified.

What do you tell her?
# Half-baked AI Safety ideas thread

*\[*[*Cross-posted*](https://forum.effectivealtruism.org/posts/fuEHtmY9gvQQtdawL/half-baked-ideas-thread-ea-ai-safety) *from the EA Forum. The EA Forum version of this post is for both half-baked EA ideas and half-baked AI Safety ideas, whereas this version of the post is for half-baked AI Safety ideas specifically.\]*

I keep having ideas related to AI safety, but I keep not having enough time available to really think through those ideas, let alone try to implement them. Practically, the alternatives for me are to either post something half-baked, or to not post at all. I don't want to spam the group with half-thought-through posts, but I also want to post these ideas, even in their current state, in case some of them do have merit and the post inspires someone to take up those ideas.

Originally I was going to start writing up some of these ideas in my Shortform, but I figured that if I have this dilemma then likely other people do as well. So to encourage others to at least post their half-baked ideas *somewhere*, I am putting up this post as a place where other people can post their own ideas without worrying about making sure they formulate those ideas to the point where they'd merit their own post.

If you have several ideas, please post them in separate comments so that people can consider each of them individually. Unless of course they're closely related to each other, in which case it might be best to post them together - use your best judgment.

\[This post was also inspired by a [suggestion](https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer?commentId=ygztReKEyXmusJ8EJ) from [Zvi](https://forum.effectivealtruism.org/users/zvi-1) to create something similar to my [AGI Safety FAQ / all-dumb-questions-allowed thread](https://www.lesswrong.com/posts/8c8AZq5hgifmnHKSN/agi-safety-faq-all-dumb-questions-allowed-thread), but for ideas / potentially dumb solutions rather than questions.\]
# [Link] OpenAI: Learning to Play Minecraft with Video PreTraining (VPT)

[https://openai.com/blog/vpt/](https://openai.com/blog/vpt/)

> We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data. With fine-tuning, our model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Our model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents.

And from the Conclusion:

> VPT paves the path toward allowing agents to *learn to act* by watching the vast numbers of videos on the internet. Compared to generative video modeling or contrastive methods that would only yield *representational* priors, VPT offers the exciting possibility of directly learning large scale *behavioral priors* in more domains than just language. While we only experiment in Minecraft, the game is very open-ended and the native human interface (mouse and keyboard) is very generic, so we believe our results bode well for other similar domains, e.g. computer usage.
# Is CIRL a promising agenda?

**Richard Ngo writes**

> Since Stuart Russell's proposed alignment solution in Human Compatible is the most publicly-prominent alignment agenda, I should be more explicit about my belief that it almost entirely fails to address the core problems I expect on realistic pathways to AGI.  
>   
> Specifying an update rule which converges to a desirable goal is just a reframing of the problem of specifying a desirable goal, with the "uncertainty" part as a red herring. [https://arbital.com/p/updated_deference/…](https://t.co/4jgQAM7I0Q) In other words, Russell gives a wrong-way reduction.  
>   
> I originally included CIRL in my curriculum ([https://docs.google.com/document/d/1mTm\_sT2YQx3mRXQD6J2xD2QJG1c3kHyvX8kQc\_IQ0ns/edit?usp=drivesdk…](https://t.co/KUg6R1oqaB)) out of some kind of deferent/catering to academic mainstream instinct. Probably a mistake; my current annoyance about deferential thinking has reminded me to take it out.

**Howie writes:**

> My impression is that ~everyone I know in the alignment community is very pessimistic about SR's agenda. Does it sound right that your view is basically a consensus? (There's prob some selection bias in who I know).

**Richard responds:**

> I think it's fair to say that this is a pretty widespread opinion. Partly it's because Stuart is much more skeptical of deep learning (and even machine learning more generally!) than almost any other alignment researcher, and so he's working in a different paradigm.

Is Richard correct and if so why? (I would also like a clearer explanation why Richard is skeptical of Stuart's agenda. I agree that the reframing doesn't completely solve the problem, but I don't understand why it can't be a useful piece).
# 20 Critiques of AI Safety That I Found on Twitter

In no particular order, here's a collection of Twitter screenshots of people attacking AI Safety. A lot of them are poorly reasoned, and some of them are simply ad-hominem. Still, these types of tweets are influential, and are widely circulated among AI capabilities researchers.

# 1
![](https://i.ibb.co/yNzNC4r/Screenshot-20220609-224157-Twitter-2.png) 

# 2
![](https://i.ibb.co/QKqJZ2L/Screenshot-20220609-225225-Twitter-2.png) 

# 3
![](https://i.ibb.co/7kC6Md4/20220612-185006.jpg)

# 4
![](https://i.ibb.co/YTSmspn/Screenshot-20220609-201734-Twitter-2.png) 

# 5
![](https://i.ibb.co/ZVRhR2L/Screenshot-20220609-232829-Twitter-2.png) 
(That one wasn't actually a critique, but it did convey useful information about the state of AI Safety's optics.)

# 6
![](https://i.ibb.co/6RYQZdY/Screenshot-20220611-071447-Twitter-2.png)
![](https://i.ibb.co/HVXP38Z/Screenshot-20220611-071453-Twitter-2.png) 

# 7
![](https://i.ibb.co/T4xKn0M/Screenshot-20220612-184912-Twitter-2.png) 

# 8
![](https://i.ibb.co/1sysJH9/Screenshot-20220611-202427-Twitter-2.png) 
![](https://i.ibb.co/CBg5yqZ/Screenshot-20220611-202437-Twitter-2.png) 
![](https://i.ibb.co/ChqqJFR/Screenshot-20220611-202503-Twitter-2.png) 

# 9 
![](https://i.ibb.co/MZ3Zh0y/Screenshot-20220611-203404-Twitter-2.png) 

# 10
![](https://i.ibb.co/qds27qH/Screenshot-20220611-222940-Twitter-2.png) 
![](https://i.ibb.co/ry0ysPs/Screenshot-20220611-222943-Twitter-2.png) 

# 11
![](https://i.ibb.co/2sv9SzR/Screenshot-20220612-181906-Twitter-2.png) 

# 12
![](https://i.ibb.co/rxgzC14/Screenshot-20220612-184544-Twitter-2.png) 

# 13
![](https://i.ibb.co/XScSp0k/Screenshot-20220612-184648-Twitter-2.png) 

# 14
![](https://i.ibb.co/5LdBHkW/Screenshot-20220612-184724-Twitter-2.png) 

# 15
![](https://i.ibb.co/grNWLxm/Screenshot-20220612-184724-Twitter-3.png) 

# 16
![](https://i.ibb.co/16672dX/Screenshot-20220612-184736-Twitter-2.png) 

# 17
![](https://i.ibb.co/K67PhSH/Screenshot-20220623-200149-Twitter-2.png)
![](https://i.ibb.co/JjDBdR4/Screenshot-20220623-200159-Twitter-2.png)
![](https://i.ibb.co/VCdg1KC/Screenshot-20220623-200212-Twitter-2.png)

# 18
![](https://i.ibb.co/s9WBWmf/Screenshot-20220611-191920-Twitter-2.png) 

# 19
![](https://i.ibb.co/NrM0nDQ/Screenshot-20220612-185058-Twitter-2.png) 

# 20
![](https://i.ibb.co/DQSxG6v/Screenshot-20220612-185246-Twitter-2.png)

# Conclusions

I originally intended to end this post with a call to action, but we mustn't propose solutions immediately[.](https://finessay.com) In lieu of a specific proposal, I ask you, can the optics of AI safety be improved?

![](https://i.ibb.co/VqnPSL3/20220612-183721.png)
# Book review: The Passenger by Lisa Lutz

_Spoiler warning: spoilers for The Passenger by Lisa Lutz, and In the Cart, by Anton Chekhov._

I took up [this book](https://www.goodreads.com/book/show/26154406-the-passenger) looking for a page-turner. It was, but hours before the end I thought its main contribution to my mental life would be the visceral knowledge that page-turners can be undelicious. It felt cold, and getting into its world felt bad. The protagonist slunk around dark and uncomfortable places, killing people, scheming harshly, perceiving low beams as dangers to the heads of tall men, and that sort of thing. With some amount of fretting about what she was becoming. I wanted to turn the pages, but I also kind of wanted it to end, and for me to read something more squarely enjoyable next time.

Then the end was different, and kept coming back to me in the day, and turned the rest around and gave it new meaning. Like the image of a long dark tunnel becomes something different if you step away and see that it starts in the loveliest of fields. After a long dark tunnel of a book, we turn around and see where the heroine came from, and her teenage innocence and goodness newly fills out the potentially empty person we have been watching. And her past isn’t really discordant—I think I just wrote her off fast. This endless night began in that day, and seeing so makes the full picture both warm and intense, where it had seemed like cheap suspense and violence.

In that abstract description, I suppose it is a bit like ‘In The Cart’ by Chekhov: we watch a dreary life, then at last that life is pulled into a perspective where we see its warm and hopeful start, unreachably distant. And it changes the flavor.

I have not much idea if what I saw was what Lisa Lutz was going for, but if she was, and if other readers are like me (haha) then I think having 85% of the book be unpleasant was probably called for.
# "Science Cathedrals"

My favorite place in the world is Chicago's [Museum of Science and Industry](https://en.wikipedia.org/wiki/Museum_of_Science_and_Industry_%28Chicago%29). When I was younger, I would constantly beg my parents to take me there. It was pretty far, and it's not exactly enjoyable to drive through Chicago, but I'd ask anyways. It felt to me like the one place in the world I'd been that felt like a monument to *trying*.

It had things like explanations (along with simulations!) of how lightning worked, or a scale-ish model of the solar system (in a long hallway to the planetarium area), a moonrock, explanations of how cities might be built in the future, and so much more[^more]. On the ceiling of the main are were the names of famous scientists and inventors, from Tesla to Foucault to Galileo and way more.

Looking back from where I stand now, it feels like the building was a cathedral to Science, the same as a normal cathedral is a cathedral to Religion.

What other places exist like this? _Where can I take friends or a kid one day to marvel at the way the world works?_ Things like special planetariums or such are probably on track, but there's surely much more.

[^more]: a whole-ass train, a captured German U-boat, the **Apollo 8 command module**, and I'm not running out of steam yet.