Harley-ml/lesswrong huggingface dataset:
# Trying to Keep the Garden Well

I used to live in city-subsidized housing. It was an apartment complex with a big garden.  
To enjoy the garden in the summer, residents started putting old chairs and a table outside. Once summer was over, they didn't bother to take them inside again. The table became weathered and didn't offer a pleasant sight anymore.  
The city noticed and put up a letter on the apartment board saying that having your own furniture in the garden was unorderly and threatened that any furniture that wasn't removed from the garden by the end of the month would be disposed of.

(You might want to quickly think about the long-run effects this letter had.)

In the following weeks, the residents dumped their old sofas, broken beds, televisions and, what-not into the garden. Normally, disposing of large items like this required you to have a car, drive the items to a disposal yard and pay a fee. However, the city had just made it free.

Or so the residents thought.

Of course, the city didn't think of following through with their threat anymore. Removing a couple of chairs and a table would have fit in a normal workday of a city janitor. Removing this pile of trash would have been costly.

The months went on, the sofas were wet, nobody used the garden anymore.  
I moved away soon after.
# Guidelines for cold messaging people

I just received a random email from a stranger in which they ask me 5-10 hard-to-answer questions on effective altruism, emotional health, and moral cluelessness that bother them.

  
Reflecting on this, I came up with a list of eight rules of thumb you might want to consider when you cold message someone. You might not want to address all of them in every message (because that contradicts no. 8), but they may be a good pointer towards what would make people keener to respond:

1\. Add a one-sentence introduction so they know how to interact with you.

2\. Make clear how you got hold of their email address.

3\. Make clear why they may want to talk to you.

4\. Make clear why you think they are the right person to solve your problem.

5\. Courteously emphasize that you don't mind if they are too busy to respond.

6\. Start or end with a note of appreciation or gratitude.

7\. ~~Ask for their permission \*before\* you dump a long list of questions or any other kind of work on them.~~ To minimize friction and time investment for them, add your list of questions to the first email right away. However, make clear that it is okay and encouraged for them to say \*no\*. For example, lead with a sentence like "Here are some questions I'm trying to answer for myself. Of course, no worries if you are too busy to help me with that."

8\. Keep the message as short and clear as possible.

Let me know if there's anything you find valuable that I overlooked.
# Speed Pasta Bake

Lily loves pasta bake: pasta, tomato sauce, and cheese baked until the cheese at the top is a bit crispy. Unfortunately, this needs about 45min in the oven. But, just like [with quesadillas](https://www.jefftk.com/p/two-minute-quesadillas), you can make it much faster by doing most of the heating in the microwave:

1.  Combine leftover pasta, jarred tomato sauce, and grated cheese in a dish and that is safe for both the microwave and the oven. Put additional grated cheese on top.
    
2.  Microwave until piping hot.
    
3.  Broil for about two minutes, or until the cheese is browned and crispy. This goes very fast, and the window between "just right" and "burnt" is small.
    

It ends up almost identical to the oven-only version, with slightly more active time and far less wall time.

I wonder why I don't see more recipes like this, which use the microwave for the "get things hot" phase before switching to a pan or oven for browning?

I forgot to take a picture before serving, but here's what it looks like after Lily had two helpings:

[![](https://www.jefftk.com/quick-pasta-bake.jpg)](https://www.jefftk.com/quick-pasta-bake-big.jpg)
# High-Impact Careers for Mathematicians (Online Workshop by EA Cambridge)

At Effective Altruism Cambridge we are organising an online careers workshop, specifically designed for maths students who might be interested in doing something useful with their quantitative skills.    
  
Full info is on the Facebook event [https://fb.me/e/2di6G25gC](https://fb.me/e/2di6G25gC) and in the registration form [https://forms.gle/31ovA6XWtriWXLwx5](https://forms.gle/31ovA6XWtriWXLwx5), but in short: 

1.  No prior knowledge of EA is assumed, so a very short introduction to EA, in particular to the 80,000 Hours framework, will be given at the start
2.  Talks + Q&As on different career paths that use maths by 4 external speakers (all were maths majors):
    1.   [Neel Nanda](https://www.neelnanda.io/about) covering AI Safety
    2.  [Oliver Crook](https://olivercrook.co.uk/) covering Quantitative Biology
    3.  Hazel Browne (MIT Economics PhD student) covering Global Priorities Research
    4.  Callum McDougall (Jane Street trader) covering Earning to Give in Quant Finance
3.  No need to be a maths student or a Cambridge student.

We expect most of the impact to come from introducing EA to students with little prior familiarity with EA. However, on the margin, it would also be helpful to those more familiar with EA, since they would benefit from the interaction speaker that they might otherwise not get. Hence, **if you run an EA student group, it'd be amazing if you could share it; or if you're a student in a quantitative subject sharing in subject uni-wide group chats would be really high value. **

In the past, we've had really great feedback on these (we run two similar ones last academic year): achieving a 3~fold increase in sign-ups, attendance and retention compared to our general career workshops.

It may well be the last time in a while that it makes sense for us to run these (most Cambridge maths students will have already had a chance to attend one), especially online (as there is less interest for online events now).   
  
P.S: 

-If you may be interested in running a similar workshop at your university, please get in touch.

-If you may be interested in speaking in future similar workshops, please do really get in touch. 

-After the workshop, I will write a follow-up post (here or on the EA forum) about our experience running these, since I think other groups should run similar ones.
# The Structure of the Guild of the Rose

This is a link post for [The Structure of the Guild](https://guildoftherose.org/articles/structure-of-the-guild), part one of a series of ongoing retrospectives on the progress of the **Guild of the Rose**, an organization for rationalist education and community.
# Descriptive vs. prescriptive optimism

*\[Cross-posting this essay from 2020 by request.\]*

I am not an “optimist”. Well, not exactly. Though I sing the praises of progress in these essays, and though I paint a [bold vision for the future](https://rootsofprogress.org/the-plight-of-the-poor), I have hesitated to apply that term to myself. I have said that I am “fundamentally” optimistic, or a “[paranoid optimist](https://twitter.com/jasoncrawford/status/602159946029903873)”; or I have reached for constructions like “[short-term pessimist, long-term optimist](https://twitter.com/jasoncrawford/status/1263525673554939906)”. But none of these are quite right.

In my [discussion with Tamara Winter and Trevor McKendrick](https://www.youtube.com/watch?v=Z8G1Tzi5gps) on optimism, the three of us converged on a formulation that I think resolves the paradox: descriptive vs. prescriptive optimism.

*Descriptive optimism* is the expectation that good things will happen, that an outcome will be positive, that the trend is in the right direction. It is a belief about the world.

*Prescriptive optimism* is the decision to work to make good outcomes happen. Whether lighthearted or desperate, cheerful or grim, it is a commitment to action and effort.

Descriptive optimism, for me, is highly contextual. It is situation-specific. I am mildly optimistic about our efforts against coronavirus. I am fairly pessimistic about US politics.

Prescriptive optimism is deeply philosophical. It is a belief about the nature of life, intelligence, and agency. It is a moral attitude, a way of living.

Descriptive optimism is determined by the external facts, but prescriptive optimism is an internal choice.

Descriptive optimism on its own can lead to complacency, Panglossianism, and other cavalier attitudes—progress as coasting. Prescriptive optimism calls for boldness, courage, and vigorous effort. When the two are combined, they call for expansive, ambitious plans. When prescriptive optimism is combined with descriptive *pessimism*, they together call for grit, determination, and fighting spirit.

The French writer and Nobel laureate Romain Rolland captured it in a passage later echoed by the Italian Marxist Antonio Gramsci, when he [wrote](https://publishing.cdlib.org/ucpressebooks/view?docId=ft538nb2x9&chunk.id=d0e6265&toc.id=&brand=ucpress) of

> this intimate alliance—which for me makes the true man—of **pessimism of the intelligence, which penetrates every illusion, and optimism of the will.** It is this natural bravery that is the flower of a good people, which “does not need to hope to undertake and to succeed to persevere,” but which lives in struggle over and above suffering, doubt, and the blasts of nothingness because his fiery life is the negation of death.

This has been distilled to a motto: “pessimism of the intellect, optimism of the will”. I am a pessimist or optimist of the intellect according to facts on the ground, but I am ever an optimist of the will.
# List of Probability Calibration Exercises

I recently spent a while looking at how different people had designed their probability calibration exercises (for ideas on how to design my own), and they turned out to be quite difficult to find. Many of the best ones were the least advertised and hardest to locate online. I figured I'd compile them all here in case anyone else finds themselves in a similar position. Please let me know about any I missed and I'll add them to the post. Many of these are old and no longer maintained, so no guarantees as to quality.

[https://bayes-up.web.app/](https://bayes-up.web.app/)

[http://acritch.com/credence-game/](http://acritch.com/credence-game/)

[http://confidence.success-equation.com/](http://confidence.success-equation.com/)

[https://calibration-practice.neocities.org/](https://calibration-practice.neocities.org/)

[http://web.archive.org/web/20100529074053/http://www.acceleratingfuture.com/tom/?p=129](http://web.archive.org/web/20100529074053/http://www.acceleratingfuture.com/tom/?p=129)

[http://credencecalibration.com/](http://credencecalibration.com/)

[https://programs.clearerthinking.org/calibrate\_your\_judgment.html](https://programs.clearerthinking.org/calibrate_your_judgment.html)

[https://www.openphilanthropy.org/calibration](https://www.openphilanthropy.org/calibration) or [https://80000hours.org/calibration-training/](https://80000hours.org/calibration-training/) (Different URLs for same application.)

[https://calibration.lazdini.lv/](https://calibration.lazdini.lv/)

[http://www.2pih.com/caltest/](http://www.2pih.com/caltest/)

[http://web.archive.org/web/20161020032514/http://calibratedprobabilityassessment.org/](http://web.archive.org/web/20161020032514/http://calibratedprobabilityassessment.org/)

[https://predictionbook.com/credence_games/try](https://predictionbook.com/credence_games/try)

[https://calibration-training.netlify.app/](https://calibration-training.netlify.app/)

[https://play.google.com/store/apps/details?id=com.the\_calibration\_game](https://play.google.com/store/apps/details?id=com.the_calibration_game)

[https://www.metaculus.com/tutorials/](https://www.metaculus.com/tutorials/)

[https://outsidetheasylum.blog/probability-calibration/](https://outsidetheasylum.blog/probability-calibration/)

[https://peterattiamd.com/confidence/](https://peterattiamd.com/confidence/)

[http://quantifiedintuitions.org/calibration](http://quantifiedintuitions.org/calibration)
# 5x5 Go Board

Lily and I have been playing some [5x5](https://senseis.xmp.net/?5x5Board) [Go](https://en.wikipedia.org/wiki/Go_(game)). We were using rubber bands to mark off a corner of a regular board, but I wanted a real one. It doesn't seem like anyone sells these, so I decided to make one:

[![](https://www.jefftk.com/playing-5x5-go.jpg)](https://www.jefftk.com/playing-5x5-go-big.jpg)

I found someone else's [writeup](https://www.jefftk.com/p/more-writeups) of [how they made a Go board](https://www.bamsoftware.com/go/board.html), and took a similar approach.

Instead of using [MDF](https://en.wikipedia.org/wiki/Medium-density_fibreboard), I used some of the three-quarter inch plywood I [keep on hand](https://www.jefftk.com/p/pantry-staples-for-diy). I cut a rectangle 6" by 6.46", following the Japanese ratio:

[![](https://www.jefftk.com/cutting-go-board.jpg)](https://www.jefftk.com/cutting-go-board-big.jpg)

I sanded it, progressively down to 220 grit:

[![](https://www.jefftk.com/sanding-go-board.jpg)](https://www.jefftk.com/sanding-go-board-big.jpg)

I sketched the initial set of lines in pencil:

[![](https://www.jefftk.com/preparing-go-board-lines.jpg)](https://www.jefftk.com/preparing-go-board-lines-big.jpg)

I drew the lines in sharpie. It would've been a bit nicer to use a finer point, but this was what I had.

[![](https://www.jefftk.com/go-board-lines-drawn.jpg)](https://www.jefftk.com/go-board-lines-drawn-big.jpg)

After two coats of polyurethane, it's ready for play:

[![](https://www.jefftk.com/finished-5x5-go-board.jpg)](https://www.jefftk.com/finished-5x5-go-board-big.jpg)
# Hyperpalatable Food Hypothesis: A LessWrong Study?

In Eliezer Yudkowsky's [recent post](https://www.lesswrong.com/posts/kjmpq33kHg7YpeRYW/briefly-radvac-and-smtm-two-things-we-should-be-doing) he discussed SlimeMoldTimeMold’s research into possible causes of obesity, and how he thinks SMTM’s theories are more convincing than the Hyperpalatable Food Hypothesis (HFH). SlimeMoldTimeMold theorizes that some kind of contamination is more likely, potentially lithium contamination in water. His work can be found here: [https://slimemoldtimemold.com/2021/07/07/a-chemical-hunger-part-i-mysteries/](https://slimemoldtimemold.com/2021/07/07/a-chemical-hunger-part-i-mysteries/) 

Matthew Barnett's [comment](https://www.lesswrong.com/posts/kjmpq33kHg7YpeRYW/?commentId=HSQyDgGJ6HrJQzxTK) supporting the hyperpalatable food hypothesis was strongly upvoted, so I’m interpreting that as indicating a decent amount of support for it among readers here. Personally, I find the HFH the most compelling of the existing theories. Why I’m posting is I think we could run our own study to test this hypothesis.  

As a group, we could have 3 months where we only eat meat, fruit, vegetables and spices. We track our weight, pool the data and see what happens at the end of it. 

Instructions would be along the lines of:

*   Eat only meat, fruit, vegetables and spices. 
*   Eat to satiation. Eat how much you feel like eating. 
*   Do not alter exercise habits greatly during these months if possible. Report if you do. 
*   Rather than record everything you eat, only record when you have broken the diet. Obviously, this diet will be very difficult to follow in certain settings, and we’ll all break it at some point. This is fine. Being allowed to break this diet for a few days at a time, or for a few meals here or there will be necessary. As long as you restart afterwards and record how much you deviated over the 3 month period, I think we'll still have a good sense of its efficacy. 

This would only make sense to do if enough people were interested, so I thought I’d make this post to see if anyone was. If it is something you'd be interested to participate in, please comment below or send me a message. 

Also, if you see any issues/improvements please comment.
# Is AI Alignment a pseudoscience?

Hi there, my background is in AI research and recently I have discovered some AI Alignment communities centered around here. The more I read about AI Alignment, the more I have a feeling that the whole field is basically a fictional-world-building exercise.

Some problems I have noticed: The basic concepts (e.g. what are the basic properties of the AI that are being discussed) are left undefined. The questions answered are build on unrealistic premises about how AI systems might work. Mathiness - using vaguely defined mathematical terms to describe complex problems and then solving them with additional vaguely defined mathematical operations. Combination of mathematical thinking and hand-wavy reasoning that lead to preferred conclusions.

Maybe I am reading it wrong. How would you steelman the argument that AI Alignment is actually a rigorous field? Do you consider AI Alignment to be scientific? If so, how is it Popper-falsifiable?
# Wireless Outlet Switching

We have lamps in our bedroom, and we were thinking it might be nice to wire up some switchable outlets. Ideally with a three-way switch, one at the doorway and one at our bed. Dimming would be nice too, though not a dealbreaker. This would be quite a lot of work, though, running new wires. And what if we wanted to move our bed at some point?

The modern solution would be [Internet of Things](https://en.wikipedia.org/wiki/Internet_of_things) smart outlets, as part of a smart home system. While it would be nice to be able to turn the lights on and off vocally, I don't want to deal with Wi-Fi issues and integration hiccups: I [just](https://www.jefftk.com/p/retrogrouch) want light switches.

It turns out this product also exists: switches that talk wirelessly with a box you put between your lamp and the wall. The switches can be battery powered, and the batteries last a long time because you aren't doing that much switching. I got two of these [GE "mySelectSmart" ones](https://smile.amazon.com/GE-mySelectSmart-Dimmable-Full-Range-37781/dp/B01N0TGKJ9?th=1) (~$25/each). I was a little bit worried about whether I would be control both sockets with both switches, but I followed the pairing process for each pair and it all works.

[![](https://www.jefftk.com/dumb-smart-lightswitch.jpg)](https://www.jefftk.com/dumb-smart-lightswitch-big.jpg)

They do what they're supposed to, and I don't have to think about them!

One downside of this setup is that the socket is always going to draw a little bit of power, even when off, because it needs to be listening for the wireless signal it tells it to turn on. But how big of an issue is this? I measured, and it draws 0.3W when off. Over a year that's ~2.6 kWh, or [$0.55](https://www.jefftk.com/p/rough-utility-costs). One of these would have to last ~45 years before the electricity cost caught up with the $25 initial purchase price. And installing hardwired switchable outlets wouldn't pay for itself, even if we kept the configuration static for a century.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100241517324182)*
# What should a student who is already a decent programmer do over the summer?

I'm majoring in computer science. However, I don't want to do a software engineering internship. Reasons:

1.  I'm already a decent programmer & have worked in big companies/startups/research lab-esque organizations. I definitely have a lot lot more to learn, but I'm not sure if doing software engineering (SWE) is the best use of my time.
2.  I'm not sure I want to be a SWE. I love programming, but it seems like the way to make the most impact in the world these days is to create an (actually meaningful) startup/organization. (I know, I know every young person says this these days).

Given that this is the case, I'm trying to figure out what to do this summer besides a normal tech internship. I've decided to ask forums like LW/Hacker News in order to get the advice of people who have lived a few more years.

Here are current options I've been considering:

1.  Try to understand virality -- I'm not particularly good at creating viral content or content that appeals to the masses (aside from the natural edge that comes from being young), which could certainly be useful when marketing a product. An option here is trying to intern at a company like Buzzfeed which is full of people who understand emotions / virality.
2.  Try to understand soft skills / business -- There are roles out there like "business development intern". I don't know anything about "business development" but negotiating with other companies / writing white-papers / etc. could be a useful skill
3.  Spend time learning on my own. To enact change these days, it's probably not sufficient to just know programming (unless you want to do a SASS). Spending time on my own learning physics/chemistry/biology could be useful. The issue here is I believe mentorship is useful and this option negates that completely.
4.  Working on my own business (not a software project). I've done this in the past & learned a lot about how to be scrappy. Not a terrible idea, but I don't want to sink 3 months into something only to have to return to school. Also negates mentor-ship. Also I'd rather spend time learning right now so I can execute on something that's not a local maximum of what my brain can imagine right now.

Anyone have thoughts?
# ELK First Round Contest Winners

Thank you to all those who have submitted proposals to the ELK proposal competition. We have evaluated all proposals submitted before January 14th[^1]. Decisions are still being made on proposals submitted after January 14th. 

[^1]: We don’t plan on spending time evaluating and awarding proposals made after February 15th but feel free to email elk@alignmentresearchcenter.org in the future if you think you have a promising proposal, and we might consider it for a prize. 


The deadline for submissions is February 15th, after which we will release summaries of the proposals and associated counterexamples.

We evaluated 30 distinct proposals from 25 people. We awarded a total of $70,000 for proposals from the following 8 people:
* $20,000 for a proposal from Sam Marks
* $10,000 for a proposal from Dmitrii Krasheninnikov
* $5,000 for a proposal from Maria Shakhova
* $10,000 for proposals from P, who asked to remain anonymous
* $10,000 for a proposal from Scott Viteri 
* $5,000 for a proposal from Jacob Hilton and Holly Mandel
* $10,000 for a proposal from Jacob Hilton

We would have also awarded $15,000 for proposals from Holden Karnofsky, however he is on ARC’s board and is ineligible to receive the prize.
# Aligned AI Needs Slack

(Half-baked)

Much has been said about [slack](https://www.lesswrong.com/tag/slack) on this site, starting with Zvi's seminal [post](https://www.lesswrong.com/posts/yLLkWMDbC9ZNKbjDG/slack). The point I couldn't find easily (probably missed) is that an aligned AI would need a fair bit of it. Having a utility function means zero slack: there is one thing you optimize, to the exclusion of everything else. And all precisely defined goals are necessarily Goodharted (or, in the DnD terms, munchkined). An AI armed with a utility function will tile the world (the whole world, or its own "mental" world, or both) with smiley paperclips. For an AI (or for a natural intelligence) to behave non-destructively it needs room to satisfice, not optimize. Optimal utility corresponds to a single state of the world among infinitely many, while adding slack to the mix expands the space of acceptable world state enough to potentially include those that are human-aligned. If an AGI is indifferent between a great many world states, it might well include some that would be acceptable to humanity and have no incentive to try to trick its creators. Not being an ML person, I have no idea how to formalize it, or if it has been formalized already. But figured it's worth writing a short note about. That is all.
# Event: Moral Foundations of Progress Studies, March 4–6 at UT Austin

I’m excited to announce a workshop on the Moral Foundations of Progress Studies.

The progress studies community has had a lot of discussion about technology, economics, history, and politics. However, there is no consensus on the moral basis for valuing or pursuing “progress,” and there are key open questions about how progress is to be judged and measured, who should benefit from it, and what type of progress we should pursue.

The goal of this workshop is to reach a consensus on what major moral/ethical questions are at the foundations of a study of progress, and what broad answers to these questions have been proposed. A few designated attendees will take notes and draft a short article afterwards summarizing the discussion. (We’re currently looking for the appropriate place to publish this; it may be in a journal or on a blog.)

[**Apply to attend here**](https://forms.gle/QhRTTcBhoKvSs9HE9)**.** Space is limited; we’ll be prioritizing people in or with a connection to academia, and public intellectuals who write about progress or adjacent topics.

**When:** March 4–6, 2022

**Where:** University of Texas at Austin

**Agenda (subject to change):**

Friday:

*   Survey of writers on progress (including Cowen, Deutsch, Pinker)
*   Theories of well-being
*   Panel: Steven Pinker, David Deutsch (via video)
*   Metrics and standards of value
*   Challenges to the claim that the last two centuries represent progress

Saturday:

*   Interrogating the idea of moral progress
*   Progress & safety (including the Precautionary Principle and existential risk)
*   Challenges in assessing possible futures

Sunday morning:

*   Wrap-up

Except for the Friday panel, each session will be a 90-minute discussion led by one or a small panel of participants who give a brief intro.

**Co-hosts:**

*   [Jason Crawford](https://jasoncrawford.org/), founder of The Roots of Progress
*   [Gregory Salmieri](https://salemcenter.org/people/gregory-salmieri/), director of the Program for Objectivity in Thought, Action, and Enterprise at the Salem Center at UT Austin

Attendees are being invited from the progress studies and Effective Altruism communities, plus moral philosophers familiar with the virtue-ethics and well-being literatures.

There is no cost to attend. We have some funding to pay for travel and lodging for a limited number of participants; you can request funding when you [apply](https://forms.gle/QhRTTcBhoKvSs9HE9).
# Vavilov Day Discussion Post

When I first [wrote up my plan](https://www.lesswrong.com/posts/puYfAEJJomeodeSsi/an-observation-of-vavilov-day) to fast in honor of Nikolai Vavilov and his institute staff, I conceived of it as basically a thing for myself, that might interest a few other people. It both got much more response than I expected, and more response (positive and negative) treating it as a bid for community structure rather than one of many things that someone, somewhere was doing. That wasn’t my goal and to be honest I feel trepidatious about the concept: I was going for a very specific thing and I’m simultaneously really happy it resonated with so many people and concerned about dilution

But the Vavilov Day concept clearly has enough energy behind it that I don't feel right blocking discussion. I continue to not want to host that discussion on a personal post about my own choices, so have created this post instead. 

Some things I think might be good to share here:

*   Your specific observation of Vavilov Day and how it felt (I was surprised a lot of people defaulted to a 36 hour fast; to the extent other people joined in I expected them to pick their own thing)
*   Feelings about the core concept and ontology
    *   Ideally keeping discussion about the larger patron saints/holiday anarchy discussion over on [that post](https://www.lesswrong.com/posts/SHokoBXaphaouyrES/patron-saint-holidays) instead.
*   What you’d like to be different or the same for you next year.

In my ideal world, this conversation is a bunch of individuals discussing actions and ideas to figure out what works best for them. But some of the pent-up discussion is about that attitude (vs. something more community built), and I don't get to declare my view correct by fiat. However I will ask that you keep that frame in mind and, if someone's responses don't make sense or seem oppressive, consider that this might be why.
# Arguments about Highly Reliable Agent Designs as a Useful Path to Artificial Intelligence Safety

This paper is a revised and expanded version of my blog post [Plausible cases for HRAD work, and locating the crux in the "realism about rationality" debate](https://www.lesswrong.com/posts/BGxTpdBGbwCWrGiCL/plausible-cases-for-hrad-work-and-locating-the-crux-in-the), now with David Manheim as co-author.

Abstract:

> Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of different approaches have made different and debated claims about the importance or usefulness of their work in the near term, and for future systems. Highly Reliable Agent Designs (HRAD) is one of the most controversial and ambitious approaches, championed by the Machine Intelligence Research Institute, among others, and various arguments have been made about whether and how it reduces risks from future AI systems. In order to reduce confusion in the debate about AI safety, here we build on a previous discussion by Rice which collects and presents four central arguments which are used to justify HRAD as a path towards safety of AI systems.
> 
> We have titled the arguments (1) incidental utility,(2) deconfusion, (3) precise specification, and (4) prediction. Each of these makes different, partly conflicting claims about how future AI systems can be risky. We have explained the assumptions and claims based on a review of published and informal literature, along with consultation with experts who have stated positions on the topic. Finally, we have briefly outlined arguments against each approach and against the agenda overall.

See also [this Twitter thread](https://twitter.com/davidmanheim/status/1480891707071905798) where David summarizes the paper.
# Mildly Photochromic Lenses?

After wearing glasses for about half my life, I decided to try [photochromic lenses](https://en.wikipedia.org/wiki/Photochromic_lens). These darken when exposed to sunlight, as if I had put on sunglasses. It's pretty neat!

[![](https://www.jefftk.com/glasses-dark.jpg)](https://www.jefftk.com/glasses-dark-big.jpg)

I got them in early summer, and I have mixed feelings. It was nice to be able to hang out in really bright sunlight without needing to squint or shade my eyes. This has never been something that bothered me very much, and I never used to wear sunglasses, but it was nice to have a bit less light.

On the other hand, as I wore them more, it was starting to feel a bit depressing. Colors were not as bright, and the world seemed less alive. Especially as we got farther into winter I was really preferring the view around the edges of my glasses. I decided to get an otherwise identical pair, but with regular lenses—it's pretty great how buying glasses online has made them so much cheaper that I can get an extra pair without feeling extravagant!

So far I like the regular glasses better, but I think I would prefer photochromics in really strong sunlight? Is there such a thing as mildly photochromic lenses, which stay clear on a typical day but darken in intense sunlight?

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100241863455532)*
# Infant Talk boxes?

I often play music with Nora (7M) sitting on my lap, and she loves to play with my [talk box](https://en.wikipedia.org/wiki/Talk_box) tube:

  
([youtube](https://youtu.be/8FppoY_xVds))

This is just something we do for fun, but it makes me wonder: could this sort of play help babies learn to talk sooner?

A talk box is a speaker with a tube that pipes its sound into your mouth. You use your mouth to filter the incoming audio, emphasizing some frequencies relative to others, and then you and others hear the modified sound. In music this can be used to make an instrument [appear to talk/sing](https://www.youtube.com/watch?v=vwf2uvWlL_c), or it can be used a bit like a [wah-wah](https://en.wikipedia.org/wiki/Wah-wah_pedal), to dynamically [adjust the tone](https://youtu.be/uB6RPefG6QI?t=135).

Now, this is very speculative and probably wrong, but here's my idea for how it might help a baby practice talking. What if most of the time in learning how to talk is learning how to shape the sounds with your mouth? Babies babble a lot, limited by some combination of vocal cord stamina and interest, but having some talk box time could let them get in additional practice?

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100242007716432)*
# Can we grow cars instead of building them?

Well that sounds like an exciting idea — imagine a car tree that grows on its own, and we need not do anything other than pick the ready-made cars off it. Perhaps we can breed faster models, or cross-breed to make new ones — in all seems like a lot less effort than having hundreds of engineers and designers spend hours planning out every detail. Ok, but what exactly do we mean when we think of growing cars, and how is it different from how they are currently built at manufacturing *plants?* (I f**ing love that “plant” is already the word we use for factories!!! I think there is some deep insight there)

![](https://cdn-images-1.medium.com/max/800/1*pX6x3Ns_BPLiLKS5kh3a2Q.png)

To dig a bit more into the distinction here, imagine this scenario: some aliens come to Earth that are so different from us that they cannot even identify humans as clearly distinct from the matter of our environment. Imagine that they don’t see us as distinct living organisms, let alone as individuals with unique personalities and interests, or anything like “consciousness.” They merely see lots of biochemical processes that lead to more biochemical processes, all intertwined, interacting, and producing some plastics. Some of these plastics combine with bits of metal into cars, as catalyzed by some more biochemical processes (which we call humans, but they don’t). From this perspective, a manufacturing plant will look to them precisely like a tree that grows cars, all on its own! Furthermore, if they tried to control what sort of cars it produces, they wouldn’t think to talk to workers and change design plans — instead they might try to cross-pollinate: bring two different manufacturing plants close together and see if they learn to produce some hybrid model of the cars.

Ok, I can see a few holes you can poke in this line of reasoning — please do so in the comments! Even so, I think the overall point holds: we can construct a reasonable “outside perspective” from which manufacturing plants will fit all the criteria above for “growing cars.” Do you buy this? What other distinctions do you see between “building” and “growing” cars that I missed in the above table? Finally, if we still want to invent a way to “grow cars” (from our perspective) what would be the main way we would know that we succeeded (our KPI)?   
I’ll continue this discussion next week, with some brainstorming (read: BS) of how we might actually achieve this in the future ;P
# South Bay ACX/LW Meetup

This is a relaxed Saturday afternoon gathering at Sylvan Park (attn regulars: different park than usual). If the weather threatens to be bad it'll get cancelled; this will be announced here the morning of.

**Location (verbose): **  
In the shade of the row of conifers just to the west of Sylvan Park parking lot. We'll have a ACX/LW sign held up by a red camping chair. [Link to the location of the meetup spot in the park.](https://www.google.com/maps/search/?api=1&query=37.381524,%20-122.062706)

**Things to maybe bring:**  
Some snacks will be available but you're welcome to bring some, too. We may run out of space on the picnic blankets so if you want to sit down you might want to bring your own. It'll likely be chilly so it probably makes sense to bundle up.
# Solar Snow Redirector?

[Three years ago](https://www.jefftk.com/p/solar-one-year-in) we put solar panels on the only part of our roof that got sunlight, which unfortunately mostly faces WNW. Since then, however, our neighbors have removed a large tree just east of us, and now the part of our roof that faces ESE gets lots of light. So I'm back to thinking about solar.

This roof is a good candidate for solar, with one problem: the panels might avalanche snow onto the neighbor's car:

[![](https://www.jefftk.com/solar-onto-car.jpg)](https://www.jefftk.com/solar-onto-car-big.jpg)

[![](https://www.jefftk.com/solar-onto-car-side.jpg)](https://www.jefftk.com/solar-onto-car-side-big.jpg)

The standard approach here is to put snow guards on the panels or the roof with the goal of holding back, slowing down, or breaking up the snow. In this case, however, I don't want the snow to stay on the roof and I don't mind it falling next to the house. I just need it not to fall out from the house, into the neighbor's driveway.

Is this a solved problem? Alternatively, is there something you can attach to the roof to redirect the snow straight down? Something like:

[![](https://www.jefftk.com/solar-snow-redirection-panel.jpg)](https://www.jefftk.com/solar-snow-redirection-panel-big.jpg)

Alternatively, people who've tried snow guards, how well do they work? Are they going to keep the snow from squashing the neighbor's car, and the neighbor?
# Help me understand the rationale of the NIH to not recommend Fluvoxamine for COVID treatment.

So the [TOGETHER trial](https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(21)00448-4/fulltext) signal boosted by Scott of slate star codex found Fluvoxamine to be effective at reducing 30% of COVID hospitalisation and fatality.

[The NIH looked at the study](https://www.covid19treatmentguidelines.nih.gov/therapies/immunomodulators/fluvoxamine/) and found it unconvicing, I am a bit confused as to the rationale.

I'll list it out as I understand it:

*   the primary outcomes \[retention in the emergency department for >6 hours or admission to a tertiary hospital\] was chosen without rationale
*   There was no significant difference in mortality between study arms in the intention-to-treat (ITT) population \[however, it's 2% in treatment arm and 3% in placebo arm as expected of the 30% reduction expectation\]
*   significant difference was only found in patients who have persisted in taking >80% of fluvoxamine doses, however there were also improved outcome for patients who have persisted in taking >80% of placebo dose, suggesting that another mechanism \[e.g. conscientiousness\] to be resposible for \[most? all?\] the improvement in outcome. 

Is my understanding correct and does NIH's critiques of the study hold merrit?
# February 2022 Open Thread

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/faq#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/faq). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lesswrong.com/tag/open-threads?sortedBy=new). The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Good books about overcoming coordination problems?

The questions I mostly frequently ask myself are "How does anything good scale?" and "How is anything good and large-scale sustained?"

The most trivial answer to this is "Markets and incentives!" which is not wrong, but also seems like an underwhelming answer. Markets and incentives do solve many coordination problems and have reduced large amounts of suffering. However, they also seem to have limits in terms of their predictive power. I would like to read books which discuss how cooperation can be leveraged when incentivized markets (free from rent-seeking!) are insufficient.

In "Mission Economy", Marianna Mazzucato claims cooperation flourishes outside of markets when the government needs to explicitly define missions. She uses The Apollo Mission as an example, but this answer felt slightly under-specified. By the end of the book, I couldn't distinguish a good "mission" from a bad one. Even worse, I can't even tell if a mission is being carried out badly, other than ham-fistedly comparing it to The Apollo Mission, even though there's a ton of vague prose spent on the topic.

What books should I read to get a better pragmatic understanding? Is there a “handbook for social change” regardless of scale?

Some candidates I'm currently considering:
- The Box: How the Shipping Container Made the World Smaller and the World Economy Bigger by Mark Levinson
- A historical book about water treatment, electricity distribution or libraries?
- A historical overview of unions. Supposedly unions have driven a number of beneficial labour reforms, but they also can result in stagnation and corruption?
# Practical Decision-Making Course: A Retrospective

This is a linkpost for [Practical Decision-Making Retrospective](https://guildoftherose.org/articles/practical-decision-making-retrospective), an article looking back on the successes and learnings from this course. This article is part two of a series of ongoing retrospectives on the progress of the **Guild of the Rose**, an organization for rationalist education and community.
# A modest spelling reform to increase autologicity, symmetry, and readability.

- o
- tw
- thr
- four 
- ffive
- ssssix
- ssseven
- eeeeight
- nnnnnnine

At this point, if one is working in a digital medium where one maybe have a blissful disregard for line length, one may happily continue onwards to tttttttten up past.. errmm.. tttttttten x tttttttten and beyond. However, if writing for the printed page, we may be force into a sadly more parsimonious form:

$$
t^{eeeeight}en \\
e^{ssssix}leven
$$
up to and progressively beyond 
$$
o\hspace{0.35em} h^{n^{e^{s^{s^{f^{f^{t^{t^{f^{nnnnnnine}ifteen}wenty  ffive}hirty ffive}ourty  ssssix}ifty  ssssix}ixty  four}eventy ffive}ighty ffive}inety four}undred. 
$$

# Appendix on negative numbers
Negatives are most naturally treated in this system by simply reversing their positive representations, thus:

- ruof
- rht
- wt

**If** *o* must be disambiguated from it's negation, positive *o* may be indicated as *.o* and negative as *o.* 

The denotation of the null quantity is still a somewhat controversial topic, with many advocating for *.*, but a vocal minority are still strong proponents of:
# Sledding Is Underrated

Sledding is pretty great:

*   No lines. You don't have to wait, just go.
    
*   Minimal cost. A basic plastic sled costs $15-$20 at a hardware store, and while they wear out pretty quickly it's still less than $5/outing. Aside from that, all you need is standard winter gear (including [snowpants](https://www.jefftk.com/p/snowpants)).
    
*   Close. Around here there are two decent places to sled within a 15min walk, and several within a 15min drive. You can go sledding after work, or as one of several activities on a Saturday.
    
*   Good learning curve. There are many hills where all you have to do is sit on your sled, no skill required. As you figure out how to steer, this opens up other hills, and more interesting options for going down existing ones.
    
*   Fast. Zooming down a hill is just really fun.
    

The competing ways to move similarly fast are much more expensive, require more travel, and you spend much more time time waiting around and less actually moving: amusement parks, skiing, etc. I'm not trying to knock those activities, they are definitely fun, but I think sledding more than holds its own here.

This morning the kids and I went with their aunt to a park near our house:

[![](https://www.jefftk.com/anna-sledding-robbins-farm-park.jpg)](https://www.jefftk.com/anna-sledding-robbins-farm-park-big.jpg)

We were there for about an hour and a half, and there were less than ten other people sledding. I'm sure there are more now, since it's a bit later in the day, but still, compared to the number of people in the area I think would enjoy sledding it's surprisingly low.

Why is that? Some guesses:

*   You have to walk back up the hill. True, but it's good exercise and it beats standing in line.
    
*   There isn't any advertising for it, beyond advertising for sleds.
    
*   In most places it can't be scheduled. You have to notice that the conditions are right and decide to go, typically on less than a days notice.
    
*   We think of it as something for kids. But amusement parks have this too and are still popular with all ages.
    
*   In places that have good sledding many times in the winter people think of it as a normal thing, and we get more excited about uncommon activities?
    
*   Maybe people don't know where to go? There are sites about the best sledding options, but they aren't great -- some sort of sledding hill review site where people describe hills and give ratings could be nice.
    

If you're in Boston and have been meaning to give it a try, conditions are good this weekend. I'd be happy to lend you a sled and/or other gear, and give suggestions of places to go!
# Metaculus launches contest for essays with quantitative predictions about AI

Metaculus has launched the AI Progress Essay Contest which aims to attract systematic thinkers to engage with the wealth of AI Forecasts that Metaculus has built up over the years. Of particular interest is rigorous and nuanced thinking about the likelihood, timing, and impacts of transformative AI that engages with quantitative probabilistic forecasts. The contest will run until April 16th, 2022, and have a $6,500 prize pool.

Many of the highlighted questions are likely to be of interest to the LW community:

*   Will the scaling up of compute be sufficient to get human-level performance on a wide-range of important domains, such as language modelling and computer vision? If so, when can we expect training runs to be sufficiently compute intensive to hit important milestones? If not, why not? Justify your views with, among other things, reference to quantitative predictions.
*   How much progress in AI algorithms and architectures has there been in the past decade, and how much should we expect in the next decade? Support your views with references to quantitative predictions, such as those involving performance on some common ML benchmarks.
*   Can we map the disagreements between notable figures in the AI risk community, as presented in the [Late 2021 MIRI Conversations](https://www.lesswrong.com/s/n945eovrA3oDueqtq), onto disagreements about quantifiable predictions? Which views imply predictions that we think are likely to fare better and why?
*   When will AI-enabled automation contribute more than 5%/25%/50%/95% to US GDP? Explain why this may or may not happen this century.
*   Is deep learning sufficient for achieving Transformative AI? (This is defined, roughly, as AI that precipitates a transition comparable to the agricultural or industrial revolution.) If not, how many new substantial insights might be needed?
*   How might the trajectory of the field of AI look if Moore’s Law were seriously stunted in the next decade? How might the rate of progress be affected, and where would the field look for new sources of improvements?
*   What fraction of the field of AI will be dedicated to working on AI Safety, broadly defined? Will the field have substantial influence on how AI systems are designed, tested, and/or deployed? Provide evidence supporting your perspective, as well as references to quantitative predictions.   
     

Further details may be found on [the contest landing page](https://www.metaculus.com/project/ai-fortified-essay-contest/).