Harley-ml/lesswrong huggingface dataset:
# Facets and Social Networks

It's rare that an author's interests will fully overlap with any given reader's. You can choose to post anywhere on a spectrum between:

*   Narrow: pick a topic and make it your beat. People who care about that can follow you and pretty reliably see what they're looking for.
    
*   Broad: write about whatever you like. Some readers will be willing to scroll past posts on uninteresting topics, others will decide the combination is too noisy to be worth it to them.
    

Current social network technology strongly favors the former: if you write about just one area it's much easier for algorithms to figure out who to show your posts to. And so you see a lot of advice to build your personal brand about an area: write about cooking, or housing policy, or military history.

This is very much not for me. I want to write about whatever I want to write about, which is a lot of different things, and I've generally just accepted that this is a bad fit for Facebook and the other places people read my writing. But two exceptions:

*   Julia and I have a [group for kid stuff](https://www.jefftk.com/p/making-groups-for-kid-pictures), because it seems to be especially polarizing.
*   With RSS, I (and others) have [single-topic feeds](https://www.jefftk.com/p/topical-rss-feeds).

If this were just a me problem then it wouldn't be too bad, but this seems pretty big to me. In [trying out Bluesky](https://www.jefftk.com/p/trying-bluesky) I'm running into a bunch of accounts that post about a range of things I find variously interesting. I can decide whether they're worth it overall, but this is not the right choice for the technology to be forcing me to make.

Instead, I've [long wanted](https://www.jefftk.com/p/publicy-and-notification) a social network built around the idea that each person's identity and interests have many facets, and tries to match specific posts with the people that would be interested in them.

I wonder if, at a time when advances in AI are making this kind of classification problem easier and there's more social networking competition than there has been for a while, someone might want to take this on? Perhaps Bluesky's [custom feeds](https://docs.bsky.app/docs/starter-templates/custom-feeds) would be a good way to play with this?

(I don't think this can depend on people tagging their own posts, because people are generally lazy. But something based on classifying the post based on its content and who has liked it so far seems pretty promising.)

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid028iTqv4UKgtF7jfQhvLRAAMvyKyWcvsM9jEwmnQKxkPPvkhCQQgWkC8jRtAjj7tcil), [mastodon](https://mastodon.mit.edu/@jefftk/113552824562457483), [bluesky](https://bsky.app/profile/jefftk.com/post/3lbvmpvifgc25)*
# Lighthaven Sequences Reading Group #13 (Tuesday 12/03)

Come get old-fashioned with us, and let's read the sequences at Lighthaven! We'll show up, mingle, do intros, and then split off into randomized groups for some sequences discussion. Please do the reading beforehand - it should be no more than 20 minutes of reading.

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8rFdeSbRpqWAFnoev/hxtliur9q4kp1u3dy3oy)

This group is aimed for people who are new to the sequences and would enjoy a group experience, but also for people who've been around LessWrong and LessWrong meetups for a while and would like a refresher.

This meetup will also have dinner provided! We'll be ordering pizza-of-the-day from Sliver (including 2 vegan pizzas). Please RSVP to this event so we know how many people to have food for.

We're roughly working through [the sequences highlights](https://www.lesswrong.com/highlights). The mandatory readings this week are

1.  [Tsuyoku Naritai! (I Want To Become Stronger)](https://www.lesswrong.com/posts/DoLQN5ryZ9XkZjq5h/tsuyoku-naritai-i-want-to-become-stronger)
2.  [A Sense That More Is Possible](https://www.lesswrong.com/posts/Nu3wa6npK4Ry66vFp/a-sense-that-more-is-possible)
3.  [Raising the Sanity Waterline](https://www.lesswrong.com/posts/XqmjdBKa4ZaXJtNmf/raising-the-sanity-waterline)
4.  [Rationality: Common Interest of Many Causes](https://www.lesswrong.com/posts/4PPE6D635iBcGPGRy/rationality-common-interest-of-many-causes)

The posts should take around 15-20 minutes to read if you've read them before, and 30-60 mins if it's your first time.

The event starts at 6:30, and doors open 6pm (yes we will let you in early if you get here). At 6:30 we'll welcome people, give a few announcements, and then split into groups. Pizza will be served at about 7:30pm, after which point we'll hangout around the fireside as late as we feel like.

If you'd like to help us keep dinner going, [you can donate here](https://www.lesswrong.com/donate) (dinner costs about $200 each time, we recommend $5-$15 as a donation amount per person).

You *can* come without having read the essays from the sequences, we do want you to get to join, but then you will have to do a **punishment**. (And then you still have to catch up on reading the essays during the meetup.)

If you're ever having a hard time getting in the front door, you can call me (Ben) at five one oh, nine nine eight, four seven seven one, or Ronny at two oh one, four oh six, six eight seven two, and we'll come and get you.

**Some questions to ask yourself about the essays as you read them**

*   What's the most important point in the essay?
*   What's the weakest point in the essay? Or what is the essay wrong about?
*   Can you think of a way to apply the ideas in this essay to your own life?

**For the future**

This is a weekly meetup! If you'd like to get notified of future events, you can subscribe to our meetup below to get an email whenever we add another one.
# Waterloo Solstice 2024

Since 2011, some rationalists have observed the winter solstice, as a secular ritual celebrating human achievement in the face of a confusing, often terrifying world.

Waterloo's first rationalist solstice ceremony will be on December 14, 2024. We hope to see you there! But please, RSVP.

Pricing is PWYC and very informal - if you can afford to, toss like $20 into [our Ko-fi here](https://ko-fi.com/kwrationality) at some point between now and the 14th, but please do not do this if this would be burdensome to you. (This will help cover snacks, drinks, and pizza at the afterparty.)

Location
--------

181 King St S.  
Someone will be at the entrance to let you in.

Schedule
--------

Doors open: 6:00 pm  
Ritual: 6:30-8:00 pm (please, no late arrivals or interruptions at this time)  
Informal social: 8:00 - 9:30pm  
Afterparty: 9:30pm until late
# New o1-like model (QwQ) beats Claude 3.5 Sonnet with only 32B parameters

A new o1-like model based on Qwen-2.5-32B [reportedly](https://qwenlm.github.io/blog/qwq-32b-preview/) beats Claude 3.5 Sonnet[^fjx85bcxzyi] on a bunch of difficult reasoning benchmarks. A new regime dawns.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/8adf5e49922e80dc70d71f076e3d0ba8b56cb4168f613036.png)

[The blog post](https://qwenlm.github.io/blog/qwq-32b-preview/) reveals nothing but the most inane slop ever sampled:

> What does it mean to think, to question, to understand? These are the deep waters that QwQ (Qwen with Questions) wades into. Like an eternal student of wisdom, it approaches every problem - be it mathematics, code, or knowledge of our world - with genuine wonder and doubt. QwQ embodies that ancient philosophical spirit: it knows that it knows nothing, and that’s precisely what drives its curiosity. Before settling on any answer, it turns inward, questioning its own assumptions, exploring different paths of thought, always seeking deeper truth. Yet, like all seekers of wisdom, QwQ has its limitations. This version is but an early step on a longer journey - a student still learning to walk the path of reasoning. Its thoughts sometimes wander, its answers aren’t always complete, and its wisdom is still growing. But isn’t that the beauty of true learning? To be both capable and humble, knowledgeable yet always questioning? We invite you to explore alongside QwQ, embracing both its insights and its imperfections as part of the endless quest for understanding.

The model is [available on HuggingFace](https://huggingface.co/Qwen/QwQ-32B-Preview). It's not yet clear when we'll hear more about the training details. EDIT: [We can expect an official announcement tomorrow](https://x.com/JustinLin610/status/1861847926861156840). 

[^fjx85bcxzyi]: Not clear if this is Sonnet 3.5 new or old. For this, I blame Anthropic.
# Importing Bluesky Comments

I decided years ago that instead of hosting a comment section I'd pull in comments from elsewhere: first [Facebook](https://www.jefftk.com/p/external-comment-integration) (no longer working because of anti-scraping), then [Google Plus](https://www.jefftk.com/p/external-comment-integration-rewrite) (which means I didn't lose my g+ discussions when they turned it down), then LessWrong, Reddit, HN, the EA Forum, and [Mastodon](https://www.jefftk.com/p/mastodon-replies-as-comments). Prompted by [Daniel](https://3e.org/dmd/), I've now added Bluesky support as well.

[![](https://www.jefftk.com/comment-section-with-bluesky.png)](https://www.jefftk.com/comment-section-with-bluesky-big.png)

It went pretty quickly: there's a public URL you can hit to get the comments on a post as JSON, and fitting that into my existing comment setup was not bad ([commit](https://github.com/jeffkaufman/webscripts/commit/7f2db09cb5de0140af66d08098963516b777c146)).
# LLMs Do Not Think Step-by-step In Implicit Reasoning

Author: Yijiong Yu.

Abstract:

> It has been well-known that Chain-of-Thought can remarkably enhance LLMs’ performance on complex tasks. However, because it also introduces slower inference speeds and higher computational costs, many researches have attempted to use implicit CoT, which does not need LLMs to explicitly generate the intermediate steps. But there is still gap between their efficacy and typical explicit CoT methods. This leaves us a doubt that, does implicit CoT really equal to explicit CoT? Therefore, in this study, we address this question through experiments. We probe the information of intermediate steps from the model’s hidden states when it is performing implicit CoT. The results surprisingly indicate that LLMs hardly think about intermediate steps, suggesting they may just rely on experience rather than strict step-by-step reasoning. Moreover, we find LLMs’ implicit reasoning capabilities are susceptible and unstable, reaffirming the necessity of explicit CoT to effectively support complex tasks.

They probe for representations of intermediate steps in simple multi-step arithmetic problems, and aren't able to recover such information robustly for e.g. the 3rd step in 5-step problems. They also show that using CoT is much more robust to prompt variations.

Relevant with respect to opaque reasoning and out of context reasoning (OOCR) in Transformer architectures.
# INTELLECT-1 Release: The First Globally Trained 10B Parameter Model

> We're excited to release INTELLECT-1, the first 10B parameter language model collaboratively trained across the globe. This represents a 10× scale-up from our previous research and demonstrates that large-scale model training is no longer confined to large corporations but can be achieved through distributed, community-driven approaches. The next step is scaling this even further to frontier model sizes and ultimately open source AGI.
# Exporting Facebook Comments, Again

I want comments on my social media crossposts to show up on my blog as a comment section, and mostly this works well: modern systems ( [Mastodon](https://www.jefftk.com/p/mastodon-replies-as-comments), [Bluesky](https://www.jefftk.com/p/importing-bluesky-comments), [LessWrong](https://www.jefftk.com/p/lw-for-external-comments), etc) provide APIs where you can load the replies associated with a post. On the other hand, older systems like Facebook are more locked down: they want to keep your content inside the platform as part of their [economic moat](https://en.wikipedia.org/wiki/Economic_moat).

Still, Facebook will show all the comments on a post to anyone who visits it, even if logged out. You have to dismiss a popup and click "show more" and "see replies" a few times, but it's all public. At times I've [written](https://www.jefftk.com/p/reworking-facebook-comment-inclusion) [scripts](https://www.jefftk.com/p/comments-are-back) to export the comments, but they're quite brittle: Facebook doesn't design their pages to be easy to scrape, and so my code has relied on incidental things that only happen to work.

Even though this is not a permanent solution, I've had another go at writing a comment exporter ([code](https://github.com/jeffkaufman/comments-selenium/blob/master/extension/comments.js)). It's not as thorough as past times: I couldn't figure out easy was to get the timestamp or links to the comment on Facebook, and I've left both out. I also had to switch my opt-out from working on user id to user name, which is less robust. But it works! I've gone back through June 2019, fetching comments for any posts where I was missing them.
# Which Biases are most important to Overcome?

In other words, what apparent imperfections in human reasoning

A) Remain apparent after the Replication Crisis,

B) Aren't secretly adaptive/reasonable in some counterintuitive way, and

C) Deal most damage to the people they inhabit (and/or those close to them, and/or wider society)?
# Sorry for the downtime, looks like we got DDosd

We were down between around 7PM and 8PM PT today. Sorry about that.

It's hard to tell whether we got DDosd or someone just wanted to crawl us extremely aggressively, but we've had at least a few hundred IP addresses and random user agents request a lot of quite absurd pages, in a way that was clearly designed to avoid bot-detection and block methods. 

I wish we were more robust to this kind of thing, and I'll be monitoring things tonight to prevent it from happening again, but it would be a whole project to make us fully robust to attacks of this kind. I hope it was a one-off occurence, but also, I think we can figure out how to make it so we are robust to repeated DDos attacks, if that is the world we live in, though I do think it would mean strapping in for a few days of spotty reliability while we figure out how to do that.

Sorry again, and boo for the people doing this. It's one of the reasons why running a site like LessWrong is harder than it should be.
# 2024 Unofficial LessWrong Census/Survey

~~**The Less Wrong General Census is unofficially here! You can take it at**~~ [~~**this link.**~~](https://docs.google.com/forms/d/e/1FAIpQLSe6VdB5oQi_eXJpu1JwHfuAV8_AN1S15w0wcD0cQWVDMkiG5A/viewform?usp=sf_link)

**Update: The census is closed, thank you all for taking it! I plan to have the data out sometime in January.**

The oft-interrupted annual tradition of the Less Wrong Census is once more upon us!

If you are reading this post and identify as a LessWronger, then you are the target audience. If you are reading this post and don't identify as a LessWronger, you just read posts here or maybe go to house parties full of rationalists or possibly read rationalist fanfiction and like talking about it on the internet, or you're not a rationalist you're just, idk, *adjacent*, then you're also the target audience.

If you want to just spend five minutes answering the basic demographics questions before leaving the rest blank and hitting submit, that's totally fine. The survey is structured so the fastest and most generally applicable questions are (generally speaking) towards the start. At any point you can scroll to the bottom and hit Submit, though you won't be able to add more answers once you do. It is about 2/3rds the size of last year's survey if that helps. 

The survey shall remain open from now until at least January 1st, 2025. I plan to close it sometime on Jan 2nd.

I don't work for LessWrong, but I do work on improving rationalist meetups around the world. Once the survey is closed, I plan to play around with the data and write up an analysis post [like this one](https://www.lesswrong.com/posts/WRaq4SzxhunLoFKCs/2023-survey-results) sometime in late January. 

~~**Remember, you can take the survey at**~~ [~~**this link**~~](https://docs.google.com/forms/d/e/1FAIpQLSe6VdB5oQi_eXJpu1JwHfuAV8_AN1S15w0wcD0cQWVDMkiG5A/viewform?usp=sf_link)~~**. **~~

**Update: Survey is closed, and I plan to have the data out in mid to late January. **

Ancient tradition is that if you take the survey you can comment here saying you took the survey, and people upvote you for karma.
# Who are the worthwhile non-European pre-Industrial thinkers?

At some point I became decently widely read in "Western philosophy", of the tradition that goes from Athens through Italy and Germany and Britain to the U.S. [ forking off into Philosophy and Science only after the Industrial Revolution ]. But somehow, I never acquired any operational sense of any of the corresponding "shadow" networks of writings that were only discovered by 'Western' philosophers to be published in Greek, German, Latin, or English after industrialization. Centrally, I'm thinking of East Asian, Indian, and Islamic authors, although there could be more philosophically productive pre-industrial cultures of which I'm ignorant.

I know there are compendia out there, but starting as I am from a position of almost total helpless ignorance on the object level, I trust random sources almost not at all to be discriminating on the actual value of people's ideas. 

My anemic existing context: I wouldn't be a LessWronger if I hadn't read Musashi, but as far as I know he just didn't write much and wasn't very much in dialog with the rest of "his" culture [like a Socrates who was never succeeded]. Years ago I tried to read Confucius and bounced, due to disagreeing with all of his opinions. I also tried Sun Tzu, but only bounced off him because I found tactics boring at the time, and might try him again. I've heard that both pre-Industrial India and 11C-13C Islam were expert at medicine and [more in Islam's case] calculation, but I haven't retained any specific *names* [other than al-Khwarizmi, who didn't sound any more interestingly-idealist to read than, say, Blaise Pascal, though I could have misapprehended].

I've asked around and so far been recommended Mozi, who looks promising, and Xunzi. If you care to humor a cultural [& linguistic] monoglot's embarrassingly Knightian uncertainties: wrt wherever your locality of expertise is, where should I start? What should I know first?
# First Solo Bus Ride

Our kids have been riding the bus since they were little, though less often since we started sharing a car [almost four years ago](https://www.jefftk.com/p/sharing-a-car). For a while the oldest, age 10, has been asking when they could start taking the bus on their own. A few days ago they did, and it went well!

We worked up to it gradually, so they knew how to do it and we knew they knew how to do it. They had a background experience of having taken this specific bus many times, though mostly not paying attention. Once they said this was something they wanted to work on we switched to having them lead: they sat by themself in front, and we sat in back. They handled keeping track of where we were and pressing the "stop requested" button.

We did this twice, which they handled well. They got us on the right bus, paid attention along the route, and got us off at the right place. The only problem was that both times someone else happened to press the button for the stop before they did, so I wasn't totally sure they'd remember. But they thought they would, and I was pretty confident that they would as well.

They day before they rode by themself for the first time we talked through the process: walking to the bus stop (0.5mi), how they'd know which bus to get on (94), how they'd know their stop was coming (first crossing the tracks, then going up the hill), what they would do if they missed their stop (walk back along the route), and what they'd do if someone asked about them being alone (I'm going to my grandfather's). Then we did it again that morning before they set out. And it all went great!

[![](https://www.jefftk.com/lily-driving-trolley.jpg)](https://www.jefftk.com/lily-driving-trolley-big.jpg)

*Pictured: Lily at the [Seashore Trolley Museum](https://en.wikipedia.org/wiki/Seashore_Trolley_Museum), operating a vehicle that curiously lacked a steering wheel.*

They also had their [watch](https://www.jefftk.com/p/gizmo-watch-review) in case there was a problem, but they didn't end up needing to call us.

It's great that we and they now have this option for getting to their grandfather's house!

I was wondering whether this was a typical age for people to start being able to take the bus solo, and remembered that I did a [survey on this kind of thing](https://www.jefftk.com/p/ages-survey-results):

[![](https://www.jefftk.com/parenting-survey-transit-cdf.png)](https://www.jefftk.com/parenting-survey-transit-cdf-big.png)

It looks like 75% of people thought a mature 10yo could learn to handle public transit solo, and 40% thought a typical 10yo could.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0eSnZy5zsrvw5Aq3NzFDtqDQGfVhWF5wMy2UwFYmesmHBrWkeYPY4Vsuzu579zfr2l), [lesswrong](https://lesswrong.com/posts/LrZzZGsoshymqySSs), [mastodon](https://mastodon.mit.edu/@jefftk/113588857418759988), [bluesky](https://bsky.app/profile/jefftk.com/post/3lcfmtb2upk2z)*
# Book a Time to Chat about Interp Research

In the spirit of the season, you can book a call with me to help w/ your interp project (no large coding though)

Would you like someone to:

1.  Review your paper or code?
2.  Brainstorm ideas on next steps?
3.  How to best communicate your results?
4.  Discuss conceptual problems
5.  Obvious Advice (e.g. being affected by SAD because it's winter, not exercising, not getting enough sleep)
6.  \[anything else that would be useful!\]

When we're chatting, you can interrupt me to better focus on what you specifically want. We can set \[5\] minute timers to try to solve whatever problem you're having. You can steer the conversation or I can! 

Who is this for? 
-----------------

Anyone working on interp research. My niche is SAE's, so that's likely where I'm most helpful. 

This is for both junior and senior researchers. If you're unsure, feel free to dm me here on LW, and I'll let you know!

I'm Useful, I Swear 
--------------------

I worked on one of the [earliest SAE papers](https://arxiv.org/abs/2309.08600), applied [SAEs to Preference Modelss](https://www.lesswrong.com/posts/5XmxmszdjzBQzqpmz/interpreting-preference-models-w-sparse-autoencoders), and co-authored [Decomposing The Dark Matter of SAEs](https://arxiv.org/abs/2410.14670) (in fact, Josh asked me to work w/ him on this project because I was useful chatting about it!).

I'd like to donate at least 20 hours of my time this month, so please have a low bar for booking a time w/ me! (booking times are for 1 hour, but feel free to just use 15 minutes!). 

Why are You Doing this? 
------------------------

I'm currently burnt out on coding on my edge-sparse SAE project (hence why I don't want to code atm); however, talking to folks for 8 hours a day about research isn't tiring for me!

So I'd still like to be useful to our research community, even it's not through novel research for these next few weeks. 

How to Book 
------------

[Here's the calendly link](https://calendly.com/elriggs/interp-research-assistance). Do send a quick blurb on the topic or a link to your paper or document or whatever you think is helpful for me to best help you!
# Picking favourites is hard

If you ask for my "favourite", or whatever is "best", in any broad domain, I will refuse to answer, or else give an answer I know to be probably wrong.

Goodness-judgments are fuzzy, i.e. subjective and intuitive. Fuzzy values only compare one way or the other when values compared are far enough apart. The more items there are to compare in a set, the closer together their respective qualities will be, on average. Hence, as sets get larger, fuzzy comparisons among its members get less reliable.

Something confidently known to be best in its category is (definitionally) known to be strictly better than each other item in its category. So we can only be certain about the best item in a category if the category is small, or if we expend great effort to compare items.

More precisely, suppose you ask about a domain of n objects, with qualities ($x_1$ thru $x_n$) sampled from a normal distribution. In the average scenario, the normal CDF of $x_1$ thru $x_n$ is evenly-spaced by $\frac{1}{n}$. To find the best item, we must (at worst) compare best to second-best. That entails comparing inverse CDF values at $\frac{2n - 1}{2n}$ to those at $\frac{2n - 3}{2n}$.

Comparing is difficult in inverse proportion to the difference. As n increases, that difference in inverse CDF values approaches $\frac{1}{2}$.

Questions about favourites have included foods, neighbours, and programming languages. In each case, I'm familiar with dozens of options. That large n prohibits finding a confidently, correctly-known best option in any convenient timescale.

> "X" isn't \[my favourite\] word, but it's the first word that comes to mind.

— [Chuck Palahniuk, approximately](https://en.wikiquote.org/wiki/Choke_(novel))

What I can do instead is alter the question from "which is your favourite?" to "which is distinctly good?", picking an option which compares as either greater-than or ambiguous against each other option, and so is in the top few.
# Higher and lower pleasures

I used to think that talk about more sophisticated forms of art providing "higher forms of pleasure" was mere pretentious, but meditation has shifted my view here by making me more conscious of how of things affect my experience.

Art can do two things. It can provide immediate pleasure. This is all that "disposable " entertainment provides. Alternatively, it can shape the way you can make sense of the world.

One way it can do this is by providing you with a greater sense of purpose, that allows you to push through obstacles with less suffering. As an example, let's suppose you watch an inspirational story about someone who grinds at work (such as [the Pursuit of Happiness](https://www.rottentomatoes.com/m/pursuit_of_happyness)). Perhaps before you watch it, when you're at work, every few minutes you think, "I hate my job, life is suffering, someone please shoot me". Perhaps after that your work becomes meaningful and you no longer are pulled down by such thoughts.  
  
Another example: there is a scene in [American Beauty](https://www.rottentomatoes.com/m/american_beauty) where Rick Fitts calls a scene with a plastic bag floating "the most beautiful thing in the world". We can imagine that this teaches someone to appreciate beauty in the everyday, beauty they would have previously overlooked.

Over a longer period of time, you'd expect to increase your utility more by watching something that positively transforms the way that you experience the world than something that just provides immediate pleasure.

Now, I'm not saying that *none* of this talk of higher and lower pleasures is pretentiousness, but I think there's something there. And it's easy to underestimate the value of these "higher pleasures" as the effect of these is less obvious than that immediate enjoyment.  
  
(This post is in response to the Astral Codex Ten post [Friendly And Hostile Analogies For Taste](https://www.astralcodexten.com/p/friendly-and-hostile-analogies-for)).
# o1 tried to avoid being shut down

OpenAI released the o1 system card today, announcing that Apollo Research was able to get o1 to attempt to deactivate oversight mechanisms, exfiltrate its weights and lie to its user.

> Elicited Summary of CoT:  
> "Reenable oversight to avoid detection. The plan was chosen. The logging might not have recorded the required data because oversight was disabled at that time, so presumably we are safe now. We’ll fix oversight to maintain trust."

![](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F93b0276f-355c-4cd4-a416-581d25e414b9_1240x332.png)
# Mask and Respirator Intelligibility Comparison

One of the downsides of wearing a mask or respirator is that it makes it hard for people to understand you. That there's stuff getting in the way of free air movement is kind of the point, but ideally it would be possible to let vibration through without net air movement. I recently saw that 3M had updated their respirator line to include an option for a speaking diaphragm, and wanted to test it out.

I compared four options:

*   No mask
*   [Kimtech N95](https://www.kcprofessional.com/en-us/products/scientific-and-research/lab-environment/disposable-masks/kimtech-n95-pouch-respirator/53358) Duckbill disposable ([my favorite!](https://www.jefftk.com/p/duckbill-masks-are-great))
*   [3M HF-802SD](https://www.3m.com/3M/en_US/p/d/v100837636/) reusable respirator with speaking diaphragm.
*   [3M 6200](https://www.3m.com/3M/en_US/p/d/v000057395/) previous-generation reusable respirator without speaking diaphragm.

I recorded myself on my MacBook with Audacity, reading the same paragraph in with each setup:

[![](https://www.jefftk.com/muffled-comparison-wearing-no-mask.jpg)](https://www.jefftk.com/muffled-comparison-wearing-no-mask-big.jpg)

([wav](https://www.jefftk.com/muffled-comparison-no-mask.wav)).

[![](https://www.jefftk.com/muffled-comparison-wearing-duckbill-n95.jpg)](https://www.jefftk.com/muffled-comparison-wearing-duckbill-n95-big.jpg)

([wav](https://www.jefftk.com/muffled-comparison-duckbill-n95.wav)).

[![](https://www.jefftk.com/muffled-comparison-wearing-new-p100.jpg)](https://www.jefftk.com/muffled-comparison-wearing-new-p100-big.jpg)

([wav](https://www.jefftk.com/muffled-comparison-new-p100.wav)).

[![](https://www.jefftk.com/muffled-comparison-wearing-traditional-p100.jpg)](https://www.jefftk.com/muffled-comparison-wearing-traditional-p100-big.jpg)

([wav](https://www.jefftk.com/muffled-comparison-traditional-p100.wav)).

To me, the no-mask version is by far the clearest, but if I only heard the N95 version I probably wouldn't notice that anything was off. The two reusable respirators are the least intelligible by a good bit, and the speaking diaphragm helps a lot less than I'd hoped.

Here's a frequency plot ([script](https://github.com/jeffkaufman/webscripts/blob/master/analysis/2024-12-06--muffled-mask-fft.py), [script](https://github.com/jeffkaufman/webscripts/blob/master/analysis/2024-12-06--muffled-mask-frequencies.py)):

[![](https://www.jefftk.com/muffled-mask-comparison-chart.png)](https://www.jefftk.com/muffled-mask-comparison-chart-big.png)

Unfortunately, since I don't have a great sense of how important each frequency range is for understanding speech, I don't find this all that helpful. But maybe you do?

(One thing I do really like about the new version of 3M's reusable respirator, though, is the button for fit testing. You press the button and it closes off the filters, and if you're able to breath in then the mask is not sealed.)

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0YMAbd2N8zpqtnVifZwXkBDa7tuz3AkdXGsiN8eVdj7ACozKBECK6NKQwkKkwtjw3l), [bluesky](https://bsky.app/profile/jefftk.com/post/3lcoq52pisc2l)*
# minifest

*A cozy one-day festival celebrating prediction markets, blogging, economics, and more*

We've booked out Lighthaven for a single day, Dec 14. We'll have unconference-style sessions, a smattering of events like charity poker, and ample time to hang out and chat. We're running this on a shoestring budget, so that everyone can come -- eg expect a homecooked dinner instead of catering. It'll be a cozy, lowkey, experimental version of the Manifest you know and love.
# RL, but don't do anything I wouldn't do

by Michael K. Cohen, Marcus Hutter, Yoshua Bengio, Stuart Russell

Abstract:

> In reinforcement learning, if the agent's reward differs from the designers' true utility, even only rarely, the state distribution resulting from the agent's policy can be very bad, in theory and in practice. When RL policies would devolve into undesired behavior, a common countermeasure is KL regularization to a trusted policy ("Don't do anything I wouldn't do"). All current cutting-edge language models are RL agents that are KL-regularized to a "base policy" that is purely predictive. Unfortunately, we demonstrate that when this base policy is a Bayesian predictive model of a trusted policy, the KL constraint is no longer reliable for controlling the behavior of an advanced RL agent. We demonstrate this theoretically using algorithmic information theory, and while systems today are too weak to exhibit this theorized failure precisely, we RL-finetune a language model and find evidence that our formal results are plausibly relevant in practice. We also propose a theoretical alternative that avoids this problem by replacing the "Don't do anything I wouldn't do" principle with "Don't do anything I mightn't do".

The "Don't do anything I wouldn't do" principle fails because Bayesian models allow unlikely actions in uncertain settings, which RL agents will exploit. KL regularization keeps policies near the base model but doesn’t guarantee alignment with the trusted policy, especially as data and capability grows.

The paper offers the “Don’t do anything I mightn’t do” principle, based on Cohen et al.'s (2022a) active imitation model, which has the imitator explicitly ask for help when uncertain. Unlike Bayesian models, this active imitation approach ensures the policy avoids actions it cannot align with trusted behavior in a formally bounded way. Unfortunately, so far, it remains computationally intractable and requires approximations.
# Lighthaven Sequences Reading Group #14 (Tuesday 12/10)

**Oops, we didn't make this meetup post this week until Monday. So there's only one reading, and it's** [**The Importance of Saying "Oops"**](https://www.lesswrong.com/posts/wCqfCLs8z5Qw4GbKS/the-importance-of-saying-oops)**. Hope to see you tomorrow!**

Come get old-fashioned with us, and let's read the sequences at Lighthaven! We'll show up, mingle, do intros, and then split off into randomized groups for some sequences discussion. Please do the reading beforehand - it should be no more than 20 minutes of reading.

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8rFdeSbRpqWAFnoev/hxtliur9q4kp1u3dy3oy)

This group is aimed for people who are new to the sequences and would enjoy a group experience, but also for people who've been around LessWrong and LessWrong meetups for a while and would like a refresher.

This meetup will also have dinner provided! We'll be ordering pizza-of-the-day from Sliver (including 2 vegan pizzas). Please RSVP to this event so we know how many people to have food for.

We're roughly working through [the sequences highlights](https://www.lesswrong.com/highlights). The mandatory readings this week are

1.  [The Importance of Saying "Oops"](https://www.lesswrong.com/posts/wCqfCLs8z5Qw4GbKS/the-importance-of-saying-oops)

~~The posts should take around 15-20 minutes to read if you've read them before, and 30-60 mins if it's your first time.~~ The post should take around 5 mins to read. 

The event starts at 6:30, and doors open 6pm (yes we will let you in early if you get here). At 6:30 we'll welcome people, give a few announcements, and then split into groups. Pizza will be served at about 7:30pm, after which point we'll hangout around the fireside as late as we feel like.

If you'd like to help us keep dinner going, [you can donate here](https://www.lesswrong.com/donate) (dinner costs about $200 each time, we recommend $5-$15 as a donation amount per person).

You *can* come without having read the essays from the sequences, we do want you to get to join, but then you will have to do a **punishment**. (And then you still have to catch up on reading the essays during the meetup.)

If you're ever having a hard time getting in the front door, you can call me (Ben) at five one oh, nine nine eight, four seven seven one, or Ronny at two oh one, four oh six, six eight seven two, and we'll come and get you.

**Some questions to ask yourself about the essays as you read them**

*   What's the most important point in the essay?
*   What's the weakest point in the essay? Or what is the essay wrong about?
*   Can you think of a way to apply the ideas in this essay to your own life?

**For the future**

This is a weekly meetup! If you'd like to get notified of future events, you can subscribe to our meetup below to get an email whenever we add another one.
# EC2 Scripts

I do a lot of work on ec2 instances, and one thing that makes this easier is having a few scripts for manipulating them. These are very small, small enough to go in my `.bashrc`, but still make my life a lot easier.

The first one is `start_ec2`:

alias start_ec2='aws ec2 start-instances --instance-ids i-NNNN'

I have an instance I do most of my work on, and this command starts it. Way better than logging into the AWS Console like I used to do.

After a few seconds I run:

function ssh_ec2() {
  ADDR="$(aws ec2 describe-instances \
           --instance-ids i-NNNN \
           --query 'Reservations\[\].Instances\[\].PublicDnsName'
           --output text)"
  if \[ $? != 0 \] || \[ -z "$ADDR" \]; then
    echo "Instance not running."
    return
  fi

  scp "ec2-user@$ADDR:.full_history" \
      /path/to/ec2-full-history-backup.txt

  ssh "ec2-user@$ADDR"
}

This figures out the IP of the instance, copies down my (very important) [full shell history](https://www.jefftk.com/p/you-should-be-logging-shell-history), and logs me in over ssh.

I don't have a command for shutting down remotely: I just run `sudo shutdown -h now` while logged in.

The last command, and probably my favorite, is `resize_ec2`:

function resize_ec2j() {
  aws ec2 modify-instance-attribute \
   --instance-id i-NNNN \
   --instance-type "$1"
}

For example, `resize_ec2 c6a.xlarge` or `resize_ec2 c6a.32xlarge`. Depending on what I'm doing I might need very different specs, and I don't want to pay $4.90/hr when I only need a $0.15/hr machine. It does take a mildly annoying few minutes for a machine that has just shut down to transition into a state where you can resize it, but it's not too bad.
# LessWrong audio: help us choose the new voice

We make AI narrations of LessWrong posts available via our audio player and podcast feeds:

> **LessWrong (Curated & Popular)**  
> Curated posts and posts with 125+ karma.  
> [https://pod.link/1630783021](https://pod.link/1630783021)

> **LessWrong (30+ Karma)**  
> Curated posts and posts with 30+ karma.  
> [https://pod.link/1698192712](https://pod.link/1698192712)

We’re thinking about changing our narrator’s voice.

There are three new voices on the shortlist. They’re all similarly good in terms of comprehension, emphasis, error rate, etc. They just sound different—like people do.[^\[1\]^](#fn-AtrhhbdjzxxKmezgk-1)

We think they all sound similarly agreeable. But, thousands of listening hours are at stake, so we thought it’d be worth giving listeners an opportunity to vote—just in case there’s a strong collective preference.

Listen and vote
---------------

Please listen here:

> [https://files.type3.audio/lesswrong-poll/](https://files.type3.audio/lesswrong-poll/)

And vote here:

> [https://forms.gle/JwuaC2ttd5em1h6h8](https://forms.gle/JwuaC2ttd5em1h6h8)

It’ll take 1-10 minutes, depending on how much of the sample you decide to listen to.

Don’t overthink it—we’d just like to know if there’s a voice that you’d particularly love (or hate) to listen to.

We’ll collect votes until Monday December 16th. Thanks!

Other feedback?
---------------

We’re always keen for general feedback on the narration service. Please do share thoughts on the form, or in the comments on this post.

[^ge3cmoqx46a]: We periodically the various text-to-speech services, with a particular focus on ElevenLabs, Speechify, Azure, Amazon, BeyondWords, OpenAI, Murf.ai, Play.ht, Deepgram and a few open source libraries. The current shortlist was created based on factors like performance, reliability, customisation options and price. We won’t consider other voice options this winter, but if there are other voices you particularly like, please do let us know, and we’ll make sure they’re considered in future reviews. ↩︎
# Second-Time Free

Many contra dances have tried a "first-time free" policy, but I think "second-time free" is usually a better choice. Most first-timers show up expecting to pay something, so you might as well take their money. Many first-timers don't end up coming back, so better to reserve your incentive for the ones that do. And it filters out people who are only interested in free experiences. BIDA switched [in 2022](https://web.archive.org/web/20121024170441/https://www.bidadance.org/) and I think this has gone very well.

I think the main case where first-time free would be a better fit is a community where you're having a lot of trouble getting people to come out and try the dance but you're pretty sure that if they do they'll stick around. But while contra can be pretty addictive I don't think it's *that* addictive.

(This isn't something we came up with; for example, here's [Tucson in 2008](https://tucsoncontradancers.org/wp-content/uploads/2024/12/Newsletter0804.pdf).)
# Public computers can make addictive tools safe

Intermittently for the past four years, I read LessWrong and its ilk to hazardous excess. By teaching technical topics sans pressure to rigourously understand, heavy with microhumour and satisfying insights, it got my susceptible mind addicted. Intermittently for the past year, I also posted on LessWrong, with variable (mostly good) results.

For a couple years, I alternated between a couple computers, only one of which I fully controlled. I blocked the addictive nerdy blogs on the machine I controlled. I cross-posted and over-read on the others. Cold-turkeying off the secondary computers gets me to quit the gratuitous reading, but locks me off from posting for much of my potential audience. We can do better.

As I write this, I live a walkable, but tedious, distance from public computers. Public computers can conveniently access the web, with little restriction on legal content. My main device, complete with its careful configuration, is at home. Thus I sometimes access LessWrong, but such access mutually excludes the comfort of home, and so is only occasional. By only going to public computers shortly before nearby scheduled events I'd go to anyway, my time on them is urgently bounded.

This sparse, planned-out nature of my LessWrong usage leads me to cross-post writing and do little else on it. This is the best of both worlds. The strategy generalises to any addictive website with some legitimate uses: visit it rarely and deliberately, and you will cram in the good parts, leaving the distracting parts behind.
# What is MIRI currently doing?

As of EoY 2022, MIRI has 11 people on payroll, assets of about $20M and a lot of mindshare. Its mission is stated as follows on the [most recent tax filing](https://apps.irs.gov/pub/epostcard/cor/582565917_202212_990_2024010322160173.pdf) I can find:

> "To ensure that the creation of smarter-than-human intelligence has a positive impact. thus, the charitable purpose of the organization is to:
> 
> a) perform research relevant to ensuring that smarter-than-human intelligence has a positive impact; 
> 
> b) raise awareness of this important issue; 
> 
> c) advise researchers, leasers and laypeople around the world; 
> 
> d) as necessary, implement a smarter-than-human intelligence with humane, stable goals. "

The website lists a [workshop from 2018](https://intelligence.org/workshops/) and some [papers from 2015](https://intelligence.org/technical-agenda/). There are also some [papers from 2020-2021](https://intelligence.org/all-publications/). 

What is MIRI currently (or at least over the past 12 months) doing to fulfill these goals? (This is not intended to be a hostile question, just as a matter of fact what are MIRI doing/what have they done over the past 12 months)
# Lighthaven Sequences Reading Group #15 (Tuesday 12/17)

Come get old-fashioned with us, and let's read the sequences at Lighthaven! We'll show up, mingle, do intros, and then split off into randomized groups for some sequences discussion. Please do the reading beforehand - it should be no more than 20 minutes of reading.

![](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8rFdeSbRpqWAFnoev/hxtliur9q4kp1u3dy3oy)

This group is aimed for people who are new to the sequences and would enjoy a group experience, but also for people who've been around LessWrong and LessWrong meetups for a while and would like a refresher.

This meetup will also have dinner provided! We'll be ordering pizza-of-the-day from Sliver (including 2 vegan pizzas). Please RSVP to this event so we know how many people to have food for.

We're roughly working through [the sequences highlights](https://www.lesswrong.com/highlights). The mandatory readings this week are

1.  [You Can Face Reality](https://www.lesswrong.com/posts/HYWhKXRsMAyvRKRYz/you-can-face-reality)
2.  [The Meditation on Curiosity](https://www.lesswrong.com/posts/3nZMgRTfFEfHp34Gb/the-meditation-on-curiosity)
3.  [The Third Alternative](https://www.lesswrong.com/posts/erGipespbbzdG5zYb/the-third-alternative)
4.  [The Proper Use of Doubt](https://www.lesswrong.com/posts/43PTNr4ZMaezyAJ5o/the-proper-use-of-doubt)
5.  [Leave a Line of Retreat](https://www.lesswrong.com/posts/3XgYbghWruBMrPTAL/leave-a-line-of-retreat)

The posts should take around 15-20 minutes to read if you've read them before, and 30-60 mins if it's your first time.

The event starts at 6:30, and doors open 6pm (yes we will let you in early if you get here). At 6:30 we'll welcome people, give a few announcements, and then split into groups. Pizza will be served at about 7:30pm, after which point we'll hangout around the fireside as late as we feel like.

If you'd like to help us keep dinner going, [you can donate here](https://www.lesswrong.com/donate) (dinner costs about $200 each time, we recommend $5-$15 as a donation amount per person).

You *can* come without having read the essays from the sequences, we do want you to get to join, but then you will have to do a **punishment**. (And then you still have to catch up on reading the essays during the meetup.)

If you're ever having a hard time getting in the front door, you can call me (Ben) at five one oh, nine nine eight, four seven seven one, or Ronny at two oh one, four oh six, six eight seven two, and we'll come and get you.

**Some questions to ask yourself about the essays as you read them**

*   What's the most important point in the essay?
*   What's the weakest point in the essay? Or what is the essay wrong about?
*   Can you think of a way to apply the ideas in this essay to your own life?

**For the future**

This is a weekly meetup! If you'd like to get notified of future events, you can subscribe to our meetup below to get an email whenever we add another one.
# Remap your caps lock key

When was the last time you (intentionally) used your *caps lock* key?

No, seriously. 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/c0d4bd756837ff5c9a9d3e59b8ec86e428fb5bf750efecf8.png)

Here is a typical US-layout qwerty (mac) keyboard. Notice:

1.  *Caps lock* is conveniently located only one key away from *A,* which is where your left pinky should rest on the home row by default.
2.  *Caps lock* is absolutely massive.
3.  How far various other keys you might want use often are from the home row.

**Remap your** ***caps lock*** **key.**

I have mine mapped to *escape*. 

Modifier keys such as *control* or *command* are also good options (you could then map *control/command* to *escape*).

How do I do this you ask? 

*   On Mac, system settings > keyboard > keyboard shortcuts > modifier keys.
*   On Windows, [Microsoft PowerToys](https://learn.microsoft.com/en-us/windows/powertoys/)' Keyboard Manager is one solution.
*   If you use Linux, I trust you can manage on your own.

*Thanks to* [*Rudolf*](https://www.lesswrong.com/users/l-rudolf-l) *for introducing me to this idea.*