Harley-ml/lesswrong huggingface dataset:
# Bay 2022 Solstice

Join us on Saturday, December 17th to celebrate humanity and reflect on our place in the universe. We'll sing songs, hear speeches, and connect with each other.

TICKETS:  
Tickets are pay-what-you-want, and are available on [EventBrite](https://www.eventbrite.com/e/bay-area-winter-solstice-2022-tickets-443735594107?fbclid=IwAR11VO6qofDqgyaIqcdQSaz5Kt9cZJQ0nhcT_xS_ELzgrXr_VIxJF7_lBTs). 

SCHEDULE:  
– Doors open at 6 PM.  
– Dinner will be served from 6:00 to 7:15.  
– The performance starts at 7:30 and will last approximately two hours.

See the [Facebook Event](https://www.facebook.com/events/3350891048507160) for communication about the afterparty, food, and the online simulcast.
# Okay, I feel it now

I’ve been coming to LessWrong for a while. I’ve read most of the arguments for how and why things might go wrong.

I’ve been keeping across most developments. I’ve been following alignment efforts. I’ve done some thinking about the challenges involved.

But now I *feel* it.

Spending time observing ChatGPT – its abilities, its quirks, its flaws – has brought my feelings into step with my beliefs. 

I already appreciated why I should be concerned about AI. Like I say, I’d read the arguments, and I’d often agreed. 

But my appreciation took a detached, ‘I can’t fault the reasoning so I should accept the conclusion’ kind of form. I was concerned in the abstract, but I was never *really* worried. At least some of my concern was second-hand; people I respected seemed to care a lot, so I probably should too. It was forced. 

I spend a lot of time thinking about catastrophic risks. When I consider something like an engineered pandemic, I’ve always *felt* the danger. It comes naturally to me. It’s intuitive. The same goes for many other threats. But only in the last few weeks has that become true for AI.

This is all a little embarrassing to admit. I know that ChatGPT isn’t an enormous leap from what existed previously. Yet it has significantly changed how I feel about AI risk. It’s made things click in a way that I can’t fully explain; all the arguments now hit with an added force. 

Those who’ve always 'felt it' are probably wondering what took me so long, and why now. I'm not sure. But I doubt my experience is unique. Hopefully some of you know exactly what I’m trying to convey here. 

Before I believed it. Now I *feel *it.

And as much as I’d like to think otherwise, that makes a big difference.
# Is the ChatGPT-simulated Linux virtual machine real?

Context: [Building A Virtual Machine inside ChatGPT](https://www.engraved.blog/building-a-virtual-machine-inside/)

It very much triggers my "roll to disbelieve" reflex, but people seem to be talking about it as if it's real and not *that* surprising.  I'd try it myself, but OpenAI is saying ChatGPT is overloaded (and I don't have an OpenAI account, although I assume that part would be solvable if not for ChatGPT being overloaded).  Can anyone confirm from personal experience that this really works?  If so, did you probe its limits, and where did it fail?  (It obviously wouldn't be that hard for it to simulate a login sequence, but the entire interaction in the blog post seems ~impossible to me without either using an actual Linux VM or massive cherry-picking of the ChatGPT results.)
# How can one literally buy time (from x-risk) with money?

[This post](https://www.lesswrong.com/posts/bkpZHXMJx3dG5waA7/ways-to-buy-time) talks about types of projects that can buy us more time to work on AI alignment. But like, I am just definitely not going to pivot to working on any of those projects right now. What pre-existing projects could someone donate money to, right now? (This seems especially relevant in the shadow of lost FTX funding.)
# COVID China Personal Advice (No mRNA vax, possible hospital overload, bug-chasing edition)

So, uh, things in China are getting a bit interesting.

There's a general consensus that the public COVID numbers no longer reflect reality. Baidu started hiding their case tracking display. Their last updates had weird inconsistencies between asymptomatic and symptomatic cases. Testing stations are going offline. While previously a single case of close contact would have been enough to shut down an entire college, now facilities are running even with multiple symptomatic cases. Every time things have loosened to this degree before, cases have spiked. Many people I know mentioned family members being infected, some of which are unreported, while official case numbers are on the decline. The security people who checked health app codes aren't even pretending to care anymore. I was explicitly told *not* to show them my health codes when I come to work today. Lines for testing booths at peak times are basically non-existent. 

I think this is it. COVID will go full endemic in China. Exponential. 

I'm in my late 20s and in great physical condition. I exercise multiple times a week (weightlifting only). Unfortunately, I only have 2 shots of Sinovac. I'm planning to get some dental work taken care of this weekend and will get a third shot if it's available.

I think I have 2 options, really. First, I fly to Hong Kong and get the mRNA vax there, which an executive at my company mentioned she planned to do. Not sure if the EV is going to be positive, and I'll have to take a week off work. Second, I go to a hospital/visit a poz family friend and intentionally poz myself ASAP before the hospitals get overloaded. 

Overall, I'm not too concerned, but am interested in what you guys think. I would also love to hear any investment advice from those familiar with Chinese markets.
# Discovering Latent Knowledge in Language Models Without Supervision

> Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect. We propose circumventing this issue by directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. Specifically, we introduce a method for accurately answering yes-no questions given only unlabeled model activations. It works by finding a direction in activation space that satisfies logical consistency properties, such as that a statement and its negation have opposite truth values. We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models: across 6 models and 10 question-answering datasets, it outperforms zero-shot accuracy by 4\% on average. We also find that it cuts prompt sensitivity in half and continues to maintain high accuracy even when models are prompted to generate incorrect answers. Our results provide an initial step toward discovering what language models know, distinct from what they say, even when we don't have access to explicit ground truth labels.

Some discussion [on Twitter]( 
https://twitter.com/CollinBurns4/status/1600892261633785856?t=yjXFHUkbSvBnsQi567AFLw&s=19) including Eliezer calling it "Very dignified work!"
# Chair Hanging Peg

Our house is now back up to full occupancy, which meant some things that had been hanging out in unused rooms needed better homes. Four of those things were [folding chairs](https://www.ikea.com/us/en/p/terje-folding-chair-black-00222440/), and now they are up and out of the way:

[![](https://www.jefftk.com/chair-hanger-complete.jpg)](https://www.jefftk.com/chair-hanger-complete-big.jpg)

I made this out of three pieces of scrap wood: two bits of 2x6 and a 1 1/4 dowel. Here are the 2x6s, painted and screwed together, with the hole for the dowel drilled:

[![](https://www.jefftk.com/chair-hanger-in-progress.jpg)](https://www.jefftk.com/chair-hanger-in-progress-big.jpg)

The outer 2x6 is screwed to the inner one, and the inner one is screwed through the wall into a stud. The dowel stays in the hole by friction: it's a close fit and it's not going anywhere. This works for these particular chairs because they have a nice round hole in the seat back; other chair designs would likely need two pegs.

There are chair-hanging products you can buy, but I didn't like any of them and this was a quick project I could do with things I had left over from earlier projects.
# Predicting GPU performance

We develop a simple model that predicts progress in the performance of field-effect transistor-based GPUs under the assumption that transistors can no longer miniaturize after scaling down to roughly the size of a single silicon atom. We construct a composite model from a performance model (a model of how GPU performance relates to the features of that GPU), and a feature model (a model of how GPU features change over time given the constraints imposed by the physical limits of miniaturization), each of which are fit on a dataset of 1948 GPUs released between 2006 and 2021. We find that almost all progress can be explained by two variables: transistor size and the number of cores. Using estimates of the physical limits informed by the relevant literature, our model predicts that GPU progress will stop roughly between 2027 and 2035, due to decreases in transistor size. In the limit, we can expect that current field-effect transistor-based GPUs, without any paradigm-altering technological advances, will be able to achieve a peak theoretical performance of 1e14 and 1e15 FLOP/s in single-precision performance.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/928754e9096e14da17736122f0f01c932a4f5964fe8b1779.png)

**Figure 1**. Model predictions of peak theoretical performance of top GPUs, assuming that transistors can no longer miniaturize after scaling down transistors to around 0.7nm. **Left:** GPU performance projections; **Top right:** GPU performance when the limit is hit; **Middle right:** Our distribution over the physical limits of transistor miniaturization; **Bottom right:** Our distribution over the transistors per core ratio with relevant historical comparisons.

While there are many other drivers of GPU performance and efficiency improvements (such as memory optimization, improved utilization, and so on), decreasing the size of transistors has historically been a great, and arguably the dominant, driver of GPU performance improvements. Our work therefore strongly suggests that it will become significantly harder to achieve GPU performance improvements around the mid-2030s within the longstanding field-effect transistor-based paradigm.

This post is just the executive summary, you can find the [full report](https://epochai.org/blog/predicting-gpu-performance) on the Epoch website.
# Is the AI timeline too short to have children?

If you had asked me at any point in my adult life (until recently) whether I wanted to have children eventually, I would've said yes, without hesitation. In recent years I've been telling myself: I don't know how likely these AI doom predictions are, but I'm going to focus on optimizing the "long path" because that's where my decisions actually matter - and so I should still have children just in case.

But now, both as I'm nearing the family-forming stage in my life, and as the AI timeline [seems](https://www.lesswrong.com/posts/zyudvnRruMQMxM3NE/okay-i-feel-it-now) to be coming into sharper focus, I'm finding it emotionally distressing to contemplate having children.

If AI kills us all, will my children suffer? Will it be my fault for having brought them into the world while knowing this would happen? Even if I think we'll all die painlessly, how can I look at my children and not already be mourning their death from day 1? If I were to die right now, I would at least have had a chance to live something like a fulfilling life - but the joy of childhood seems inextricable from a sense of hope for the future. Even if my children's short lives are happy, wouldn't their happiness be [fundamentally false](https://www.lesswrong.com/posts/SqF8cHjJv43mvJJzx/feeling-rational) and devoid of meaning?
# How is ARC planning to use ELK?

Let's say we arrive at a worst-case solution for ELK, how are we planning to use it? My initial guess was that ELK is meant to help make IDA viable so that we may be able to use it for some automated alignment-type approach. However, this might not be it. Can someone clarify this? Thanks.
# Masking to Avoid Missing Things

About a year ago I [wrote](https://www.jefftk.com/p/personal-response-to-omicron) that I'd now:

> follow official guidance and mask regulations, cheerfully go along with precautions others need, and test+isolate when sick, but I'm not going to go above and beyond to attempt to reduce spread the way I did for earlier parts of the pandemic.

And yet I'm often wearing an N95 in situations where they're not required and a large majority of other people aren't masked. Are these in tension?

My views haven't changed. I still think that if my region and the communities I'm part of aren't going to put a serious effort into keeping covid from spreading then it doesn't make sense for me to make large sacrifices on my own for pro-social reasons. But if I get covid, or one of the several other things going around, I'll need to isolate to avoid getting other people sick. That means missing things I'd be very sad to miss, and I'll take steps to avoid that:

*   I masked at the [MetaSUB](http://metasub.org/2022-agenda/) conference the weekend before Thanksgiving.
    
*   I masked at Sunday's [EA dinner](https://www.eaboston.org/) and my work's holiday party because I'm running music for a [secular solstice](https://www.facebook.com/events/3403227779922411) this Saturday (sort of an atheist church service inspired by ideas from the effective altruism and rationalist communities).
    
*   I'm planning to mask at the solstice event because it's just before relatives come to town for Christmas.
    

On the other hand, I didn't mask at Thanksgiving and I'm not planning on masking at Christmas because being sick right after that is, while not ideal, basically fine.

While this does have pro-social effects, slowing down the rate of transmission of airborne diseases, my motivation here is primarily selfish: I don't want to miss things.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0btqdK3Wrv9aAFFW6Gfaqhuq7D2tHGuRBQ7wG4ZJriNBaSoxavAMTssiu3mXGxpNCl), [mastodon](https://mastodon.mit.edu/@jefftk/109519692827743907)*
# College Selection Advice for Technical Alignment

Hi all, I'm a highschool senior trying to make some college-related decisions, and I'd like to ask for some advice.

My current situation is:

*   I want to work on technical alignment. For exogenous reasons, not going to college (e.g., taking a year off, just being an autodidact/independent researcher) is not an available option, so I'll have to leverage my undergraduate experience as much as possible to upskill on technical alignment.
    *   I'll probably double major somewhere along CS/Math and maybe CompBio.
*   Accepted to Harvard (non-binding REA). Was planning to apply to Stanford, MIT, Harvey Mudd for RD, but ... 
*   ... I truly *despise* the application writing process, every single second of it, and it has taken a significant toll on my mental health. I'd prefer not to go through that again, although I can if necessary.

My considerations are:

*   Flexibility - Is it possible to take advanced (under)graduate courses while skipping prerequisites? I've been (and currently am) self-studying a bunch of (under)graduate subjects that I think would be helpful (mainly from the [Study Guide](https://www.lesswrong.com/posts/bjjbp5i5G8bekJuxv/study-guide)) and it'd really suck to have to take them all again just for meeting prereqs for advanced classes.
    *   I don't really care much about getting class credits as long as (1) I don't get kicked out of school for low credit and (2) the low credits or lack of prereqs won't prevent me from taking advanced subjects later on.
*   Are there any alignment research community/group/event nearby?
*   No need for financial aid right now.

The impression I got about Harvard (probably not so well-justified, just from anecdotes across reddit/etc) is that they're much less flexible in terms of class choices or prereqs compared to more traditionally "engineering" colleges like eg MIT. I also think the alignment community is mostly centered around the Bay area and that it hasn't really developed much around Harvard yet (I know about HAIST, though!)

Would Harvard be a good option to just go with, or is there enough additional value from Stanford/MIT/Harvey Mudd that it would be worth applying to any one of those colleges? Thanks!

(Apologies in advance if I broke any posting norms.)
# Vaguely interested in Effective Altruism? Please Take the Official 2022 EA Survey

**The 2022 Effective Altruism Survey is now live at the following link:** [**https://rethinkpriorities.qualtrics.com/jfe/form/SV_1NfgYhwzvlNGUom?source=lw**](https://rethinkpriorities.qualtrics.com/jfe/form/SV_1NfgYhwzvlNGUom?source=lw)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The survey will close at the end of the year.

### **What is the Effective Altruism Survey?**

The Effective Altruism (EA) Survey is the official survey run by Rethink Priorities and the Center for Effective Altruism to provide valuable information about the demographics of the EA community and other vital data used for EA community building. **LessWrong users remain an important part of this community and we want to ensure they are represented.**

We appreciate it when EAs share the survey with others. If you would like to do so, please use this link ([https://rethinkpriorities.qualtrics.com/jfe/form/SV_1NfgYhwzvlNGUom?source=shared](https://rethinkpriorities.qualtrics.com/jfe/form/SV_1NfgYhwzvlNGUom?source=shared)) so that we can track where our sample is recruited from.  
 

### **What's new this year?**

*   **The EA Survey is substantially shorter.** Our testers completed the survey in 10 minutes or less. 
*   **We worked with CEA to make it** **possible for some of your answers to be pre-filled with your previous responses,** **to save you even more time**. At present, this is only possible if you took the 2020 EA Survey and shared your data with CEA. This is because your responses are identified using your EffectiveAltruism.org log-in. In future years, we may be able to email you a custom link which would allow you to pre-fill, or simply not be shown, certain questions which you have answered before, whether or not you share your data with CEA, and there is an option to opt-in to this in this year’s survey.

### **Why take the EA Survey?**

The EA Survey provides valuable information about the EA community and how it is changing over time. Every year the survey is used to inform the decisions of a number of different EA orgs. And, despite the survey being much shorter this year, this year we have included requests from a wider variety of decision-makers than ever before.

###   
**Prize**

This year the Centre for Effective Altruism has, again, generously donated a prize of $1000 USD that will be awarded to a randomly selected respondent to the EA Survey, for them to donate to any of the organizations listed on EA Funds. Please note that to be eligible, you need to provide a valid e-mail address so that we can contact you.
# Paper: Constitutional AI: Harmlessness from AI Feedback (Anthropic)

The authors propose a method for training a harmless AI assistant that can supervise other AIs, using only a list of rules (a "constitution") as human oversight. The method involves two phases: first, the AI improves itself by generating and revising its own outputs; second, the AI learns from preference feedback, using a model that compares different outputs and rewards the better ones. The authors show that this method can produce a non-evasive AI that can explain why it rejects harmful queries, and that can reason in a transparent way, better than standard RLHF:![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/eef38f7cf657118b1edf535aa3702c790b6b71b9138fb406.png)

Paper abstract:

> As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through selfimprovement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as ‘Constitutional AI’. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use ‘RL from AI Feedback’ (RLAIF). As a result we are able to train a harmless but nonevasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.

See also Anthropic's tweet thread:  
[https://twitter.com/AnthropicAI/status/1603791161419698181](https://twitter.com/AnthropicAI/status/1603791161419698181)
# "Starry Night" Solstice Cookies

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/7505c26e40621e5998429697275ed91a2312c2140328d105.jpg)

There are no traditional aspiring-rationalist foods that I know of. Here's a potential candidate for one. I started with a friend's family recipe and adjusted it to be vegan and slightly more thematic.

The white mini chocolate chips symbolize humanity (the light in the world) in the darkness of the universe (the bitter chocolate cookie). The dried cherries can be... nebulae.

Makes around 12 large cookies.

### Ingredients

*   1 Tbsp instant espresso powder
*   1 Tbsp boiling water
*   133g (5/8c) coconut oil
*   176g (1c packed) brown sugar
*   80g (1/4c) mashed banana
*   1/4 tsp vanilla extract
*   120g (1c) flour
*   90g (3/4c)  unsweeted cocoa powder
*   1 tsp baking soda
*   2 tsp baking powder
*   1/4 tsp salt
*   100g (3.75oz, 3/4c) chopped sweetened dried cherries
*   80g (3oz, 1/2c) mini white chocolate chips (vegan)

### Directions

Preheat oven to 350 degrees and line baking sheets with parchment paper.

Mix instant espresso powder and water in a small bowl, and reserve.

Cream the coconut oil, then add brown sugar and cream together. Add mashed banana and vanilla and beat until thoroughly combined. Add reserved espresso/water mixture and beat until combined.

In a separate bowl, mix the flour, cocoa powder, baking soda, baking powder, and salt, then combine with the wet ingredients. (Or if you're lazy like me, put them on top of the wet ingredients and mix the dry ingredients with a fork before mixing fully.) Fully combine until a dough forms.

Mix in chopped dried cherries and mini white chocolate chips.

Form dough into flattened balls and place on cookie sheets. Bake for about 13 minutes.

Remove from oven and cool for 10 minutes before transferring cookies to a wire rack to cool completely.

These taste slightly better the next day; they get a bit chewier.
# Looking for an alignment tutor

Hey, this is [me](https://oatml.cs.ox.ac.uk//members/jan_brauner/). I’d like to understand AI X-risk better. Is anyone interested in being my “alignment tutor”, for maybe 1 h per week, or 1 h every two weeks? I’m happy to pay.

Fields I want to understand better:

*   Anything related to prosaic AI alignment/existential ML safety
*   Failure stories/threat models

Fields I’m not interested in (right now):

*   agent foundation
*   decision theory
*   other very mathsy stuff that’s not related to ML

My level of understanding:

*   I have a decent knowledge of ML/deep learning (I’m in the last year of my PhD)
*   I haven’t done the [AGI Safety Fundamentals](https://www.agisafetyfundamentals.com/ai-alignment-curriculum) course, but I just skimmed it, and I think I had independently read essentially all the core readings (which means I probably have also read many things not on the curriculum). I’d say I have a relatively deep understanding of a majority (but not all) of the content in this curriculum.
*   Similarly for the AGI Safety Fundamentals 201, excluding the tracks

Example questions I wrestled with recently, and I might have brought up during the tutoring:

*   It seems to me that our currently level of outer alignment tools (RLHF + easy augmentation) is enough to solve the outer alignment problem sufficiently well so that humans don’t end up dead or disempowered (conditional on slow takeoff); and then we can solve further outer alignment problem as the come up, with iteration and regulation. So I basically think that the core of the alignment problem, at the moment, is inner alignment + deceptive alignment. What am I missing? (I read Christiano’s “Another Outer Alignment Failure Story”, but I still have this question.)
*   I understand that a reward maximiser would wire-head (take control over the reward provision mechanism), but I don’t see why training an RL agent would necessarily end up in a reward-maximising agent? Turntrout’s [Reward is Not the Optimisation Target](https://www.lesswrong.com/posts/pdaGN6pQyQarFHXF4/reward-is-not-the-optimization-target) shed some clarity on this, but I definitely have remaining questions.
*   Is the failure mode describe in Ajeya’s [Without Specific Countermeasures](https://www.lesswrong.com/posts/pRkFkzwKZ2zfa3R6H/without-specific-countermeasures-the-easiest-path-to) an inner alignment failure, or an outer alignment failure (I think it’s both).

You don’t need to have very crisps answers to these to be my tutor, but you should probably have at least some good thoughts.
# Event [Berkeley]: Alignment Collaborator Speed-Meeting

If you’re working in alignment research, or would like to be, and you would like help finding researchers with whom to collaborate, this event is for you.

Inspired by the structure of a speed-dating event, participants will spend the first two hours in short one-on-one conversations with each other, changing partners at regular intervals, all for the goal of assessing fit as collaborators. The event will end with one hour of free-for-all socializing.

Please RSVP through Eventbrite or Facebook:  
Facebook: [https://www.facebook.com/events/688222332896562/](https://www.facebook.com/events/688222332896562/)  
Eventbrite: [https://www.eventbrite.com/e/alignment-collaborator-speed-meeting-tickets-490832281567](https://www.eventbrite.com/e/alignment-collaborator-speed-meeting-tickets-490832281567)

The event will take place at Constellation, on Jan 14, 2023.

Instructions for access:  
Upon arrival at the building (at the Shattuck Ave. entrance) text phone number [(510) 213-8521](tel:5102138521), and one of the event hosts will come down to let you up. Feel free to arrive as soon as 2:45 PM.
# CEA Disambiguation

In 2011 effective altruists in Oxford had two main organizations: [Giving What We Can](https://www.givingwhatwecan.org/) and [80,000 Hours](https://80000hours.org/). They wanted to incorporate, and created the [Centre for Effective Altruism](https://www.centreforeffectivealtruism.org/) as an umbrella organization to host them: \[1\]

[![](https://www.jefftk.com/cea-gwwc-80k.png)](https://www.jefftk.com/cea-gwwc-80k-big.png)

Over time they started running other projects: [conferences](https://www.eaglobal.org/), supporting local groups, the [EA forum](https://forum.effectivealtruism.org/), [community health](https://www.centreforeffectivealtruism.org/community-health), etc. There was effectively a "CEA" community-focused organization within the "CEA" umbrella organization:

[![](https://www.jefftk.com/cea-gwwc-80k-cea.png)](https://www.jefftk.com/cea-gwwc-80k-cea-big.png)

This was pretty confusing: when someone said "CEA" did they mean the organization focused on the EA community ("CEA runs EA Global") or the umbrella organization ("80k is part of CEA")? This got even more confusing as there started to be more organizations and projects:

[![](https://www.jefftk.com/cea-gwwc-80k-cea-etc.png)](https://www.jefftk.com/cea-gwwc-80k-cea-etc-big.png)

In September 2022 the umbrella organization renamed itself to the [Effective Ventures Foundation](https://ev.org/):

[![](https://www.jefftk.com/evf-gwwc-80k-cea-etc.png)](https://www.jefftk.comev-gwwc-80k-cea-etc-big.png)

Unfortunately the [announcement](https://forum.effectivealtruism.org/posts/9gHTYC5qbSH9E37vx/cea-ops-is-now-ev-ops) wasn't very clear about what specifically was changing, and a lot of people are [still](https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/why-did-cea-buy-wytham-abbey?commentId=F9SNzY8T6oTwhXZdq#F9SNzY8T6oTwhXZdq) [confused](https://forum.effectivealtruism.org/posts/xBfp3HGapGycuSa6H/i-m-less-approving-of-the-ea-community-now-than-before-the?commentId=hfibmmctEsuwBKqp3#hfibmmctEsuwBKqp3) about when to say "CEA" and when to say "EV". Hopefully this history and the diagrams clear things up a bit!

\[EDIT: changed 'EVF' to 'EV'; [Shakeel says](https://forum.effectivealtruism.org/posts/xaMLvzbaBMFjX88Z2/cea-disambiguation?commentId=sbxFkjEKBTFT5mbgh) they prefer the latter.\]

  

\[1\] This is also the origin of using [effective altruism](https://forum.effectivealtruism.org/posts/9a7xMXoSiQs3EYPA2/the-history-of-the-term-effective-altruism) to refer to the movement.
# Next Level Seinfeld

Thanks to [Jordan Schneider](https://twitter.com/jordanschnyc) gifting a last minute ticket for an amazing seat, serendipity led me to go to Friday night’s performance by Jerry Seinfeld at the Beacon Theater.

If I had to describe the show in one word, I’d say it was _unsurprising._ Jerry was Jerry. He had thoughts. Mostly he had complaints. They were all Very Seinfeld.

If I had two words I might say _mildly amusing._ Which was good enough for a worthwhile evening. Live performances are something special. Every time I’ve gone out to a comedy show, even if a bunch of it was kind of lame, I have been happy I came. The correct bar for worth watching is actually _lower_ in person than at home.

The thought I couldn’t shake as I went home was, _what GPT level was that on?_

This first came to me during the opening act. His opening act was if anything _too_ on the nose. Very Jerry. Much Seinfeld.

It felt _very much_ like he had GPT-X put together a package of Standard Mildly Amusing Jokes and Perspectives, with the prompt that the performer was black, old, male, never married and the opening act for Jerry Seinfeld. Given the material, delivery was solid.

The question was, what was the X in GPT-X?

I am confident our current incarnation, which is GPT-3.5 or so, can’t do it.

What about GPT-4? Could go either way, if you could try out bits and select winners.

What about GPT-5? Yes, absolutely, if it is what I’d expect, that should work.

So I’d give that act a GPT-level rating of 4.25.

Jerry himself was on a higher level. Not as high as his peak. I’d give him a 5.

What about the best stand-up shows and specials?

I’ve seen 6s. I worry I may have never seen a 7.
# Notice when you stop reading right before you understand

Here is a bad habit I noticed in myself. 

*   I encounter something I want to learn about. For example, let's pick transformers. 
*   I scroll through the tweet/paper/article and get some sort of general gist. *"Okay, there's positional encoding, and an encoder-decoder setup, and something called self-attention which people seem to think is important, and people claim this somehow makes it so that the model understands how words in a sentence relate to each other." *
*   Then, I think to myself: *"Understanding this topic seems relevant to my goals! But, I don't feel like putting in the mental energy to fully understand it. Maybe later." *
*   And then I abandon the topic. If transformers come up later, my knowledge is limited to a couple of related keywords ("self-attention", "encoder"/"decoder") with no real gears-level understanding of anything. 

It's like intellectual edging. I read just enough so that I almost understand the topic, but then I quit and never reach full understanding. 

I believe the source of this habit is 1) social media and 2) school. On social media, it's easy to get in the habit of scrolling past dense intellectual content, because the fluff goes down so much easier. With school, often the way to get adequate grades with minimal time spent is to learn just enough so that you can do shallow keyword matching—full understanding is not needed. 

I think this habit is bad because I end up spending too much time skimming content I don't need and won't remember, and not enough time trying to deeply understand content I do need and want to remember. 

I have noticed this habit in myself, and I intend to unlearn it. Here is the process I intend to follow instead:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/faa41fb904360ee7aff3131e0236280003f48f40b8196926.png)

My goal is to follow the "YES" branch at least once per day.

I consider this to be attainable because the topics I am envisioning are quite small ("understand how a basic transformer works", not "understand all variants of transformers ever used") and my bar for understanding is not too high ("be able to explain how transformers work to a technical friend", not "be able to implement a transformer from scratch with no references").
# Logging Shell History in Zsh

By default most shells don't log your history in a detailed durable way, which gives up one of the big advantages of working on the command line. Good history lets you look back at things you did months or years ago, in a searchable and skimmable fashion, so you can answer questions like "how did I generate this number?", "what was that trick I used?", or "if I'm doing something similar now what should I recall from last time?"

I use `bash` as my shell and have [long](shell-history-and-the-value-of-text-only-environments) [used](https://www.jefftk.com/p/you-should-be-logging-shell-history):

.bashrc:
  promptFunc() {
    echo "$(date +%Y-%m-%d--%H-%M-%S) $(hostname) $PWD $(history 1)" \
      >\> ~/.full_history
  }
  PROMPT_COMMAND=promptFunc

I was recommending this to folks at work, but they use `zsh`. In `zsh` you can get a lot of the way here by setting `INC_APPEND_HISTORY` (append to history immediately instead of waiting for the shell to exit), `SAVEHIST=1000000000` (effectively don't limit the history size on disk), and `EXTENDED_HISTORY` (to store timestamps with history entries). But you risk losing most of your history if you ever accidentally invoke `zsh` without these set, and it doesn't write the directory you ran the command in (which is metadata I reference a lot).

[Mike](https://mikemc.cc/) tweaked my snippet to run in `zsh`:

.zshrc:
  precmd() {
    echo "$(date +%Y-%m-%d--%H-%M-%S) $(hostname) $PWD $(history -1)" \
      >\> ~/.full_history
  }

The two changes are that `zsh` uses `precmd` instead of `PROMPT_COMMAND`, and that you need `history -1` instead of `history 1`.

You can still use the same `histgrep` command on both:

function histgrep {
  local n_lines=10
  if \[\[ "$1" =~ ^\[0-9\]*$ \]\]; then
    n_lines="$1"
    shift
  fi
  grep "$@" ~/.full\_history | tail -n "$n\_lines"
}

Note that this doesn't replace your shell's built-in history tooling, and I wouldn't recommend turning that off. This just a very cheap additional layer of logging with additional metadata and less risk of accidental deletion.
# Google Search loses to ChatGPT fair and square

I mentioned before that Google search is less than satisfactory even on uncomplicated queries: [https://www.lesswrong.com/posts/naAs59xiGfr7fPjej/google-search-as-a-washed-up-service-dog-i-halp](https://www.lesswrong.com/posts/naAs59xiGfr7fPjej/google-search-as-a-washed-up-service-dog-i-halp)

The linked article compares accuracy and usefulness of Google search with ChatGPT on some quite reasonable queries, and the latter seems to win hands down, even gimped by the lack of internet access (h/t [https://twitter.com/emollick/status/1605420852635488258).](https://twitter.com/emollick/status/1605420852635488258).) I am not sure what to make of it, since Google has its own AI lab that is apparently at least as advanced and with better staffing and funding...
# What readings did you consider best for the happy parts of the secular solstice?

I'm currently thinking about creating a new rationalist ritual day that focuses just on happiness. I'm thinking about reusing some elements that worked well at the solstice. Which readings did you enjoy very much during the happy parts of the solstice?
# Applied Linear Algebra Lecture Series

Over the past couple months, I gave weekly lectures on applied linear algebra. The lectures cover a grab-bag of topics which I've needed to know for my own work, but which typically either aren't covered in courses or are covered only briefly in advanced courses which use them (like e.g. quantum). The series is now complete, and recordings of all the lectures are available [here](https://www.youtube.com/playlist?list=PLiayR7yJx8-aCfBlccBjF1t-UO86fZJVu).

Be warned: all of the lectures were given with zero review and minimal prep. There are errors. There are poor explanations and too few examples. There are places where I only vaguely gesture at an idea and then say to google it if and when you need it. The flip side is that you will see only things I know off the top of my head - and therefore things which I've found useful enough often enough to remember.

Outline of Topics
-----------------

Lecture 1

*   Prototypical use cases of linear algebra
    *   First-order approximation of systems of equations for solving or stability analysis
    *   Second-order approximation of a scalar function in many dimensions for optimization or characterizing of peak/bowl shape
    *   First-order approximation of a dynamical system near a steady state
    *   Principal components of a covariance matrix

Lecture 2

*   Working with efficient representations of large matrices
    *   Tricks for jacobian and hessian matrices
    *   Prototypical API for implicit matrix representations: scipy's LinearOperator

Lecture 3

*   Suppose we look at a matrix (e.g. using pyplot.matshow()). What patterns are we most likely to see, and what can we do with them?
    *   Recognizing sparse & low-rank structure
    *   Interpreting sparse & low-rank structure
    *   Leveraging sparse & low-rank structure

Lecture 4

*   Matrix calculus, with a focus on stability of eigendecomposition
    *   Basics: tensor notation
    *   Differentiating eigendecomposition
    *   Instability of eigenvectors of (approximately) repeated eigenvalues

Lecture 5

*   Leveraging symmetry
    *   Suppose my system is invariant under some permutation (e.g. a PDE with wraparound boundary, or exchangeable variables in a covariance matrix). How can I leverage that to more efficiently find an eigendecomposition (and invert the matrix etc)?
    *   What Fourier transforms have to do with symmetry, and how to compute them quickly
*   How to represent rotations/orthogonal matrices

Lecture 6

*   Wedge products: those "dx dy dz" things in integrals
    *   How to do coordinate transformations with things like "dx dy", even when embedded in a higher-dimensional space
*   Map between function operations/properties and matrix operations/properties
# Mental acceptance and reflection

Naive investigation of the world often works, especially when we have direct access, fast feedback loops, and can try many times. Sports, games, and even parts of science often have properties. When we investigate, say, microorganisms, we can directly experiment on what we are interested in and can rapidly iterate. On the other hand, broadly investigating ourselves can be very difficult. **Biases, trauma, and all of the messy human stuff can interfere with directly accessing our motivations or reasons for decision-making. When we try to reflect on our actions, it is easy to become absorbed in our emotions and then unable to access the underlying reasons.**

For example, I often have trouble initiating communication with people I don’t know. This hasn’t been crippling, but [has prevented me from optimizing at times](https://www.lesswrong.com/s/EL2YvcrPNHrGgzPnZ/p/ddd3eBucFN3ZdbFzx). Reflecting on this can make me feel guilty about how this has interfered with decent decision-making on my part. **Reflecting is miserable because it brings up those feelings of guilt, and those very same feelings make it hard to consider what I should do to ameliorate these anxieties going forward or fix any mistakes. Yet, dealing with trauma or biases in how we view ourselves is also very difficult. It is hard not to feel guilt!**

A potential way of bypassing this is by shifting our perspective of ourselves to that of a somewhat easier problem: science. [We are all part of physics](https://www.lesswrong.com/posts/NEeW7eSXThPz7o4Ne/thou-art-physics). We are not special. If we can see ourselves as a physical process, we can apply our epistemology to ourselves to better understand and shape the actions we take. Accepting that we are part of reality, just as much as a falling rock, we can investigate ourselves like any other object in science. By applying the scientific method to ourselves, we can observe phenomena and extract regularities in our behavior. By simply looking at causal relationships, we can step back and make ourselves more detached, but detached in a way that allows us to still consider our actions. 

So instead of wrestling with these feelings of guilt, I can examine the causes of my anxieties. I’m fine talking with people once we start talking, so this anxiety is just isolated to the step of first contact. Once I isolate the cause, I can devise strategies of slowly acclimating myself, starting with, say, interacting with new people anonymously online. This approach of viewing myself as just another object of my epistemology helps sidestep all the messy biases and feelings of guilt.

**Naively applying epistemology to ourselves can be very hard. One approach to doing this is to see yourself as part of the world, and applying your science and epistemology to yourself can be an effective method of stepping back and allowing you to reflect.**
# "Search" is dead. What is the new paradigm?

For over two decades internet search (which has been largely synonymous with Google for most of that time) has been the main way to augment the human brain with the digital information outside of it. First the virtual assistants and now LLMs are challenging and changing the way people access, process and use the data. It is becoming more like a true capability augmentation, though still clunky and unreliable, but improving at a dizzying rate. Specifically, it is becoming amazingly easier to create, rather than consume, though consumption is greatly facilitated, too. 

Assuming we are still ways away from the prophesied AI doom, what might be a better name for the way people interact with information?
# Why is "Argument Mapping" Not More Common in EA/Rationality (And What Objections Should I Address in a Post on the Topic?)

I recently published [an article](https://georgetownsecuritystudiesreview.org/2022/11/30/complexity-demands-adaptation-two-proposals-for-facilitating-better-debate-in-international-relations-and-conflict-research/) in the Georgetown Security Studies Review (GSSR) on the use of "argument management systems" (e.g., [Kialo](https://www.kialo.com/should-jury-trials-be-abolished-9360)) for the complex debates that arise in fields where it's often impractical to resolve disagreements through standard empirical methods (e.g., RCTs). I've long been confused why this method of discussion is not more widely supported in EA and Rationalist circles, and am considering adapting my GSSR [article](https://georgetownsecuritystudiesreview.org/2022/11/30/complexity-demands-adaptation-two-proposals-for-facilitating-better-debate-in-international-relations-and-conflict-research/) to AI policy/safety research (or Longtermism more generally, which various people have criticized for being too speculative/theoretical rather than being based on empirical tests) and posting it here. However, before doing that I would love to get a sense of people's reasons for skepticism or apathy towards such methods, so I could potentially address them in the post.

For what it's worth, I have seen Leverage Research's report on the topic, and I am aware of the criticism that "argument mapping" (in some formats) is overly formal and too complicated. (I plan to respond to these points)

In short, I expect my argument to be fairly similar to what I laid out in my GSSR article: the way we currently present arguments (i.e., predominantly through prose/paragraph text) seems rife with points of failure and inefficiencies, especially given how debates are often not linear but rather branch and have cross-cutting points. In fields like international relations and peace/conflict studies I've repeatedly encountered instances where people fail to (seriously) address existing counterarguments, and more generally it is hard for audiences to determine who has or hasn't addressed counterarguments. In contrast, I think that making one's arguments more explicit and keeping track of the arguments in a more-searchable and more-permanent format than "memory" or prose would help to mitigate some of these problems. 

To me, better methods of argumentation seems like a natural extension of norms that promote statistics/experimental methods in science, but thus far I've found the EA/Rationalist communities fairly lukewarm towards the ideas (even if they are more receptive on average than the general public).
# Non-Elevated Air Purifiers

Last year for Christmas gatherings at my dad's house I put together [wall-mounted filter cubes](https://www.jefftk.com/p/elevated-air-purifier-cubes), to get them up and out of the way. This time around this seemed like too much hassle, so I put one together on top of the piano, one in a bookcase, and used one commercial purifier:

[![](https://www.jefftk.com/filters-on-piano.jpg)](https://www.jefftk.com/filters-on-piano-big.jpg)

It's double-ended because I had an extra fan. You do need to remember to turn on both fans. Probably would be slightly more effective if I'd put a partition in the middle to make it actually be two independent cubes. I used the 12x20 filters from my [ceiling fan air purifier](https://www.jefftk.com/p/ceiling-air-purifier) project.

The kitchen has a commercial purifier, which has a similar capacity to the filter cubes while taking less space. If I wanted to run these all the time I might buy several commercial purifiers, but since these are just for occasional use that seems less worth it.

[![](https://www.jefftk.com/commercial-purifier-in-kitchen.jpg)](https://www.jefftk.com/commercial-purifier-in-kitchen-big.jpg)

For the living room, [Julia](https://juliawise.net/) had the clever idea of using a part of the built-in shelving that was exactly the right size:

[![](https://www.jefftk.com/bookcase-filter-cube-construction.jpg)](https://www.jefftk.com/bookcase-filter-cube-construction-big.jpg)

[![](https://www.jefftk.com/bookcase-filter-cube.jpg)](https://www.jefftk.com/bookcase-filter-cube-big.jpg)

This is only two filters worth, but I still feel a lot of airflow from the fans. It's a good location for not being in the way, though we did need to talk to the kids about leaving it alone since it would be easily damaged if bumped.

I should make some shrouds for the fans.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/pfbid0HeNzPcJPEVwSu2A6Vn568eziURcoimREToc2v6rjLr28EHLb16gFxhxqF5pAsrxLl), [mastodon](https://mastodon.mit.edu/@jefftk/109568663475873856)*