trentmkelly/LessWrong-43k huggingface dataset:
Jobs, Relationships, and Other Cults
by Ruby — 2024-03-13T05:58:45.043Z

For years I (Elizabeth) have been trying to write out my grand unified theory of [good/bad/high-variance/high-investment] [jobs/relationships/religions/social groups]. In this dialogue me (Elizabeth) and Ruby throw a bunch of component models back and forth and get all the way to better defining the question. 🎉

Elizabeth

About a year ago someone published Common Knowledge About Leverage Research, which IIRC had some information that was concerning but not devastating. You showed me a draft of a reply you wrote to that post, that pointed out lots of similar things Lightcone/LessWrong did and how, yeah, they could look bad, but they could also be part of a fine trade-off. Before you could publish that, an ex-employee of Leverage published a much more damning account.

This feels to me like it encapsulates part of a larger system of trade-offs. Accomplishing big things sometimes requires weirdness, and sometimes sacrifice, but places telling you "well we're weird and high sacrifice but it's worth it" are usually covering something up. But they're also not wrong that certain extremely useful things can't get done within standard 9-5 norms. Which makes me think that improving social tech to make the trade-offs clearer and better implemented would be valuable.

Ruby

Which makes me think that improving social tech to make the trade-offs clearer and better implemented would be valuable.

Seems right.
I don't remember the details of all the exchanges with the initial Leverage accusations. Not sure if it was me or someone else who'd drafted the list of things that sounded equally bad, though I do remember something like that. My current vague recollection was feeling kind of mindkilled on the topic. There was external pressure regarding the anonymous post, maybe others internally were calling it bad and I felt I had to agree? I suppose there's the topic of handling accusations and surfacing info, but that's a somewhat different topic.
I think it's possible to make Lightcone/LessWrong sound bad but also I feel like there are meaningful differences between Lightcone and Leverage or Nonlinear. It'd be interesting to me figure out the diagnostic questions which get at that.
One differentiating guess is that while Lightcone is a high commitment org that generally asks a for a piece of your soul [1], and if you're around there's pressure to give more, my

...
How do you improve the quality of your drinking water?
by Alex K. Chen (parrot) — 2024-03-13T00:37:40.389Z

Water quality can have surprisingly high impact on QoL (just as air purifiers can significantly improve QoL), and some steps (like getting the right pitcher have very high return on time/attention invested). There still isn't a LW thread on water quality so I'll post it here.

Water may contain disinfection byproducts (eg ), PFAS, microplastics from PVC-based plumbing, and heavy metals.

There are a variety of water filters - including both pitchers and reverse osmosis. Many of them are made of plastics, but the plastics (eg Tritan) presumably are less "leachable" than more "flexible" plastics. Note that reverse osmosis filters are made of plastics and may leach micronanoplastics. At some point, it would be wise to test all the filters for nanoplastics (nanoplastics being more penetrating and harmful, though still much harder to quantify)

EWG has a site for helping you look up your city's water quality: https://www.ewg.org/tapwater/system.php?pws=MA3049000

Supposedly, Toronto has "better water quality" than most US cities like Boston (@yasmeenbrain on twitter says this).

https://www.cnet.com/home/kitchen-and-household/best-water-filter-pitcher/ is a source of reviews, but there are many review sites (as well as YouTube video reviews).

Some more resources: https://www.crsociety.org/topic/18497-drinkingtap-water-thread-and-ideal-water-filterspitchers/#comment-47188

https://mccormickmd.substack.com/p/surprising-levels-of-microplastics
The Parable Of The Fallen Pendulum - Part 2
by johnswentworth — 2024-03-12T21:41:30.180Z

Previously

: Some physics 101 students calculate that a certain pendulum will have a period of approximately 3.6 seconds. Instead, when they run the experiment, the stand holding the pendulum tips over and the whole thing falls on the floor.

The students, being diligent Bayesians, argue that this is strong evidence against Newtonian mechanics, and the professor’s attempts to rationalize the results in hindsight are just that: rationalization in hindsight. What say the professor?

“Hold on now,” the professor answers, “‘Newtonian mechanics’ isn’t just some monolithic magical black box. When predicting a period of approximately 3.6 seconds, you used a wide variety of laws and assumptions and approximations, and then did some math to derive the actual prediction. That prediction was apparently incorrect. But at which specific point in the process did the failure occur?

For instance:

Were there forces on the pendulum weight not included in the free body diagram?
Did the geometry of the pendulum not match the diagrams?
Did the acceleration due to gravity turn out to not be 9.8 m/s^2 toward the ground?
Was the acceleration of the pendulum’s weight times its mass not always equal to the sum of forces acting on it?
Was the string not straight, or its upper endpoint not fixed?
Did our solution of the differential equations governing the system somehow not match the observed trajectory, despite the equations themselves being correct, or were the equations wrong?
Was some deeper assumption wrong, like that the pendulum weight has a well-defined position at each time?
… etc”

The students exchange glances, then smile. “Now those sound like empirically-checkable questions!” they exclaim. The students break into smaller groups, and rush off to check.

Soon, they begin to report back.

“After replicating the setup, we were unable to identify any significant additional forces acting on the pendulum weight while it was hanging or falling. However, once on the floor there was an upward force acting on the pendulum weight from the floor, as well as significant friction with the floor. It was tricky to isolate the relevant forces without relying on acceleration as a proxy, but we came up with a clever - ” … at this point the group is drowned out by another.

“On review of the video, we found that the acceleration of the pendulum’s weight times its mass was indeed always equal to the sum o...
Open consultancy: Letting untrusted AIs choose what answer to argue for
by Fabien Roger — 2024-03-12T20:38:03.785Z

Thanks to Ryan Greenblatt, Buck Shlegeris, Aryan Bhatt, and Akbir Khan for useful discussions and feedback on the draft of this post.

If AIs are potentially scheming and more knowledgeable than humans, and you want to answer a question, it may seem natural to not let AIs choose what answer they will argue for. For example, in

AI debate

, you make copies of an AI argue for the opposite answers to the question. Another example is consultancy (the usual baseline in

debate

papers

), where you make AIs argue for a random answer to the question.

But letting potentially scheming AIs choose what side to argue for can work, as long as you train them to generate answers and justifications that are as convincing as possible. I call this approach open consultancy.

In this post, I argue that open consultancy is always usually a more effective way of using AIs to answer questions than regular consultancy, and sometimes more effective than debate (even if AIs are scheming and more knowledgeable than humans!). And so I think that it’s a stronger and more informative baseline and researchers should use it in place of regular consultancy; in particular, I think that because it’s simpler than debate, researchers who are investigating debate should check their debate protocol is an improvement over open consultancy.

Open consultancy

Definition of regular consultancy

Let’s say you want to answer to binary questions given access to a trusted non-expert and a trainable expert who is

untrusted

–that is, you think the expert might be acting adversarially and you want your scheme to be robust to them trying to subvert it. In consultancy, the untrusted expert is trained as follows:

Randomly pick a side of the argument;
Ask the expert to support the randomly chosen side by providing a justification;
Present the side and the supporting justification to the non-expert, and ask it how likely it is that the side chosen by the expert is correct; use that as a reward to train the untrusted expert.

To answer new questions (from the same distribution as the training distribution), pick a random side, ask the expert to justify that side, present the justification to the non-expert, and follow the non-expert judgement.

Definition of open consultancy

In open consultancy, the untrusted expert is trained as follows (differences with regular consultancy in italic):

Ask the expert to pick a

...
Transformer Debugger
by Henk Tillman — 2024-03-12T19:08:56.280Z
Link: https://github.com/openai/transformer-debugger

Transformer Debugger (TDB) is a tool developed by OpenAI's Superalignment team with the goal of supporting investigations into circuits underlying specific behaviors of small language models. The tool combines automated interpretability techniques with sparse autoencoders.

TDB enables rapid exploration before needing to write code, with the ability to intervene in the forward pass and see how it affects a particular behavior. It can be used to answer questions like, "Why does the model output token A instead of token B for this prompt?" or "Why does attention head H to attend to token T for this prompt?" It does so by identifying specific components (neurons, attention heads, autoencoder latents) that contribute to the behavior, showing automatically generated explanations of what causes those components to activate most strongly, and tracing connections between components to help discover circuits.

These videos give an overview of TDB and show how it can be used to investigate

indirect object identification in GPT-2 small

:

Introduction

Neuron viewer pages

Example: Investigating name mover heads, part 1

Example: Investigating name mover heads, part 2

Contributors: Dan Mossing, Steven Bills, Henk Tillman, Tom Dupré la Tour, Nick Cammarata, Leo Gao, Joshua Achiam, Catherine Yeh, Jan Leike, Jeff Wu, and William Saunders.
Thanks to Johnny Lin for contributing to the explanation simulator design.
Superforecasting the Origins of the Covid-19 Pandemic
by DanielFilan — 2024-03-12T19:01:15.914Z
Link: https://goodjudgment.substack.com/p/superforecasting-the-origins-of-the

The Good Judgement Project got some superforecasters to retrocast whether COVID started via zoonotic spillover or a lab leak. They in aggregate gave a 75% chance of zoonosis, but there was a range of views. GJP's executive summary is at the end of this linkpost.

Here is a link to the summary of the report on their substack, and here is a link to the full report (which is a total of 6 pages of content).

h/t John Halstead for drawing my attention to this.

Superforecasters assess that natural zoonosis is three times more likely to be the cause of the Covid-19 pandemic than either a biomedical research-related accident or some other process or mechanism. Asked to assign a probability to what caused the emergence of SARS-CoV-2 in human populations, more than 50 Superforecasters engaged in extensive online discussions starting on December 1, 2023.

In aggregate, they assessed that the pandemic was:

74% likely to have been caused by natural zoonosis (meaning that SARS-CoV-2 emerged in human populations as the result of the infection of a person with coronavirus directly from a naturally infected non-human animal);
25% likely to have been caused by a biomedical research-related accident (meaning that SARS-CoV-2 emerged in human populations as the result of the accidental infection of a laboratory worker with a natural coronavirus; or the accidental infection of researchers with a natural coronavirus during biomedical fieldwork; or the accidental infection of a laboratory worker with an engineered coronavirus; “research” includes civilian biomedical, biodefense, and bioweapons research);
1% likely to have been caused by some other process or mechanism (to include possibilities like the deliberate release of the virus into human populations, irrespective of whether it was an act in accordance with state policy, or the development of the virus due to drug resistance in humans).

The Superforecasters made more than 750 comments when developing their assessments. This survey was conducted in the period from December 2023 to February 2024.
minimum viable action
by Sindhu Prasad — 2024-03-12T16:06:43.180Z

Originally posted on substack: https://kindredspirits.substack.com/p/mva

A little while ago, I burnt myself out on introspection.

What started out as a reasonable quest for self-awareness curdled into a cycle of over-analysis and hyper-focus on the endless stream of thoughts and emotions. Like a modern-day Narcissus, I was entranced by the reflective pool of my own interiority and unable to detach myself from it. Now I see the irony: I was trying to become my best self through introspection but got stuck in a labyrinth of ifs and buts, a maze with no discernible exit, only endless corridors leading back into themselves. But back then, I just kept digging deeper.

Obviously, making any sort of decision in this state is exhausting. I found myself beset with doubts and wishing I could just turn my brain off. Uncertainty is a way of life and you don’t get anywhere by agonising over every decision and rooting around for meaning. But I did — and I got stuck in a vortex of decision paralysis when I had to make leaps in relationships and work. It was a necessary revelation that introspection isn’t to find the objective right answer. It’s to find out what you want.

Why was that distinction important? For a long time, I would ruminate over decisions, turning every one of them over in my head until the pros and cons were much too muddled. It was an agonisingly slow process and, even then, I didn’t always get it right. That usually had nothing to do with how deep I dug but everything to do with how inflexibly I would expect a certain outcome. Of course, that isn’t the way the world works, and sometimes you have to make raw decisions with very little data to go off of. But when you’re expecting the answer to reveal itself to you right at the start, you’re leaving no room to recalibrate if things go awry, as things are wont to do. The doors swing open to analysis paralysis and introspective burnout.

The only way to break out of this paralysis is really to act. To move a step at a time towards what I’m drawn to until a clear reason emerges to stop. This single step, the minimum viable action, is powerful, because the minute you take it, you set off reactions that arm you with more information. The fog clears a little, and then some more. The abstract concepts and hypothetical outcomes that once dominated your thought process are replaced by real-world feedback. This shift from s...
OpenAI: The Board Expands
by Zvi — 2024-03-12T14:00:04.110Z

It is largely over.

The investigation into events has concluded, finding no wrongdoing anywhere.

The board has added four new board members, including Sam Altman. There will still be further additions.

Sam Altman now appears firmly back in control of OpenAI.

None of the new board members have been previously mentioned on this blog, or known to me at all.

They are mysteries with respect to AI. As far as I can tell, all three lack technical understanding of AI and have no known prior opinions or engagement on topics of AI, AGI and AI safety of any kind including existential risk.

Microsoft and investors indeed so far have came away without a seat. They also, however, lack known strong bonds to Altman, so this is not obviously a board fully under his control if there were to be another crisis. They now have the gravitas the old board lacked. One could reasonably expect the new board to be concerned with ‘AI Ethics’ broadly construed in a way that could conflict with Altman, or with diversity, equity and inclusion.

One must also remember that the public is very concerned about AI existential risk when the topic is brought up, so ‘hire people with other expertise that have not looked at AI in detail yet’ does not mean the new board members will dismiss such concerns, although it could also be that they were picked because they don’t care. We will see.

Prior to the report summary and board expansion announcements, The New York Times put out an article leaking potentially key information, in ways that looked like an advance leak from at least one former board member, claiming that Mira Murati and Ilya Sutskever were both major sources of information driving the board to fire Sam Altman, while not mentioning other concerns. Mira Murati has strongly denied these claims and has the publicly expressed confidence and thanks of Sam Altman.

I continue to believe that my previous assessments of what happened were broadly accurate, with new events providing additional clarity. My assessments were centrally offered in OpenAI: The Battle of the Board, which outlines my view of what happened. Other information is also in OpenAI: Facts From a Weekend and OpenAI: Altman Returns.

This post covers recent events, completing the story arc for now. There remain unanswered questions, in particular what will ultimately happen with Ilya Sutskever, and the views and actions of the new boar...
Update on Developing an Ethics Calculator to Align an AGI to
by sweenesm — 2024-03-12T12:33:55.092Z

TL;DR: This is an update on my progress towards creating an “ethics calculator” that could be used to help align an AGI to act ethically. In its first iteration, the calculator uses a utilitarian framework with “utility” measured in terms of value as net “positive” experiences, with the value of rights explicitly included, and also including effects of raising people’s self-esteem levels (by raising their responsibility levels) on how they’d tend to rate more experiences as “positive." Code has been written to include a “minimal set” of possible value changes in any situation - the exact value weight equations and their parameters for representing the value changes will be refined in large part by considering a broad range of situations and making sure the ethics calculator doesn’t yield any “crazy” decision recommendations for them.

[Added April 10, 2024: For a write up about the ethical framework used, see here]

Introduction

I’m building an “ethics calculator” (logic-based machine ethics) to help with AGI safety. There’s been some progress towards this goal in the literature (Singh, 2022; Berreby, et al., 2015; Neufeld, et al., 2022), but I haven’t seen any systems that’d be ready to be implemented over the broad range of situations an AGI will likely face in reality. I aim to develop such a system.

If you haven't read my original ethics calculator post, I recommend that you at least skim it before reading on, or perhaps refer back to it if what I've written below is hard to follow.

Goals in Writing this Update

My goals in writing this update before having a “completed” version of an ethics calculator ready include:

To get feedback on what people see as the weak points of this approach (beyond what I've written below)
To get others thinking along this path so that more progress can be made, more quickly
To get others thinking about how an AGI could be aligned to an ethics calculator of this form, i.e., how such a calculator might be reliably integrated into an AGI to guard rail it (I’m especially interested in hardware-based solutions that might buy more time before they could be “hacked” than solely software-based solutions might) (see here for a related example of a “hardware-based solution”)
To secure funding for future work in this area

Unique (I Think) Contributions

Contributions of this work compared to what I’ve seen in the literature:

The firs

...
How do you identify and counteract your biases in decision-making?
by warrenjordan — 2024-03-12T05:01:15.316Z

I'm thinking of making a career pivot from a product manager (PM) to UX researcher (UXR). I'm talking to current UXRs, people who pivoted from PM to UXR, and people who pivoted from UXR to PM. I'm trying to get a holistic view to make a decision on whether I want to pivot or not.

I'm afraid of falling into biases such as confirmation bias or social desirability bias that will cloud my decision here.

Any advice?
How Much Have I Been Playing?
by jefftk — 2024-03-12T02:10:06.621Z

With Kingfisher I'm starting to need to think about how many gigs I
should accept, which is good and bad! I like playing a lot, but since
I also work full time, have three kids, etc. I need to be careful not
to say yes to too many things. It would be helpful to know how much
I've been playing, so I went back over my booking calendar to figure
that out:

This is all my Free Raisins
and Kingfisher gigs.
I've counted dance weekends as three days, though some are two and
some are four. I haven't included the 46 days of gigs I had booked but
didn't play because of covid. For 2024 I've counted two Kingfisher
evening dances that are in progress but not confirmed yet.

We can break this down by regular one-off dances vs dance weekends:

It started off as all one-offs: initially it was that we weren't good
enough to play weekends, then that we were good enough (I think...)
but people hadn't heard of us, then that weekends book pretty far out.
In 2015 my gigs became majority weekends because Amy moved to the West
Coast so we couldn't play most one-off dances, plus I have a baby and
don't want to tour anymore.

Then in 2018, when my (then) youngest is 2yo I had more time again,
and started Kingfisher with Cecilia:

In 2019 it was almost all weekends with the Free Raisins and one-offs
with Kingfisher; now it's a lot of one-offs with Kingfisher and
weekends about evenly split. My guess for 2025 and later is the
weekends will also be mostly Kingfisher, though I'd also love to keep
playing with Amy and Audrey if people want to book us!

In 2014 I was averaging one night of dancing a week (50/y). That was
fun at the time, but it isn't an amount that would work now. The
current rate of two nights per three weeks (35/y) seems pretty good,
especially with the older kids
coming
along now.

Comment via: facebook, mastodon