trentmkelly/LessWrong-43k huggingface dataset:
Underappreciated content on LessWrong
by Ege Erdil — 2022-04-11T17:40:15.487Z

It's likely that there is some mismatch between your preferences and the preferences of the LessWrong community about which posts or comments are good or deserving of upvotes. In addition to these differences, good content sometimes doesn't receive much attention from the community because it's posted in times of high traffic (such as Good Hearts Week), because too much context is required to understand it, or just because of other sources of noise that we don't understand.

What are some pieces of content posted on LessWrong that you think more people should read but that didn't get much in the way of upvotes and engagement? Questions, personal blog posts, frontpage posts, and comments are all admissible responses.
Editing Advice for LessWrong Users
by JustisMills — 2022-04-11T16:32:17.530Z

Who am I?

I'm an editor who does feedback requests for LessWrong. Many of these feedback requests are for proofreading/clarity of language, so I have lots of experience reading community draft posts and helping improve them. I want to share some high level takeaways - low effort self-editing techniques that, based on my[1] experience, I think many to most LW posters could benefit from.

Disclaimers

I wrote all the example text in this post myself. I make no claims that any of it is reasonable on the object level - it's just meant to resemble lots of the stuff I edit.

Bolded text in examples is for emphasis of the points under discussion, not a sign that it would make sense to actually bold those words or phrases.

As usual, reverse advice as necessary.

What's the advice?

Here's my main advice, in decreasing order of confidence - I'll elaborate on each:

Beware "this"
Don't let hedging be a tic
Break up run-on sentences
Use more links, images, examples, and commas

Beware "this"

The problem: many sentences on LessWrong begin with "this". At worst, the thing "this" is pointing to is fully ambiguous between multiple live options, making it impossible to parse the author's intended meaning. But even in milder cases, it can waste a few seconds for the reader to trace out what a post is talking about.

The solution: scan through your draft for words like "this" or "that" (but for whatever reason overwhelmingly "this" specifically), and when in doubt about clarity, just replace them with whatever their intended antecedents are.

An example:

Many key figures in the community have been shortening their timelines recently, since major advances just keep coming one after the other. This is especially worrying, especially compared to discussions of "AI winter" just a few years ago.

What, precisely, does "this" point to in the sentence above? The fact of key figures shortening their timelines? The major advances? When we get later in the sentence and see that we're comparing whatever "this" is to "discussions", we can infer that we're talking about the shortening of timelines (vs. the actual technical advances), but it takes up precious reader bandwidth to figure this out, or, worse, the reader becomes overloaded by ambiguity and unconsciously switches to "skimming mode".

Individually, overuse of "this" isn't a big deal. But it adds up. It's the single greatest contribution to a hand...
Post-history is written by the martyrs
by Veedrac — 2022-04-11T15:45:14.997Z
Link: https://www.royalroad.com/fiction/35360/post-history-is-written-by-the-martyrs/chapter/544607/post-history-is-written-by-the-martyrs

This is a story I wrote in late 2020, a few months after GPT-3 was released. Speculative fiction seems to be well received here, and although this is not as serious a story as my other recent post, it covers serious ideas and attempts to be hard science fiction. I'm also moving it here to more easily reference it in the future.

People who have been closely following tech news might be aware of GPT-3, a Machine Learning based language model by OpenAI that has incredible abilities. Watch GPT-3 write a small app based on a description, play 19 degrees of Kevin Bacon, give coherent reasoning on meaningful questions, diagnose a patient from a description, perform grounded world-modelling, rephrase messages for politeness, write non-literal poetry translations with explanatory annotations, and write a response to philosophers discussing its importance. Most importantly, this model was only trained to predict text, and its only major innovation was scale to a slightly larger sliver of the size of the human brain. The natural conclusion is that machine models are capable of learning more, and better, than they have historically been given credit.

This story is the result of thinking earnestly about the implications of GPT-3. Relative to the scale of a human brain, or to even larger model sizes potentially within reach, what aspects of human cognition can we reasonably argue will not be replicated by scaled-up neural networks? Which aspects are we now required to start to seriously expect to be within short reach? The answers to these questions do not neatly fit in a foreword.

This story is not serious speculation, but indulgence in a silly hypothetical that got itself stuck in my head, asking to be written. This is not the course the future will run, with fortuitous coincidences and cooperative governments, but I hope, at least, it makes for a fun tale.

This story was not written using GPT-3.

My father sometimes told me about the internet before GPT-3. Some time ago—I don’t remember when, I think I was young—he told me that, before GPT-3, the internet was alive. That there was something about its rawness and its viscerality that made it a place you could become a part of, somewhere you could defend as if it was your homeland. I don’t really know what that means. You’d have to be hurt in the head to treat the internet like that today. The internet I know is but peat...
What Chords Do You Need?
by jefftk — 2022-04-11T15:00:01.716Z

You famously only need

four
chords to play everything, or maybe it's only

three. But how true is this really?

I decided to go look and see what was common. I looked at a large
number of transcribed songs, writing down which chords were used.
This isn't perfect: some chords are more optional than others, but
it's about right. I only wrote down the triad: C7 and Cmaj7 are just
C. I used three sources:

Rise Up
Singing: a songbook popular at singalongs. I flipped through and
took the chords the 277 songs that seemed most familiar to me.

The Portland
Collection: a tunebook popular among contra dance musicians. I
did the same flipping through, with all three volumes, and took the
chords from 89 tunes.

Farm and
Wilderness String Band Tunebook, Third Edition. All 42 tunes.

Out of these 408 songs, here's how many include each chord:

This isn't a great way to look at it, though, because it is absolute
and music is (mostly) relative. We perceive a song that goes "C F G"
very similarly to one that goes "A D E". Musicians will often use
"Roman Numeral notation" to talk about chords in a purely relative
sense. If we look at "C F G" relative to "C", or "A D E" relative to "A",
we can call them "I IV V": they are each three major chords using the first,
fourth, and fifth notes of the major scale. We indicate minor chords
by using a lowercase number: "iv" would be "Am" relative to "C", or
"F#m" relative to "A".

Here's what this looks like in C, for example:

root
major
minor

C
I
i

C#
bII
bii

D
II
ii

D#
bIII
biii

E
III
iii

F
IV
iv

F#
bV
bv

G
V
v

Ab
bVI
bvi

A
VI
vi

Bb
bVII
bvii

B
VII
vii

Here's the same chart as above, but with relative chords:

This is mechanically converted, and assumes every song is using the
standard major scale, which of course they aren't: see all those "i"
chords? Instead of thinking of a song with "Am C F G" as being "i
bIII bVI bVII", we should think of it as "vi I IV V", as if it is in C
(the relative major). This focuses our distribution:

Some songs are neither major nor minor, but in other modes, typically
mixolydian. These have chords like "A G D". Instead of "A G D" being
"I bVII IV", we can think of it using the corresponding major and use "V IV I":

While it's almost always I, IV, V, and vi, we have both II/ii and
III/iii, diffe...
What can people not smart/technical/"competent" enough for AI research/AI risk work do to reduce AI-risk/maximize AI safety? (which is most people?)
by Alex K. Chen (parrot) — 2022-04-11T14:05:33.979Z

(aka the vast majority of people in the world). I know that many of them can still benefit from those who are in support roles for them. The speed at which top AI researchers need to learn new AI content is.. just.. extraordinary even by the standards of any other academic field.

I've found that people who have "taste/context" for the most important things are still extraordinarily rare, and that when someone is in the right communities, having taste/context more than makes up for 1.5 SDs in deficits of intelligence (most at the middle-high range).

I think it's first of all important to note the reading streams of AI researchers or AI risk workers (eg see if they follow health nuts like Brad Stanfield or Mike Lustgarten). Fluid intelligence, brain size, memory, and neuroplasticity all decline after the 20s (but there is a very high amount of variation in how fast people's brain sizes decline with age, especially given how poor the western diet is), so doing anything to slow the decline in AI researchers could help [eg EAG events already serve super-healthy vegan food for meals, so have a personal chef for them who continually serves them the healthiest food possible -I know several AI safety people at EAG Boston last week commented "I never could imagine that vegan food could taste so good"]. Simply having high-normal blood sugar levels shrinks brain sizes by age 60 (and SGLT2 inhibitors, newly available on unitedpharmacies.md, do a lot to reduce them, esp when paired with metformin).

Reducing lead/microplastic pollution rates is huge (where lead went down, now we have an explosive increase in microplastics)

There are some newly released interventions (eg rapamycin+SGLT2 inhibitors (like empagliflozin/canagliflozin)+metformin) that can do A LOT to reduce aging rate and the risk of diseases later on, esp as most AI researchers (esp the ones at Anthropic) aren't getting any younger. There are health coaches like those at Apeiron Health who can monitor biometrics (including EEG data, Kernel data, and Health Nucleus data) of AI researchers to ensure that they stay at "peak performance" for their lives (athletes like bengreenfieldfitness.com use it - so we need to port it to AI researchers).

AI researchers also tend to be in a monoculture where they hang around other people who are academically successful AI researchers, but not with empaths who might not be aca...
Goodhart's Law Causal Diagrams
by JustinShovelain — 2022-04-11T13:52:33.575Z

Goodhart's law is closely related to both

inner

and outer alignment problems and the principal agent problem. Understanding it better should help us solve these problems. This post is an attempt to work out a more precise, unified way of thinking about part of Goodhart’s law.

More specifically, we will interpret Goodhart's law using causal diagrams. We will introduce this framing and look at one class of situations where a Goodhart problem might arise. In future posts, we will describe a more complete categorisation of these situations and how they relate to the AI alignment problem.

Definition of Goodhart’s law

The standard definition of Goodhart’s law is: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

What does this mean precisely and in what ways does this happen? Let’s zoom in on each part of this definition.

What does “Any observed statistical regularity” mean? In the language of causal diagrams, this statement can be translated as some set of causal relationships between observed and controlled variables. In the case of Goodhart’s law, one of these variables is a target (measures what you actually want), and one is a proxy (the variable you measure while optimizing for the target). There may be a complex set of connections and latent variables between the proxy, the target, and any variables under your control when taking actions. Let’s call any variable under your direct control an action variable.

Given this interpretation of “observed statistical regularity” what does “will tend to collapse once pressure is placed upon it for control purposes” mean? At a high level, it means that when an attempt is made to maximize the proxy variable by changing the action variables, then something about the relationship between the proxy and the target will (usually) change. Understanding Goodhart problems requires understanding both why this general situation comes up regularly, and the mechanics of how the problem occurs.

Causal Diagrams

In this diagram, arrows are causality, and wavy line is correlation without any causal link

In this causal diagram, we have the true target goal Value, which represents the variable that we want to maximize, and four potential proxy variables (Parent, Sibling, Child and Disconnected) that are each correlate...
China Covid Update #1
by Zvi — 2022-04-11T13:40:01.663Z

An increasingly large percentage of new developments are coming from China and in particular from Shanghai. It seems sensible to post them on their own rather than waiting for Thursday.

When they locked down Shanghai by halves, it meant they did not have the resources to lock down Shanghai. Then they locked down Shanghai anyway, file under ‘how to quite possibly lose the mandate of heaven if you keep this up.’

Concretely, what is going on?

His speculation that the test are what is spreading Covid is interesting. This draconian level of restriction otherwise really should be enough to turn the tide within Shanghai while it is in place. Yet is seems even within Shanghai things continue to get steadily worse in terms of infections, while putting the citizens through hell.

If you think that all sounds bad, it’s worse. Quoting in full, seems important, remember this is the censored version. Not that it seems hard to tell what the real message is here.

Bloomberg’s account is that the government is doing a poor job of directly helping and delivery services are overwhelmed to the point of uselessness, but you can buy in bulk if you can make a deal with wholesalers and that still kind of works. Executives are spending their days arranging bread deliveries, but for them it can work. For those who don’t know how to work such systems, it is a lot harder.

Residents scream out of their windows after a week of not being allowed to leave their apartments for any reason. Cases continue rising.

Protests are being censored, never a good sign.

In the footage viewed and independently verified by Bloomberg News, dozens of residents behind the gates of the Jiangnan Xinyuan housing estate in Shanghai’s Minhang district chant phrases including “we want to eat,” “we want the right to know,” and “we want freedom.” People there have been confined to their homes since March 2 and residents have undergone more than 10 rounds of mass virus testing, according to official notices from the compound’s WeChat account.

One could call those reasonable requests.

China claims that out of 130,000 cases in Shanghai, only one person is in serious condition. That is an odd combination of claims to make. Also highly impossible.

Meanwhile, every day somehow Shanghai, under st...
Is it time to start thinking about what AI Friendliness means?
by Victor Novikov — 2022-04-11T09:32:59.684Z

Some background:

I have followed the writing of Eliezer on AI and AI safety with great interest (and mostly, I agree with his conclusions).

I have done my share of programming.

But, I confess, most of the technical side of AI alignment is beyond my current level of understanding (currently I am reading and trying to understand the sequence on brain-like AGI safety).

I do, however, find the ethical side of AI alignment very interesting.

In 2004, Eliezer Yudkowsky wrote a 38-page paper on Coherent Extrapolated Volition, or CEV. An attempt to create a philosophy of Friendliness, to somewhat formalize our understanding of how we would want a Friendly AI to behave.

In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to generate the AI's utility function.

There are many objections to CEV. I have browsed the posts tagged CEV, and in particular enjoyed the list of CEV-tropes, a slightly tongue-in-cheek categorization of common speculations (or possibly misconceptions) about CEV.

So I think it is rather uncontroversial to say that we do not understand Friendliness yet. Not enough to actually say what we would want a Friendly AI to do once is is created and becomes a superintelligence.

Or perhaps we do have a decent idea of what we would want it do, but not how we would formalize that understanding in a way that doesn't result in some perverse instantiation of our ethics (as some people argue CEV would. Some versions of CEV, anyway - CEV is underspecified. There are many possible ways to implement CEV).

In the above-mentioned paper on CEV, Eliezer Yudkowsky writes the following warning.

Arguing about Friendliness is easy, fun, and distracting. Without a technical solution to FAI, it doesn’t matter what the would-be designer of a superintelligence wants; those intentions will be irrelevant to the outcome. Arguing over Friendliness content is planning the Victory Party After The Revolution—not just before winning the battle for Friendly AI, but before there is any prospect of the human species putting up a fight before we go down. The goal is not to put up a good fight, but to win, which i

...
Is there an equivalent of the CDF for grading predictions?
by Optimization Process — 2022-04-11T05:30:01.780Z

When I see people grading their predictions, it's always by: (a) bucketing their predictions by probability (into a "46-55%" bucket, a "56-75%" bucket, ...), and then (b) plotting each bucket's nominal probability vs empirical frequency-of-correctness. See e.g. Scott Alexander here.

This seems... fine... but the bucketing step has a certain inelegance to it: just as you can build many different-looking histograms for the same dataset, you can build many different-looking calibration curves for the same predictions, based on a semi-arbitrary choice of bucketing algorithm. Also, by bucketing datapoints together and then aggregating over the bucket, information is destroyed.

For histograms, there's an information-preserving, zero-degree-of-freedom alternative: the CDF. The CDF isn't perfect, but it at least has a different set of problems from histograms.

Is there any similar tool for grading predictions?
Impactful data science projects
by Valentin2026 — 2022-04-11T04:27:51.133Z

I am considering switching my career to more data science/machine learning (possibly, AI safety if I happen to be good at it). As a start, I want to complete a data science project (to add to my resume and to practice new skills). Is there anything like the Effective Thesis organization (or at least a website or forum or blog) where I can find ideas for the projects that would be impactful? There are many people switching to data science and doing a couple of projects for this purpose, so this should be relevant not only to me.
Could we set a resolution/stopper for the upper bound of the utility function of an AI?
by FinalFormal2 — 2022-04-11T03:10:25.346Z

My thought is that, instead of an AI purely trying to maximize a utility function, we give it a goal of reaching a certain utility, and then each time it reaches that goal, it shuts off and we can decide if we want to set the utility ceiling higher. Clearly, this could have some negative effects, but it could be useful if we used it in concert with other safety precautions.
Epistemic Slipperiness
by Raemon — 2022-04-11T01:48:46.541Z

(epistemic status: thinking out loud, kinda confused)

Sometimes I interact with a person, and something about their thinking feels... slippery.

Probably, sometimes my mind is slippery. But, I can't quite pin down when.

This is probably actually several different phenomena, but they blur together in my current mind. I'm confused about it. This post is me trying to deconfuse myself. Let's explore a few examples and see what shakes out.

Subtly Bad Jokes and Slipping Sideways

I first got the concept of 'epistemic slipping' from this Eliezer facebook post in 2017, which argued that there was something off about people who made jokes about how Trump winning the presidency meant we must be living in a simulation.

When I'm asking whether some person known to be in the Advanced Epistemology 101 club is *strongly* trustworthy, one of the primary questions I ask is: Does this person slip sideways in reality, even just a little, in order to resolve their internal tensions?

You don't know in advance what crisis is going to hit. But when a crisis hits, it often creates some mental tension that can be diminished by slipping sideways in reality to a world that is false but not *blatantly* false. No, person Y couldn't really be lying to me!

And since I see this sideways-slipping as this huge remaining foundational sin even for many people in the AE101 club... well, you can see why I was worried about hearing some people making Simulation Hypothesis jokes about Trump. It expresses a wish for this world not to be real, an impulse to slip sideways out of the tension, even if only by saying that this world is a simulation and so it doesn't count. The conceptual link to the Simulation Hypothesis seems to show a forming fault-line--like the person's brain now has a new way to escape from reality in case of crisis. I worry both that the general pressure exists and hasn't been defeated, and that the Simulation Hypothesis in particular is a fault-line that could crack if the person gets put under enough pressure.

(ADDED: It's not that the fault-line says a Trump victory is okay because simulations are ontologically unreal. The feeling is rather that simulations don't count toward the Laws that Must Be Obeyed, the socially-valent generalizations whose violation feels uncomfortable. It was mandatory for Hillary Clinton to win in the real world, the world that *counts*; but she doesn't hav

...