trentmkelly/LessWrong-43k huggingface dataset:
New: use The Nonlinear Library to listen to the top LessWrong posts of all time
by KatWoods — 2022-04-09T20:50:14.371Z

Update #1: It’s a rite of passage to binge the top LessWrong posts of all time, and now you can do it on your podcast app.

We (

Nonlinear

) made “top of all time” playlists for LessWrong, the EA Forum, and the Alignment Forum. Each is around ~400 of the most upvoted posts:

LessWrong

EA Forum

Alignment Forum

Update #2: The original

Nonlinear Library

feed includes top posts from the EA Forum, LessWrong, and the Alignment Forum. Now, by popular demand, you can get forum-specific feeds:

LessWrong

EA Forum

Alignment Forum

Stay tuned for more features. We’ll soon be launching channels by tag, so you can listen to specific subjects, such as longtermism, rationality, animal welfare, or global health.

Enter your email here

to get notified as we add more channels.

Below is the original explanation of The Nonlinear Library and its theory of change.

We are excited to announce the launch of The Nonlinear Library, which allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs.

In the rest of this post, we’ll explain our reasoning for the audio library, why it’s useful, why it’s potentially high impact, its limitations, and our plans. You can read it here or listen to the post in podcast form here.

Listen here: Spotify, Google Podcasts, Pocket Casts, Apple, or elsewhere

Or, just search for it in your preferred podcasting app.

Goal: increase the number of people who read EA research

A koan: if your research is high quality, but nobody reads it, does it have an impact?

Generally speaking, the theory of change of research is that you investigate an area, come to better conclusions, people read those conclusions, they make better decisions, all ultimately leading to a better world. So the answer is no. Barring some edge cases (1), if nobody reads your research, you usually won’t have any impact.

Research → Better conclusion → People learn about conclusion → People make better decisions → The world is better

Nonlinear is working on the third step of this pipeline: increasing the number of people engaging with the research. By increasing the total number of EA and rationalist articles read, we’re increasing the impact of all of that content.

This is often relatively neglected because researchers typically prefer d...
140 Cognitive Biases You Should Know
by André Ferretti — 2022-04-09T17:15:27.239Z

I'm excited to announce my list of ∼140 cognitive biases.

Become aware of your biases. Make better decisions. I adapted most biases from Wikipedia's

list of cognitive biases

. Download my biases as Anki flashcards

here

.

My favourite biases

Backfire Effect
You reject evidence opposing your opinion, and you strengthen your original opinion.

Curse of Knowledge
You assume that the listener has the background knowledge to understand you.

Dilution Effect
You under-utilise useful information when useless information is present.

False Consensus Effect
You see your actions as more common than they are.

Google Effect
You forget information that you can find online.

Illusion of Transparency
You overestimate how much others are aware of your mental state.

Overjustification Effect
You have less intrinsic motivation to act if you are externally rewarded for it.

Self-Reference Effect
You remember information better if you are related to it.

Zeigarnik Effect
You remember unfinished tasks better than finished tasks.

What you can do

Share the list of biases with a friend.
Check out my website for more.
Suggest a new bias in the comments.
Please give me feedback in the comments or directly in the Google Doc.
Strategies for keeping AIs narrow in the short term
by Rossin — 2022-04-09T16:42:28.884Z

Disclaimer: I don’t have any particular expertise in AI safety, I just had some thoughts and this seemed like the place to put them.

The bleak outlook that Eliezer presents here and elsewhere seems to in large part be driven by the expectation that AGI will be developed before the technical problem of how to robustly align it is solved, and that poorly aligned AGIs tend to destroy the world.

One way to potentially get more time to solve the technical problem is to expend more effort on activism with the goal of convincing the groups that are trying to create increasingly advanced AIs to slow down/stop and at least not attempt to create AGI without a rigorous theory of AI alignment. Realistically, I think suggestions to stop AI development completely at these places or to get world governments to enact effective regulations are doomed to fail. However, convincing the top researchers at the most advanced groups that AGI is incredibly dangerous and that we should try to keep AIs as narrow as possible until we're really confident that we know what they will do seems more plausible.

I don't think most researchers can be convinced to stop completely, but if they are left a line of retreat that goes something like 'just try to keep AIs as narrow as possible using some specific techniques', that might be much easier to swallow. Maybe I am underestimating the difficulty of convincing researchers on this point, but it should at least be easier to sell the AI alignment story to people if it does not require them to quit their jobs and instead just asks that they employ techniques to keep AIs they're working on as narrow as possible. I've had some admittedly inexpert thoughts on what sort of things could be done to try to prevent a machine learning model from acquiring a more general understanding than the programmer wants, which I go over below. Even if these are impractical, I hope that they can at least be a jumping off point for discussion on buying time type strategies.

A common first step on the path of an AI developing more generality than desired seems like it might often be developing the concept that it is undergoing an ongoing cycle of reward - that it will have many future chances to earn reward. This is important because why would you spend resources exploring your environment in detail if you have no expectation of any opportunity for r...
Hyperbolic takeoff
by Ege Erdil — 2022-04-09T15:57:16.098Z
Link: https://www.metaculus.com/notebooks/10617/hyperbolic-takeoff/

The debate over "slow" versus "fast" takeoff is one of the more controversial subjects in the AI safety community. The question is roughly about whether AI development will be more gradual, or whether once an AI achieves what could be called "general intelligence", it will be able to rapidly grow its capabilities and within a very short time transform the world in some way, either desirable or not.

The central problem when it comes to thinking about takeoff scenarios is that the reference class is empty: we've never seen anything like an AGI, so it seems very difficult to say anything about what might happen when takeoff gets going. I'll argue that this is not true: while we don't have anything exactly like an AGI, we have lots of different pieces of information which when put together can allow us to pin down what happens after AGI is developed.

The key property of AGI that matters for what happens after takeoff is that AGI is very likely going to be much easier to improve than humans are. Humans have a relatively fixed hardware that's difficult to change much and making more humans is a slow and expensive process. In contrast, an AGI could grow its capabilities both by recursive self-improvement at a much higher frequency compared to humans and by manufacturing or otherwise acquiring new processors to expand its computing power. Both of these processes can happen on timescales much faster than humans are able to make changes to human civilization, so we expect AGI to accelerate all kinds of change once it arrives.

This is already true of contemporary AI systems: we can't run Terence Tao's brain twice as fast even if we throw a billion dollars at accomplishing this, but for any deep learning system that's currently around a billion dollars of compute would be enough to give us enormous speedups. Therefore the expectation that AGI will be much easier to improve is not only theoretical; it's also based on the properties of current AI systems.

The model

The basic model I'll use throughout this notebook is a deterministic hyperbolic growth model. These models are well-known and have been a source of singularity forecasts for a long time, but here I'll use them for a somewhat different purpose.

First, let's motivate the model. Right now, we have a world in which the primary bottleneck in AI development is human effort and time. Insofar as compute and hardware are bo...
Elicit: Language Models as Research Assistants
by stuhlmueller — 2022-04-09T14:56:37.763Z

Ought is an applied machine learning lab. We’re building Elicit, the AI research assistant. Our mission is to automate and scale open-ended reasoning. To get there, we train language models by supervising reasoning processes, not outcomes. This is better for reasoning capabilities in the short run and better for alignment in the long run.

In this post, we review the progress we’ve made over the last year and lay out our plan for Elicit. We'd love to get feedback on how to make Elicit more useful for LW and to get thoughts on our plans more generally. To make this easier, we've erred on the side of giving detail even where we know it will change.

Progress in 2021:

We built Elicit to support researchers because high-quality research is a bottleneck to important progress and because researchers care about good reasoning processes.
We identified some building blocks of research (e.g. search, summarization, classification), operationalized them as language model tasks, and connected them in the Elicit literature review workflow.
On the infrastructure side, we built a streaming task execution engine for running compositions of language model tasks This engine is supporting the literature review workflow in production.
About 1,500 people use Elicit every month.

Roadmap for 2022+:

We expand literature review to digest the full text of papers, extract evidence, judge methodological robustness, and help researchers do deeper evaluations by decomposing questions like “What are the assumptions behind this experimental result?”
After literature review, we add other research workflows, e.g. evaluating project directions, decomposing research questions, and augmented reading.
To support these workflows, we refine the primitive tasks through verifier models and human feedback, and expand our infrastructure for running complex task pipelines, quickly adding new tasks, and efficiently gathering human data.
Over time, Elicit becomes a general-purpose reasoning assistant, transforming any task involving evidence, arguments, plans and decisions.

How we think about success

Our mission is to automate and scale open-ended reasoning. If we can improve the world’s ability to reason, we’ll unlock positive impact across many domains including AI governance & alignment, psychological well-being, economic development, and climate change.

As AI advances, the raw cognitive capabilities of the world ...
AMA Conjecture, A New Alignment Startup
by adamShimi — 2022-04-09T09:43:02.739Z

Conjecture is a new alignment startup founded by Connor Leahy, Sid Black and Gabriel Alfour, which aims to scale alignment research. We have VC backing from, among others, Nat Friedman, Daniel Gross, Patrick and John Collison, Arthur Breitman, Andrej Karpathy, and Sam Bankman-Fried. Our founders and early staff are mostly

EleutherAI

alumni and previously independent researchers like

Adam Shimi

. We are located in London.

As described in our announcement post, we are running an AMA this week-end, from Today (Saturday 9th April) to Sunday 10th of April. We will answer any question asked before the end of Sunday Anywhere on Earth. We might answer later questions, but no guarantees.

If you asked question on our announcement post, we would prefer that you repost them here if possible. Thanks!

Looking forward to your questions!
What advice do you have for someone struggling to detach their grim-o-meter?
by Zorger74 — 2022-04-09T07:35:02.534Z

From detach the grim-o-meter:

[S]top calibrating it against the state of the world. That's a terrible plan!

The quote, and the rest of the essay, seem to imply that once someone notices that their grim-o-meter is connected to the state of the whole world, they can detach it on command. Personally, I'm having some trouble, and I imagine others are too, so I'm asking the community: What advice do you have for someone struggling to detach their grim-o-meter?
Can AI systems have extremely impressive outputs and also not need to be aligned because they aren't general enough or something?
by WilliamKiely — 2022-04-09T06:03:10.068Z

Layperson question here; I appreciate any answers/comments that might help me understand this topic better.

When thinking about DALL·E 2 and Google's PaLM this week I wondered if in the future we could hypothetically have AI systems like this...

Input: "an award-winning 2-hour sci-fi drama feature film about blah blah blah"

Output: an incredible film that people living in the 1970s would assume was made by future humans, not future AI, and deserves to win Best Picture because it made them laugh and cry and was amazing

...all without having to solve the alignment problem because the system that produces the film isn't actually intelligent in the ways that matter to AI risk.

That is, can we get crazy impressive outputs from an AI system without that AI system posing an existential risk or being aligned?

If so, what feature distinguishes AI systems that do pose existential risks and need to be aligned from those that don't?

If not, what necessary aspect of the AI system that produces the output above is it that makes it so the system now poses an existential risk and needs to be aligned?
Buy-in Before Randomization
by jefftk — 2022-04-09T01:30:02.206Z

With kids, some problems are a good fit for randomization. Perhaps
you're doing a bath and both kids want to be the second one to have
their hair washed, so you propose flipping a coin. The kids understand
that they can't both be second, and flipping for it seems fair.

Or, at least, it seems fair right up until you flip the coin and one
of them loses. Then there are suddenly all sorts of overlooked issues
and last-minute considerations ("I had to go first last time!", "I
have longer hair!").

The way I deal with this is by confirming buy-in before randomization.
I won't randomize unless they agree that we should resolve this
randomly, that they will abide by the results without fussing, and
that if they do fuss it's time-out. Even though they know this rule
with me, I still repeat it every time: it doesn't take that long
compared to the rest of the process, and it being fresh in their minds
when we randomize works much better.

In cases where the randomization is for their benefit
("which park should we go to?", "who gets the first Nora cuddle?"), if
they don't accept the terms we can fall back to neither. In other
cases, though, where I need to impose randomization, I'll be mildly
coercive: if one of them won't agree to these terms I say I'll resolve
it in the other's favor. At which point they prefer half a
chance of getting their way to no chance, decide to participate, and
confirm they won't fuss.
Why Instrumental Goals are not a big AI Safety Problem
by Jonathan Paulson — 2022-04-09T00:10:40.949Z

For concreteness, I’ll focus on the “off button” problem, which is that an AI (supposedly) will not let you turn it off. Why not? The AI will have some goal. ~Whatever that goal is, the AI will be better able to achieve it if it is “on” and able to act. Therefore, ~all AIs will resist being turned off.
Why is this wrong? First, I’ll offer an empirical argument. No actually existing AIs exhibit this problem, even the most agentic and capable ones. Go-playing AIs are superhuman in pursuing their “goal” of winning a Go game, but they do not offer any resistance to being turned off. Language models have a seemingly impressive world model, and are fairly capable at their “goal” of predicting the next token, but they do not resist being turned off. (Indeed, it’s not even clear that language models are “on” in any sense, since different executions of the model share no state, and any individual execution is short-lived). Automated trading systems make stateful economically significant decisions, but they do not resist being turned off (actually, being easy to turn off is an important design consideration).

So that’s the empirical case. How about a thought experiment? Suppose you scaled up a game-playing AI with 1000x more data/compute/model-size. It would get much better at achieving its “goal” of playing the game well. Would it start to resist being turned off? Obviously it would not. It’s not even going to be able to represent the idea of being “turned off”, since it’s just thinking very hard about e.g. Go.

So what about the theory? How come theory says ~all AIs should resist being turned off, and yet ~no actually existing AIs (or even powered-up versions of existing AIs), do resist being turned off? The reason is that “agents with a goal” is not a good way of thinking about AIs.
It’s a tempting way to think about AIs, because it is a decent (although imperfect) way of thinking about humans. And “reward functions” do look an awful lot like goals. But there are some key differences:

1) Training defines the “world” that the AI is thinking about. This was the problem with the super-Go-playing AI - we just told it to think hard about the world of Go. So even though it is superhuman, even godlike, *in that domain*, it hasn’t thought about the real world. It can’t even interact with the real world, since it takes input and provides output in the very specific way that it wa...
A method of writing content easily with little anxiety
by jessicata — 2022-04-08T22:11:47.298Z
Link: https://unstableontology.com/2022/04/08/a-method-of-writing-content-easily-with-little-anxiety/

Hello, I have today been informed about a method for writing content that is relatively easy and painless. The idea is that by default a human will when reading predict what word is going to come next. Studies show that humans can predict what letter is going to come next and get pretty good entropy scores. That means that even the process of reading involves generating plausible next words.

And so it is possible to similarly do the same when writing. When writing a sequence of words, after the end there will be a feeling that some word will come next. It is unexpected and abrupt for the sentence to immediately end, so there is a feeling of continuation, that something will come after. One can while writing simply look at the space right to the right of the text that has already been written and there will be a feeling there about what word will come next.

Do not worry if this is difficult and slow at first. A practice exercise is to write a few words and then take some time to get a feeling of what word is going to come next. With meditation it is possible to notice experiential phenomena that are in one's experience and sometimes have a spatial location. Well, maybe someone who knows how to find these phenomena is going to be able to find some sense of "next word" ness when focusing on the end of a sentence fragment. That will tell them what word they think is going to come next.

It is possible to simply, when noticing what word is going to come next, type that word. That is in fact what I am doing right now and what I have been doing for this entire post. It is rather easy and painless to do. It is not anxiety-provoking because it is a meditative, instruction-following exercise where I simply get a sense of what word will come next and type it. I am not responsible for the output of my text predictive model. I am not to blame. I am not unworthy.

This shortcuts various anxiety loops that are possible when writing in a more traditional manner. At that point, while there are many sentence continuations that are invisibly generated, all are filtered out because it would feel somehow bad to have written that sentence. Maybe the sentence would be cringeworthy or stylistically bad or offensive or embarrassing or I-don't-know-what, it might just be off, and someone with enough of an aesthetic taste might simply be repulsed by the idea of actually writing ou...
Good Heart Donation Lottery Winner
by Gordon Seidoh Worley — 2022-04-08T20:34:41.104Z

The Good Heart Donation Lottery is closed. And the winner is...

0 [ggworley (master) ~/lottery](1649449423 2022-04-08 13:23:43)
$ cat lottery.rb
puts Hash[
"EA Funds - Long-Term Future Fund" => 21,
"Against Malaria Foundation" => 18,
"EA Funds - Animal Welfare Fund" => 13,
"GiveWell Maximum Impact Fund" => 11,
"EA Funds - EA Infrastructure Fund" => 11,
"Machine Intelligence Research Institute" => 11,
"EA Funds - Global Health and Development Fund" => 9,
"EA Funds - Even distribution" => 9,
"FTX Future Fund" => 5,
"Planned Parenthood" => 1
].collect { |k,v|
[k] * v
}.flatten.sample(1)
0 [ggworley (master) ~/lottery](1649449430 2022-04-08 13:23:50)
$ ruby lottery.rb
EA Funds - EA Infrastructure Fund

The EA Infrastructure Fund!

I've updated my payment info in LessWrong so that my GHT will be cashed out to the EA Infrastructure Fun. I believe the payout should be $837 based on what I'm seeing, which means I'll be making a donation of $163 to the EA Infrastructure Fun to round up to the $1000 pool minimum I promised. I'll make payment shortly after I'm sure of the amount once LessWrong pays out on their side.

Thanks to everyone who voted and suggested charities as part of this fun extension to this year's April Fool's joke.
Roam Research Mobile is Out!
by Logan Riggs — 2022-04-08T19:05:40.211Z

I really enjoy Rome Research and now I can use it on my phone when I want to record a new set of thoughts on the go and not just "QuickCapture".

Download the app and click the bottom-middle tab between "graph List" and "quickCapture". The syncronization between mobile and online is near-instant as well.

I recommend Rome Research in general as a notetaking tool, but also, spending 2 hours figuring out all the shortcuts and features is useful for any new software tool you incorporate (There's several youtube videos for Rome you can watch as well)