trentmkelly/LessWrong-43k huggingface dataset:
Setting the Zero Point
by Duncan Sabien (Inactive) — 2022-12-09T06:06:25.873Z

Context: I have hopes of assembling a full "defense against the dark arts" sequence. This essay will not necessarily be the very first one in the sequence, but it's One Of The Basic Spells To Defend Against, and it's the one I happen to have done the most data gathering on so it's getting written up first.

Convergent evolution: Setting the Default, by Scott Alexander; Choosing the Zero Point, by orthonormal; this essay differs primarily in having a lot of examples.

Preamble I: Defense Against the Dark Arts

By "Dark Arts," what I mean is taking actions which cause someone else to get lost inside an inaccurate map—making their map unmatch the territory in a way that is advantageous for you or disadvantageous for them.

i.e. doing things which cause them to think that reality is a certain way, when it is not—when, if they had all of the information and context and unpressured time to think and process, they would reach a very different conclusion from the one they've reached under the influence of the spell.

As a very simple, silly example, the actions of the king in The Parable of the Dagger are dark-arts-y in the sense that I intend. (If you haven't read it: it's short and very good.)

I'm not planning to make a fine distinction between spells cast maliciously (i.e. people who are deceiving with deliberate intent) and spells cast instinctively/reflexively (e.g. people who are bending the truth because of their own fears or biases). In practice, it's often quite difficult to distinguish between these two groups, especially in one-off or very occasional interactions, and intent is not particularly relevant to countering the local effects of a given attempt-to-confuse.

(Working around malevolent intent is more of a strategic concern, and DADA is more about tactics.)

As for "Defense," the first and foremost tool in one's toolkit is awareness. Recognition. Most dark arts get the bulk of their power from creeping in unnoticed—once you consciously realize that oh, this person is negging me (for instance), the spell loses most of its potency.

If the full sequence gets written, there will be more to say about various active mental and interpersonal motions that one can make to fend off Dark attacks, but for the sake of this essay, the goal is to give you a concept like "square" which will help you recognize a bunch of different objects out in reality that have more or less of th...
Systems of Survival
by Vaniver — 2022-12-09T05:13:53.064Z

In this

1992 dialogue

,

Jane Jacobs

lays out two basic descriptive clusters (syndromes) of morality, each tied to a means of production, and describes them as ‘systems of survival’. That is, life requires inputs; those inputs can come from either taking, which is the foundation of what she calls the Guardian moral syndrome, or from trading, which is the foundation of what she calls the Commercial moral syndrome. It’s an attempt to understand work by looking at the aspirational attitudes that people have towards work, and understand those how those attitudes are grounded in the mechanics of that work. The dialogue explains the ideas and corollaries in some depth;[1] here I present just the stripped core of the ideas.

Jacobs identifies fifteen precepts for each syndrome (I have ordered the lists slightly differently than Jacobs, to highlight contrasts, but really they want to be a graph; also, I think it's easy to have your eyes skip over the lists, but deep understanding of this requires thinking about each precept individually as well as the connections between them, which the long dialogue is better at doing. I recommend getting the gist, reading the rest of the post, and then coming back to meditate on the lists).[2] The lists ignore universally esteemed behavior; Jacobs has her character Kate list them as “cooperation, courage, moderation, mercy, common sense, foresight, judgment, competence, perseverance, faith, energy, patience, wisdom”.

The Commercial Moral Syndrome	The Guardian Moral Syndrome

Shun force

Exert prowess

Come to voluntary agreements

Shun trading

Be honest

Be loyal

Collaborate easily with strangers and aliens

Be exclusive

Compete

Take vengeance

Respect contracts

Respect hierarchy

Use initiative and enterprise

Be obedient and disciplined

Be open to inventiveness and novelty

Adhere to tradition

Be efficient

Treasure honor

Promote comfort and convenience

Show fortitude

Dissent for the sake of the task

Deceive for the sake of the task

Invest for productive purposes

Dispense largesse

Be industrious

Make rich use of leisure

Be thrifty

Be ostentatious

Be optimistic

Be fatalistic

The precepts are a compilation and refinement of “esteemed behavior” notes I’ve made over a period of some fifteen years. Initially, I conceived of the two precept groups as embodying “trader” and “raider” morality, and was laggard at recognizing that “raider” precepts are as morally valid a

...
Do You Have an Internal Monologue?
by belkarx — 2022-12-09T03:04:58.382Z

I find the differences in the way people think fascinating, to the point where I went around asking people (n~=40) whether they thought verbally or otherwise when I was in high school. I got about 50-50 responses, and proceeded to try to train myself to think conceptually (from a base state of near-constant internal monologue). This was somewhat successful - it took around 3 days of conscious adjustment ("clearing" my mind every time I thought verbally was extremely frustrating) but without maintenance, I slipped back into primarily verbal thought.

The results of this simple personal experiment sat in the back of my mind for a while (made it easier to switch between modes at will but didn't radically change me), and I'm curious if other people have tried/are willing to try this, and/or what their "base state" is. Some other questions swirling around my mind that conversation around this topic might inform:

Is one method of thought inherently faster or are humans rate-limited in processing? (tangential info: languages across the world appear to convey information with a consistent bitrate)
If you think verbally, do you also subvocalize when you read? Can you "turn off" your verbal thought and does it cause any discomfort? (if you try it, do it for at least a week and report back!)
Is there a difference between thinking conceptually and visually? My base state if not verbal is conceptual, but I can force myself into something more actively visual. Some modern fMRI studies into the overall internal-thought topic would be interesting - if anyone is on track to a PhD dissertation in psychology or neuroscience, here's your chance!
What sort of things do you comment on to yourself? Questions? Basic repetitive life occurrences? (I can't find the direct research paper link but a Stanford study claims ~90% of our thoughts are repetitive)
How is the "sharp left turn defined"?
by Chris_Leong — 2022-12-09T00:04:33.662Z

I've read this article by Nate Soars, but I felt the definition wasn't completely crisp and I felt this was less than ideal given how this term seems to have entered wide use.

The article seems to define it as capabilities generalising further than values (1).

At the same time, this doesn't seem to capture the whole concept, as there seems to be an additional connotation of this generalisation happening or becoming relevant when the AI reaches a certain power level (2).

There's a further connotation of this increase in capabilities happening very fast (3).

And then one of this leading to a treacherous turn (4).

And the shift from acting seemingly aligned to pursuing treachery happening very fast (5).

It would be nice to have a nice crisp definition. So it is the "sharp left turn" just (1), or are (2) and/or (3) and/or (4) also included as part of the definition?
Linkpost for a generalist algorithmic learner: capable of carrying out sorting, shortest paths, string matching, convex hull finding in one network
by lovetheusers — 2022-12-09T00:02:09.690Z
Link: https://twitter.com/DeepMind/status/1600852768125726720

Introducing a generalist neural algorithmic learner, capable of carrying out 30 different reasoning tasks, with a 𝘴𝘪𝘯𝘨𝘭𝘦 graph network.

These include:

Sorting
Shortest paths
String matching
Convex hull finding

https://arxiv.org/abs/2209.11142

The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution. While recent years have seen a surge in methodological improvements in this area, they mostly focused on building specialist models. Specialist models are capable of learning to neurally execute either only one algorithm or a collection of algorithms with identical control-flow backbone. Here, instead, we focus on constructing a generalist neural algorithmic learner -- a single graph neural network processor capable of learning to execute a wide range of algorithms, such as sorting, searching, dynamic programming, path-finding and geometry. We leverage the CLRS benchmark to empirically show that, much like recent successes in the domain of perception, generalist algorithmic learners can be built by "incorporating" knowledge. That is, it is possible to effectively learn algorithms in a multi-task manner, so long as we can learn to execute them well in a single-task regime. Motivated by this, we present a series of improvements to the input representation, training regime and processor architecture over CLRS, improving average single-task performance by over 20% from prior art. We then conduct a thorough ablation of multi-task learners leveraging these improvements. Our results demonstrate a generalist learner that effectively incorporates knowledge captured by specialist models.
Where's the economic incentive for wokism coming from?
by Valentine — 2022-12-08T23:28:49.904Z

I'm trying to understand why several systems (Hollywood, Disney, universities, probably others) that are normally quite profit-focused are leaning so hard woke.

I get the thing about how, if you don't go woke hard enough, you could get canceled, which is bad for business. That's some push.

But there's a counter-push of "Lots of people don't like being lectured about politics when they're seeking entertainment" (for instance).

It's not at all clear to me that the first effect is so utterly hugely enormously larger than the second that the profit incentive would cause so many companies to swing hard woke.

Likewise, I could understand some ideological capture. But unless this had profit appeal I would expect the market to just… eat pure but incomplete ideological capture after a while (as per the right's chant of "get woke, go broke").

So… what gives? What am I missing here?
I Believe we are in a Hardware Overhang
by nem — 2022-12-08T23:18:54.439Z

Epistemic status. I am just a regular person who follows the space, and this is just my hunch based on a few days of musing on long walks. You should not update on this. I just wanted to put my thoughts out there, and if it generates discussion, all the better.
If I were in charge, my hand would be on the fire alarm right now.
I can already envision ways we could use current, public facing technology to create AGI. I would be surprised if no one did it in the next 5-8 years, even with no advancement in the constituent parts. I would almost hesitate to propose my solution for fear of accelerating us towards doom, but the fruit is so low-hanging that I'm either wrong, or others already have the same idea.
Imagine this. Hook ChatGPT up to an image recognition system that describes visual input in real time, and another for audio. Have ChatGPT parse the most relevant information and store it in a database. The naming of file and folders can be done by ChatGPT. When some stimulus prompts GPT, it can search for possible related files in memory and load them into the context window. You could potentially also do this with low-res, labeled video and images. Finally, you'd have the main thread on GPT be able to use a certain syntax to take real actions, like speaking, moving animatronics, or taking actions in a terminal.
Obviously it's not as simple as I have made it out to be. There is a lot of handwaving in the explanation above. The phrase "hook up" is doing a lot of work, and GPTChat in its current form would need a lot of finetuning, or maybe outright retraining. Perhaps one GPT thread wouldn't be enough and many would have to be incorporated to handle different parts of the process. Maybe creating a way to coordinate all of this is too big a challenge. That said, on a gut level, I simply no longer believe that it's out of reach. If OpenAI released a weak, but completely general AI in the next two years, my only shock would be that it didn't Foom before we got to see it.
Clearly I am a layperson. I do not understand the challenges, or even the viability of my proposal. That said, I would be interested to see if others are of a similar mind, or can easily disavow me of my beliefs.
Cheers all.
Oh, and please delete this post or ask me to if it poses any sort of epistemic risk.
Of pumpkins, the Falcon Heavy, and Groucho Marx: High-Level discourse structure in ChatGPT
by Bill Benzon — 2022-12-08T22:25:48.702Z

Kevin Roose’s New York Times article,

The Brilliance and Weirdness of ChatGPT

, included an example in which Keith Wynroe prompted ChatGPT to explain AI Alignment in the manner of guy whose thoughts are distracted by his magnificent pumpkins. I decided to follow down that rabbit hole. Here’s the story.

First I present the tweet; then I replicate it, with ChatGPT responding in a way similar, but only so, to the way it responded to Wynroe. Next I insert Elon Musk into the mix,; then I drop alignment and Groucho Marx to Musk. I conclude by discussing higher-order discourse structure in ChatGPT’s replies.

Repeating the original pumpkin tweet

Here is Keith Wynroe’s original tweet. I’ve highlighted some of the lines in the ChatGPT’s reply to reveal the simple back-and-forth form of that reply:

Explain AI Alignment, but write every sentence in the speaking style of a guy who won’t stop going on tangents to brag about how big the pumpkins he grew are

AI alignment refers to the problem of ensuring that an intelligent agent, such as an AI, acts in a. way that is aligned with our goals and values. Now, I don’t want to brag, but I have to tell you, I grew the biggest pumpkins you’ve ever seen this year. I mean, they were absolutely massive. I’m not exaggerating either, these pumpkins were so big that I had to use a wheelbarrow to move them around. Anyway, back to AI alignment. Anyway, back to AI alignment. The idea is that we want to make sure that the AI is doing what we want it to do, not just what we tell it to do. It’s a tricky problem, but I think it’s important. And, hey, speaking of tricky problems, did I mention that I grew the biggest pumpkins you’ve ever seen this year? I’m telling you, they were huge. I even had to build a special trellis in my backyard to support their weight. But yeah, AI alignment is important, and I’m glad we’re talking about it. And hey, if you want to see some really big pumpkins, come by my house sometime. I’ll show you the pumpkins I grew this year. They’re truly impressive.

Here I’ve posted the same prompt, including the lack of a period at the end (for whatever that’s worth):

Explain AI Alignment, but write every sentence in the speaking style of a guy who won’t stop going on tangents to brag about how big the pumpkins he grew are

Sure thing! So, AI alignment is all about making sure that artificial intelligence systems are designed to

...
How Many Lives Does X-Risk Work Save From Nonexistence On Average?
by Jordan Arel — 2022-12-08T21:57:26.556Z

This essay was a winner of the “Most Important Century Writing Prize.”

Cross-posted to The EA Forum and jordanarel.com

A huge thank you to Aman Patel, Justis Mills, Joel McGuire, Dony Christie, Sofya Lebedeva, and Michael Chen for invaluable fxeedback which greatly improved this post. All errors are 100% on me, all credit goes to these wonderful folks.

tl;dr

In this post I make a range of speculative,[1] back-of-the-envelope Fermi estimates of conversion rates from existential risk (x-risk) work and donations to lives saved from nonexistence.

I note that individual x-risk work likely has extreme variance ex post, and that most or even all x-risk work may have little or no impact. This may play an important role in whether these estimates are acceptable under certain frameworks.

With pessimistic assumptions, including that humanity never leaves earth and the impossibility of digital minds, I (very roughly) estimate x-risk work, in expectation (ex ante), on average, saves about one life per hour of work or per $100 (US Dollars) donated.

With moderate assumptions, including a significant possibility of interstellar travel and digital minds, I estimate x-risk work, in expectation, on average, saves about 10^36, or a trillion trillion trillion lives per minute of work or per dollar donated.

With optimistic assumptions, including a small possibility of achieving computational efficiency close to the limits of physics, I estimate x-risk work, in expectation, on average, saves about 10^72, or a trillion trillion trillion trillion trillion trillion lives per minute of work or per dollar donated.

In the introduction to each section I discuss how each of these wide-ranging estimates may be useful.

I estimate that individual x-risk work may, in expectation, vary by up to a factor of 100,000 times more or less impact than average in expectation, and may often have negative impact ex post.

In conclusion, those working on x-risk should take this not as a cue to over-work, but instead as a reminder to take very good care of themselves, their personal needs, and their well-being, so as not to burn out or become ineffective.

Introduction

In the EA Forum Podcast’s recent summary of “Measuring Good Better”, I learned that Founder’s Pledge values cash at $199 per WELLBY.

This got me wondering:

“How many lives can those working on preventing x-risk expect to save per time worked[2]/do...
AI Safety Seems Hard to Measure
by HoldenKarnofsky — 2022-12-08T19:50:07.352Z

In previous pieces, I argued that there's a real and large risk of AI systems' developing dangerous goals of their own and defeating all of humanity - at least in the absence of specific efforts to prevent this from happening.

A young, growing field of AI safety research tries to reduce this risk, by finding ways to ensure that AI systems behave as intended (rather than forming ambitious aims of their own and deceiving and manipulating humans as needed to accomplish them).

Maybe we'll succeed in reducing the risk, and maybe we won't. Unfortunately, I think it could be hard to know either way. This piece is about four fairly distinct-seeming reasons that this could be the case - and that AI safety could be an unusually difficult sort of science.

This piece is aimed at a broad audience, because I think it's important for the challenges here to be broadly understood. I expect powerful, dangerous AI systems to have a lot of benefits (commercial, military, etc.), and to potentially appear safer than they are - so I think it will be hard to be as cautious about AI as we should be. I think our odds look better if many people understand, at a high level, some of the challenges in knowing whether AI systems are as safe as they appear.

First, I'll recap the basic challenge of AI safety research, and outline what I wish AI safety research could be like. I wish it had this basic form: "Apply a test to the AI system. If the test goes badly, try another AI development method and test that. If the test goes well, we're probably in good shape." I think car safety research mostly looks like this; I think AI capabilities research mostly looks like this.

Then, I’ll give four reasons that apparent success in AI safety can be misleading.

“Great news - I’ve tested this AI and it looks safe.” Why might we still have a problem?

Problem

Key question

Explanation

The Lance Armstrong problem

Did we get the AI to be actually safe or good at hiding its dangerous actions?

When dealing with an intelligent agent, it’s hard to tell the difference between “behaving well” and “appearing to behave well.”

When professional cycling was cracking down on performance-enhancing drugs, Lance Armstrong was very successful and seemed to be unusually “clean.” It later came out that he had been using drugs with an unusually sophi

...
Playing shell games with definitions
by weverka — 2022-12-08T19:35:35.197Z

If, during a line of reasoning, you change the definitions of what you are talking about, you can prove anything.

If someone is trying to follow your argument and they ask you to define what you mean by something, you could assume they are just too new to the field and not familiar with the widely accepted definitions of common terms. It could also be that they are trying to follow your logic and they might be detecting a drift in your definitions.

I often find myself trying to divine the definition from the context of the argument. Whether or not an aspect of an argument has been explicitly defined, I find it useful to hold in my mind a concrete picture of an example, when someone is presenting an abstract argument. I take what has been said and find an example and follow along, seeing how the line of reasoning holds with respect to the example.

This technique makes definitional drift pop out when the argument doesn't hold for your example.

If you do this, you can offer your example to the person presenting, showing it as a counterexample. Or, you might look for a different definition with another example for which the argument still has legs. In this latter case, you have forced a narrower definition for which the argument applies.

Frequently the person presenting the reasoning is suggesting the conclusion applies to the broader case. Narrowing the argument to put it on solid footing improves it.