trentmkelly/LessWrong-43k huggingface dataset:
Dath Ilani Rule of Law
by David Udell — 2022-05-10T06:17:30.951Z

Minor spoilers for mad investor chaos and the woman of asmodeus (planecrash Book 1).

Also, be warned: citation links in this post link to a NSFW subthread in the story.

Criminal Law and Dath Ilan

When Keltham was very young indeed, it was explained to him that if somebody old enough to know better were to deliberately kill somebody, Civilization would send them to the Last Resort (an island landmass that another world might call 'Japan'), and that if Keltham deliberately killed somebody and destroyed their brain, Civilization would just put him into cryonic suspension immediately.

It was carefully and rigorously emphasized to Keltham, in a distinction whose tremendous importance he would not understand until a few years later, that this was not a threat. It was not a promise of conditional punishment. Civilization was not trying to extort him into not killing people, into doing what Civilization wanted instead of what Keltham wanted, based on a prediction that Keltham would obey if placed into a counterfactual payoff matrix where Civilization would send him to the Last Resort if and only if he killed. It was just that, if Keltham demonstrated a tendency to kill people, the other people in Civilization would have a natural incentive to transport Keltham to the Last Resort, so he wouldn't kill any others of their number; Civilization would have that incentive to exile him regardless of whether Keltham responded to that prospective payoff structure. If Keltham deliberately killed somebody and let their brain-soul perish, Keltham would be immediately put into cryonic suspension, not to further escalate the threat against the more undesired behavior, but because he'd demonstrated a level of danger to which Civilization didn't want to expose the other exiles in the Last Resort.

Because, of course, if you try to make a threat against somebody, the only reason why you'd do that, is if you believed they'd respond to the threat; that, intuitively, is what the definition of a threat is.

It's why Iomedae can't just alter herself to be a kind of god who'll release Rovagug unless Hell gets shut down, and threaten Pharasma with that; Pharasma, and indeed all the other gods, are the kinds of entity who will predictably just ignore that, even if that means the multiverse actually gets destroyed. And then, given that, Iomedae doesn't have an

...
AI safety should be made more accessible using non text-based media
by Massimog — 2022-05-10T03:14:13.259Z

I've been doing some thinking on AI safety's awareness problem, after a quick search I found that this post summarizes my thoughts pretty well. In short, AI safety has an awareness problem in a way that other major crises do not (I'll draw parallels specifically with climate change in my analysis). Most ordinary people have not even heard of the problem. Of those that have, most do not understand the potential risks. They cannot concretely imagine the ways that things could go horribly wrong. I'll outline a few reasons I think this is an undesirable state of affairs, but on a surface level I feel it should be obvious to most people convinced of the severity of the issue why the alignment problem should be garnering at least as much attention as climate change, if not more.

The reason I'm writing this up in the first place though is to point out what I see as a critical obstacle for raising awareness that I feel is somewhat overlooked, namely, that virtually all of the best entry-level material on the alignment problem is text-based. It is sadly the case that many many people are simply unwilling to read anything longer than a short blog post or article, ever. Of those that are, getting them to consume lengthy non-fiction on what appears at first glance to be a fairly dry technical topic is still an almost insurmountable challenge. It seems to simply be that for people like that there's currently no easy way to get them invested in the problem, but it really doesn't have to be that way.

Climate change's media landscape

In sharp contrast with AI safety, google brings up a number of non text-based material on climate change aimed at a general audience:

Dozens of movies (including, ironically, one titled "Artificial Intelligence"). I will point out that this includes both non-fiction documentaries and, crucially, fiction that features climate change as a major worldbuilding or plot element.
An endless pile of short, highly produced youtube videos providing good explainers supported by engaging animated content.
Illustrated children's books
A few obscure videogames, though climate change was recently worked into a major mechanic in the very popular Civilization series.
Captain planet. If someone figures out how to make captain planet for AI safety that would be pretty incredible.

AI safety's media landscape

AI safety does, at least, have a small amount of decen...
LessWrong Now Has Dark Mode
by jimrandomh — 2022-05-10T01:21:44.065Z

One of the more commonly requested features on LessWrong is a dark mode. Well, now we have it! To activate, hover over the user-menu in the top right, open the Themes submenu, and click on Dark Mode there.

Notes:

This feature is beta, by which I mean there may be aesthetic problems, especially in less-used areas of the site. Please give feedback in the comments here. Hopefully there is no black-on-black text anywhere, but this is the sort of thing that could happen, so if you do come across any then please report it.
You need to be logged in to pick a theme, for UI reasons (the theme-select option is in the user menu because there wasn't a good place to put it). If you log in, switch to dark mode, and then log back out, it will remember your choice, though, so you don't have to stay logged in if you don't want.
Posts that use custom colors will mostly have their colors overridden to defaults, to prevent black-on-black text. We do have a way to provide dark-mode overrides for specific custom colors which we've used on a few specific posts. If there are important historical posts with black-on-black problems, or which use custom colors in an important way that didn't come through, please let me know in the comments here and I'll try to fix it.
For technical reasons, we don't inherit the operating system dark-mode preference. Let us know in the comments if this is important to you and we might prioritize it.
Conditions for mathematical equivalence of Stochastic Gradient Descent and Natural Selection
by Oliver Sourbut — 2022-05-09T21:38:59.772Z
Link: https://www.oliversourbut.net/p/conditions-for-mathematical-equivalence-of-stochastic

Many thanks to Peter Barnett, my alpha interlocutor for the first version of the proof presented, and draft reader.

Analogies are often drawn between natural selection and gradient descent (and other training procedures for parametric algorithms). It is important to understand to what extent these are useful and applicable analogies.

Here, under some modest (but ultimately approximating) simplifying assumptions, natural selection is found to be mathematically equivalent to an implementation of stochastic gradient descent.

The simplifying assumptions are presented first, followed by a proof of equivalence. Finally, a first attempt is made to consider the effect of relaxing each of these assumptions and what departure it causes from the equivalence, and an alternative set of assumptions which retrieves a similar equivalence is presented.

Summary of simplifying assumptions

It is essential to understand that the equivalence rests on some simplifying assumptions, none of which is wholly true in real natural selection.

Fixed 'fitness function' or objective function mapping genome to continuous 'fitness score'

Continuous fixed-dimensional genome

Radially symmetric mutation probability density

Limit case to infinitesimal mutation

A degenerate population of 1 or 2

No recombination or horizontal transfer

(NB assumptions 5 and 6 will be addressed in detail and amended later)

Proof

Setup and assumptions

Let us make some substantial modelling simplifications, while retaining the spirit of natural selection, to yield an 'annealing style' selection process.

We have continuous genome xt∈Rn with fixed fitness/objective function f:Rn→R

Mutations are distributed according to probability density Pm:Rn→[0,1] which is radially symmetric

so ∀ϵ∈Rn.Pm(ϵ)=g(|ϵ|) for some g:R+→[0,1] (a density, but not a probability density)

a mutation in one direction is just as likely as a mutation in another direction

We also consider the case of mutations very small relative to the genome, tending to infinitesimal.

Selection is stochastic, but monotonically determined by the fitness differential, according to selection function Ps:R→[0,1]

so the probability of x′ being selected over x is Ps(f(x′)−f(x))

Ps is a monotonically nondecreasing function: a greater 'fitness score' differential can not reduce chances of selection

i.e. ∀Δ,δ∈R.Δ>δ⟹Ps(Δ)≥Ps(δ)

...
AI Alignment YouTube Playlists
by jacquesthibs — 2022-05-09T21:33:54.574Z

I created two AI Alignment playlists on Youtube. One that is slide-heavy and the other is not. I separated them into two playlists for two reasons.

It’s useful to separate for a dataset I am working on.
Media is easier to consume when you don’t have to pay attention to the slides and pictures someone is describing.

Not slide-heavy (currently 216 videos): https://youtube.com/playlist?list=PLTYHZYmxohXp0xvVJmMmpT_eFJovlzn0l

Slide-heavy (currently 366 videos): https://youtube.com/playlist?list=PLTYHZYmxohXpn5uf8JZ2OouB1PsDJAk-x

If you would like to contribute and add more videos to the playlists or create new Alignment-relevant playlists, let me know!

If you like access to the audio and youtube auto-generated subs in .txt format, I have stored them here: https://drive.google.com/drive/folders/1qVo4TyHKrsJvbJ3UrIOLW45j_7_wwnbZ?usp=sharing

I've batched up the files into buckets of 90-ish hours (except for the final bucket which is less) since I plan on loading them into otter.ai and that website only accepts 100 hours per user (per month). Additionally, if you would like to help load some of the audio files in your own otter.ai account, please let me know! I want to create transcripts of the audio files and add them to a dataset very soon.
When is AI safety research harmful?
by  — 2022-05-09T18:19:31.008Z

Cross posted to the EA Forum and The Good Blog

Summary

AI safety research improves capability by making AIs done what humans want
Having more capability means that AI is more likely to be deployed
If AI safety is really hard then AI we think is safe at deployment is likely to be unsafe
This effect is mitigated if safety failures are continuous - in this world the more total safety research done the better
Highly theoretical AI safety research is plausibly not going to be done anyway and so adds to the total amount of safety research done
Empirical safety research has a smaller counterfactual impact
The effect of this could go either way depending on weather safety failures and discrete or continuous

What do we mean by capability

There is an argument that safety research is bad because getting a utility function which is close to one that kind, sensible humans would endorse is worse than missing completely. This argument won’t be the focus of this blog post but is well covered

here

.

I will argue that another harm could be that safety research leads to an unsafe AI being deployed more quickly, or at all, than without the safety research being done.

The core of this argument is that AI safety and AI capability is not orthogonal. There are two ways capability can be understood: firstly as the sorts of things an AI system is able to do and secondly as the ability of people to get what they want using an AI system.

Safety is very clearly not orthogonal under the second definition. The key claim made by AI safety as a field is that it’s possible to get AIs which can do a lot of things but will end up doing things that are radically different from what a human principal wants it to do. Therefore improving safety improves this dimension of capability in the sense that ideally a safer AI is less likely to cause catastrophic outcomes which presumably their principals don’t want.

It’s also plausible that under the second definition of capability that AI safety and capabilities are not orthogonal. The problem that value-learning approaches to AI safety is trying to solve is one of attempting to understand what human preferences are from examples. Plausibly this requires understanding how humans work at some very deep level which may require substantial advances in the sorts of things an AI can do. For instance it ...
A Bird's Eye View of the ML Field [Pragmatic AI Safety #2]
by Dan H — 2022-05-09T17:18:53.978Z

This is the second post in

a sequence of posts

that describe our models for Pragmatic AI Safety.

The internal dynamics of the ML field are not immediately obvious to the casual observer. This post will present some important high-level points that are critical to beginning to understand the field, and is meant as background for our later posts.

Driving dynamics of the ML field

How is progress made in ML? While the exact dynamics of progress are not always predictable, we will present three basic properties of ML research that are important to understand.

The importance of defining the problem

A problem well-defined is a problem half solved.

—John Dewey (apocryphal)

The mere formulation of a problem is often more essential than its solution, which [...] requires creative imagination and marks real advances in science.

—

Albert Einstein

I have been struck by how important measurement is... This may seem basic, but it is amazing how often it is not done and how hard it is to get right.

—

Bill Gates

If you cannot measure it, you cannot improve it.

—Lord Kelvin (paraphrase)

For better or worse, benchmarks shape a field.

—

David Patterson

, Turing award winner

Progress in AI arises from objective evaluation metrics.

—

David McAllester

Science requires that we clarify the question and then refine the answer: it is impossible to solve a problem until we know what it is. Empirical ML research, which is the majority of the field, progresses through well-defined metrics for progress towards well-defined goals. Once a goal is defined empirically, is tractable, and is incentivized properly, the ML field is well-equipped to make progress towards it.

A variation on this model is that artists (writers, directors, etc.) come first. They help give ideas, and philosophers add more logical constraints to those ideas to come up with goals or questions, and finally scientists can help make iterative progress towards those goals. To give an example: golems, animate beings created from clay, were a common symbol in Jewish folklore, and at times could create evil. There are many other historical stories of automatons creating problems for humans (

Pandora

,

Frankenstein

, etc.). More recent stories, like Terminator, made the ideas more concrete, even as they included fantasy elements not grounded in reality. More recently,

Bostrom

(2002) recognized the possibility for existentia...
Introduction to Pragmatic AI Safety [Pragmatic AI Safety #1]
by Dan H — 2022-05-09T17:06:00.409Z

This is the introduction to a sequence of posts that describe our models for Pragmatic AI Safety. Thanks to Oliver Zhang, Mantas Mazeika, Scott Emmons, Neel Nanda, Cameron Berg, Michael Chen, Vael Gates, Joe Kwon, Jacob Steinhardt, Steven Basart, and Jacob Hilton for feedback on this sequence (note: acknowledgements here may be updated as more reviewers are added to future posts).

Machine learning has been outpacing safety. Ten years ago, AlexNet pushed the boundaries of machine learning, and it was trained using only two GPUs. Now state-of-the-art models are trained on thousands of GPUs. GPT-2 was released only around three years ago, and today, we have models capable of answering bar exam questions, writing code, and explaining jokes.

Meanwhile, existing approaches to AI safety have not seen similar strides. Many older approaches are still pre-paradigmatic, uncertain about what concrete research directions should be pursued and still aiming to get their bearings. Centered on math and theory, this research focuses on studying strictly futuristic risks that result from potential systems. Unfortunately, not much progress has been made, and deep learning resists the precise and universal mathematical characterizations preferred by some safety approaches.

Recently, some established safety teams have focused more on safety in the context of deep learning systems, which has the benefit of being more concrete and having faster experimental feedback loops. However, many approaches often exhibit the downside of blurring the lines between general capabilities research and safety, as there appear to be few other options.

Finally, neither the pre-paradigmatic nor industry deep learning-based approaches seriously emphasize the broad range of sociotechnical factors that are critical for reducing risk from AI systems.

Given that ML is progressing quickly, that pre-paradigmatic research is not highly scalable to many researchers, and that safety research that advances capabilities is not safely scalable to a broader research community, we suggest an approach that some of us have been developing in academia over the past several years. We propose a simple, underrated, and complementary research paradigm, which we call Pragmatic AI Safety (PAIS). By complementary, we mean that we intend for it to stand alongside current approaches, rather than replace them.

Pragmatic AI Safety res...
Jobs: Help scale up LM alignment research at NYU
by Sam Bowman — 2022-05-09T14:12:22.938Z

NYU is hiring alignment-interested researchers!

I'll be working on setting up a CHAI-inspired research center at NYU that'll focus on empirical alignment work, primarily on large language models. I'm looking for researchers to join and help set it up.
The alignment group at NYU is still small, but should be growing quickly over the next year. I'll also be less of a hands-on mentor next year than in future years, because I'll simultaneously be doing visiting position at Anthropic. So, for the first few hires, I'm looking for people who are relatively independent, and have some track record of doing alignment-relevant work.
That said, I'm not necessarily looking for a lot of experience, as long as you think you're in a position to work productively on some relevant topic with a few peer collaborators. For the pre-PhD position, a few thoughtful forum posts or research write-ups can be a sufficient qualification. We're looking for ML experimental skills and/or conceptual alignment skills/knowledge that could be relevant to empirical work, not necessarily both.
Our initial funding is coming from Open Philanthropy, for a starting project inspired by AI Safety Via Debate. Very early results from us (and a generally-encouraging note from Paul C.) here.
Pay and benefits really are negotiable, and we're willing to match industry offers if there's a great fit. Don't let it stop you from applying.
Microphone on Electric Mandolin
by jefftk — 2022-05-09T14:00:02.723Z

A large portion of what I'm doing when I play rhythm mandolin is
percussive: the sound of the pick hitting the strings. After
listening back to the

comparison between my electric
and acoustic mandolins, this is the thing I find most lacking in
the sound of the electric. What's interesting, however, is that I
don't really notice this when playing at home. I realized a few days
ago that it's hearing the electric mandolin acoustically that makes
the difference, because the pickups are not really capturing these
pick sounds. And if I can hear it acoustically, I can fix it with a
microphone!

I made some recordings, and I really like the sound I get when I mix a
little external microphone into the main sound from the pickup. I also
really like that this means when I run the pickups through distortion
or a talkbox the crisp sound
of the percussion is unaffected.

Here's what it sounds like, recorded on an AudioBox VSL1818. The
clip-on mic is an Audio-Technica Pro-35. In cases where I recorded
the same thing last time, I've included the acoustic version for
comparison. I've also included EvanY's
EQ-matched versions from last time.

Overall, I feel like this is a huge improvement for chords and other
percussive things, while it doesn't have much of an impact on melody
either way.

Chords

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

(

Logic EQ-matched mp3)

(

Pro-Q EQ-matched mp3)

More Chords

(

pickup only mp3)

(

dual mp3)

Percussive

(

pickup only mp3)

(

dual mp3)

High Riff

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

Low Riff

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

High Melody

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

Medium Melody

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

Low Melody

(

pickup only mp3)

(

dual mp3)

(

acoustic mp3)

I'm happy with this, and while this doesn't get the sound of the
electric to where I would leave the acoustic behind on a Free Raisins gig, I think it
will help Kingfisher
mandolin sets a lot.

Comment via: facebook
Thought experiment: Imagine you were assigned to help a random person in your community become as peaceful and joyful as the most peaceful and joyful person you'd ever met. What would you try?
by nonzerosum — 2022-05-09T13:53:57.161Z

What general methods, tools, approaches might you take?

Certain kinds of therapy, certain kinds of meditation, certain types of medication, certain types of coaching, certain activities, certain processes, and so on. Specific things, and/or meta things.

For constraints, imagine that you can spend 5 or 6 figures of US dollars per month, you're there in person full time with them. You don't know who it's going to be, but you know they don't have any notable medical conditions or past history. They're a semi willing participant. You can try anything that exists as of May 2022.
Willing to be your music mentor in exchange for video editing mentorship
by monkymind — 2022-05-09T11:57:37.633Z

Hey,
I’m looking to trade expertise! I’m knowledgable in music theory, composition, improvisation, and music practice. I’m most proficient in guitar but I could teach piano as well.

I want to learn everything I can about video editing and would like a mentor. Here’s a list of some stuff I already know I need to learn about video editing: what it is, what tools I need, how to improve my taste, how to do it well, etc.

Open to having a video chat with anyone, don’t hesitate to ping me :)