trentmkelly/LessWrong-43k huggingface dataset:
[AN #130]: A new AI x-risk podcast, and reviews of the field
by Rohin Shah — 2020-12-24T18:20:05.289Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

Announcing AXRP, the AI X-risk Research Podcast (Daniel Filan) (summarized by Rohin): Episodes of this new podcast will involve Daniel interviewing a researcher about a paper they’ve written, talking about the ideas in the paper and why they matter. Three episodes have already been released; I summarize them later in this newsletter.

2020 AI Alignment Literature Review and Charity Comparison (Larks) (summarized by Rohin): The tradition continues (AN #78)! I'll say nearly the same thing as I did last year:

This mammoth post goes through the work done within AI alignment from December 2019 - November 2020, from the perspective of someone trying to decide which of several AI alignment organizations to donate to. As part of this endeavor, Larks summarizes a ton of papers that were published at various organizations, and compares them to their budget and room for more funding.

Rohin's opinion: I look forward to this post every year. It continues to be a stark demonstration of how much work doesn't get covered in this newsletter -- while I tend to focus on the technical alignment problem, with some focus on AI governance and AI capabilities, this literature review spans many organizations working on existential risk, and as such has many papers that were never covered in this newsletter. Anyone who wants to donate to an organization working on AI alignment and/or x-risk should read this post.

Last year I mentioned I might write an overview for the sake of building inside view models (rather than donation decisions), this went out shortly afterward (AN #84). I don't expect to write a similar post this year, partly because I think last year's post is still quite good as an overview of the discussion that's been happening.

TAI Safety Bibliographic Database (Jess Riedel et al) (summarized by Rohin): Related to the previous summary, we also have a database of a bunch of papers on tran...
Covid 12/24: We’re F***ed, It’s Over
by Zvi — 2020-12-24T15:10:02.975Z

UPDATE 7/21/2021: As you doubtless know at this point, it was not over. Given the visibility of this post, I'm going to note here at the top that the prediction of a potential large wave of infections between March and May did not happen, no matter what ultimately happens with Delta (and the prediction was not made with Delta in mind anyway, only Alpha). Some more reflections on that at the bottom of this post here.

A year ago, there were reports coming out of China about a new coronavirus. Various people were saying things about exponential growth and the inevitability of a new pandemic, and urging action be taken. The media told us it was nothing to worry about, right up until hospitals got overwhelmed and enough people started dying.

This past week, it likely happened again.

A new strain of Covid-19 has emerged from southern England, along with a similar one in South Africa. The new strain has rapidly taken over the region, and all signs point to it being about 65% more infectious than the old one, albeit with large uncertainty and error bars around that.

I give it a 70% chance that these reports are largely correct.

There is no plausible way that a Western country can sustain restrictions that can overcome that via anything other than widespread immunity. This would be the level required to previously cut new infections in half every week. And all that would do is stabilize the rate of new infections.

Like last time, the media is mostly assuring us that there is nothing to worry about, and not extrapolating exponential growth into the future.

Like last time, there are attempts to slow down travel, that are both not tight enough to plausibly work even if they were implemented soon enough, and also clearly not implemented soon enough.

Like last time, no one is responding with a rush to get us prepared for what is about to happen. There are no additional pushes to improve our ability to test, or our supplies of equipment, or to speed our vaccine efforts or distribute the vaccine more efficiently (in any sense), or to lift restrictions on useful private action.

Like last time, the actions urged upon us to contain spread clearly have little or no chance of actually doing that.

The first time, I made the mistake of not thinking hard enough early enough, or taking enough action. I also didn’t think through the implications, and didn’t do things like buying pu...
Open Christmas card to my mother
by KatjaGrace — 2020-12-24T07:00:19.081Z

Dear Mama,

Happy Christmas!

Rather than be so bold as to give you a further object to take care of, I gathered for you a collection of things I think you might like on the internet. I CC the world, in case they are interested, or want to add anything.

Lots of love,
Katja
xox

(Picture: Geboorte van Christus, Robert van den Hoecke, after Jan van den Hoecke, 1632 - 1668)

Baby elephants

A nice video of a 1965 living room clog dance

Round seals that I gathered earlier

Dance your PhD winner (though perhaps you showed that to me originally..)

The complete works of Alfred Sisley

Archive of Olegas Truchanas’ photography

The 1890s, around the world, on film

Positly: Perhaps you have enough opportunities to interview people, but if you want to run your own surveys of strangerse online, this streamlines the process a lot (i.e. you basically write some questions and put money in the slot and they handle getting strangers to answer your questions and paying them). For instance, I asked some Americans why they voted as they did, and some other things about their experience of the election.

Hamilton the Musical: I know I probably recommended this before, but a) I should note that it might be the strongest artistic recommendation I’ve ever given, and b) if you did listen to it, I recommend listening to it again.

How to live for much longer in expectation (I know you don’t believe in this, but I encourage you to think about it more, because its important to me that you avoid dying if feasible.)

Rijksmuseum: the national museum of the Netherlands has a pleasing website for their pleasing art collection.

Here are some that I like today:

Morning Ride along the Beach, Anton Mauve, 1876

Fishing Pinks in Breaking Waves, Hendrik Willem Mesdag, c. 1875 - c. 1885

Landschap in de omgeving van Den Haag, Jacob Maris, 1891

Cellar of the Artist’s Home in The Hague, Johan Hendrik Weissenbruch, 1888
How to practice rationality?
by warrenjordan — 2020-12-23T23:15:25.204Z

Newbie to rationality and to LW. I'm currently going through the Hammertime Sequence to help provide structure to my rationality practice. Once I'm done, I'm not sure how I can integrate a daily/routine practice for rationality. Does anyone have suggestions how I can go about practicing?

Reading some of the Sequences and the CFAR handbook have helped, but I'm definitely an experiential learner and find myself not knowing how to put rationality into routine practice. Any suggestions or resources would be helpful!
My favorite essays of life advice
by benkuhn — 2020-12-23T23:00:10.832Z

I start each of my weekly reviews by re-reading one of my favorite essays of life advice—a different one each week. It’s useful for a few different reasons:

It helps me get into the right reflective frame of mind.

The best essays are dense enough with useful advice that I find new interesting bits every time I read them.

Much good advice is easy to understand, but hard to implement. So to get the most benefit from it, you should find whatever version of it most resonates you and then re-read it frequently to keep yourself on track.

I’ve collected my favorite essays for re-reading below. I’ll keep this updated as I find more great essays, and I’d welcome other contributions—please suggest your own favorites in the comments!

There's a lot of essays here! If you'd like, I can email you one essay every weekend, so you can read it before your weekly review:
(sign up on site)

Paul Graham, Life is Short. Inspire yourself never to waste time on bullshit again:

Having kids showed me how to convert a continuous quantity, time, into discrete quantities. You only get 52 weekends with your 2 year old. If Christmas-as-magic lasts from say ages 3 to 10, you only get to watch your child experience it 8 times. And while it’s impossible to say what is a lot or a little of a continuous quantity like time, 8 is not a lot of something. If you had a handful of 8 peanuts, or a shelf of 8 books to choose from, the quantity would definitely seem limited, no matter what your lifespan was.

Ok, so life actually is short. Does it make any difference to know that?

It has for me. It means arguments of the form “Life is too short for x” have great force. It’s not just a figure of speech to say that life is too short for something. It’s not just a synonym for annoying. If you find yourself thinking that life is too short for something, you should try to eliminate it if you can.

When I ask myself what I’ve found life is too short for, the word that pops into my head is “bullshit.” I realize that answer is somewhat tautological. It’s almost the definition of bullshit that it’s the stuff that life is too short for. And yet bullshit does have a distinctive character. There’s something fake about it. It’s the junk food of experience. [1]

If you ask yourself what you spend your time on that’s bullshit, you probably already know the answer. Unnecessary meetings, pointless disputes, bureaucrac

...
Sunzi's《Methods of War》- Planning Attacks
by lsusr — 2020-12-23T21:52:34.217Z

This is a translation of Chapter 3 of The Art of War by Sunzi. No English sources were used.

孙子曰：夫用兵之法，

全国为上，破国次之，

全军为上，破军次之；

全旅为上，破旅次之；

全卒为上，破卒次之；

全伍为上，破伍次之。

It is best to conquer a state whole rather than breaking it.

It is best to conquer an army whole rather than breaking it.

It is best to conquer a regiment whole rather than breaking it.

It is better to conquer a company whole rather than breaking it.

It is better to conquer a squad whole rather than breaking it.

是故百战百胜，非善之善者也；

不战而屈人之兵，善之善者也。

A hundred victories in as many battles is imperfect perfection.

To defeat an enemy without a fight is perfect perfection

故上兵伐谋，

其次伐交，

其次伐兵，

其下攻城。

The best use of soldiers is to dispatch plans,

followed by dispatching ambassadors,

followed by dispatching troops,

followed by attacking cities.

攻城之法为不得已。

Attack cities reluctantly.

修橹轒辒，具器械，三月而后成，距堙，又三月而后已。

Furbish your mortuary tools. In three months, dig a grave. Then dig another grave three months after that.

将不胜其忿而蚁附之，杀士三分之一而城不拔者，此攻之灾也。

A general who grows impatient, angers and storms a city will kill one third of the attacking force and fail to take the city.

故善用兵者，

屈人之兵而非战也，

拔人之城而非攻也，

毁人之国而非久也，

必以全争于天下。

The perfect general

subdues the enemy without battle,

draws enemies out of cities to be attacked,

destroys people quickly,

utilizes every attack under the sun.

故兵不顿而利可全，此谋攻之法也。

Total benefit without halt, this is the strategem of war.

故用兵之法，

十则围之，

五则攻之，

倍则分之，

敌则能战之，

少则能逃之，

不若则能避之。

If you outnumber the enemy

10× surround

5× attack

2× divide

1× engage, if possible

<1× run, if possible

≪1× escape, if possible

故小敌之坚，

大敌之擒也。

A small stubborn force will be captured by a large enemy.

夫将者，国之辅也。

辅周，则国必强；

辅隙，则国必弱。

The general complements the country. If the complement is whole then the country will be mighty. If the complement is chinked then the country will be weak.

故君之所以患于军者三：

There are three obstructions to commanding armies:

不知军之不可以进而谓之进，不知军之不可以退而谓之退，是谓“縻军”；

If you do not know the army cannot advance then the army will not advance at your command. If you do not know the army cannot retreat then the army cannot retreat at your command. This is called a "tied up" army.

不知三军之事，而同三军之政者，则军士惑矣；

If you do not know the three armies[1] and their command structure then soldiers will be confused.

不知三军之权，而同三军之任，则军士疑矣。

If you contradict the commands already issued by officers then they will be hesitant.

三军既惑且疑，则诸

...
MIRI: 2020 Updates and Strategy
by Rob Bensinger — 2020-12-23T21:27:39.206Z
Link: https://intelligence.org/2020/12/21/2020-updates-and-strategy

Highlights from MIRI's year-end strategy update:

MIRI executive director Nate Soares reports that the "new research directions" we initiated in 2017 have "at this point, largely failed, in the sense that neither Eliezer nor I have sufficient hope in it for us to continue focusing our main efforts there. [...] We are currently in a state of regrouping, weighing our options, and searching for plans that we believe may yet have a shot at working."
Our most interesting (to us) research we've written up publicly this year is Scott Garrabrant's Cartesian Frames and Vanessa Kosoy and Alex Appel's Infra-Bayesianism.
Our funding situation is good, and we aren't running a fundraiser this winter.
We're (still) seriously thinking about the pros and cons of moving out of the Bay Area, and thinking about what other locations have to offer.

You can find the full post here.
New Bucket Brigade UI
by jefftk — 2020-12-23T20:30:03.185Z

After using the UI that the

Ritual Engine folks
built for

Solstice, I was
feeling a bit embarrassed with how awkward and confusing stand-alone
Bucket Brigade was:

I took a pass over it to make the UI more coherent:

There are a lot of changes, but the biggest ones:

There is now an explicit concept of 'bucket' in the UI:

Names are drawn in the appropriate bucket for their position.

You can join a bucket by clicking a button.

You shouldn't have to deal with numeric positions.

If you do need it there is still a box where you
can manually enter a position, and positions are displayed in the
mixing console

It uses the whole page instead of making you scroll around:

Links are across the top

Controls are across the bottom

If there is too much to display in the main panel, just not
panel scrolls.

There are multiple main panels, with a tabbed interface.

Settings for configuring roundsinging are out of the main flow,
on the "Advanced Settings" tab.

Debug settings are divided into ones that only affect you
versus ones that affect everyone.

There's a setting for adjusting how loud the backing
track / metronome is, if you're using one.

Instead of looking at the user agent string to determine
whether the browser supports what we need, we specifically
test for them (if (window.AudioWorklet) ...).

It is a (debatably) tasteful blue, and has a logo.

Since the app only works in modern browsers anyway, I styled the UI
with flexbox.
It's wonderful that there is now a way to use CSS that is solidly
better than tables.

I'm pretty happy with it, and while we still don't have automated
tests it seems to work correctly with my poking at it. If you find any
problems or have suggestions, please let me know either in the
comments here or by filing
an issue.

Comment via: facebook
Announcing AXRP, the AI X-risk Research Podcast
by DanielFilan — 2020-12-23T20:00:00.841Z

Happy holidays! Today, I’m launching AXRP (pronounced axe-urp), the AI X-risk Research Podcast. The episodes involve me interviewing a researcher about a paper they’ve written, talking about the ideas in the paper and why they matter. The first three guests are Adam Gleave, Rohin Shah, and Andrew Critch. You can listen on all major podcast distribution networks by searching “AXRP”, or read transcripts online at axrp.net. If you have comments about the show, feel free to leave them here, or to email me at feedback@axrp.net.
2019 Review Rewrite: Seeking Power is Often Robustly Instrumental in MDPs
by TurnTrout — 2020-12-23T17:16:10.174Z
Link: https://www.lesswrong.com/posts/6DuJxY8X45Sco4bS2/seeking-power-is-often-robustly-instrumental-in-mdps

For the 2019 LessWrong review, I've completely rewritten my post Seeking Power is Often Robustly Instrumental in MDPs. The post explains the key insights of my theorems on power-seeking and instrumental convergence / robust instrumentality. The new version is more substantial, more nuanced, and better motivated, without sacrificing the broad accessibility or the cute drawings of the original.

Big thanks to diffractor, Emma Fickel, Vanessa Kosoy, Steve Omohundro, Neale Ratzlaff, and Mark Xu for reading / giving feedback on this new version.

Here's my review, which I also posted as a comment.

Self-review

One year later, I remain excited about this post, from its ideas, to its formalisms, to its implications. I think it helps us formally understand

part of the difficulty of the alignment problem

. This formalization of power and the

Attainable Utility Landscape

have together given me a

novel frame for understanding alignment and corrigibility

.

Since last December, I’ve spent several hundred hours expanding the formal results and rewriting

the paper

; I’ve generalized the theorems, added rigor, and taken great pains to spell out what the theorems do and do not imply. For example, the main paper is 9 pages long; in Appendix B, I further dedicated 3.5 pages to exploring the nuances of the formal definition of ‘power-seeking’ (Definition 6.1).

However, there are a few things I wish I’d gotten right the first time around. Therefore, I’ve restructured and rewritten much of the post. Let’s walk through some of the changes.

‘Instrumentally convergent’ replaced by ‘robustly instrumental’

Like

many

good things, this terminological shift was prompted by a critique from Andrew Critch.

Roughly speaking, this work considered an action to be ‘instrumentally convergent’ if it’s very probably optimal, with respect to a probability distribution on a set of reward functions. For the formal definition, see Definition 5.8 in the paper.

This definition is natural. You can even find it echoed by Tony Zador in the

Debate on Instrumental Convergence

:

So i would say that killing all humans is not only not likely to be an optimal strategy under most scenarios, the set of scenarios under which it is optimal is probably close to a set of measure 0.

(Zador uses “set of scenarios” instead of “set of reward functions”, but he is implicitly reasoning: “with respect to my beliefs...
Notes on Patience & Forbearance
by David Gross — 2020-12-23T16:31:04.680Z

This post examines the virtues of patience and forbearance. I mean to explore what other people have learned about these virtues, rather express my own opinions about them, though I’ve been selective about what I found interesting or credible, according to my own inclinations. I wrote this not as an expert, but as someone who wants to learn. I hope it helps people who want to know more about these virtues and how to nurture them.

What is this virtue?

If you have patience and forbearance you gracefully put up with the usual slings and arrows that beset a human life. You can show patience in the face of boredom, suffering, difficulty, pain, wait, and annoyance. Forbearance usually means, more specifically, restraint in the face of difficult people. Someone with forbearance suffers fools gladly, because, well, why let their foolishness stop your gladness? A person with forbearance is slow to anger, and tolerant of insults and idiocy.

Patience and forbearance include self-regulating your emotions and moods, as well as moderating your behavior.

These virtues are related to self-control / serenity / good temper / moderation / balance / harmony (you don’t lose your cool), perspective and humility (it’s not all about you), charity / forgiveness / mercy / clemency / tolerance (you give other people more slack), dignity and grace (patience is a good look), endurance, perseverance, and stoic acceptance. The virtue (or “pāramitā”) of kṣānti includes both patience and forbearance.

What good is it?

Levius fit patientia, quicquid corrigere est nefas
(“What cannot be quite cured, is made easier by patience”)

―Horace[1]

Patience makes it easier to acquire skills that require practice and to accomplish things that are difficult or time-consuming. It helps us to decline immediate gratification in exchange for future rewards. It helps us wait for opportune moments to act.[2]

Impatience can make a bad situation worse, both by being unpleasant and by prompting us to make suboptimal decisions.

Patience is also a factor in the “attention span” that everybody seems to complain about in this day and age. By helping us resist the eyeball-capturing techniques of clickbait instant gratification and peripheral ⓴ notifications, patience helps us more rationally prioritize our time.

Patience makes it easier to avoid jumping to premature conclusions, so it can help you consider nuance, change your mind...
Moral moralizing
by KatjaGrace — 2020-12-23T06:50:12.509Z

Most people get their ethics from a combination of trusting what is normally done in their society and doing what they feel is right.

It seems to me that this has been utterly discredited as a reliable source of ethical advice, because it is the same one that recommended to the average person slavery as labor, and genocide as standard geopolitical strategy, and rape as prize, and torture as entertainment.

I don’t know of a clearly better source, for a typical person who isn’t a professional ethicist. But given the intense fallibility of this one, I’m inclined to say that the resulting moral views should be held with uncertainty and modesty.

From this I’m inclined to infer that people should not be moralizing about moral disagreements.

I do share the urge to moralize when I’m in a moral disagreement, but is this wrong?
Will AGI have "human" flaws?
by Agustinus Theodorus — 2020-12-23T03:43:41.910Z

In the wake of GPT 3, it seems that AGI is possible. No question about that. But my question is can it be more "human" than we think? I watched an interview with Eric Elliot and GPT 3. The machine says that it can lie, I don't think it is impossible for a machine to lie (as far as I know, but I could be wrong). But then it begs the question, how many human traits will AGI pick up? And if picking up negative human traits is a consequence of AGI itself how "general" do we want to make it?

I am questioning the purpose of making AGI not because I fear it, I just don't understand where the direction of this entire concept is going. Any references to the discussions and ethics regarding this matter will be very appreciated.