trentmkelly/LessWrong-43k huggingface dataset:
Digital Dinner Signup
by jefftk — 2022-08-23T02:20:04.404Z

For

house dinner we need a way of
coordinating who is going to cook what days and marking who
will/won't be at dinner. For the past ~7y we've used a piece of paper
on the fridge with a row for each day. This is not a bad system, and
it's generally worked well, but it has a few downsides:

Some of our housemates live downstairs, and coming up to read
or modify the sheet is a bit annoying.

One of our housemates will soon move to a nearby house, making
this even harder.

Same issue if you are traveling or want to make a change while
you're out.

Some housemates are "default present" while others are "default
absent", and it's annoying remembering which.

We're adding a new housemate who will only be here some weeks.

So we're trying out a new system, a digital one. Like with any post
you make right when you start a new system there's some risk that it
isn't a good one and you don't know yet, but that beats never
describing it at all.

We now have one Google Calendar event for each day of
the week, each marked as recurring weekly. Each housemate is invited,
and can RSVP to individual instances or the repeated event. This lets
you communicate "I'll be gone this Friday", or "I'm not going to make
Fridays", or "I'm not going to make Fridays, but I'll be here this
Friday". To cook you edit the title to add "Name cooking" and
if you're bringing a guest you can either invite them to the instance
of the event or add their name to the title.

It's also helpful to see cooking and eating plans at a glance, so I
made a web page that summarizes the current state. You log in with
Google, give it read access to your calendar, and it summarizes events
named "Dinner":

The page is here and
you're welcome to look at the source, but it won't work for you unless
you live here. If you want to tweak and use for your house, though,
go ahead! There are comments in the HTML source saying how.

Overall this is almost the way I like it, except that login only lasts
for an hour. The authentication token I get from Google is good for
one hour, after which you need to OAuth again (though you don't need
to reconsent to Calendar API access). I haven't figured out how to
fix this, and I've asked on
StackOverflow.
Stable Diffusion has been released
by P. — 2022-08-22T19:42:20.120Z
Link: https://stability.ai/blog/stable-diffusion-public-release

It's an open-source text-to-image model capable of producing NSFW content. I, for one, I'm very excited to see the consequences this has on society, which I expect to be mostly positive (except if it leads to an increase in AI funding).
Any thoughts?

Other relevant links:

https://www.reddit.com/r/StableDiffusion/

https://www.reddit.com/r/StableDiffusion/comments/wqaizj/list_of_stable_diffusion_systems/

https://news.ycombinator.com/item?id=32555028

https://www.lesswrong.com/posts/DhDAXQw4PsWXnmwPS/ai-art-isn-t-about-to-shake-things-up-it-s-already-here
Finding Goals in the World Model
by Jeremy Gillen — 2022-08-22T18:06:48.213Z

Produced As Part Of The SERI ML Alignment Theory Scholars Program 2022 Under John Wentworth

Introduction

This post works off the assumption that the first AGI comes relatively soon, and has an architecture which looks basically like EfficientZero, with a few improvements: a significantly larger world model and a significantly more capable search process. How would we align such an AGI?

Our pitch is to identify human values within the AGI’s world model, and use this to direct the policy selector through IRL (inverse reinforcement learning). We take a lot of inspiration from Vanessa’s PreDCA proposal [

comment

][

video

], as well as ideas developed in Infra-Bayesian Physicalism. We have stripped these down to what we saw as the core insights, meaning that there are significant differences between this and PreDCA. We initially arrived at this proposal by thinking about an idea similar to

"retarget the search"

, except we’re using hard-coded search instead of learned optimizers, and doing the "identify human values" part using a mathematical definition of agents & goals.

Source: DALLE-2

We think that this proposal directly gets at what we view as the core of the alignment problem: pointing to human values in a way that is robust as capabilities scale. Naturally this all depends on the research outlined in the 'Research Required' section succeeding. See the last few sections to see many of the difficulties of this approach.

Architecture assumptions

The most important assumption that we are making is that the agent is designed to explicitly search over actions that maximize a utility function. This is opposed to the model where an AGI is a single component trained end-to-end by RL (or self-supervised learning), and where the AGI learns its own mesa-objective (or the mesa-objectives of simulacra[1]) internally. We will lay out a concrete vision of this model, but keep in mind that the exact details don't matter much.[2]

We are also structuring this proposal to point this agent to maximize the preferences of a single human (henceforth "the operator"), as a simplifying assumption, because we think the difficulty of this proposal is getting it to work for a single human, and scaling to recognize many people's preferences and aggregate them is not too hard.

IRL training procedure for an aligned EfficientZero style model

We train a generative world model. Thi

...
Please (re)explain your personal jargon
by Nathan Helm-Burger — 2022-08-22T14:30:46.774Z

Technical jargon is quite useful. It allows us to put sharp definitions onto natural phenomena useful for precise scientific discussion. However, it can also be confusing, since you are creating or re-using words to mean this new specific precise thing. So, whenever you write an independent piece of text (e.g. a scientific article or an internet post), you need to define any jargon you use which is not yet accepted as common knowledge within the field you are writing for.

Once upon a time, long before personal computers were invented (much less anything like 'deep neural nets'), some scientists were discussing a concept. This concept was something like "obvious and useful features in an environment which tend to independently reoccur as concepts in the minds of intelligent agents". In fact, these scientists realized that these two attributes, obvious and useful, were scalar values which had positive predictive value for whether the concepts would arise.

This was in the time before ubiquitous computers, so the intelligent agents being studied were humans. In this disconnected pre-'Information Era' age there were still many groups of humans who had never communicated with each other. They had infinite degrees of Kevin Bacon, if you can imagine that! The scientists, who today we would call 'social anthropologists' or 'linguists' went to these isolated groups and took careful note of which concepts these people had words for. A group of people having a commonly understood word for a thing is a good proxy for them having that concept in their minds.

One example of a set of concepts that were thoroughly studied was color. Not all colors are equally obvious and useful. Some sets of colors are harder for some humans to distinguish between, such as blue/green or red/green or yellow/green. Nevertheless, most humans can distinguish reasonably well between strong hue differences in natural environments, so there isn't a huge difference in 'obviousness' between colors. This provides a nice natural experiment isolating 'usefulness'. Colors which are more useful in the an environment are much more likely to exist as concepts in the minds of intelligent agents interacting with that environment. The scientists found that 'red' tended to be a more useful color in most of the environments studied, and more consistently represented in independent languages than other colors....
On Car Seats as Contraception
by Zvi — 2022-08-22T14:10:01.299Z

(Note: This post takes the perspective of strongly preferring higher American fertility to lower American fertility on anything close to current margins. I endorse this view and consider it very overdetermined, but this post does not defend or justify it.)

(Note: This post has been edited to include consideration of two studies arguing against the main study.)

It is often said that governments can do little to increase the birth rate.

I find this implausible. We do lots of things as the government that decrease the birth rate. We could Stop It. Or do the opposite.

A paper that came to my attention recently looks at a concrete example of such a policy: Car seat requirements.

You can see the various rules by state here. Some are relatively reasonable. Others, not so much.

Several commenters have disputed the conclusion that these seats have minimal effects beyond the age of two, including pointing to this study claiming 55% injury reduction in ages 4-8 (sci-hub), and this study claiming 29% reduction in injuries in ages 8-12. I examine them in an added area of the safety section. There are reasons to worry the controls are insufficient, in ways the main study examined does not need to worry about. Neither study has the usefully power to look at deaths, and the death data does not leave room for much effect there. There is still plausibly substantial benefit on preventing lesser injuries.

The first best solution to this particular issue is to eliminate all car seat requirements, or at least limit such requirements to children under the age of two.

Failing an age two limit, limiting to the first four years would confidently capture most of the safety benefits, while eliminating more than 80% (and likely more than 90%) of the negative effects.

Failing that, we could even exempt any child with two younger siblings (or even two younger children present in the car), for obvious practical reasons, and still most of the negative effects (at least on births) go away while most of the safety benefits clearly remain.

Car seat requirements also allow us to get an estimate of what impact a direct front-loaded economic subsidy might have on American fertility. I estimate that various incentives will generate or prevent one additional birth per ~$270,000 in effective size, with the note that front-loading likely increases this effect while spreading it out over a longer period will li...
Is there a Lesswrong newsletter I can subscribe to?
by emily.fan — 2022-08-22T13:28:59.308Z

I've gone on EA forum much more often when I get regular emails about new posts on the EA forum. I want to check LessWrong more often, but... I don't...

The EA forum had a mailing list that was hidden and very hard to search for; took me around 6 months after it launched for me to know that it even existed: https://forum.effectivealtruism.org/posts/zRoXctENAsLchwryn/sign-up-for-the-forum-s-email-digest

On the home page of LessWrong, on the side bar, I clicked subscribe, and email, but I'm not getting any emails from less wrong.

I prefer not to have RSS; I think I tried figuring out how to get RSS to work, but then got confused / it had too much activation energy to set up properly?

Other things I can try

using tabsnooze / snoozz to open windows on a scheduled basis

setting up a recurring email to remind me to open up LessWrong

However the newsletter is better because I can see interesting posts that I want to read and directly click on them.
AI art isn't "about to shake things up". It's already here.
by Davis_Kingsley — 2022-08-22T11:17:55.415Z

For a while, I've been seeing people commenting about how AI art is on the cusp of shaking up the art world. Quite frankly, at this point I consider such sentiments to be behind the times. AI art is not "on the cusp" of disrupting such things. It is already here. Using only capabilities that are straightforwardly and publicly available right now, AI art has radically transformed the budget for certain types of project that require art.
Let's give a basic example. I play card games of various types -- games like Magic: the Gathering or similar -- on a highly competitive level. I've even been involved in testing and development for some such games, so I'm familiar with that process as well. I am not a professional game designer, but am fairly involved with some aspects of the field.
For one of these games, a typical budget for an individual card illustration of sufficient quality is, as I understand it, in the realm of $200-1000 USD. A single "set" of cards might require 100+ such illustrations. That means that you're looking at paying $20k in art costs on the low end, and that this is a recurring cost every time you want to make a new set that isn't just reprinting old cards -- and even then, sometimes new art is used for reprints!
Except, well, that was then and this is now. Now, if I were in the business of making a card game set, my art budget wouldn't be $20k-100k. It would be... a $30/month Midjourney subscription with $20/month private visibility enabled, and quite frankly high quality Midjourney images look better than many of the images already being used for art in these games. [1]

I'm going to highlight that again. The price for the art needed to create a set of a hundred cards just went from twenty thousand dollars-- at the low end -- to fifty bucks a month. This is an extreme shift, and it is already here. This is not something that is based on a press release or future development that hasn't arrived yet. This is something that I could do today, using only techniques that are widely known and publicly available. If you assume you can get the art needed in one month of Midjourney time, that's four hundred times cheaper.
There are many other areas where this applies. What's the price for a book cover? Quite frankly, that's not my field -- but whatever it is, I'm going to bet that Midjourney is often going to be cheaper and better. What's the price for t...
What if we solve AI Safety but no one cares
by 142857 — 2022-08-22T05:38:02.894Z

Suppose that next year, AI Safety is solved, the solution is approved by Eliezer Yudkowsky, etc.

How do we actually get people to follow this solution?

It seems to me a lot of people or companies will ignore any AI Safety solutions for the same reasons they are currently ignoring AI Safety:
- They think AGI is still very far away, so AI Safety methods won't need to be applied to the development of current narrow AI systems
- The concepts of AI Safety are difficult to understand, leading to incorrect application of an AI Safety solution or failure to apply it at all
- They will fall behind their competitors or make less money if they adhere to the AI Safety solution
- They have a new cool idea they want to test out and just don't care about or believe in the concerns raised by AI Safety

Thoughts?
AXRP Episode 17 - Training for Very High Reliability with Daniel Ziegler
by DanielFilan — 2022-08-21T23:50:20.513Z

YouTube link

Sometimes, people talk about making AI systems safe by taking examples where they fail and training them to do well on those. But how can we actually do this well, especially when we can’t use a computer program to say what a ‘failure’ is? In this episode, I speak with Daniel Ziegler about his research group’s efforts to try doing this with present-day language models, and what they learned.

Listeners beware: this episode contains a spoiler for the Animorphs franchise around minute 41 (in the ‘Fanfiction’ section of the transcript).

Topics we discuss:

Summary of the paper

Alignment as scalable oversight and catastrophe minimization

Novel contributions

Evaluating adversarial robustness

Adversary construction

The task

Fanfiction

Estimators to reduce labelling burden

Future work

About Redwood Research

In this transcript, to improve readability, first names are omitted from speaker tags.

Filan:
Hello everybody. Today, I’ll be speaking with Daniel Ziegler. After spending time as an ML engineer on the alignment team at OpenAI, Daniel is now the lead of the adversarial training team at Redwood Research. In this episode, we’ll be speaking about the paper ‘Adversarial Training for High-stakes Reliability’, on which he’s the first author. For links to what we’re discussing, you can check the description of this episode, and you can read the transcript at axrp.net. Welcome to the show, Daniel.

Ziegler:
Thanks.

Summary of the paper

Filan:
So this paper: first of all, could you just summarize for us what’s in it, what it is, what it does?

Ziegler:
Sure. So basically, we took a pretty simple task that we think has some important analogous features to the kinds of AGI alignment situations we’re worried about in the future. And then we tried to attack it with adversarial training and see whether we could get really good adversarial robustness. So basically, we had a generator that was trained to generate fanfiction stories. It sees three sentences of an existing story, and it wants to generate one more sentence. And its task, or the task that we’re interested in, is to generate that sentence without introducing any new injuries that weren’t already present. And we’re using this as a stand-in for some catastrophic behavior that a future AI system could do. And so, our hope was: let’s take some generator that is pretty good at producing text, ...
Volunteer to host a meetup!
by mingyuan — 2022-08-21T22:43:06.468Z

Once (or twice) a year, Scott Alexander promotes a bunch of meetups on his blog, and it's that time of year again! If you've ever wanted to start a meetup, this is a great time — these events get a ton of visibility. If you feel like you're 'not qualified' to run a meetup, don't worry, almost no one is 'qualified' but their meetups go fine anyway! All you have to do is set a time and place, and show up to meet other people who also like reading Scott's blog.

You can volunteer to host a meetup in your city by filling out this form. The deadline is the end of the day on Tuesday (two days from now).

If you want more details, check out the ACX post announcing the meetups, or ask me questions here — I do all the administrative work for these big meetup announcements.
The Alignment Problem Needs More Positive Fiction
by Netcentrica — 2022-08-21T22:01:59.878Z

What follows is the draft of a letter I wrote but did not send to UC Berkeley Center for Human-Compatible AI aka CHAI. Whoever I sent it to might feel put on the spot which would limit its dissemination. So I thought I would post it here and see what the LessWrong community, thought in regards to the issue I raise.

[Beginning of draft email]

To whom it may concern:

Given CHAI’s efforts at communicating its message and reading through your 2020 Progress Report I am surprised there are no channels other than non-fiction.

There are three reasons I suggest why being open to members of your community writing and sharing related fictional stories among themselves would be of benefit.

Firstly, we learn well from stories. They are a proven, high quality teaching and communications method.

Secondly they show if something is properly understood or not by providing complex, “real life” settings and scenarios. For example, I have been writing daily about AI now for two years. I am not an academic but this has required a great deal of research. Reading through my short story, “The Alignment Problem”, do I get Stuart Russell’s essential concepts right? Using non-technical language, did I communicate the issue correctly? Did the ending raise an issue with any merit within the current paradigm?
https://acompanionanthology.wordpress.com/the-alignment-problem/

Thirdly, a great deal of evidence with regards to the dangers of advanced AI is provided by CHAI and related organizations and communities but very little in the way of showing what the positive results of your efforts would look like. It would seem that Professor Russell would agree with the idea that fiction has value to at least to some degree given his involvement in the short video Slaughterbots. That video shows what the concern is but how would you show what successfully addressing the concern would look like? What might “the new model for AI” look like in story form? What specific story details would demonstrate the concepts involved?

As far as what the members of your community might base such stories on there are already plenty of potential ideas in your 2020 progress report.

I am not suggesting the general public will have any interest in such stories, their tastes lie elsewhere. Nor am I suggesting that anyone write with the intention of becoming a successful writer of fiction but within your community there may be va...
The Bunny: An EA Short Story
by JohnGreer — 2022-08-21T20:59:26.154Z

This is cross-posted from my blog.

“Eight villages in our district have had significant to total structural damage from mudslides. Furthermore, the weather conditions conducive to mudslides have only been increasing in the past decade.”

Professor Cristian Avendaño clicked the button and a list of temperatures, wind conditions, and annual rainfall popped up on the screen. He looked every bit the absent-minded professor in his weathered sport coat with elbow patches. But his mind was far from absent.

“As you can see, the water accumulation is a serious problem.”

More graphs and figures appeared.

The professor glanced up and saw half the council nodding politely and the other half trying to stifle yawns. Most of the villagers were on their phones or peering around the auditorium.

If they would have read the slide, they would have seen that the professor’s model predicted the chances of a mudslide destroying their village were one in three in the next five years.

The head of the council pressed down on his mic.

“Thank you for your presentation, Professor. We’ll take your suggestions under advisement. Now, Ms. Zelada, you have something you’d like to speak to us about?” his voice perking up at the end.

Eliza Zelada stood up confidently and smiled.

“Yes, thank you, Councilman Ricardo.”

Her stroll up to the lectern would have been equally at home on the runway with her long, red dress flowing around her. She leaned forward and tapped the mic.

“I am here today to talk to you about a grave problem that requires our immediate action,” she said, spreading her arms out for effect.

The villagers began sitting up in their seats and clearing their throats. The room was suddenly at attention.

She clicked the remote.

A giant photograph appeared on the screen. It was of a bunny huddled in a trash pile, surrounded by plastic bottles and dirty newspapers.

Gasps filled the room.

“Is the bunny okay, Mamá?” a little girl quietly sobbed.

Eliza paused and took in the moment before continuing.

“How can this be our lovely village? How can this be our Salgar? We must do something about this! Immediately!”

Cheers filled the room. Councilman Ricardo clapped the loudest of anyone. It had been rumored the councilman and Eliza were having an affair.

Gerardo the restaurant owner stood up. “From now on, I will only use sustainable products!”

Applause. Eliza nodded approvingly.

Mrs. Domingo, the matronly figure who ...
If you know you are unlikely to change your mind, should you be lazier when researching?
by Double — 2022-08-21T17:16:53.343Z

I'm currently doing research into political candidates to decide how I'm going to vote. It's tedious because I don't know of a good system that prescribes how much research to do and what exactly to factor into my voting decision. I'm wondering whether the information in Yudkowsky's "We Change Our Minds Less Often Than We Think" would justify a lazy approach: once I start leaning one way or the other, assume that more information won't change that (because We Change Our Minds Less Often Than We Think) and move on.