trentmkelly/LessWrong-43k huggingface dataset:
Will OpenAI's work unintentionally increase existential risks related to AI?
by adamShimi — 2020-08-11T18:16:56.414Z

[The original question was "Is OpenAI increasing the existential risks related to AI?" I changed it to the current one following a discussion with Rohin in the comments. It clarifies that my question asks about the consequences of OpenAI's work will assuming positive and aligned intentions.]

This is a question I've been asked recently by friends interested in AI Safety and EA. Usually this question comes from discussions around GPT-3 and the tendency of OpenAI to invest a lot in capabilities research.

[Following this answer by Vaniver, I propose for a baseline/counterfactual the world where OpenAI doesn't exists but the researchers there still do.]

Yet I haven't seen it discussed here. Is it a debate we failed to have, or has there already been some discussion around it? I found a post from 3 years ago, but I think the situation probably changed in the meantime.

A couple of arguments for and against to prompt your thinking:

OpenAI is increasing the existential risks related to AI because:

They are doing far more capability research than safety research;
They are pushing the state of the art of capability research;
Their results will motivate many people to go work on AI capabilities, whether out of wonder or out of fear of unemployment.

OpenAI is not increasing the existential risks related to AI because:

They have a top-notch safety team;
They restrict the access to their models, by either not releasing them outright (GPT-2) or bottlenecking access through their API (GPT-3);
Their results are showing the potential dangers of AI, and pushing many people to go work on AI safety.Is OpenAI increasing the existential risks related to AI?
Book review: Architects of Intelligence by Martin Ford (2018)
by Ofer — 2020-08-11T17:30:21.247Z

Cross-posted from the EA Forum.

The 2018 book Architects of Intelligence by Martin Ford is a collection of 23 interviews about progress in AI and future impacts thereof, including the prospect of developing AGI and existential risks. The interviewees include some of the most prominent and influential AI researchers, and they vary a lot in the degree to which they take existential risk from AI seriously. They include Andrew Ng and Yann LeCun who, at least historically, were some of the loudest skeptics about existential risk from AI. They also include Nick Bostrom and Stuart Russell, who both played a major role in establishing the field of AI safety.

I highly recommend this book to anyone who's interested in AI safety (both AI alignment and AI policy) and doesn't yet have a good model for how key figures in AI talk about AGI and existential risk from AI, what is socially acceptable to say in what circles, how do people in different circles frame things, etc. AGI and existential risk from AI are sensitive topics, especially to people whose career involves making progress in AI, and if you want to work in related areas it's very useful to have a good model for all the above. Just be aware that the book is from 2018 and at least a bit outdated; there may have been important shifts in some of those aspects since its publication.

Applying the “80/20 rule” when reading this book

Every interview is a separate chapter, titled by the name of the interviewee, including in the Audible version of the book (so it's easy to jump to certain interviews); and each interview is self-contained. If you're interested in the book mainly from an AI safety perspective, I think you would get ~90% of the value from reading just the following 8 interviews (listed in the order they appear in the book):

Yoshua Bengio, who won a Turing Award (with Geoffrey Hinton and Yann LeCun) for his work in deep learning.

Note: In his interview he seems to argue that we shouldn't currently warry about existential risk from AI, but I think his thinking on this topic changed since then. In 2019, he tried to recruit a postdoc to work on AI alignment and he endorsed the 2019 book Human Compatible by Stuart Russell, writing: "This beautifully written book addresses a fundamental challenge for humanity: increasingly intelligent machines that do what we ask but not what we really intend. Essential reading

...
Utility functions without a maximum
by Abhimanyu Pallavi Sudhir — 2020-08-11T12:58:44.354Z

An elementary question that has probably been discussed for 300 years, but I don't know what the right keyword to use to google it might be.

How, theoretically, do you deal with (in decision theory/AI alignment) a "noncompact" utility function, e.g. suppose your set of actions is parameterized by t in (0, 1], and U(t) = {t for t < 1, 0 for t = 1}. Which t should the agent choose?

E.g. consider: the agent gains utility f(t) from expending a resource at time t and f(t) is a (sufficiently fast-growing) increasing function. When does the agent expend the resource?

I guess the obvious answer is "such a utility function cannot exist, because the agent obviously does something, and that demonstrates what the agent's true utility function is", but it seems like it would be difficult to hard-code a utility function to be compact in a way that doesn't cause the agent to be stupid.
Moloch and multi-level selection
by thoughtfulmadison — 2020-08-11T07:08:48.217Z

I'm new to the LW community, and I've been reading the seminal works. After reading Scott Alexander’s Meditations on Moloch and Zvi's Moloch Hasn't Won, I wanted to add a couple thoughts - mostly additional rebuttal of "Meditations...".

To summarize Scott’s argument:

With a few exceptions, because of game theoretic concerns, Malthusian scenarios are inevitable
We should be grateful to have a relatively happy existence, and fight to keep the good things in society
We should generally be pessimistic that “the good life” can last forever

I think that Zvi covers most of the key issues with "Mediations..." in "Moloch Hasn’t Won”. To summarize Zvi’s rebuttals:

Empirically, we see that things actually are pretty good and the worst Malthusian scenarios have not come to pass. This isn’t because of dumb luck.
Perfect competition does not exist in the real world
Uncertainty and environmental change means that there are no truly optimal decisions or static equilibria
Groups are still just collections of individuals - even soulless, profit-optimizing corporations are influenced by individual human concerns

I agree with Zvi. I also think he comes tantalizingly close to a couple of points that are worth calling out explicitly.

I.

Competition is continually playing out at many scales. Individual humans are competing against individual humans. Groups of humans are competing against other groups. Nations compete. Within humans, cells compete. The upshot is that the optimal decision at one scale may be suboptimal at a different scale (Zvi hints at this when he talks about antifragility).

Consider two competing tribes, the Athenians and the Spartans. Individual Athenians are more prone to altruism and cooperation, while individual Spartans are more prone to selfishness and competition.

In Sparta, the more competitive Spartan individuals are selected for vs. the less competitive ones. One spring, Athens and Sparta go to war. In the lead up to the pivotal battle, the Spartan generals cannot agree on a unified battle plan. As a result, the entire Spartan tribe dies.

In victorious Athens, Athenians now specially recognize the need for group solidarity and altruism. They coordinate to prevent a Malthusian race-to-the-bottom. Over the next 1000 years, Athens does not experience selection for strong,...
Swiss Political System: More than You ever Wanted to Know (III.)
by Martin Sustrik — 2020-08-11T05:09:00.368Z

Previous part

When I've mentioned the failed referendum to limit urban sprawl to a Swiss friend he nodded and casually noted that government is already introducing some anti-suburbanization legislation.

Wait! What?

The people have voted against it and the government is still making it happen?

Where's the famed Swiss rule by the people?

And how come that he was not fuming with rage at the government so blatantly ignoring the will of the people?

Well, it turns out that vote against a proposal is not a vote for the opposite extreme. Vote against stricter zoning doesn't mean that the zoning should be relaxed. All it means is that status quo is preserved. The referendum has failed and the government could safely ignore it.

But while that explains why the Swiss haven't stormed Bundeshaus with pitchforks, it doesn't explain why did the government go an extra mile and introduced legislation clearly inspired by a failed popular initiative.

It's not that hard to explain though. You just have to put yourself in the government's shoes.

The result of the initiative haven't been spectacular (36.3% in favor) but the preliminary opinion polls have shown much larger support. At times it even looked like the initiative may succeed. So, presumably, a lot of people felt that something should have been done against the urban sprawl, but they disliked the particulars of the initiative.

If the underlying tension was not relieved, it would probably lead to similar initiatives in the future. And one of those may succeed.

But the government is not particularly thrilled about successful popular initiatives. First, they have no say in the exact wording. Silly things may get in, not because of ill will but because the text of the initiative was written without fully understanding the full scope of the problem and its on-the-ground consequences. Consider one of the government's arguments against the urban sprawl initiative, one, which the authors have clearly overlooked: The initiative would not allow to build greenhouses or poultry halls on agricultural land. The farmers would have to move them to the construction zones where the land is much more expensive. That, in turn, would harm the agriculture.

Second, the text of a successful initiative is incorporated into the constitution and is therefore set in stone. From that point on the government has to carefully navigate around its provisions and th...
Writing Piano Songs: A Journey
by Logan Riggs — 2020-08-10T21:50:25.099Z

This is an account of how I got good at writing piano songs. My first instinct is to write a perfect formalization and algorithm of “how to write a good piano song”, but that would be a lie. Instead, this is a story of slow progression and learned heuristics. I use some music jargon, but the main points can be grokked w/o understanding them.

(If you’d first like some proof of my claimed prowess, go to the bottom for links to songs I’ve written. Or listen while you read!)

Heuristic 1: Trial and Error

In 4th grade, my friend wanted to start up a band. I had a piano at my house, so I was keys. I went home and played different notes in various orders until I came up with a 7-note melody that sounded good. We never actually wrote any songs.

Years later (middle school), I wrote a 2-minute song just by writing 3 melodies that flowed into each other (by lots of trial and error). I even incorporated the left hand, again, by lots of trial and error.

Heuristic 2: Jealousy

In high school/middle school, I took piano lessons for 2 years. My mother paid for them. I only practiced the night before lessons, and really only knew ~4 songs? Not even all of Fur Elise!

But then, I saw a classmate play a cover of Daft Punk’s “Working Harder Makes Us Stronger” for a pretty lady. I’ve had lessons for TWO YEARS! I could impress a pretty lady too! So I went home and found the 4-part tutorial on youtube and learned it.

[A friend shared that he also got into piano to try to impress a girl]

Heuristic 3: Blatant Plagiarism

Somehow I realized that the “fancy left hand” pattern of the Daft Punk cover was repeating like several other songs were [it was a chord progression using a specific arpeggio for each chord]. So I played that same pattern using different chords and voila, everything I played sounded amazing (read, made me appear talented). I used that fancy arpeggio in Fur Elise, Heart & Soul, that song I wrote in Middle School that I still haven’t named, and Pop! Goes the Weasel. I was unstoppable.

But I didn’t stop there; there was a fun “hopping” melody in the Daft Punk song that I took the first 4 notes and then trial-and-errored into a new melody. I’ve since stolen so many chord progressions, melodies, left-hand patterns/ arpeggios, and no one has ever called me on it!

[Most people...
Social Dynamics
by curi — 2020-08-10T19:29:02.767Z

These are some summary notes on social dynamics.

conformity

try to fit in

pandering

pleasing people

avoiding conflict

do whatever the group thinks is high status

follow trends

you need to already have friends. people are impressed by people who other people already like (pre-selection).

have standard interests like the right TV shows, music, movies and sports. talk about those. don’t say weird stuff.

you’re allowed to have interests people think they should have. lots of people think they should be more into art galleries and operas. you can talk about that stuff to people who don’t actually like it but pretend they want to. they’ll be impressed you actually do that stuff which seems a bit inaccessible but valuable to them.

law of least effort

being chased, not chasing

people come to you

opposite of tryhard

less reactive

don’t get defensive or threatened (important for confidence too)

hold frame without showing visible effort

but also don’t let people get away with attacking you

when you attack people, it should seem like the conflict isn’t your fault, was just a minor aside, no big deal to you, preferably you weren’t even trying to attack them

people do what you say

you don’t have to do what other people say

you generally aren’t supposed to care that much about stuff. instead, be kinda chill about life

if you get ahead while appearing this way, it looks like success comes naturally to you. that impresses people. (it should not look like you got lucky)

confidence

hide weakness

pretend to be strong

know what you’re doing, having a strong frame, have goals

be able to lead

best to already be leader of your social group, or at least high up like second in command

value

DHVs (demonstrations of higher value, e.g. mentioning high value things in passing while telling a story)

money, popularity, fame, social media followers, loyal friends, skills, knowledge, SMV (sexual market value, e.g. looks)

abundance mentality

well spoken, know other languages, can play an instrument or sing, cultured, can cook, etc.

virtues like being moral and enlightened are important. these are group specific. some groups value environmentalism, being woke, anti-racist signaling, inclusive attitudes towards various out groups and low status people (poor people, immigrants, disabled, homeless, drug addicts), etc. other groups value e.g. patriotism, toughness, guns, Chr

...
UK Predicted Grades
by cwg1g16 — 2020-08-10T17:36:51.640Z

An issue arising in current UK politics is the issuing of high school grades for students who weren't able to take their exams this summer as a result of the COVID-19 pandemic. This could be an interesting experiment in Game Theory; how does one decide what grade a student should get? Should one decide at all?

Firstly, the main victims of this unfortunate position are those expecting to go to university in September, who are unable to delay by waiting to take exams later this year or next year. The current UK policy was to consider previous and predicted grades of students and to consider the performance of the schools to help make an educated guess as to what the students would have achieved. The issue with this is that some schools overestimate what their students would have achieved and their students predicted grades, either honestly (through error) or dishonestly (to improve their performance records). This means those schools who are honest and correct have their student's grades moderated downwards, meaning many students are disappointed in their scores and may miss out on a place at university. Ultimately, if too many students aren't meeting their university admission offer, universities will have to lower their standards to meet their required course numbers.

Is there a similar situation in Game Theory in which one must guess the outcome of multiple games based on the correlation of past results? Is there a Game Theory solution to this problem?

More practically speaking, if universities make the ultimate decision to admit a student, what is the point in using an arbitrary guess, made by organisations with a conflict of interest, as a basis of admission? Should universities reconsider their process for this year's intake and allow all who have an offer into university, regardless of their "grades"? How would that affect universities?
What's the evidence on falling testosteron and sperm counts in men?
by Alexander Gietelink Oldenziel — 2020-08-10T08:58:47.851Z

One occasionally hears some concerns about falling testosteron/sperm counts, usually in some narrative about the Good Olden days when Men were still real Men, etc. It sounds a little like ' they are turning the frogs gay' type of stuff, but perhaps there is something to the scientific claim after all.

EDIT: It seems my phrasing has unnecessarily antagonized people. Please believe this is a good-faith question. Also, after researching the issue I have become significantly more concerned than before.
Forecasting AI Progress: A Research Agenda
by rossg — 2020-08-10T01:04:21.207Z

AI forecasting is an important research area but lacking in a general direction. To address this issue we present a research agenda for AI forecasting that has been generated using the Delphi technique to elicit opinion from 15 leading researchers on the topic (the majority of whom are members of this community). The research agenda can be found on arXiv through this link:

Forecasting AI Progress: A Research Agenda (link to arXiv)

The agenda was framed so that it can be useful to both members of this community as well as the technological forecasting community more broadly. To these ends we plan to submit the arXiv manuscript to Technological Forecasting and Social Change, however, we will wait for roughly a month to receive comments. Please feel free to give us your thoughts here.
Effect of Numpy
by jefftk — 2020-08-10T01:00:09.373Z

In the bucket brigade singing project I talked about

yesterday, the server is Python. We wrote
it in a very straightforward "it won't be fast but at least it's
correct" style. In simple stress testing, I found that it could
handle about 10 clients, which is definitely not enough.

My first thought was to switch it to C with libmicrohttpd,
and for a program which is just a web interface to a circular buffer C
isn't terrible. But how far can we get if we do the hard stuff in
Numpy?

In the initial version there were four parts that did work in
proportion to either the input or the whole buffer:

Clearing the buffer: this is a circular buffer, and as we wrap
around we want to be following the lead singer. We should not hear
things from the last time around. So we need to zero out the buffer
ahead of the lead person.

Incoming audio: we receive a large block of samples from the
network, and need to sum them with what's already there at the right
part of the circular buffer.

Outgoing audio: we need to send a section of the buffer out to
the client to play.

Computing metadata: We draw a debugging chart at the bottom
of the page, and it needs to know what parts of the queue have audio
data. We send information about which frames (128 samples) are
non-zero.

If this were a large project, I would start with profiling, but this
was small enough that I was pretty sure we were just going to want to
convert everything to an efficient implementation. David and I
pair-programmed for a couple hours and turned this
naive
code
into this
numpy-using code.

The refactor brought a single request from 218ms to 74ms. This was
ok, but still not great. The problem was that computing metadata
was really very slow. The other three operations are in proportion to
the size of a request, but the metadata one was in proportion to the
size of the queue. And worse, it involved a Python loop over
128-sample frames.

Since the metadata was only used to populate a debugging chart, I
measured the effect of removing it. In the pure Python version this
brought us from 218ms to 23ms, while with Numpy it brought us from
74ms to 0.06ms. I won't credit Numpy with the effect of removing the
chart, but 23 ms to 0.06 ms is still an excellent 383x speedup.

I doubt this is enough to move us from 10 clients to 3830 clients, but
it helps enough that I expect encoding a...
Fantasy-Forbidding Expert Opinion
by yamabiko — 2020-08-10T00:01:57.009Z

I believe in fairies! I do! I do!

...Except I don't, and neither do you.

"Why don't you believe in fairies?" I ask.

"What an absurd question," you retort, "it's because fairies don't exist." However, the many times when you thought something so-obviously-didn't-exist and were proven absolutely wrong suddenly pop into your mind. "Okay, let's do this exercise then. Why do I believe fairies do not exist?" For the sake of argument, let's equate fairies to diminutive humanoids with insect-like wings that can fly, and leave magic and wonder and weird golden powder out of the equation.

There are certain beliefs you hold about the real world that you deem incompatible with the existence of fairies. For instance,

The widespread knowledge of the existence of another species different from humans but with equal cognitive capabilities would mean that everybody would know about them and whole institutions would be built around them (maybe even countries).

If fairies existed but there was not widespread knowledge about them, it wouldn't take long for them to gain worldwide recognition after their discovery. The only way that would not happen is for the existence of fairies to be intentionally shrouded in secrecy, or for them to not have been discovered at all.

Even if the existence of fairies could be kept secret, it would be extremely rare for nature to spontaneously generate humans AND fairies. Given the extreme likeness and cognitive abilities of fairies and humans, humans would likely be closer to fairies than to any other primate. The divergence would have likely happened very recently, after nature invented big smart human brain, so you cannot help but wonder what selective pressures and coincident random mutations could have led to such a drastic reduction in size and change in function (wings) in such a short time.

"Whew," you think, "I can say for sure now that fairies don't exist." Your belief in the nonexistence of fairies actually paid rent. "But, did I really have to go through all of this just to prove to myself that fairies didn't exist?" Never in your life did you need to go through this tedious exercise to actually believe fairies didn't exist. This is the first time you had even thought of these arguments. Why exactly, then, did you believe that fairies do not exist?

I would place my bet on two reasons:

Everyone else seems to think that fairies don'

...