trentmkelly/LessWrong-43k huggingface dataset:
Is MIRI actually hiring and does Buck Shlegeris still work for you?
by seed — 2021-02-13T10:26:23.362Z

I applied to MIRI software engineer position last year on 26 February 2020. The first stage of the interview was a Triplebyte quiz. Triplebyte said I passed it "particularly well" and qualify for Fast Track program. However, Buck Shlegeris said he's not interested in proceding with an interview, with no explanation. Well, maybe my test score wasn't good enough. Anyway, he provided me with a list of ways to impress him and get back on track, and I did one of them (solve 8 chapters of Lean exercises). Buck said that I've managed to impress him and he'll get back to me about next steps. Two months later he said that MIRI is not hiring because of the pandemic. Buck promised to contact me when the hiring starts again.

I have recently learned from a friend that Buck doesn't work for MIRI anymore, and his Linkedin page also says that he left in 2020. So I decided to reapply for the job, but I got back an automated reply from Buck Shlegeris with a link to the same Triplebyte quiz.

I am confused. Does Buck still work for MIRI? If you aren't hiring, why are there job ads on your website?
Things a Katja-society might try (Part 2)
by KatjaGrace — 2021-02-13T09:20:14.706Z

(Part 1)

Carefully structured and maintained arguments for any interesting claims that people believe. For instance, I would like to see the argument for any of the causes generally considered Effective Altruist carefully laid out (I’m not claiming that these don’t exist, just that they aren’t known to me).

A wider variety of accommodations. For instance, you could rent houses in cheap versions of this sort of style:

(A view of the interior of Nasir ol Molk Mosque located in Shiraz, Iran. Image: Ayyoubsabawiki, CC BY-SA 4.0, via Wikimedia Commons)

(Interior view of the dome of the Shah Mosque, Isfahan, Iran. Photo: Amir Pashaei, CC BY-SA 4.0, via Wikimedia Commons)

(Sheikh Lotfallah Mosque, Isfahan, Iran. Photo by Nicolas Hadjisavvas, CC BY 2.5, via Wikimedia Commons)

Adult dorms. An organization would buy a complex that could house something like a few hundred people, and had common areas and such. They would decide on the kind of community they wanted, and what services they would provide (e.g. cleaning, food, nice common areas). There would be a selection process to get in. If you lived there, you would be assumed part of the community, like in school.

Well directed quantitative contests and assessments for adults, that put numbers on things that the adults would like to care about. If there were a Save The World From X-Risk Olympiad, or an ‘expected relative value of your intellectual contributions’ number that was frequently updating, it would be easier to optimize for those things relative to e.g. optimizing for number of internet-dogs who visited your page, or success at memorizing Anki cards.

Social-implication-limited socializing services. There are many reasons to want to be around other people, and not all of them are strongly correlated with wanting the entire cluster of attributes that come with the specific personal relationships that you want. For instance, if you want some social pressure to have your act together sometimes, but the kinds of people you make friends with are too forgiving, you can have someone with their act together stop by sometimes and totally expect you to be on top of things. Or if you are sick and want someone nice to take care of you, yet none of your specific personal relationships are at a point where getting vomit on them would be a plus? Or if you just sometimes wan

...
Your Cheerful Price
by Eliezer Yudkowsky — 2021-02-13T05:41:53.511Z

There's a concept I draw on often in social interactions. I've been calling it the "happy price", but that is originally terminology by Michael Ellsberg with subtly different implications. So I now fork off the term "cheerful price", and specialize it anew. Earlier Facebook discussion here.

Tl;dr:

When I ask you for your Cheerful Price for doing something, I'm asking you for the price that:

Gives you a cheerful feeling about the transaction;
Makes you feel energized about doing the thing;
Doesn't generate an ouch feeling to be associated with me;
Means I'm not expending social capital or friendship capital to make the thing happen;
Doesn't require the executive part of you, that knows you need money in the long-term, to shout down and override other parts of you.

The Cheerful Price is not:

A "fair" price;
The price you would pay somebody else to do similar work;
The lowest price such that you'd feel sad about learning the transaction was canceled;
The price that you'd charge a non-friend, though this is a good thing to check (see below);
A price you're willing to repeat for future transactions, though this is a good thing to check (see below);
The bare minimum amount of money such that you feel cheerful. It should include some safety margin to account for fluctuating feelings.

The point of a Cheerful Price, from my perspective as somebody who's usually the one trying to emit money, is that:

It lets me avoid the nightmare of accidentally inflicting small ouches on people;
It lets me avoid the nightmare of spending social capital while having no idea how much I'm spending;
It lets me feel good instead of bad about asking other people to do things.

Warnings:

Not everybody was raised with an attitude of "money is the unit of caring and the medium of cooperation" towards exchanges with an overt financial element. Some people may just not have a monetary price for some things, such that the exchange would boost rather than hurt their friendship, and their feelings too are valid. Not as valid as mine, of course, but still valid.
"I don't have a cheerful price for that, would you like a non-cheerful price" is a valid response.
Any time you ask somebody for a Cheerful Price, you are implicitly promising not to hold any price they name against them, even if it's a billion dollars.
If Tell Culture doesn't work for someone, Cheerful Prices may not work for

...
Watching themselves
by jefftk — 2021-02-12T22:20:06.159Z

We've been very lucky with childcare this pandemic. We had an au pair,
so when schools closed suddenly on a Wednesday in early March we had
live-in child care for Lily (6y) and Anna (4y). Having our children
well taken care of without risking exposure outside the home or
requiring either of the parents to take time off work has been
incredibly helpful. Going into winter, with restrictions still in
full, however, our au pair was not interested in renewing for another
year.

We were again lucky, and one of our housemates has been available to
watch the kids three days a week. They're really great, and the kids
are having a good time, but this does leave two days a week without
care. We decided to draw on some of the independence we've been
cultivating, and I talked to the kids about how they would be
watching themselves two days a week.

Over the next few weeks the kids and I worked through some plans. We
went over their day, talking through the different places where they
currently rely on adults, and we figured out how we were going to
handle each one. Some examples:

Lunch. This wasn't something they were going to be able to do
on their own, so I agreed I would still make them lunch. When
possible, I eat lunch with them and read to them. It's nice to have
time together, and I think they also do eat more that way.

Snacks. We talked about what food they would like to have
available in case they got hungry (peanut butter pretzels, Ritz
crackers) and made a small shelf for them to use.

Drinks. They can already get themselves water whenever they
want to, but they like to drink milk. The milk jug is heavy and easy
to spill, so we decided that each day I would fill a cup with milk and
leave it in the fridge. I'm still doing this, though mostly they've
ended up just drinking milk at meals when an adult would be able to
give them milk anyway.

Classes. Lily is in school remotely, and has various classes at
different times during the day. She knows how to get into her classes
herself, but she was relying on grown-ups to let her know when the
classes would happen. I wanted to use this as an opportunity for
practice at telling time, but practically she wasn't going to get good
enough quickly enough. She has a tablet that she uses to listen to podcasts, and I set up
alarms on it to go off two minutes before each class was supposed to
start. S

...
Potential factors in Bell Labs' intellectual progress, Pt. 1 
by Ruby — 2021-02-12T19:26:42.167Z

Epistemic status: these are notes from a popular account filtered through my own existing beliefs. Here, I am largely trusting the book to report true things and not misrepresent them, though, in fact, I suspect the book is trying to create a certain narrative that might be misleading. If I were to get very serious about Bell Labs, I'd have to look at more source material.

Over the years, I've heard various people reference Bell Labs as a place where a uniquely large amount of intellectual progress was made, making it a worthy target of investigation if you're interested in intellectual progress.

A few days ago, I started reading The Idea Factory: Bell Labs and the Great Age of American Innovation. I'm only 20% of the way through, but I've started to note various factors that might explain their output.

Many of the factors that are salient to me were already in my bag of hypotheses and could just represent confirmation bias on my part. A few were surprising. I suppose I should also look for factors I expected to see but haven't yet (look into the dark).

Note: the most significant invention to come out of Bell Labs was the transistor and a lot of the book has focused on that, but they did other notable things too.

Factors Salient to Me

While the work often tended in the direction of basic science that was distant from practical application, by dint of it occurring within AT&T, it was expected that all the research done might somehow benefit AT&T and I sense a degree of backchaining from that in all they did.
Ten years before the achievement of the transistor, Mervin Kelly said to William Shockley that they really needed something solid state that could amplify and switch to replace fragile and undurable vacuum tubes and relays. It took a tonne of basic research in solid state physics to get there, but ultimately that was the driving motivation. (Only after its announcement did some folks from MIT write to say it might be applicable to electronic computer circuits.)
A lot of the innovation was motivated by concrete problem solving, e.g., needing to figure out better cable insulation and sheathing so they could run underwater cables – that and related projects required a lot of materials innovation.
Their work was highly empirical, highly directed, and highly applied. There was a lot of feedback. Often they were trying to build devices that did certain things an

...
Tournesol, YouTube and AI Risk
by adamShimi — 2021-02-12T18:56:18.446Z

Introduction

Tournesol

is a research project and an app aiming at building a large and varied database of preference judgements by experts on YouTube videos, in order to align YouTube’s recommendation algorithm towards videos according to different criteria, like scientific accuracy and entertainment value.

The researchers involved launched the

website

for participating last month, and hope to ratchet a lot of contributions by the end of the year, so that they have a usable and useful database of comparison between YouTube videos. For more details on the functioning of Tournesol, I recommend the video on the

front page

of the project, the

white paper

and

this talk

by one of the main researchers.

What I want to explore in this post is the relevance of Tournesol and the research around it to AI Alignment. Lê Nguyên Hoang, the main research on Tournesol, believes that it is very relevant. And whether or not he is right, I think the questions he raises should be discussed here in more detail.

This post focuses on AI Alignment, but there are also a lot of benefits to get from Tournesol on the more general problem of recommender systems and social media. To see how Tournesol should help solve these problems, see the

white paper

.

Thanks to Lê Nguyên Hoang and Jérémy Perret for feedback on this post.

AI Risk or Not AI Risk

There are two main ways to argue about Tournesol’s usefulness and importance for AI Alignment, depending on a central question: is YouTube’s algorithm a likely candidate for a short timeline AGI or not? So let’s start with it.

YouTube and Predict-O-Matic

Lê believes that YouTube’s algorithm has a high probability of reaching AGI level in the near future -- something like the next ten years. While I’ve been updating to shorter timelines after seeing the GPT models and talking with

Daniel Kokotajlo

, I was initially rather dismissive of the idea that YouTube’s algorithm could become an AGI, and a dangerous one at that.

Now I’m less sure of how ridiculous it is. I’m still not putting as much probability as Lê does, but our discussion was one of the reasons I wanted to write such a post and have a public exchange about it.

So, in what way could YouTube’s algorithm reach an AGI level?

(Economic pressure) Recommending videos that are seen more and more is very profitable for YouTube (and its parent company Google). So there is an economic incentive to push the under

...
Super-forecasters as a service

by frcassarino — 2021-02-12T13:35:19.062Z
Link: https://federicorcassarino.substack.com/p/super-forecasters-as-a-service

I've just started building a website yesterday that I think would be super interesting, but I'm not sure if it's legal or if Metaculus/Elicit would be fine with it.
The idea: Super-forecasters as a Service.

Say I wanted to know if I should book a flight to Costa Rica this summer, but I'm hesitant because of Covid flight restriction uncertainty. I could create the following question using

Elicit

: "Will commercial flight BA2490 from the UK to Costa Rica in July 24 be cancelled?"

The website would let you embed your elicit question, and pay people for predicting. People get paid based on their

Points-per-question in Metaculus

. You can "auction" 100 dollars on a question, and pick a base price per prediction multiplied by the Points-per-question metric.

The more money you auction, the more predictions you'll get. The higher the multiplier, the more predictions will tend to come from top-predictors.
Benefits:

It would give all of us access to predictions from the top forecasters in Metaculus.
It would incentivize people to become better predictors in Metaculus, so they can get paid more per prediction in my website.
It would

lend credibility to prediction markets

, if the best predictors are making money for predicting.

Drawbacks:

It could encourage more people to game Metaculus.
It would not be as efficient as a straight prediction market, as people have no incentive to make an effort to make good predictions in this website, as there's no scoring. They are incentivized to make as many predictions as possible to make more money.

Questions:

I'm I right to assume that this would be legal, as I'm paying people a fixed fee per prediction (based on their Metaculus score), rather than paying based on correctly predicting the question?
Would Metaculus be fine with this idea? Would they be fine with me scraping

https://metaculusextras.com/points_per_question?page=1

and using that data to determine the score for each person?
Would Elicit be fine with this idea? Would they be willing to give me access to their API, or a way to embed questions directly on my Website? By the way, I tried to reach out to you folks by joining your Slack, but it seems to be invite only.
Originally posted to Substack: https://federicorcassarino.substack.com
Mapping the Conceptual Territory in AI Existential Safety and Alignment
by jbkjr — 2021-02-12T07:55:54.438Z

(Crossposted from my blog)

Throughout my studies in alignment and AI-related existential risks, I’ve found it helpful to build a mental map of the field and how its various questions and considerations interrelate, so that when I read a new paper, a post on the Alignment Forum, or similar material, I have some idea of how it might contribute to the overall goal of making our deployment of AI technology go as well as possible for humanity. I’m writing this post to communicate what I’ve learned through this process, in order to help others trying to build their own mental maps and provide them with links to relevant resources for further, more detailed information. This post was largely inspired by (and would not be possible without) two talks by Paul Christiano and Rohin Shah, respectively, that give very similar overviews of the field,[1] as well as a few posts on the Alignment Forum that will be discussed below. This post is not intended to replace these talks but is instead an attempt to coherently integrate their ideas with ideas from other sources attempting to clarify various aspects of the field. You should nonetheless watch these presentations and read some of the resources provided below if you’re trying to build your mental map as completely as possible.

(Primer: If you’re not already convinced of the possibility that advanced AI could represent an existential threat to humanity, it may be hard to understand the motivation for much of the following discussion. In this case, a good starting point might be Richard Ngo’s sequence AGI Safety from First Principles on the Alignment Forum, which makes the case for taking these issues seriously without taking any previous claims for granted. Others in the field might make the case differently or be motivated by different considerations,[2] but this still provides a good starting point for newcomers.)

Clarifying the objective

First, I feel it is important to note that both the scope of the discussion and the relative importance of different research areas change somewhat depending on whether our high-level objective is “reduce or eliminate AI-related existential risks” or “ensure the best possible outcome for humanity as it deploys AI technology.” Of course, most people thinking about AI-related existential risks are probably doing so because they care about ensuring a good long-term future for humanity, but t...
The art of caring what people think
by KatjaGrace — 2021-02-12T05:40:12.633Z

People care what people think. People often strive to not care what people think. People sometimes appear to succeed.

My working model though is that it is nearly impossible for a normal person to not care what people think in a prolonged way, but that ‘people’ doesn’t mean all people, and that it is tractable and common to change who falls into this category or who in it is salient and taken to represent ‘people’. And thus it is possible to control the forces of outside perception even as they control you. Which can do a lot of the job of not caring what other people think.

To put it the other way around, most people don’t care what other people think, for almost all values of ‘other people’. They care what some subset of people think. So if there are particular views from other people that you wish to not care about, it can be realistic to stop caring about them, as long as you care what some different set of people think.

Ten (mostly fictional) examples:

You feel like ‘people’ think you should be knowledgeable about politics and current events, because they are always talking about such things. You read some philosophers through the ages, and instead feel like ‘everyone’ thinks you should be basically contributing to the timeless philosophical problems of the ages. (Also, everyone else has some kind of famous treatise - where is yours?)

You haven’t really thought through which causes are important, but ‘people’ all seem to think it’s nuclear disarmament, so looking into it feels a bit pointless. You go to a weekend conference on soil depletion and experience the sense that ‘people’ basically agree that soil degradation is THE problem, and that it would be embarrassing to ask if it isn’t nuclear disarmament, without having a much better case.

You are kind of fat. You wish you didn’t care what ‘people’ thought, but you suspect they think you’re ugly, because you’ve seen ‘people’ say that or imply it. You read about all the people who appreciate curviness, and recalibrate your sense of what ‘people’ think when they see you.

You can hardly think about the issue of gun regulation because you feel so guilty when you aren’t immediately convinced by the arguments on your side, or don’t have an eloquent retort for any arguments the other side comes up with. You wish you were brave enough to think clearly on any topic, but knowing everyone agrees that you wou

...
Institute for Assured Autonomy (IAA) newsletter
by Aryeh Englander — 2021-02-11T21:48:52.420Z

Just a very brief link to a resource that people may not know about:

Johns Hopkins Institute for Assured Autonomy newsletter

For anybody interested in nearer-term AI safety / assured autonomy issues, I have found this recently-started newsletter to be a useful aggregator of relevant news and articles.

(Full disclosure: Much of my work is for the IAA.)
Participating in a Covid-19 Vaccine Trial
by ejacob — 2021-02-11T20:35:37.740Z
Link: https://eidan.substack.com/p/the-vaccine-trial-chronicles-1

I live near Stony Brook University, which has a well-known hospital and medical school. About a month ago I signed up to participate in research related to Covid-19. As a work-from-homer, this was a way I could reasonably get a vaccine without resorting to “skipping the line” or pretending I had a pre-existing condition. Today I received my first dose. Here is the story to date. I plan to post at least one follow-up to this.

The study I'm participating in is a phase III trial for the Novavax vaccine. Studies in the UK and South Africa already have demonstrated it to be safe and effective (at least to a similar standard as the Pfizer and Moderna options). Novavax also has the added advantage that it can be kept stable at higher temperatures than the mRNA vaccines. I'm not opposed in principle to studying its efficacy more, but I think it's silly that this trial has to happen before it can be distributed or used in the US. FDA Delenda Est.

Day -2: Registration

I receive a phone call from a lady calling herself Bella. Bella asks if I am still interested in participating in Covid-19 vaccine trials. I am. She takes my name, address, medical history, etc. She says to expect an email with directions to the clinic and some videos to watch.

Day -1: Orientation

In the morning there is an email from Bella with links to a required series of introductory videos. After each video there are a few questions to make sure I was paying attention. The videos are old. I can tell they are old because the second one says “There is no approved vaccine available for Covid-19.” Unforced error count: 1. One of the questions for this video asks if there are any approved vaccines available for Covid-19. I know what will happen if I get a question “wrong,” and I don’t want to try asking anyone to update the form, so I select “no.” I’ve already entered a simulacrum. I learn that if I participate in the study, I will receive the Novavax Covid-19 vaccine with probability 2/3 or a placebo with probability 1/3, and earn up to $1700 depending on how diligently I participate in follow-up. Sounds great.

Day 0: First Visit and Vaccination

I get in the car and drive to Stony Brook University Medical Center’s satellite clinic in Commack, NY. I walk in to a large lobby. A sign on the information desk says “COVID-19 STUDY: ROOM 201.” I go to suite 201 and enter another lobby. I walk up to the receptionist.

A...