trentmkelly/LessWrong-43k huggingface dataset:
No nonsense version of the "racial algorithm bias"
by Yuxi_Liu — 2019-07-13T15:39:12.545Z

In discussions of algorithm bias, the COMPAS scandal has been too often quoted out of context. This post gives the facts, and the interpretation, as quickly as possible. See this for details.

The fight

The COMPAS system is a statistical decision algorithm trained on past statistical data on American convicts. It takes as inputs features about the convict and outputs a "risk score" that indicates how likely the convict would reoffend if released.

In 2016, ProPublica organization claimed that COMPAS is clearly unfair for blacks in one way. Northpointe replied that it is approximately fair in another way. ProPublica rebukes with many statistical details that I didn't read.

The basic paradox at the heart of the contention is very simple and is not a simple "machines are biased because it learns from history and history is biased". It's just that there are many kinds of fairness, each may sound reasonable, but they are not compatible in realistic circumstances. Northpointe chose one and ProPublica chose another.

The math

The actual COMPAS gives a risk score from 1-10, but there's no need. Consider the toy example where we have a decider (COMPAS, a jury, or a judge) judging whether a group of convicts would reoffend or not. How well the decider is doing can be measured in at least three ways:

False negative rate = (false negative)/(actual positive)

False positive rate = (false positive)/(actual negative)

Calibration = (true positive)/(test positive)

A good decider should have false negative rate close to 0, false positive rate close to 0, and calibration close to 1.

Visually, we can draw a "square" with four blocks:

false negative rate = the "height" of the false negative block,

false positive rate = the "height" of the false positive block,

calibration = (true positive block)/(total area of the yellow blocks)

Now consider black convicts and white convicts. Now we have two squares. Since they have different reoffend rates for some reason, the central vertical line of the two squares are different.

The decider tries to be fair by making sure that the false negative rate and false positive rates are the same in both squares, but then it will be forced to make the calibration in the Whites lower than the calibration in the Blacks.

Then suppose the decider try to increase the calibration in the Whites, then the decider must somehow decrease the false negative rate o

...
Reclaiming Eddie Willers
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2019-07-13T15:32:01.040Z

[Content: personal ramble, not trying to be rigorous]

When I read Atlas Shrugged a few years ago, it was one of the more intensely disturbing experiences I’ve had.

I remember that Eddie Willers was the only character I resonated or identified with much. He’s also, as far as I can tell, the only (even slightly positively portrayed) Hufflepuff character in the story. And the last we see is of him alone in the wilderness, as the last train breaks down – mistakenly loyal to the train company, an entity that isn’t capable of loyalty in return, and not agenty or cool enough to join the main protagonists in their escape from the collapse of civilization.

That...really got to me. I won’t make any claims about whether Atlas Shrugged is a particularly well-written book, or whether it even contains this message on purpose, but at that moment in my life, it painted a very vivid, compelling picture of a world in which to be Hufflepuff is to be unsafe, useless, unwanted. Incapable of agency or of doing the right thing when it matters. Eddie is an earnest idealist, trying to do his best by Dagny Taggart and her company, and that trait is his doom.

(I was recently quoted a friend of mine saying “a Hufflepuff among Slytherins will die as surely as among [snakes? Don’t remember exact quote]”. Right now, this feels like an example of that phenomenon.)

I notice a desire to push back against that interpretation. I claim that Eddie is flawed, imperfect, and his last choice ends up being ineffective, but not because of his earnest idealism. He’s being unstrategic, not paying attention to the patterns of his world and what will actually work – but I refuse to say that his caring about the train reaching its destination is a mistake.

Loyalty isn’t necessarily strategic, and blind loyalty can lead into disaster, but I refuse to say that having a drive towards it is inevitably a character flaw.

In the real world, it matters if trains reach their destinations. It’s a bad thing if civilization collapses because all the people who could have stopped it walked away. And it doesn’t make someone a fool, or pitiable, or merely a foil for the true protagonists, if they genuinely and earnestly care.

If I were in Eddie Willers’ shoes, transplanted as I am now into the world of Atlas Shrugged – I don’t think I would be blindly loyal to Dagny Taggart, or to her company. I hope I would actually ta...
Job description for an independent AI alignment researcher
by rmoehn — 2019-07-13T09:47:54.502Z

This is the job description that I've written for myself in order to clarify
what I'm supposed to be doing.

I'm posting it here in order to get feedback on my understanding of the job.
Also, if you're thinking of becoming an independent researcher, you might find
it useful to know what it takes.

Admin

Job Title: Independent AI alignment researcher

Location: anywhere (in my case: Kagoshima, Japan)

Reports To: nobody (in a sense: funders, mentors, givers of feedback)

Position Status: not applicable

Responsibilities

Define AI alignment research projects. Includes finding questions,
gauging their significance and devising ways to answer them.

Execute research projects by reading, thinking and experimenting.

Write and publish results in the form of blog entries, contributions to
discussions and conferences (conference paper, presentation, poster), journal
articles, public datasets, software.

Solicit feedback and use it to improve processes and results.

Find potential junior (in the sense of (slightly) less experienced in the
field) researchers and help them grow.

Help other researchers with their work.

Make sure that the money doesn't run out.

Any other activities required by funders or givers of feedback.

Hiring requirements

Entry level:

Strong desire to do good for the world by contributing to AI alignment.

Undergrad degree or equivalent skill level in computer science, maths or
machine learning. Includes having researched, written and presented a
scientific paper or thesis.

Ability to define, plan and complete novel projects with little supervision.

Ability to collaborate remotely.

Initiative.

Discipline.

Ability to speak and write clearly.

Ability to identify and close gaps in knowledge or skills.

Ability to write job or funding applications.

Ability to figure out things that are usually taken care of for employees:
taxes, insurance, payments, bookkeeping, budgeting.

Ability to deal with uncertainty and financial stress.

Resources used

Manager Tools: Writing a Job
Description

Booth et al.: The Craft of
Research
Raw Post: Talking With My Brother
by DirectedEvolution — 2019-07-13T02:57:42.142Z

The Circumstances

I am sitting down to write this immediately after one of the most honest conversations I’ve ever had with my brother. The reason I’m posting it to LessWrong is because I think it is a case study in rationality, emotion, personality, political partisanship, and methods of conversation and debate, all topics that are of interest to segments of this community. We spoke for about an hour, first while driving, and then for a long time at the curb.

We started talking about my brother’s interest in getting involved with the local socialist party. He is not the most talkative person, and is a deeply thoughtful and very well-read person. One of his strong interests is in politics and economics, so I decided to ask him about his thoughts on socialism. I am no lassaiz-faire capitalist, but my political preferences are for well-regulated free markets.

Rising Tension

Our conversation became tense quickly. As I tried to ask him critical questions in a neutral, genuine, and thoughtful manner, he would dismiss them using words like “silly,” “stupid,” “artificial binary,” “lack of imagination,” and so on. This didn’t feel good, but I continued, because my hope was that by maintaining my composure and demonstrating repeatedly that I was really listening and responding with my valid questions and concerns, he would see that I really wanted to engage with him and wasn’t trying to shut him down. I used techniques like trying to find our cruxes of disagreement, framing them as respectfully and clearly as I could, but he would swat them down. He grew audibly angrier as the car ride went along.

I could have tried to divert the conversation to some other topic, but I don’t think that’s healthy, and our family dynamic is such that I feel very confident that this would not have led to a happy atmosphere, but to unaddressed simmering resentment that would have lingered beyond our car ride. So I pressed on, all the way until we got to Seattle.

When he accused me of silliness, I offered what I thought was a detailed and thoughtful description of how I thought things might go under his proposed system. When he accused me of disingenuously demanding that every minor detail be worked out in advance to stifle a basic and obviously good shift that needs to happen, I told him that this was my attempt to really put my imagination to work, thinking through the implications of an idea...
What are we predicting for Neuralink event?
by Dr_Manhattan — 2019-07-12T19:33:57.759Z

Interesting exercise in AI-adjacent forecasting area (brain-computer interfaces). Curious if people want to specify some possible reveals+probabilities. https://twitter.com/neuralink/status/1149133717048188929

(if in the somewhat likely scenario you're relying on inside info please mention it)
My Conversion from LW to Pragmatism: Steelman
by hunterglennzero — 2019-07-12T19:17:55.144Z

Some think of humans as the "scientific species." But also as failures, because, well, humans mostly do just fail when it comes to thinking and acting scientifically.

It's a little odd, isn't it, that we'd be so bad at doing the thing we were built for?
So, maybe we weren't built for it after all. Because whatever we're really built for, we should probably be pretty good at it.

Actually, when you think about it, it doesn't really make sense that we would be scientifically-oriented. Science has only really been around for a few hundred years. If you could look at humans from 10,000 years ago, they wouldn't be trying to be scientists and then failing at it.
They just wouldn't be trying to be scientific at all. They had never even heard of the idea.

And it would really be kind of weird to look at a species that has never heard of science, and isn't trying to do it, and call them a "scientific species," wouldn't it.

So, what are we built for? What are we succeeding at?

I think it's more like magic and tribalism.

Tribalism has been talked a lot about, so what about magic?

What I mean by magic is *mental tools*. Ideas in your head that help you handle problems, even if they're not factual.

Factual ideas. That's the "scientific species" idea, right? They say that we're trying to have factual ideas in our heads, but most of us are pretty bad at it.
But what exactly is it that we have in our heads instead of facts? For we clearly have ideas in our heads, just not factual ones.
If we're not choosing our ideas so we can have factual accuracy, what are we choosing them for?

For magic. For their tool-like nature. For their ability to empower us to navigate the treacherous terrain in our life.

And that should sound a little familiar, because, after all, what is it that people cling to instead of to factual ideas? What kind of ideas do people protect from science and refuse to give up?
It's the ideas that help them get through the day, and it's the ideas that help them get through their fiery trials, and it's the ideas that help them get through life itself!

Useful ideas.

So, there it is. Humans are a species really good at finding ideas that work for them, "magic." In other words, humans aren't failed scientists, they're successful magicians. They di...
Largest open collection quotes about AI
by teradimich — 2019-07-12T17:18:20.401Z
Link: https://docs.google.com/spreadsheets/d/19edstyZBkWu26PoB5LpmZR3iVKCrFENcjruTj7zCe5k/edit?gid=1448563947#gid=1448563947

I apologize for my bad English, this is not my native language. And probably I will make some mistakes when posting.
For over 2 years I have been reading materials on the topic of AI Safety. I don't have the appropriate education, cognitive abilities, knowledge. I do not even have time to learn the language. So I didn't hope to do something useful myself.

But once I tried to systematize quotations from one show in order to understand when the experts represented there are waiting for AGI and how likely they consider the extinction of humanity.

I thought it would be interesting to do so with the rest of the experts.
In addition, I have already seen and studied with interest such collections of quotes. It seemed to me that the best thing I could do was try to do something similar.
Therefore, I began to collect quotes from people who can be attributed to the experts. It turned out to be harder than I thought.

I have compiled a table with quotes from more than 800 experts. I tried not to distort the opinion of forecasters and simply copied from sources, sometimes deleting or slightly editing. My edits can be recognized by square brackets :)
1) The first column of the table is the name of the expert.
2) The second column is the year of the forecast. The table is built in chronological order.
3) The third column is the predicted time for AGI. Unfortunately, most people did not speak directly about time and probability. Because of this, many quotes came out rather vague. For example, “Machines are very far from being intelligent” or “And we can reach it in a close time”.
4) The fourth column is an opinion about Takeoff Speed. About how much progress will be accelerated after create of AGI.
5) The fifth column is the expert's opinion about the future of mankind with AGI. Choosing a quote here was the hardest. Most of all I was interested in the risk of extinction or serious shocks due to AI, and I tried to provide quotes that most fully reveal this particular topic.
6) The sixth column indicates the source of the quote.
That is, to the right of the forecaster's name, you can find out the date of the given quotes, his opinion about the time of the creation of AI, about the intellectual explosion and about the future of humanity, as well as get acquainted with the source.
Of course, cases where the expert spoke on the topic of time, the speed of self-improvement and the influence ...
Bystander effect false?
by Ben Pace — 2019-07-12T06:30:02.277Z

Can someone check this link out and see whether the methodology is actually sound?

One-line summary is that Surveillance Cameras Debunk the Bystander Effect H/T Hacker News.

More broadly I'm interested in anyone's sense of whether the bystander effect replicates, and whether the corresponding concept is misleading (and I should use something else instead).
If I knew how to make an omohundru optimizer, would I be able to do anything good with that knowledge?
by mako yass — 2019-07-12T01:40:48.999Z

I'd bet we're going to figure out how to make an omohundro optimiser - a fitness-maximizing AGI - before we figure out how to make AGI that can rescue the utility function, preserve a goal, or significantly optimise any metric other than its own survival, such as paperclip production, or Good.

(Arguing for that is a bit beyond the scope of the question, but I know this position has a lot of support already. I've heard Eliezer say, if not this exactly, something very similar. Nick Land especially believes that only the omohundro drives could animate self-improving AGI. I don't think Nick Land understands how agency needs to intercede in prediction - that it needs to consider all of the competing self-fulfilling prophesies and only profess the prophesy it really wants to live in, instead of immediately siding with the prophesy that seems the most hellish, and most easiest to stumble into. The prophesies he tends to choose do seem like the easiest prophesies to stumble into, so he provides a useful service as a hazard alarm, for we who are trying to learn not to stumble))

What would you advise we do, when one of us finds ourselves in the position of knowing how to build an omohundro optimiser? Delete the code and forget it?

If we had a fitness-optimising program, is there anything good it could be used for?
How much background technical knowledge do LW readers have?
by johnswentworth — 2019-07-11T17:38:37.839Z

When writing posts, it would be useful to know how much background technical knowledge LW readers have in various areas.

To that end, I set up a short six-question survey. Please take it, so I (and others) can write posts better fit to your level of technical background. If your answer to all of the questions is "zero-ish technical knowledge", please take it, so you're not inundated with mathy posts. If your answer to all of the questions is "I am secretly John Von Neumann", please take the survey, so the rest of us know there's someone like that around. If you are somewhere in the middle, please take the survey. It'll take, like, maybe sixty seconds.

Here's that link again: survey.
New SSC meetup group in Lisbon
by tamkin&popkin — 2019-07-11T12:19:52.540Z

After years of reading Slate Star Codex and LessWrong (more the former than the latter), I've decided to take a more active role in the so-called rationalist community by creating an account on Less Wrong and meeting people in real life for whom the Sequences have been important in their critical thinking.

The ideal way for me would be to join a SSC/LW meetup group in Lisbon, where I live, but, unfortunately, there wasn't one in Lisbon until now! So, if you live in Lisbon or you're just passing by, let's meet and have a chat about some SSC posts or any topic that relates to the rationalist community. I'm very curious to know how many people read regularly SSC or LW and how they relate that in their lives. If you're interested I'm easy to find on twitter at @m8popkin, send me a message on LessWrong or send a comment on the meetup group.
Are we certain that gpt-2 and similar algorithms are not self-aware?
by Ozyrus — 2019-07-11T08:37:59.606Z

Have someone even started this conversation? This is f*cked up. I'm really, really freaked out lately with some of those. https://old.reddit.com/r/SubSimulatorGPT2/comments/cbauf3/i_am_an_ai/

And I read up a lot about cognition and AI, and I'm pretty certain that they are not. But shouldn't we give it a lot more consideration just because there's a small chance they are? Because the ramifications are that dire.

But then again, I'm not sure that knowledge will help us in any way.
Modeling AI milestones to adjust AGI arrival estimates?
by Ozyrus — 2019-07-11T08:17:55.914Z

Is there a roadmap of major milestones AI researchers have to achieve through before AGI is here? I feel like this would be beneficial to update on AGI arrival timelines and adjust actions accordingly. Lately, a lot of things that can be classified as a milestone have been achieved -- GLUE benchmark, StarCraft II, DOTA 2 etc. Should we adjust our estimates on AGI arrival based on those or are they moving according to plan? It would be cool to have a place with all forecasts on AGI in one place.
Please give your links speaking names!
by rmoehn — 2019-07-11T07:47:07.981Z

Bad:

Shrimping is a fundamental drill for grappling, according to this article.

Good:

Shrimping is a fundamental drill for grappling, according to The Ultimate Guide to Developing BJJ Hip Movement Through Shrimping.

Why is this important? So that people who print your articles know what you're referencing.

Why would anyone print an article? In order to be able to annotate it more easily, and to read with more context. Try it! Instead of scrolling back and forth in that complicated essay you're reading on a thirteen inch laptop, print it and spread six pages on the desk. I read many articles on AI alignment issues and printing them helps a lot.
AI Alignment "Scaffolding" Project Ideas (Request for Advice)
by DirectedEvolution — 2019-07-11T04:39:11.401Z

I believe that AI Alignment is almost certainly the most pressing issue for the future of humanity. It seems to me that the greatest thing that could happen for AI alignment research is probably receiving a whole lot more brains and money and political sponsorship. The public benefit is extraordinary, and the potential for private profit very small, and so this will need substantial private or government subsidy in order to receive optimal resource allocation.

In order to start thinking about strategies for achieving this, I picture scientific work as a sort of signalling system between research, the educational system, government, and industry, as diagrammed below. I want to apply the neglected/tractable/important framework to this diagram to explore potential projects.

The Technical Side

a) Professional technical work on AI alignment

b) Amateur and student learning contributing or leading to technical work

c) Meta-analysis of the state of the art, risks and rewards, milestones, and big questions of AI and AI alignment, via surveys, forecasts, overviews of different perspectives, etc.

d) Awareness-raising discussion and expert advice for policy-makers, the public, and potential career changers/donors

e) Laws, regulations, and projects created by legislators, public policy makers, and private institutions

f) Pressure by industry lobbyists and geopolitical tension to soften AI alignment concerns and go full steam ahead with AI development.

The Political Side

Questions

1) Do any of the following exist?

A comprehensive AI alignment introductory web hub that could serve as a "funnel" to turn the curious into the aware, the aware into amateur learners, amateurs into formal machine learning PhD students, and PhDs into professional AI alignment researchers. I'm imagining one that does a great job of organizing books, blogs, videos, curriculum, forums, institutions, MOOCs, career advising, and so on working on machine learning and AI alignment.
A formal curriculum of any kind on AI alignment
A department or even a single outspokenly sympathetic official in any government of any industrialized nation
Any government sponsorship of AI alignment research whatsoever
A list of concrete and detailed policy proposals related to AI alignment

2) I am organizing an online community for older career changers, and several of us are interested in working on this issue on either the...