trentmkelly/LessWrong-43k huggingface dataset:
Intelligence Amplification and Friendly AI
by lukeprog — 2013-09-27T01:09:15.978Z

Part of the series AI Risk and Opportunity: A Strategic Analaysis. Previous articles on this topic: Some Thoughts on Singularity Strategies, Intelligence enhancement as existential risk mitigation, Outline of possible Singularity scenarios that are not completely disastrous.

Below are my quickly-sketched thoughts on intelligence amplification and FAI, without much effort put into organization or clarity, and without many references.[1] But first, I briefly review some strategies for increasing the odds of FAI, one of which is to work on intelligence amplification (IA).

Some possible “best current options” for increasing the odds of FAI

Suppose you find yourself in a pre-AGI world,[2] and you’ve been convinced that the status quo world is unstable, and within the next couple centuries we’ll likely[3] settle into one of four stable outcomes: FAI, uFAI, non-AI extinction, or a sufficiently powerful global government which can prevent AGI development[4]. And you totally prefer the FAI option. What should you do to get there?

Obvious direct approach: start solving the technical problems that must be solved to get FAI: goal stability under self-modification, decision algorithms that handle counterfactuals and logical uncertainty properly, indirect normativity, and so on. (MIRI’s work, some FHI work.)

Do strategy research, to potentially identify superior alternatives to the other items on this list, or superior versions of the things on this list already. (FHI’s work, some MIRI work, etc.)

Accelerate IA technologies, so that smarter humans can tackle FAI. (E.g. cognitive genomics.)

Try to make sure we get high-fidelity WBEs before AGI, without WBE work first enabling dangerous neuromorphic AGI. (Dalyrmple’s work?)

Improve political and scientific institutions so that the world is more likely to handle AGI wisely when it comes. (Prediction markets? Vannevar Group?)

Capacity-building. Grow the rationality community, the x-risk reduction community, the effective altruism movement, etc.

Other stuff. (More in later posts).

The IA route

Below are some key considerations about the IA route. I’ve numbered them so they’re easy to refer to later. My discussion assumes MIRI’s basic assumptions, including timelines similar to my own AGI timelines.

Maybe FAI is so hard that we can only get FAI with a large team of

...
AI ebook cover design brainstorming
by lukeprog — 2013-09-26T23:49:03.319Z

Thanks to everyone who brainstormed possible titles for MIRI’s upcoming ebook on machine intelligence. Our leading contender for the book title is Smarter than Us: The Rise of Machine Intelligence.

What we need now are suggestions for a book cover design. AI is hard to depict without falling back on cliches, such as a brain image mixed with computer circuitry, a humanoid robot, HAL, an imitation of Creation of Adam with human and robot fingers touching, or an imitation of March of Progress with an AI at the far right.

A few ideas/examples:

Something that conveys ‘AI’ in the middle (a computer screen? a server tower?) connected by arrow/wires/something to various ‘skills/actions/influences’, like giving a speech, flying unmanned spacecraft, doing science, predicting the stock market, etc., in an attempt to convey the diverse superpowers of a machine intelligence.

A more minimalist text-only cover.

A fairly minimal cover with just an ominous-looking server rack in the middle, with a few blinking lights and submerged in darkness around it. A bit like this cover.

Similar to the above, except a server farm along the bottom fading into the background, with a frame composition similar to this.

A darkened, machine-gunned room with a laptop sitting alone on a desk, displaying the text of the title on the screen. (This is the scene from the first chapter, about a Terminator who encounters an unthreatening-looking laptop which ends up being way more powerful and dangerous than the Terminator because it is more intelligent.)

Alex Vermeer sketched the first four of these ideas:

Some general inspiration may be found here.

We think we want something kinda dramatic, rather than cartoony, but less epic and unbelievable than the Facing the Intelligence Explosion cover.

Thoughts?
Meetup : Atlanta Pre-Meetup Museum Field Trip
by Nova_Division — 2013-09-26T20:17:43.030Z

Discussion article for the meetup : Atlanta Pre-Meetup Museum Field Trip

WHEN:
28 September 2013 03:00:00PM (-0400)

WHERE:
Atlanta

In celebration of National Free Museum Day, Atlanta Lesswrongers are going to go museuming it up before heading over to our 6 o'clock regularly scheduled meetup.

The exact museum is TBD, check out our facebook post on the matter in our facebook group: https://www.facebook.com/groups/100137206844878/

Discussion article for the meetup : Atlanta Pre-Meetup Museum Field Trip
Meetup : Montreal LessWrong - Return from summer break
by bartimaeus — 2013-09-26T17:49:05.172Z

Discussion article for the meetup : Montreal LessWrong - Return from summer break

WHEN:
01 October 2013 06:00:00PM (-0400)

WHERE:
McGill University, Montreal, QC, Canada

After taking a break for the summer, Montreal LessWrong is once again going to start meeting up regularly. We'll be moving towards more structured content this time around, based on feedback from members. First meetup will be more meta-level stuff, planning location, times, ideas for meetups and so on.
The exact location hasn't been determined yet. Message me and I'll let you know ASAP.

Edit: The meetup will take place in Music - Room A301 (527 Sherbrooke Street West). Message me if you have any difficulty finding it.

Discussion article for the meetup : Montreal LessWrong - Return from summer break
Meetup : Helsinki Meetup
by PyryP — 2013-09-26T17:40:17.413Z

Discussion article for the meetup : Helsinki Meetup

WHEN:
06 October 2013 02:00:00PM (+0300)

WHERE:
Vilhonkatu 4, 00100 Helsinki, Finland

The theme of this meetup will be cognitive bias, and we’ll learn about them through serious games.
We’ll meet in Oluthuone Kaisla. To find us, look for someone wearing a pink elephant hat.

Discussion article for the meetup : Helsinki Meetup
Rationalizing: looking for the wrong kind of loopholes
by bokov — 2013-09-26T15:05:34.827Z

This morning I found what I think is an interesting way to explain rationalizing to my son, and I thought I'd share it:

Physical reality has rules that you can game to your advantage (natural laws).

People have another set of rules that you can game to your advantage (preferences, biases, cultural norms).

Rationalization is when you are trying to overcome an obstacle based in physical reality by trying to game human rules.

Two subsequent thoughts that ocurred to me:

If you're rationalizing-- the magic excuse fairy might not be there to hear you, but your subconsciousness is. And you will often convince your subconsciousness... to believe that the problem you're trying to solve is impossible, that it won't do any good anyway, that people are out to get you, and any number of other non-factual things that are directly antagonistic to your goals. This is why rationalizing is a bad habit.

By this definition, the opposite of rationalizing is using the constraints physical reality to convince people that you are right. This is something you _can_ use effectively and should always try to do. It's called presenting evidence.
Bayesian probability as an approximate theory of uncertainty?
by cousin_it — 2013-09-26T09:16:04.448Z

Many people believe that Bayesian probability is an exact theory of uncertainty, and other theories are imperfect approximations. In this post I'd like to tentatively argue the opposite: that Bayesian probability is an imperfect approximation of what we want from a theory of uncertainty. This post won't contain any new results, and is probably very confused anyway.

I agree that Bayesian probability is provably the only correct theory for dealing with a certain idealized kind of uncertainty. But what kinds of uncertainty actually exist in our world, and how closely do they agree with what's needed for Bayesianism to work?

In a Tegmark Level IV world (thanks to pragmatist for pointing out this assumption), uncertainty seems to be either indexical or logical. When I flip a coin, the information in my mind is either enough or not enough to determine the outcome in advance. If I have enough information - if it's mathematically possible to determine which way the coin will fall, given the bits of information that I have received - then I have logical uncertainty, which is no different in principle from being uncertain about the trillionth digit of pi. On the other hand, if I don't have enough information even given infinite mathematical power, it implies that the world must contain copies of me that will see different coinflip outcomes (if there was just one copy, mathematics would be able to pin it down), so I have indexical uncertainty.

The trouble is that both indexical and logical uncertainty are puzzling in their own ways.

With indexical uncertainty, the usual example that breaks probabilistic reasoning is the Absent-Minded Driver problem. When the probability of you being this or that copy depends on the decision you're about to make, these probabilities are unusable for decision-making. Since Bayesian probability is in large part justified by decision-making, we're in trouble. And the AMD is not an isolated problem. In many imaginable situations faced by humans or idealized agents, there's a nonzero chance of returning to the same state of mind in the future, and that chance slightly depends on the current action. To the extent that's true, Bayesian probability is an imperfect approximation.

With logical uncertainty, the situation is even worse. We don't have a good theory of how logical uncertainty should work. (Though there have been several attempts, like Ben...
What makes us think _any_ of our terminal values aren't based on a misunderstanding of reality?
by bokov — 2013-09-25T23:09:17.215Z

Let's say Bob's terminal value is to travel back in time and ride a dinosaur.

It is instrumentally rational for Bob to study physics so he can learn how to build a time machine. As he learns more physics, Bob realizes that his terminal value is not only utterly impossible but meaningless. By definition, someone in Bob's past riding a dinosaur is not a future evolution of the present Bob.

There are a number of ways to create the subjective experience of having gone into the past and ridden a dinosaur. But to Bob, it's not the same because he wanted both the subjective experience and the knowledge that it corresponded to objective fact. Without the latter, he might as well have just watched a movie or played a video game.

So if we took the original, innocent-of-physics Bob and somehow calculated his coherent extrapolated volition, we would end up with a Bob who has given up on time travel. The original Bob would not want to be this Bob.

But, how do we know that _anything_ we value won't similarly dissolve under sufficiently thorough deconstruction? Let's suppose for a minute that all "human values" are dangling units; that everything we want is as possible and makes as much sense as wanting to hear the sound of blue or taste the flavor of a prime number. What is the rational course of action in such a situation?

PS: If your response resembles "keep attempting to XXX anyway", please explain what privileges XXX over any number of other alternatives other than your current preference. Are you using some kind of pre-commitment strategy to a subset of your current goals? Do you now wish you had used the same strategy to precommit to goals you had when you were a toddler?
Doing Important Research on Amazon's Mechanical Turk?
by Peter Wildeford — 2013-09-25T17:04:25.692Z

There seems to be many important questions that need research, from the mundane (say, which of four slogans for 80,000 Hours people like best) to the interesting (say, how to convince people to donate more than they otherwise would). Unfortunately, it's difficult to collect data in a quick, reliable, and affordable way. We generally lack access to easily survey-able populations and a lot of research has high barriers to entry for completing (such as needing to enroll in graduate school).

However, since the 2005 creation of Amazon's Mechanical Turk, some of this has changed. Mechanical Turk is a website where anyone can create tasks for people to complete at a certain wage. These tasks can be anything, from identifying pictures to transcribing interviews to social science research.

Best of all, this is quick and cheap -- for example, you could offer $0.25 to complete a short survey, put $75 in a pot, and get 300 responses within a day or two, and this should be quicker and cheaper than any other option available to you for collecting data.

But could Mechanical Turk actually be useable for answering important questions? Could running studies on Mechanical Turk be a competitive use of altruistic funds?

What Questions Would We Be Interested in Asking?

There are a variety of questions we might be interested in that would be appropriate to ask via Mechanical Turk. I don't believe you could make a longitudinal study, so testing the effects of vegetarian ads on diets in a useful way wouldn't be able to happen. But less conclusive diet studies could be run in this area.

Additionally, we could test to see how people respond to various marketing materials in EA space. We could explore how people think about charity and see what would make them more likely to donate and how changes in the marketing affect a willingness to donate. We could find out which arguments are more compelling. We could even test various memes against each other and see what people think of them.

Is Mechanical Turk a Reliable Source of Data?

It would only be good to use MTurk if the data you could get is useful. But is it?

Diversity of the Sample

The first question we might ask is whether MTurk produces a sample that is sufficiently diverse and representative of the United States. Unfortunately, this isn'...
Inferential silence
by Kaj_Sotala — 2013-09-25T12:45:34.692Z

Every now and then, I write an LW comment on some topic and feel that the contents of my comment pretty much settles the issue decisively. Instead, the comment seems to get ignored entirely - it either gets very few votes or none, nobody responds to it, and the discussion generally continues as if it had never been posted.

Similarly, every now and then I see somebody else make a post or comment that they clearly feel is decisive, but which doesn't seem very interesting to me. Either it seems to be saying something obvious, or I don't get its connection to the topic at hand in the first place.

This seems like it would be about inferential distance: either the writer doesn't know the things that make the reader experience the comment as uninteresting, or the reader doesn't know the things that make the writer experience the comment as interesting. So there's inferential silence - a sufficiently long inferential distance that a claim doesn't provoke even objections, just uncomprehending or indifferent silence.

But "explain your reasoning in more detail" doesn't seem like it would help with the issue. For one, we often don't know beforehand when people don't share our assumptions. Also, some of the comments or posts that seem to encounter this kind of a fate are already relatively long. For example, Wei Dai wondered why MIRI-affiliated people don't often respond to his posts that raise criticisms, and I essentially replied that I found the content of his post relatively obvious so didn't have much to say.

Perhaps people could more often explicitly comment if they notice that something that a poster seems to consider a big thing doesn't seem very interesting or meaningful to them, and briefly explain why? Even a sentence or two might be helpful for the original poster.
Prisoner's Dilemma vs the Afterlife
by DataPacRat — 2013-09-24T16:59:54.510Z

I've had a thought I don't recall having encountered before described quite this way; but, given my past experiences with such thoughts, and the fact that it involves evo-psych, I currently peg my confidence in this idea at around 10%. But just in case this particular idea rose to my attention out of all the other possible ideas that didn't for a reason, I'll post it here.

One of the simpler analyses of the Prisoner's Dilemma points out that if you know that the round you're facing is the last round, then there's no reason not to defect; your choice no longer has any influence over future rounds, and whatever your opponent does, you gain a higher score by defecting on this particular round than by cooperating. Thus, any rational algorithm which is attempting to maximize its score, and can identify which round is the last round, will gain a higher score by adding a codicil to defect on the last round.

Expanding that idea implies that if such a "rational" algorithm is facing other seemingly rational algorithms, it will assume that they will also defect on the last round; and thus, such an algorithm faced with the /second/-last round will be able to assume that its actions will have no influence on the actions of the last round; and, by a similar logic, will choose to defect on the second-last round; and the third-last; and so forth. In fact, if the whole game has a maximum length, then this chain of logic applies, leading to programs that are, in effect, always-defect. Cooperative strategies such as tit-for-tat thus tend to arise when the competing algorithms lack a particular piece of information: the length of the game they are playing.

Depending on where a person is born and lives (and various other details), they have roughly a fifty percent chance of living to 80 years of age, a one-in-a-million chance of making it to 100 years, and using LaPlace's Sunrise Formula, somewhere under one-in-a-hundred-billion odds of making it to 130 years. If a person assumes that their death is the end of them, then they have a very good idea of what their maximum lifespan will be; and depending on how rational they are, they could follow a similar line of reasoning to the above and plan their actions around an "always defect" style of morality. (Eg, stealing whenever the profit outweighs the risk times the punishment.)

However, introducing even an extremely vague c...
Meetup : Durham NC/Triangle Area: Cognitive Biases, Part 3
by therufs — 2013-09-24T13:40:45.253Z

Discussion article for the meetup : Durham NC/Triangle Area: Cognitive Biases, Part 3

WHEN:
26 September 2013 07:00:00PM (-0400)

WHERE:
420 West Geer St., Durham NC 27701

Moar bias identification and discussion of debiasing strategies!

7:00 coffee + talk

7:30 discussion

~9:30 adjourn to Fullsteam

Discussion article for the meetup : Durham NC/Triangle Area: Cognitive Biases, Part 3
Autism, Watson, the Turing test, and General Intelligence
by Stuart_Armstrong — 2013-09-24T11:00:47.736Z

Thinking aloud:

Humans are examples of general intelligence - the only example we're sure of. Some humans have various degrees of autism (low level versions are quite common in the circles I've moved in), impairing their social skills. Mild autists nevertheless remain general intelligences, capable of demonstrating strong cross domain optimisation. Psychology is full of other examples of mental pathologies that impair certain skills, but nevertheless leave their sufferers as full fledged general intelligences. This general intelligence is not enough, however, to solve their impairments.

Watson triumphed on Jeopardy. AI scientists in previous decades would have concluded that to do so, a general intelligence would have been needed. But that was not the case at all - Watson is blatantly not a general intelligence. Big data and clever algorithms were all that were needed. Computers are demonstrating more and more skills, besting humans in more and more domains - but still no sign of general intelligence. I've recently developed the suspicion that the Turing test (comparing AI with a standard human) could get passed by a narrow AI finely tuned to that task.

The general thread is that the link between narrow skills and general intelligence may not be as clear as we sometimes think. It may be that narrow skills are sufficiently diverse and unique that a mid-level general intelligence may not be able to develop them to a large extent. Or, put another way, an above-human social intelligence may not be able to control a robot body or do decent image recognition. A super-intelligence likely could: ultimately, general intelligence includes the specific skills. But his "ultimately" may take a long time to come.

So the questions I'm wondering about are:

How likely is it that a general intelligence, above human in some domain not related to AI development, will acquire high level skills in unrelated areas?

By building high-performance narrow AIs, are we making it much easier for such an intelligence to develop such skills, by co-opting or copying these programs?
[Link] - No evidence of intelligence improvement after working memory training
by pinyaka — 2013-09-23T21:06:58.518Z

This article critically examines previous studies that showed a link between working memory training (specifically via n-back training) and fluid intelligence, finding that the results may not have been as positive as reported owing to a number of factors including the use of a no-contact rather than active control group, and difficulty selecting tests that isolate the impact of working memory on fluid intelligence. The authors also present findings from a new study that show no improvement in fluid intelligence from dual n-back training, visual search training (active placebo) and no training (no contact placebo).

PubMed

Journal Challenged
Meetup : Moscow, Beliefs
by Yuu — 2013-09-23T18:14:26.743Z

Discussion article for the meetup : Moscow, Beliefs

WHEN:
29 September 2013 04:00:00PM (+0400)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

Please use the following guide to get to the meetup: link. You need the second revolving door near the swing or just look for group of geek-looking people. We will meet you at 16:00 inside.

Main topics:

Applied rationality exercise: beliefs finding, analysis, correction and adoption of new ones.

Game session.

Important notice: these exercises form a complete set. It means that if you miss first one or two exercises you can not complete others. So if you want to study new skills you should not be late.

If you are going for the first time, you can fill this one minute form (in Russian), to share your contact information. You can also use personal messages here, or drop a message at lw@lesswrong.ru to contact me for any reason.

Discussion article for the meetup : Moscow, Beliefs