trentmkelly/LessWrong-43k huggingface dataset:
Studying Psychology - Which path should I take to best help our cause? Suggestions please.
by Friendly-HI — 2011-11-23T19:52:10.551Z

If you solve the problem of human-friendly self-improving AI, you have indirectly solved every problem. After spending a decent amount of time on LW, I have been convinced of this premise and now I would like to devote my life to that cause.

Currently I'm living in Germany and I'm studying psychology in the first semester. The university I'm studying at has a great reputation (even internationally if I can believe the rankings) for the quality of its scientific psychology research and it ranks about #2nd or #3rd place when it comes to various psy-science-related criteria out of about 55 German universities where one can study psychology. Five semesters of statistics in my Bachelor of Science might also hint at that.

I want to finish my Bachelor of Science and then move on to my Master, so in about 5 years I might hit my "phase of actual productivity" in the working field. I'm flirting with cognitive neuroscience, but haven't made my decision yet - however, I am pretty sure that I want to move towards research and a scientific career rather than one in a therapeutic field.

Before discovering lesswrong my most dominant personal interest in psychology has been in the field of "positive psychology" or plainly speaking the "what makes humans happy" field. This interest hasn't really changed through the discovery of LW, as much as it has evolved into: "how can we distill what makes human life worthwhile and put it into terms a machine could execute for our benefit"?

As the title suggests, I'm writing all this because I want some creative input from you in order to expand my sense of possibilities concerning how I can help the development of friendly AI from the field of psychology most effectively.

To give you a better idea of what might fit me, a bit more background-info about myself and my abilities seems in order:

I like talking and writing a lot, mathematically I am a loser (whether due to early disgust or incompetence I can't really tell). I value and enjoy human contact and have constantly moved from being an introvert towards being an extrovert by several cognitive developments I can only speculate on. I would probably easily rank in the middle field of any positive extroversion scale nowadays. My IQ seems to be around 134 if one can trust the "International High IQ Society" (www.highiqsociety.org), but as...
[SEQ RERUN] Argument Screens Off Authority
by MinibearRex — 2011-11-23T04:06:53.362Z

Today's post, Argument Screens Off Authority was originally published on 14 December 2007. A summary (taken from the LW wiki):

There are many cases in which we should take the authority of experts into account, when we decide whether or not to believe their claims. But, if there are technical arguments that are available, these can screen off the authority of experts.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Reversed Stupidity Is Not Intelligence, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Value of Information: Four Examples
by Vaniver — 2011-11-22T23:02:43.098Z

Value of Information (VoI) is a concept from decision analysis: how much answering a question allows a decision-maker to improve its decision. Like opportunity cost, it's easy to define but often hard to internalize; and so instead of belaboring the definition let's look at some examples.

Gambling with Biased Coins

Normal coins are approximately fair.1 Suppose you and your friend want to gamble, and fair coins are boring, so he takes out a quarter and some gum and sticks the gum to the face of the quarter near the edge. He then offers to pay you $24 if the coin lands gum down, so long as you pay him $12 to play the game. Should you take that bet?

First, let's assume risk neutrality for the amount of money you're wagering. Your expected profit is $24p-12, where p is the probability the coin lands gum down. This is a good deal if p>.5, but a bad deal if p<.5. So... what's p? More importantly, how much should you pay to figure out p?

A Bayesian reasoner looking at this problem first tries to put a prior on p. An easy choice is a uniform distribution between 0 and 1, but there are a lot of reasons to be uncomfortable with that distribution. It might be that the gum will be more likely to be on the bottom- but it also might be more likely to be on the top. The gum might not skew the results very much- or it might skew them massively. You could choose a different prior, but you'd have trouble justifying it because you don't have any solid evidence to update on yet.2

If you had a uniform prior and no additional evidence, then the deal as offered is neutral. But before you choose to accept or reject, your friend offers you another deal- he'll flip the coin once and let you see the result before you choose to take the $12 deal, but you can't win anything on this first flip. How much should you pay to see one flip?

Start by modeling yourself after you see one flip. It'll either come up gum or no gum, and you'll update and produce a posterior for each case. In the first case, your posterior on p is P(p)=2p; in the second, P(p)=2-2p. Your expected profit for playing in the first case is $4;3 your expected profit for playing in the second case is negative $4. You think there's a half chance it'll land gum side up, and a half chance it'll land gum side down, and if it lands gum side down you can choose not to play. There's a half chance you get $4 from seein...
Intelligence Explosion analysis draft: Why designing digital intelligence gets easier over time
by lukeprog — 2011-11-22T22:28:03.868Z

Again, I invite your feedback on this snippet from an intelligence explosion analysis Anna Salamon and myself have been working on. This section is less complete than the others; missing text is indicated with brackets: [].

_____

Many predictions of human-level digital intelligence have been wrong.1 On the other hand, machines surpass human ability at new tasks with some regularity (Kurzweil 2005). For example, machines recently achieved superiority at visually identifying traffic signs at low resolution (Sermanet and LeCun 2011), diagnosing cardiovascular problems from some types of MRI scan images (Li et al. 2009), and playing Jeopardy! (Markoff 2011). Below, we consider several factors that, considered together, appear to increase the odds that we will develop digital intelligence as the century progresses.

More hardware. For at least four decades, computing power2 has increased exponentially, in accordance with Moore’s law.3 Experts disagree on how much longer Moore’s law will hold (e.g. Mack 2011; Lundstrom 2003), but if it holds for two more decades then we may have enough computing power to emulate human brains by 2029.4 Even if Moore’s law fails to hold, our hardware should become much more powerful in the coming decades.5 More hardware doesn’t by itself give us digital intelligence, but it contributes to the development of digital intelligence in several ways:

Powerful hardware may improve performance simply by allowing existing “brute force” solutions to run faster (Moravec, 1976). Where such solutions do not yet exist, researchers might be incentivized to quickly develop them given abundant hardware to exploit. Cheap computing may enable much more extensive experimentation in algorithm design, tweaking parameters or using methods such as genetic algorithms. Indirectly, computing may enable the production and processing of enormous datasets to improve AI performance (Halevi et al., 2009), or result in an expansion of the information technology industry and the quantity of researchers in the field.6

Massive datasets. The greatest leaps forward in speech recognition and translation software have come not from faster hardware or smarter hand-coded algorithms, but from access to massive data sets of human-transcribed and human-translated words (Halevy, Norvig, and Pereira 2009). [add sentence about h...
FAI FAQ draft: What is nanotechnology?
by lukeprog — 2011-11-22T22:13:23.977Z

I invite your feedback on this snippet from the forthcoming Friendly AI FAQ. This one is an answer to the question "What is nanotechnology?" For references, see here.

_____

Nanotechnology is the study of materials and devices built at the scale of 1-100 nanometers (“nano-” means “one billionth of”). A hydrogen atom is about 0.24nm across, so we’re talking about materials and devices built atom by atom.

One famous piece of nanotechnology is the carbon nanotube. A carbon nanotube is a one-atom-thick sheet of graphite that is rolled into a seamless tube. Because of their physical properties, carbon nanotubes usually allow ballistic conduction, meaning that electrons can flow through the tube without collisions (Lin & Shung 1995), which allows the carbon nanotubes to conduct electricity without heat dissipation (Chico et al. 1996)! Carbon nanotubes are also much stronger than diamond or steel (Popov et al. 2002). Easton Bell Sports uses carbon nanotubes to build tougher bicycles, doctors use carbon nanotubes as scaffolding for bone growth in tissue engineering applications (Zanello et al. 2006), and one company uses carbon nanotubes to produce a special kind of high-conductance heater.

New nanomaterials are being developed every year, and may see applications in nearly every field of technology (Allhoff et al. 2010). Nanotechnology has already given us stain-free pants, larger-capacity hard drives, stronger cement, longer-lasting tennis balls, the world’s first sale of a quantum computer, a new method for fighting cancer, and much more.

An even more radical technology was described in Eric Drexler’s (1987) Engines of Creation. As Allhoff et al. (2010, p. 7) explain, Drexler predicted

a new form of technology based on molecular “assemblers,” which would be able to “place atoms in almost any reasonable arrangement” and thereby allow the formation of “almost anything the laws of nature allow.” This may sound like a fanciful and fantastical idea but, as Drexler points out, this is something that nature already does, unaided by human design, with the biologically based machines inside our own bodies (and those of any biological species).

Tiny molecular machines called “nanobots” would be a particularly revolutionary invention. For example in nanomedicine they would allo...
New website on careers for optimal philanthropy
by lukeprog — 2011-11-22T20:06:10.340Z

80,000 hours (eightythousand.org) is a new website associated with High Impact Careers, a Giving What We Can-associated effort to inform the public about "professional philanthropy" and the fact that you can do more good as a banker or entrepreneur than as an aid worker. It recently got some BBC press, and there's a neat new video.

Related to efficient charity and optimal philanthropy. Also see scope insensitivity.
[Link] The Decision Education Foundation
by curiousepic — 2011-11-22T19:51:17.068Z

Among Michael Anissimov's recent Delicious links, I spotted The Decision Education Foundation. It seems to be relevant to our interest in spreading rationality and bringing up young rationalists, but I hadn't seen it shared or discussed before and just thought I'd share it.

There is an urgent need for young people to understand how to make better decisions. High school dropout numbers, substance abuse, and violence are some of the visible signs of poor decision making among many young people in the U.S. In addition, mainstream students are failing to reach their full academic and personal potential because they lack good decision-making skills. They're facing an increasingly complex, risky, and uncertain future in which these skills will be more important than ever.

Yet decision skills training is missing in the majority of today's school curricula.

To answer this urgent and critical need and to provide this missing link in schools and youth programs, the Decision Education Foundation (DEF) was created in 2001 by educators, decision scientists, and management consultants with a passion for education and a commitment to bring decision education to today's youth.
Since then, DEF’s founders, staff, and volunteers have been designing and delivering training opportunities and teaching materials for educators and youth mentors who desire to increase self-esteem, build better critical thinking skills, and give students more power over their academic and personal lives through better decision skills.
Criticisms of intelligence explosion
by lukeprog — 2011-11-22T17:42:06.036Z

On this page I will collect criticisms of (1) the claim that intelligence explosion is plausible, (2) the claim that intelligence explosion is likely to occur within the next 150 years, and (3) the claim that intelligence explosion would have a massive impact on civilization. Please suggest your own, citing the original source when possible.

[Under construction.]

"AGI won't be a big deal; we already have 6 billion general intelligences on Earth."

Example: "I see no reason to single out AI as a mould-breaking technology: we already have billions of humans." (Deutsch, The Beginning of Infinity, p. 456.)

Response: The advantages of mere digitality (speed, copyability, goal coordination) alone are transformative, and will increase the odds of rapid recursive self-improvement in intelligence. Meat brains are badly constrained in ways that non-meat brains need not be.

"Intelligence requires experience and learning, so there is a limit to the speed at which even a machine can improve its own intelligence."

Example: "If you define the singularity as a point in time when intelligent machines are designing intelligent machines in such a way that machines get extremely intelligent in a short period of time--an exponential increase in intelligence--then it will never happen. Intelligence is largely defined by experience and training, not just by brain size or algorithms. It isn't a matter of writing software. Intelligent machines, like humans, will need to be trained in particular domains of expertise. This takes time and deliberate attention to the kind of knowledge you want the machine to have." (Hawkins, Tech Luminaries Address Singularity)

Response: Intelligence defined as optimization power doesn't necessarily need experience or learning from the external world. Even if it did, a superintelligent machine spread throughout the internet could gain experience and learning from billions of sub-agents all around the world simultaneously, while near-instantaneously propagating these updates to its other sub-agents.

"There are hard limits to how intelligent a machine can get."

Example: "The term 'singularity' applied to intelligent machines refers to the idea that when intelligent machines can design intelligent machines smarter than themselves, it will cause an exponential growth in machine intelligence leading to a singularity of infinite (or at leas...
The ethics of randomized computation in the multiverse
by lukeprog — 2011-11-22T16:31:31.508Z

From David Deutsch's The Beginning of Infinity:

Take a powerful computer and set each bit randomly to 0 or 1 using a quantum randomizer. (That means that 0 and 1 occur in histories of equal measure.) At that point all possible contents of the computer’s memory exist in the multiverse. So there are necessarily histories present in which the computer contains an AI program – indeed, all possible AI programs in all possible states, up to the size that the computer’s memory can hold. Some of them are fairly accurate representations of you, living in a virtual-reality environment crudely resembling your actual environment. (Present-day computers do not have enough memory to simulate a realistic environment accurately, but, as I said in Chapter 7, I am sure that they have more than enough to simulate a person.) There are also people in every possible state of suffering. So my question is: is it wrong to switch the computer on, setting it executing all those programs simultaneously in different histories? Is it, in fact, the worst crime ever committed? Or is it merely inadvisable, because the combined measure of all the histories containing suffering is very tiny? Or is it innocent and trivial?

I'm not so sure we have the computing power to "simulate a person," but suppose we did. (Perhaps we will soon.) How would you respond to this worry?
5-second level case study: Value of information
by Kaj_Sotala — 2011-11-22T13:44:37.817Z

This post started off as a comment to Vaniver's post Value of Information: Four Examples. This post also heavily builds on Eliezer's post The 5-Second Level. The five second level is the idea that to develop a rationality skill, you need to automatically recognize a problem and then apply a stored, actionable procedural skill to deal with it, all in about five seconds or so. In here, I take the value of information concept and develop it into a five second skill, summarizing my thought process as I do so. Hopefully this will help others develop things into five second skills.

So upon reading this, I thought "the value of information seems like a valuable concept", but didn't do much more. A little later, I thought, "I want to make sure that I actually apply this concept when it is warranted. How do I make sure of that?" In other words, "how do I get this concept to the five second level?" Then I decided to document my thought process in the hopes of it being useful to others. This is quite stream-of-consciousness, but I hope that seeing my thought process helps to learn from it. (Or to offer me valuable criticism on how I should have thought.)

First off, "how do I apply this concept?" is too vague to be useful. A better question would be, "in what kinds of situations might this concept be useful?". With a bit of thought, it was easy to find at least three situations, ones where I am:

1. ...tempted to act now without gathering more information, despite the VoI being high.
2. ...tempted to gather more information, despite the VoI being low.
3. ...not sure of whether I should seek information or not.

#3 implies that I'm already reflecting on the situation, and am therefore relatively likely to remember VoI as a possible mental tool anyway. So developing a five-second level reaction for that one isn't as important. But in #1 and #2 I might just proceed by default, never realizing that I could do better. So I'll leave #3 aside, concentrating on #1 and #2.

Now in these situations, the relevant thing is that the VoI might be "high" or "low". Time to get more concrete - what does that mean? Looking at Vaniver's post, the VoI is high if 1) extra information is likely to make me choose B when I had intended on choosing A, and 2) there's a high payoff in choosing correctly between A and B. If at least 2 is false, VoI is low. The intermediate case is the one where 2 is true bu...
How to prove anything with a review article
by Richard_Kennaway — 2011-11-22T12:04:33.041Z

Thus the subtitle of this blog posting at PLoS, referencing this article on "Cigarette smoking: an underused tool in high-performance endurance training". The point being that you can write a review article to argue anything you want, with sufficient cherry-picking and chains of links.

If you are doing actual experiments and making observations or proving theorems, then to a large extent -- larger in some sciences than in others -- you are constrained by the brute facts. But when writing secondary literature, especially in areas where data is generally fuzzier, it is easy, whether deliberately or not, to write to a bottom line, including findings you like and excluding those you don't.

Something to bear in mind when reading or writing any review article.
Objections to Coherent Extrapolated Volition
by XiXiDu — 2011-11-22T10:32:13.175Z

In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

— Eliezer Yudkowsky, May 2004, Coherent Extrapolated Volition

Foragers versus industry era folks

Consider the difference between a hunter-gatherer, who cares about his hunting success and to become the new tribal chief, and a modern computer scientist who wants to determine if a “sufficiently large randomized Conway board could turn out to converge to a barren ‘all off’ state.”

The utility of the success in hunting down animals and proving abstract conjectures about cellular automata is largely determined by factors such as your education, culture and environmental circumstances. The same forager who cared to kill a lot of animals, to get the best ladies in its clan, might have under different circumstances turned out to be a vegetarian mathematician solely caring about his understanding of the nature of reality. Both sets of values are to some extent mutually exclusive or at least disjoint. Yet both sets of values are what the person wants, given the circumstances. Change the circumstances dramatically and you change the persons values.

What do you really want?

You might conclude that what the hunter-gatherer really wants is to solve abstract mathematical problems, he just doesn’t know it. But there is no set of values that a person “really” wants. Humans are largely defined by the circumstances they reside in.

If you already knew a movie, you wouldn’t watch it.

To be able to get your meat from the supermarket changes the value of hunting.

If “we knew more, thought faster, were more the people we wished we were, and had grown up closer together” then we would stop to desire what we learnt, wish to think even faster, become even different people and get bored of and rise up from the people similar to us.

A singleton is an attractor

A singleton will inevitably change everything by causing a feedback loop between itself as an attractor and humans and their values.

Much of our values and goals, what we want, are culturally induc...