trentmkelly/LessWrong-43k huggingface dataset:
The Urgent Meta-Ethics of Friendly Artificial Intelligence
by lukeprog — 2011-02-01T14:15:33.740Z

Barring a major collapse of human civilization (due to nuclear war, asteroid impact, etc.), many experts expect the intelligence explosion Singularity to occur within 50-200 years.

That fact means that many philosophical problems, about which philosophers have argued for millennia, are suddenly very urgent.

Those concerned with the fate of the galaxy must say to the philosophers: "Too slow! Stop screwing around with transcendental ethics and qualitative epistemologies! Start thinking with the precision of an AI researcher and solve these problems!"

If a near-future AI will determine the fate of the galaxy, we need to figure out what values we ought to give it. Should it ensure animal welfare? Is growing the human population a good thing?

But those are questions of applied ethics. More fundamental are the questions about which normative ethics to give the AI: How would the AI decide if animal welfare or large human populations were good? What rulebook should it use to answer novel moral questions that arise in the future?

But even more fundamental are the questions of meta-ethics. What do moral terms mean? Do moral facts exist? What justifies one normative rulebook over the other?

The answers to these meta-ethical questions will determine the answers to the questions of normative ethics, which, if we are successful in planning the intelligence explosion, will determine the fate of the galaxy.

Eliezer Yudkowsky has put forward one meta-ethical theory, which informs his plan for Friendly AI: Coherent Extrapolated Volition. But what if that meta-ethical theory is wrong? The galaxy is at stake.

Princeton philosopher Richard Chappell worries about how Eliezer's meta-ethical theory depends on rigid designation, which in this context may amount to something like a semantic "trick." Previously and independently, an Oxford philosopher expressed the same worry to me in private.

Eliezer's theory also employs something like the method of reflective equilibrium, about which there are many grave concerns from Eliezer's fellow naturalists, including Richard Brandt, Richard Hare, Robert Cummins, Stephen Stich, and others.

My point is not to beat up on Eliezer's meta-ethical views. I don't even know if they're wrong. Eliezer is wickedly smart. He is highly trained in the skills of overcoming biases and properly proportioning beliefs to the evidence. H...
Approaching Infinity
by Psychohistorian — 2011-02-01T08:11:53.822Z

[This is from a very neat example my real analysis professor used some years ago. While I'm fairly confident it's neat, I'm not certain it's top-level-post-worthy. The general point is about problems with applying concepts involving infinity to reality; any advice on content (or formatting!) would be greatly welcomed. My math education basically ended after a few upper division courses, so it's possible there are some notational schemes or methods I am ignorant of.

I think this is a fun little exercise, if nothing more.]

The concept of "infinity" and "infinite series" and sets get thrown around a lot in mathematics and some of philosophy. It's worth trying to put the concept of infinity in perspective before we try to think of things in the real world being "infinite." Warning: this post will involve numbers that are literally too large to comprehend. But that's the point.

Let us define an operator, /X\ ("triangle-X"). /X\ = X raised to the X power X times. Thus, /2\ = 22^2 = 24 = 16.

//2\\ ("2-triangle-2") would do this operation twice. Thus, it would equal /16\, the value of which we'll get to in a minute.

We now introduce a new operator, [X] ("square-X"). [X] = triangle-X-triangle-X, i.e. X inside of /X\ triangles. [2] = ////////////////2\\\\\\\\\\\\\\\\
We can introduce another operator, [X> ("pentagon-X"). [X> = X inside of [X] squares. I believe this would be "square-X-square-X").
...

[Edited for clarity]
I'll spare the next [X] operators, and go right to (X) ("circle-X"). Technically, it's whole-lot-of-sides-polygon-X - we could continue this process indefinitely - but we'll call it circle-X, because that's as far as we're going. (X) follows the process that took us from triangle to square to pentagon, iterated an additional [X] times.

I'll be honest. This got kind of meaningless a bit before [X]. Let's start trying to construct what [2] equals, and you'll see why.

/2\=16. So //2\\ = /16\ = 1616^16^16^16^16^16^16^16^16^16^16^16^16^16^16^16. Using some very rough approximations, we can say this is about 102x10^19, or one followed by twenty billion billion zeroes. //16\\ is thus one followed by twenty billion billion zeroes, raised to the power of one followed by twenty billion billion zeroes one followed by twenty billion billion zeroes times. My math education could be more complete, but I am not aware of another way to denote such a number....
Starting a LW meet-up is easy.
by AnnaSalamon — 2011-02-01T04:05:43.179Z

All you need to do is:

Pick a time. Weekend afternoons or evenings work well.

Pick a place. This can be a coffee shop or casual restaurant (e.g., a pizza place or pub) or a classroom or other on-campus location. Best if it isn’t too noisy.

Announce the time and place on LW, a week or so ahead of time, using the "Add new meetup" link near your username.

Show up yourself, with a sign that says “Less Wrong Meet-up”.

That’s all -- anything else is optional. If folks come, just say “Hi, my name’s [whatever your name is]”, and see where the conversation goes. Most major cities, and many minor ones, have some LW-ers. And if no one comes, all it cost you was a few hours of reading a book in a restaurant. You don’t need to have a LW history; many a lurker has enjoyed in-person LW conversation (and the folks who actually show up to meet-ups are often less intimidating than those who post on the main site).

Meet-ups are fun, and the simple act of talking to other LW-ers (in person, where your primate brain can see that they’re real) can help: (a) you become a better rationalist; (b) other attendees become better rationalists; and (c) LW become a stronger community.

Also, if anyone is interested in starting a meet-up but wants to discuss it with someone first, I'd be happy to help. There is also a good meet-up resources page.

(This was discussed a bit in this comment thread, but it seems worth repeating it somewhere where more people might see it, especially since the idea was new to someone at Saturday's H+ conference who now plans to start a meet-up.)
Counterfactual Calculation and Observational Knowledge
by Vladimir_Nesov — 2011-01-31T16:28:15.334Z

Consider the following thought experiment ("Counterfactual Calculation"):

You are taking a test, which includes a question: "Is Q an even number?", where Q is a complicated formula that resolves to some natural number. There is no a priori reason for you to expect that Q is more likely even or odd, and the formula is too complicated to compute the number (or its parity) on your own. Fortunately, you have an old calculator, which you can use to type in the formula and observe the parity of the result on display. This calculator is not very reliable, and is only correct 99% of the time, furthermore its errors are stochastic (or even involve quantum randomness), so for any given problem statement, it's probably correct but has a chance of making an error. You type in the formula and observe the result (it's "even"). You're now 99% sure that the answer is "even", so naturally you write that down on the test sheet.

Then, unsurprisingly, Omega (a trustworthy all-powerful device) appears and presents you with the following decision. Consider the counterfactual where the calculator displayed "odd" instead of "even", after you've just typed in the (same) formula Q, on the same occasion (i.e. all possible worlds that fit this description). The counterfactual diverges only in the calculator showing a different result (and what follows). You are to determine what is to be written (by Omega, at your command) as the final answer to the same question on the test sheet in that counterfactual (the actions of your counterfactual self who takes the test in the counterfactual are ignored).

Should you write "even" on the counterfactual test sheet, given that you're 99% sure that the answer is "even"?

This thought experiment contrasts "logical knowledge" (the usual kind) and "observational knowledge" (what you get when you look at a calculator display). The kind of knowledge you obtain by observing things is not like the kind of knowledge you obtain by thinking yourself. What is the difference (if there actually is a difference)? Why does observational knowledge work in your own possible worlds, but not in counterfactuals? How much of logical knowledge is like observational knowledge, and what are the conditions of its applicability? Can things that we consider "logical knowledge" fail to apply to some counterfactuals?

(Updateless analysis would say "observational knowledge is not...
Optimal Employment
by Louie — 2011-01-31T12:50:17.783Z

Related to: Best career models for doing research?, (Virtual) Employment Open Thread

In the spirit of offering some practical real world advice, let's talk about employment rationality. Let’s talk about optimal employment.1

You're young, smart, and hoping to have a positive impact on the world. Maybe you finished college, maybe you didn't. You want to pay your bills but also have time to pursue your intellectual goals. You want a low-stress job that doesn't leave you drained at the end of the day. And it would be nice to earn lots of extra money, because whatever you value, money tends to be a good way to get it.

And it is possible to find easily obtained, low-stress jobs with flexible hours that allow you to save as much money as someone in the USA making $100,000/yr... if you leave the USA to look for them.

Your instinctive reaction is probably that there’s no free lunch, so I must be mistaken or dishonest. And while you may have the right prior, I hope to persuade you that these jobs exist and tell you how to get one if you're interested.

This, I think, is a special opportunity for rationalists, an illustration that we can get better life outcomes from our investment in rationality - better outcomes such as low-stress jobs that leave us with ample discretionary income and enough free time to pursue whatever else we're interested in, obtained by being willing to break habits and think in numbers.

Employment Biases

First, consider some cognitive biases that may be leading people away from optimal employment.

Status Quo Bias - As a rule, we work the same jobs we worked the day before regardless of whether it's still the best option for us. Almost everyone you know is doing the same thing. So you shouldn't expect to be able to just copy others' behavior and end up with optimal employment. Most people stop searching too soon.

Money Illusion - We reason in terms of nominal salaries rather than in actual buying power. This causes us to chase high nominal salaries ($100,000/yr!) even when those salaries are coupled with an exceptionally high cost of living that decreases our overall buying power.

Ostrich Effect - Regardless of income, the average American ends up paying close to 40% in taxes yet consistently self-reports as paying only 3%. In other words, we tend to ignore or deny obviously negative situations when we feel we can't change them.

...
Reply to Stuart on anthropics
by cousin_it — 2011-01-31T07:06:39.673Z

You wake up in a hospital bed, remembering nothing of your past life. A stranger sits beside the bed, smiling. He says:

"I happen to know an amusing story about you. Many years ago, before you were born, your parents were arguing about how many kids to have. They settled on flipping a coin. If the coin came up heads, they would have one child. If it came up tails, they would have ten."

"I will tell you which way the coin came up in a minute. But first let's play a little game. Would you like a small piece of chocolate, or a big tasty cake? There's a catch though: if you choose the cake, you will only receive it if you're the only child of your parents."

Stuart Armstrong has proposed a solution to this problem (see the fourth model in his post). Namely, you switch to caring about the average that all kids receive in your branch. This doesn't change the utility all kids get in all possible worlds, but makes the problem amenable to UDT, which says all agents would have precommitted to choosing cake as long as it's better than two pieces of chocolate (the first model in Stuart's post).

But.

Creating two physically separate worlds with probability 50% should be decision-theoretically equivalent to creating them both with probability 100%. In other words, a correct solution should still work if the coin is quantum. In other words, the problem should be equivalent to creating 11 kids, offering them chocolate or cake, and giving cake only if you're the first kid. But would you really choose cake in this case, knowing that you could get the chocolate for certain? What if there were 1001 kids? This is a hard bullet to swallow, and it seems to suggest that Stuart's analysis of his first model may be incorrect.

I await comments from Stuart or anyone else who can figure this out.
Berkeley LW Meet-Up Saturday February 5
by LucasSloan — 2011-01-31T06:06:36.097Z

EDIT: We are not at the VLSB, we are at the Free Speech Cafe.

After this month's hiatus in Berkeley Meetups, I'm quite sure that people are itching to get together with their fellow LWers. As usual, we'll meet at the Starbucks at 2128 Oxford Street at 7 pm, then move into the atrium in the Valley Life Sciences Building on the Berkeley Campus. There'll be an expedition to get food, so don't feel that you have to eat before hand.

I want to make a special plea to the lurkers on lesswrong to come, I promise no one will think poorly of you if you're still in High School, aren't a member of Mensa, or haven't read the sequences. This is a social event where we get together with people who like rationality, not a way for the shadowy cabal of beisutsukai to puzzle out the secret of quantum gravity. Even if you only just linked here from Methods of Rationality, I and everyone else here wants to get to know you.

Hope to see you all there!
Avoid interruptions by time-shifting them
by Dr_Manhattan — 2011-01-30T19:14:21.439Z

A lot of times you may come upon a juicy-smelling "information scent". Now you have a decision of whether to extend your lunch break and cut into work by following the scent (bad) or abandoning it (painful, and possibly bad).

One thing that has worked for me is using either delicious or more recently instapaper app to save the link (instapaper will actually sync the entire article back your iphone/pad/android). Pang of pain is largely avoided, and I can decide later whether this bit of info was worth reading, by comparing it to my other saved articles in the app at the appropriate time.
Looking for information on scoring calibration
by Scott Alexander — 2011-01-29T22:24:02.768Z

There are lots of scoring rules for probability assessments. Log scoring is popular here, and squared error also works.

But if I understand these correctly, they are combined measurements of both domain-ability and calibration. For example, if several people took a test on which they had to estimate their confidence in their answers to certain true or false questions about history, then well-calibrated people would have a low squared error, but so would people who know a lot about history.

So (I think) someone who always said 70% confidence and got 70% of the questions right would get a higher score than someone who always said 60% confidence and got 60% of the questions right, even though they are both equally well calibrated.

The only pure calibration estimates I've ever seen are calibration curves in the form of a set of ordered pairs, or those limited to a specific point on the cuve (eg "if ey says ey's 90% sure, ey's only right 60% of the time"). There should be a way to take the area under (or over) the curve to get a single value representing total calibration, but I'm not familiar with the method or whether it's been done before. Is there an accepted way to get single-number calibration scores separate from domain knowledge?
What is Eliezer Yudkowsky's meta-ethical theory?
by lukeprog — 2011-01-29T19:58:18.670Z

In You Provably Can't Trust Yourself, Eliezer tried to figured out why his audience didn't understand his meta-ethics sequence even after they had followed him through philosophy of language and quantum physics. Meta-ethics is my specialty, and I can't figure out what Eliezer's meta-ethical position is. And at least at this point, professionals like Robin Hanson and Toby Ord couldn't figure it out, either.

Part of the problem is that because Eliezer has gotten little value from professional philosophy, he writes about morality in a highly idiosyncratic way, using terms that would require reading hundreds of posts to understand. I might understand Eliezer's meta-ethics better if he would just cough up his positions on standard meta-ethical debates like cognitivism, motivation, the sources of normativity, moral epistemology, and so on. Nick Beckstead recently told me he thinks Eliezer's meta-ethical views are similar to those of Michael Smith, but I'm not seeing it.

If you think you can help me (and others) understand Eliezer's meta-ethical theory, please leave a comment!

Update: This comment by Richard Chappell made sense of Eliezer's meta-ethics for me.
David Chalmers' "The Singularity: A Philosophical Analysis"
by lukeprog — 2011-01-29T02:52:41.480Z

David Chalmers is a leading philosopher of mind, and the first to publish a major philosophy journal article on the singularity:

Chalmers, D. (2010). "The Singularity: A Philosophical Analysis." Journal of Consciousness Studies 17:7-65.

Chalmers' article is a "survey" article in that it doesn't cover any arguments in depth, but quickly surveys a large number of positions and arguments in order to give the reader a "lay of the land." (Compare to Philosophy Compass, an entire journal of philosophy survey articles.) Because of this, Chalmers' paper is a remarkably broad and clear introduction to the singularity.

Singularitarian authors will also be pleased that they can now cite a peer-reviewed article by a leading philosopher of mind who takes the singularity seriously.

Below is a CliffsNotes of the paper for those who don't have time to read all 58 pages of it.

The Singularity: Is It Likely?

Chalmers focuses on the "intelligence explosion" kind of singularity, and his first project is to formalize and defend I.J. Good's 1965 argument. Defining AI as being "of human level intelligence," AI+ as AI "of greater than human level" and AI++ as "AI of far greater than human level" (superintelligence), Chalmers updates Good's argument to the following:

There will be AI (before long, absent defeaters).

If there is AI, there will be AI+ (soon after, absent defeaters).

If there is AI+, there will be AI++ (soon after, absent defeaters).

Therefore, there will be AI++ (before too long, absent defeaters).

By "defeaters," Chalmers means global catastrophes like nuclear war or a major asteroid impact. One way to satisfy premise (1) is to achieve AI through brain emulation (Sandberg & Bostrom, 2008). Against this suggestion, Lucas (1961), Dreyfus (1972), and Penrose (1994) argue that human cognition is not the sort of thing that could be emulated. Chalmers (1995; 1996, chapter 9) has responded to these criticisms at length. Briefly, Chalmers notes that even if the brain is not a rule-following algorithmic symbol system, we can still emulate it if it is mechanical. (Some say the brain is not mechanical, but Chalmers dismisses this as being discordant with the evidence.)
Searle (1980) and Block (1981) argue instead that even if we can emulate the human brain, it doesn't follow that the emulation is intelligent or has a mind. Chalmers says we can set the...
AIXI-style IQ tests
by gwern — 2011-01-29T00:49:55.051Z

"Measuring universal intelligence: Towards an anytime intelligence test"; abstract:

In this paper, we develop the idea of a universal anytime intelligence test. The meaning of the terms “universal” and “anytime” is manifold here: the test should be able to measure the intelligence of any biological or artificial system that exists at this time or in the future. It should also be able to evaluate both inept and brilliant systems (any intelligence level) as well as very slow to very fast systems (any time scale). Also, the test may be interrupted at any time, producing an approximation to the intelligence score, in such a way that the more time is left for the test, the better the assessment will be. In order to do this, our test proposal is based on previous works on the measurement of machine intelligence based on Kolmogorov complexity and universal distributions, which were developed in the late 1990s (C-tests and compression-enhanced Turing tests). It is also based on the more recent idea of measuring intelligence through dynamic/interactive tests held against a universal distribution of environments. We discuss some of these tests and highlight their limitations since we want to construct a test that is both general and practical. Consequently, we introduce many new ideas that develop early “compression tests” and the more recent definition of “universal intelligence” in order to design new “universal intelligence tests”, where a feasible implementation has been a design requirement. One of these tests is the “anytime intelligence test”, which adapts to the examinee's level of intelligence in order to obtain an intelligence score within a limited time.

http://www.csse.monash.edu.au/~dld/Publications/HernandezOrallo+DoweArtificialIntelligenceJArticle.pdf

Example popular media coverage: http://www.sciencedaily.com/releases/2011/01/110127131122.htm

The group's homepage: http://users.dsic.upv.es/proy/anynt/

(There's an applet but it seems to be about constructing a simple agent and stepping through various environments, and no working IQ test.)

The basic idea, if you already know your AIXI*, is to start with simple programs** and then test the subject on increasingly harder ones. To save time, boring games such as random environments or one where the agent can 'die'*** are excluded an...