trentmkelly/LessWrong-43k huggingface dataset:
Minds that make optimal use of small amounts of sensory data
by SforSingularity — 2009-08-15T14:11:34.964Z

In That alien message, Eliezer made some pretty wild claims:

My moral - that even Einstein did not come within a million light-years of making efficient use of sensory data.

Riemann invented his geometries before Einstein had a use for them; the physics of our universe is not that complicated in an absolute sense. A Bayesian superintelligence, hooked up to a webcam, would invent General Relativity as a hypothesis - perhaps not the dominant hypothesis, compared to Newtonian mechanics, but still a hypothesis under direct consideration - by the time it had seen the third frame of a falling apple. It might guess it from the first frame, if it saw the statics of a bent blade of grass.

They never suspected a thing. They weren't very smart, you see, even before taking into account their slower rate of time. Their primitive equivalents of rationalists went around saying things like, "There's a bound to how much information you can extract from sensory data." And they never quite realized what it meant, that we were smarter than them, and thought faster.

In the comments, Will Pearson asked for "some form of proof of concept". It seems that researchers at Cornell - Schmidt and Lipson - have done exactly that. See their video on Guardian Science:

'Eureka machine' can discover laws of nature - The machine formulates laws by observing the world and detecting patterns in the vast quantities of data it has collected

Researchers at Cambridge and Aberystwith have gone one step further and implemented an AI system/robot to perform scientific experiments:

Researchers at Aberystwyth University in Wales and England's University of Cambridge report in Science today that they designed Adam - they describe how the bot operates by relating how he carried out one of his tasks, in this case to find out more about the genetic makeup of baker's yeast Saccharomyces cerevisiae, an organism that scientists use to model more complex life systems. Using artificial intelligence, Adam hypothesized that certain genes in baker's yeast code for specific enzymes that catalyze biochemical reactions. The robot devised experiments to test these beliefs, ran the experiments, and interpreted the results.

The crucial question is: what can we learn about the likely effectiveness of a "superintelligent" AI from the behavior of these AI programs? First of all, let us...
Fighting Akrasia:  Survey Design Help Request
by Gordon Seidoh Worley — 2009-08-14T19:48:29.420Z

Follow-up to: Fighting Akrasia: Finding the Source

In the last post in this series I posted a link to a Google Docs survey to try to gather some data on what techniques, if any, work for people in conquering akrasia, but we haven't gotten very much information so far: the response pool is fairly homogeneous in terms of age, sex, and personality type. In part this is because we need to get more responses outside of the LW readership, but probably also because I'm not asking the right questions. So, my challenge this weekend is to come up with some good revisions for the survey.

In order to maximize comment usefulness, please suggest one revision per top level comment and then any discussion of that revision can take place in the replies.

In the interest of keeping the comments on topic, I request a moratorium on discussions of whether or not akrasia exists and whether or not we can or should do something about it in the comments on this article. It's not that I want to exclude or silence opinions contrary to what I'm trying to accomplish: it's just that I would like to keep this article on the topic of revising the akrasia fighting survey. By all means, if my posting about akrasia really bothers you, write up an article explaining why I'm wrong and we'll discuss the issue more there.

Thanks!
Mistakes with nonexistent people
by KatjaGrace — 2009-08-13T22:28:35.000Z

Who is better off if you live and I die? Is one morally obliged to go around impregnating women? Is the repugnant conclusion repugnant? Is secret genocide OK? Does it matter if humanity goes extinct? Why shouldn’t we kill people? Is pity for the dead warranted?

All these discussions come down to the same question often: whether to care about the interests of people who don’t exist but could.

I shan’t directly argue either way; care about whatever you like. I want to show that most of the arguments against caring about the non-existent which repeatedly come up in casual discussion rely on two errors.

Here are common arguments (paraphrased from real discussions):

There are infinitely many potential people, so caring about them is utterly impractical.

The utility that a non-existent person experiences is undefined, not zero. You are calculating some amount of utility and attributing it to zero people. This means utility per person is x/0 = undefined.

Causing a person to not exist is a victimless crime. Stop pretending these people are real just because you imagine them!

If someone doesn’t exist, they don’t have preferences, so you can’t fulfil them. This includes not caring if they exist or not. The dead do not suffer, only their friends and relatives do that.

Life alone isn’t worth anything – what matters is what happens in it, so creating a new life is a neutral act.

You can’t be dead. It’s not something you can be. So you can’t say whether life is better.

Potential happiness is immeasurable; the person could have been happy, they could have been sad. Their life doesn’t exist, so it doesn’t have characteristics.

How can you calculate loss of future life? Maybe they’d live another hundred years, if you’re going to imagine they don’t die now.

All of these arguments spring from two misunderstandings:

Thinking of value as being a property of particular circumstances rather than of the comparison between choices of circumstances.

People who won't exist under any of our choices are of no importance (picture: Michelangelo)

We need never be concerned with the infinite people who don’t exist. All those who won’t exist under any choice we might make are irrelevant. The question is whether those who do exist under one choice we can make and don’t exist un...
Towards a New Decision Theory
by Wei Dai — 2009-08-13T05:31:41.197Z

It commonly acknowledged here that current decision theories have deficiencies that show up in the form of various paradoxes. Since there seems to be little hope that Eliezer will publish his Timeless Decision Theory any time soon, I decided to try to synthesize some of the ideas discussed in this forum, along with a few of my own, into a coherent alternative that is hopefully not so paradox-prone.

I'll start with a way of framing the question. Put yourself in the place of an AI, or more specifically, the decision algorithm of an AI. You have access to your own source code S, plus a bit string X representing all of your memories and sensory data. You have to choose an output string Y. That’s the decision. The question is, how? (The answer isn't “Run S,” because what we want to know is what S should be in the first place.)

Let’s proceed by asking the question, “What are the consequences of S, on input X, returning Y as the output, instead of Z?” To begin with, we'll consider just the consequences of that choice in the realm of abstract computations (i.e. computations considered as mathematical objects rather than as implemented in physical systems). The most immediate consequence is that any program that calls S as a subroutine with X as input, will receive Y as output, instead of Z. What happens next is a bit harder to tell, but supposing that you know something about a program P that call S as a subroutine, you can further deduce the effects of choosing Y versus Z by tracing the difference between the two choices in P’s subsequent execution. We could call these the computational consequences of Y. Suppose you have preferences about the execution of a set of programs, some of which call S as a subroutine, then you can satisfy your preferences directly by choosing the output of S so that those programs will run the way you most prefer.

A more general class of consequences might be called logical consequences. Consider a program P’ that doesn’t call S, but a different subroutine S’ that’s logically equivalent to S. In other words, S’ always prod

...
Are abortion views sexist?
by KatjaGrace — 2009-08-12T00:01:11.000Z

Indian girls are born on 500,000 fewer occasions per year than Indian boys (2006).(Photo: Steve Evans)

Abortion isn’t too bad according to half of Americans, and most of liberals and the irreligious and that bunch. The fetus never really got as far as being a child, and virtually nobody thinks failing to have children is as bad as murder.

Selective abortion of female fetuses, on the other hand, is horrific according to both ends of the ideological spectrum. And the reasons given are almost always to do with it being bad for the females who aren’t born. It’s “discrimination“, a “gross violation of women’s rights“, “an extreme manifestation of violence against women” . As my pro-choice friend (among others) complains, ‘There are all these females who should exist and are missing!’

So confirmed females have a right to exist if they are conceived, and have suffered a grave loss if they cease to be, but fetuses who might be male may as well not exist? This is either hypocritical or extremely sexist. Why are the same people adamant about both views often?

They both appear to be applications of general pro-female sympathy. When supporting the pro-choice side, the concern is for a woman’s rights over her own body. When condemning gender-specific abortion, the concern is for the females who won’t be born. Siding with the females becomes complicated when females are conspicuous as aborters one day and abortees the next. So it looks like this isn’t hypocrisy via accidental oversight, but policy choice biased by sympathies to a specific gender. If ‘whether an aborted fetus has been done a terrible wrong’ were the important point, we should expect to see more consistency on that.

When I asked about this previously my friend suggested that the motivations were importantly different in the two cases. Aborting someone because they are female is wrong. Aborting someone because you don’t want to look after them is compassionate. This doesn’t apply here, even if it were true. Gender specific abortions are common for economic and other pragmatic reasons too, not because people hate females especially. Moreover one could argue consistently that gender specific abortions are bad because they harm to others who do exist, such as the males who will go lonely. This is rarel...
Sense, Denotation and Semantics
by Vladimir_Nesov — 2009-08-11T12:47:06.014Z

J. Y. Girard, et al. (1989). Proofs and types. Cambridge University Press, New York, NY, USA. (PDF)

I found introductory description of a number of ideas given in the beginning of this book very intuitively clear, and these ideas should be relevant to our discussion, preoccupied with the meaning of meaning as we are. Though the book itself is quite technical, the first chapter should be accessible to many readers.

From the beginning of the chapter:

Let us start with an example. There is a standard procedure for multiplication, which yields for the inputs 27 and 37 the result 999. What can we say about that?

A first attempt is to say that we have an equality

27 × 37 = 999

This equality makes sense in the mainstream of mathematics by saying that the two sides denote the same integer and that × is a function in the Cantorian sense of a graph.

This is the denotational aspect, which is undoubtedly correct, but it misses the essential point:

There is a finite computation process which shows that the denotations are equal. It is an abuse (and this is not cheap philosophy — it is a concrete question) to say that 27 × 37 equals 999, since if the two things we have were the same then we would never feel the need to state their equality. Concretely we ask a question, 27 × 37, and get an answer, 999. The two expressions have different senses and we must do something (make a proof or a calculation, or at least look in an encyclopedia) to show that these two senses have the same denotation.

Concerning ×, it is incorrect to say that this is a function (as a graph) since the computer in which the program is loaded has no room for an infinite graph. Hence we have to conclude that we are in the presence of a finitary dynamics related to this question of sense.

Whereas denotation was modelled at a very early stage, sense has been pushed towards subjectivism, with the result that the present mathematical treatment of sense is more or less reduced to syntactic manipulation. This is not a priori in the essence of the subject, and we can expect in the next decades to find a treatment of computation that would combine the advantages of denotational semantics (mathematical clarity) with those of syntax (finite dynamics). This book clearly rests on a tradition that is based on this unfortunate current state of affairs: in the dichotomy between infinite, s

...
Deleting paradoxes with fuzzy logic
by  — 2009-08-11T04:27:43.258Z

You've all seen it. Sentences like "this sentence is false": if they're false, they're true, and vice versa, so they can't be either true or false. Some people solve this problem by doing something really complicated: they introduce infinite type hierarchies wherein every sentence you can express is given a "type", which is an ordinal number, and every sentence can only refer to sentences of lower type. "This sentence is false" is not a valid sentence there, because it refers to itself, but no ordinal number is less than itself. Eliezer Yudkowsky mentions but says little about such things. What he does say, I agree with: ick!

In addition to the sheer icky factor involved in this complicated method of making sure sentences can't refer to themselves, we have deeper problems. In English, sentences can refer to themselves. Heck, this sentence refers to itself. And this is not a flaw in English, but something useful: sentences ought to be able to refer to themselves. I want to be able to write stuff like "All complete sentences written in English contain at least one vowel" without having to write it in Spanish or as an incomplete sentence.1 How can we have self-referential sentences without having paradoxes that result in the universe doing what cheese does at the bottom of the oven? Easy: use fuzzy logic.

Now, take a nice look at the sentence "this sentence is false". If your intuition is like mine, this sentence seems false. (If your intuition is unlike mine, it doesn't matter.) But obviously, it isn't false. At least, it's not completely false. Of course, it's not true, either. So it's not true or false. Nor is it the mythical third truth value, clem2, as clem is not false, making the sentence indeed false, which is a paradox again. Rather, it's something in between true and false--"of medium truth", if you will.

So, how do we represent "of medium truth" formally? Well, the obvious way to do that is using a real number. Say that a completely false sentence has a truth value of 0, a completely true sentence has a truth value of 1, and the things in between have truth values in between.3 Will this work? Why, yes, and I can prove it! Well, no, I actually can't. Still, the following, trust me, is a theorem:

Suppose there is a set of sentences, and there are N of them, where N is some (possibly infinite) cardinal number, and each sentence's truth value is a continuou...
Utilons vs. Hedons
by Psychohistorian — 2009-08-10T19:20:20.968Z

Related to: Would Your Real Preferences Please Stand Up?

I have to admit, there are a lot of people I don't care about. Comfortably over six billion, I would bet. It's not that I'm a callous person; I simply don't know that many people, and even if I did I hardly have time to process that much information. Every day hundreds of millions of incredibly wonderful and terrible things happen to people out there, and if they didn't, I wouldn't even know it.

On the other hand, my professional goals deal with economics, policy, and improving decision making for the purpose of making millions of people I'll never meet happier. Their happiness does not affect my experience of life one bit, but I believe it's a good thing and I plan to work hard to figure out how to create more happiness.

This underscores an essential distinction in understanding any utilitarian viewpoint: the difference between experience and values. One can value unweighted total utility. One cannot experience unweighted total utility. It will always hurt more if a friend or loved one dies than if someone you never knew in a place you never heard of dies. I would be truly amazed to meet someone who is an exception to this rule and is not an absolute stoic. Your experiential utility function may have coefficients for other people's happiness (or at least your perception of such), but there's no way it has an identical coefficient for everyone everywhere, unless that coefficient is zero. On the other hand, you probably care in an abstract way about whether people you don't know die or are enslaved or imprisoned, and may even contribute some money or effort to prevent such from happening. I'm going to use "utilons" to refer to value utility units and "hedons" to refer to experiential utility units; I'll demonstrate that this is a meaningful distinction shortly, and that we value utilons over hedons explains much of our moral reasoning appearing to fail.

Let's try a hypothetical to illustrate the difference between experiential and value utility. An employee of Omega, LLC,1 offers you a deal to absolutely double your hedons but kill five people in, say, rural China, then wipe your memory of the deal. Do you take it? What about five hundred? Five hundred thousand?

I can't speak for you, so I'll go through my evaluation of this deal and hope it generalizes reasonably well. I don't take it at any of these ...
Misleading the witness
by Bo102010 — 2009-08-09T20:13:52.895Z

Related: Trust in Math

I was reading John Allen Paulos' A Mathematician Plays the Stock Market, in which Paulos relates a version of the well-known "missing dollar" riddle. I had heard it once before, but only vaguely remembered it. If you don't remember it, here it is:

Three people stay in a hotel overnight. The innkeeper tells them that the price for three rooms is $30, so each pays $10.

After the guests go to their rooms, the innkeeper realizes that there is a special discount for groups, and that the guests' total should have only been $25.

The innkeeper gives a bellhop $5 with the instructions to return it to the guests.

The bellhop, not wanting to get change, gives each guest $1 and keeps $2.

Later, the bellhop thinks "Wait - something isn't right. Each guest paid $10. I gave them each back $1, so they each paid $9. $9 times 3 is $27. I kept $2. $27 + $2 is $29. Where did the missing dollar go?"

I remembered that the solution involves trickery, but it still took me a minute or two to figure out where it is. At first, I started mentally keeping track of the dollars in the riddle, trying to see where one got dropped so their sum would be 30.

Then I figured it out. The story should end:

Later, the bellhop thinks "Wait - something isn't right. Each guest paid $10. I gave them each back $1, so they each paid $9. $9 times 3 is $27. The cost for their rooms was $25. $27 - $25 = $2, so they collectively overpaid by $2, which is the amount I kept. Why am I such a jerk?"

I told my fiance the riddle, and asked her where the missing dollar went. She went through the same process as I did, looking for a place in the story where $1 could go missing.

It's remarkable to me how blatantly deceptive the riddle is. The riddler states or implies at the end of the story that the dollars paid by the guests and the dollars kept by the bellhop should be summed, and that that sum should be $30. In fact, there's no reason to sum the dollars paid by the guests and the dollars kept by the bellhop, and no reason for any accounting we do to end up with $30.

The contrasts somewhat with the various proofs that 1 = 2, in which the misstep is hidden somewhere within a chain of reasoning, not boldly announced at the end of the narrative.

Both Paulos and Wikipedia give examples with different numbers that make the deception in the missing dollar riddle more obvious (and less effective). In the...
Guess Again
by Alicorn — 2009-08-09T19:11:41.763Z

In Bead Jar Guesses, I made a slightly clumsy attempt at carving out a kind of guess based on so little information that even a rationally-supposed, very small probability of some outcome doesn't confer a commensurate level of surprise when that outcome occurs. Here are several categories of probability assignment (including a re-statement of the bead jar thing) that I think might be worth considering separately. (I'm open to changing their names if other people have better ideas.)

Bewilderment: You don't even have enough information to understand the question. What is your probability that any given shren is a darkling? What is your probability that Safaitic is sometimes recorded in boustrophedon? What is your probability that 你有鼻子? (Ignore question 1 if you have read Elcenia, especially if you've seen more than is currently published; ignore question 2 if you know what either of those funny words mean; ignore question 3 if you can read Chinese.) In this case, you might find yourself in a situation where you have to make a guess, but even if you were then told the answer it wouldn't tell you much in the absence of further context.1 You would have no reason to be surprised by such an answer, no matter what probability you'd assigned.

Bead Jar: You understand the question, but have no information about anything that causally interacts with the answer. To guess, you have to grasp at the flimsiest of straws in the wording of the question and the motivations of the asker, or much broader beliefs about the general kind of question. What is your probability that Omega will pull out a red bead? What's your probability that I'm making the peace sign as I type this question with the other hand? What's your probability that the fruit on the tree in my best friend's backyard is delicious? Like Bewilderment questions, Bead Jar guesses come with no significant chance of surprise. Even if you have a tiny probability that the bead is lilac, it should not surprise you.

Bingo: You understand the question and you know something about what causes the answer, but the mechanism by which those conditions come about is known to be random (in a practical epistemic sense, not necessarily in the sense of being physically undetermined). You can have an excellent, well-calibrated probability.&...
Calibration fail
by PhilGoetz — 2009-08-09T17:15:34.708Z

I was talking on Friday with two people who've been to Italy recently and saw the leaning tower of Pisa. One of them was surprised at how short it was: "about 30 feet tall", she said. Then the other person, who'd also seen it, agreed that it was surprisingly short, and said it was "only about as tall as the Washington Monument".

"Wait," I said. "You just said it's 30 feet tall; and you just said it's as tall as the Washington monument, which has got to be at least 100 yards tall. And you agreed with each other."

And they both shook their heads, and said, "No, the Washington Monument isn't anywhere near 100 yards tall." We all live near Washington DC, and have seen the Washington Monument many times.

I said that the Washington Monument must be taller than that. One of them said that it was just as tiring to walk to the top of the Leaning Tower as to walk to the top of the Washington Monument; which was odd, since the WM stairs have been closed since before he was born.

They finally agreed that both structures were about 30 yards tall.

The Leaning Tower is 183 feet tall, and the Washington Monument is 555 feet tall.

WTF?