trentmkelly/LessWrong-43k huggingface dataset:
Reacting to Inadequate Data
by MrHen — 2009-12-18T16:25:08.831Z

Two Scenarios

Alice must answer the multiple-choice question, "What color is the ball?" The two choices are "Red" and "Blue." Alice has no relevant memories of The Ball other than she knows it exists. She cannot see The Ball or interact with it in any way; she cannot do anything but think until she answers the question.

In an independent scenario, Bob has the same question but Bob has two memories of The Ball. In one of the memories, The Ball is red. In the other memory, The Ball is blue. There are no "timestamps" associated with the memories and no way of determining if one came before the other. Bob just has two memories and he, somehow, knows the memories are of the same ball.

If you were Alice, what would you do?

If you were Bob, what would you do?

Variations

More questions to ponder:

Should they do anything at all?

Should Alice and Bob act differently?

If Alice and Bob could circle more than one color, should they?

Would either answer change if the option "Green" was added to the choice list?

If the question was fill-in-the-blank, what should they write?

If Bob's memories were of different balls but he didn't know which ball was The Ball, should his actions change?

If Alice and Bob could coordinate, should it affect their answers?

Further Discussion

The basic question I was initially pondering was how to resolve conflicting sensory inputs. If I were a brain in a vat and I received two simultaneous sensory inputs that conflicted (such as the color of a ball), how should I process them?

Another related topic is whether a brain in a vat with absolutely no sensory inputs should be considered intelligent. These two questions were reduced into the above two scenarios and I am asking for help in resolving them. I think they are similar to questions asked here before but their relation to these two brain-in-a-vat questions seemed relevant to me.

Realistic Scenarios

These scenarios are cute but there are similar real-world examples. When asked if a visible ball was red or green and you happened to be unable to distinguish between red and green, how do you interpret what you see?

Abstracting a bit, any input (sensory or otherwise) that is indistinguishable from another input can really muck with your head. Most optical illusions are tricks on eye-hardware (software?).

This post is not intended to be clever or teach anything new. Rather, the topic confuses me and I am...
An account of what I believe to be inconsistent behavior on the part of our editor
by PeterS — 2009-12-17T01:33:15.694Z

There was recently a submission here posing criticism a well-known contributor, Eliezer Yudkowsky. Admittedly, whatever question the poster intended to ask was embedded within a post which was evidently designed to be a test of our rationality. Eliezer didn't seem to care too much, stating that the issue was

it being a top-level post instead of Open Thread comment. Probably would've been a lot more forgiving if it'd been an Open Thread comment. . .

The question would, under other circumstances, be just - but I don't care to justify myself here. Elsewhere, perhaps. . .

This belongs as a comment on the SIAI blog, not a post on Less Wrong.

I asked Eliezer why mormon2's post belonged on the SIAI blog and not here. He responded thus:

Because Less Wrong is about human rationality, not the Singularity Institute, and not me.

This response is unsatisfactory. Either certain posts belong on the SIAI blog and not here, or they don't and can be posted here. It can't be both ways

Note that I do not approve of mormon2's submission, as of the recent statement he made in an edit. I do, however, approve of the idea of such a submission. Somebody should be able to make a top-level post directing questions and criticism towards another author, under certain circumstances. I can't fully pin down just what the precise circumstances should be - it's not up for me to decide in any case.

But consider a high-profile contributor, who already has many posts about himself (several self-submitted) and his work, who has at times responded to off-site comments with top-level posts on Less Wrong, and who has recently given his blessing to a post entitled Less Wrong Q&A With Eliezer Yudkowsky - when such a person suggests that a submission concerning himself and his work belongs as a comment in the monthly Open Thread, or as a comment on an off-site blog, I find it very outlandish.

Eliezer is not an ordinary contributor. In the beginning, it was apparently envisioned that there would be a limit to the number of non-Yudkowsky/Hanson posts submitted per day. Obviously that policy has not been enacted. In any case, by my count there have been 682 submissions to Less Wrong as of December 14th. Eliezer has contributed 108 of those (the median number of posts per author being 2.5)1. He is not a "specific person" being asked to justify "specific decisions", as he would hypothetically suggest ...
Getting Over Dust Theory
by jhuffman — 2009-12-15T22:40:55.602Z

It has been well over a year since I first read Permutation City and relating writings on the internet on Greg Egan's dust theory. It still haunts me. The theory has been discussed tangentially in this community, but I haven't found an article that directly addresses the rationality of Egan's own dismissal of the theory.

In the FAQ, Egan says things like:

I wrote the ending as a way of dramatising[sic] a dissatisfaction I had with the “pure” Dust Theory that I never could (and still haven't) made precise (see Q5): the universe we live in is more coherent than the Dust Theory demands, so there must be something else going on.

and:

I have yet to hear a convincing refutation of it on purely logical grounds...

However, I think the universe we live in provides strong empirical evidence against the “pure” Dust Theory, because it is far too orderly and obeys far simpler and more homogeneous physical laws than it would need to, merely in order to contain observers with an enduring sense of their own existence. If every arrangement of the dust that contained such observers was realised, then there would be billions of times more arrangements in which the observers were surrounded by chaotic events, than arrangements in which there were uniform physical laws.

Isn't this, along with so many other problems, a candidate for our sometime friend the anthropic principle? That is: only in a conscious configuration field which has memories of perceptions of an orderly universe is the dust theory controversial or doubted? In the vastly more numerous conscious configuration fields with memories of perceptions of a chaotic and disorderly universe lacking a rational way to support the observer the dust theory could be accepted a priori or at least be a favored theory.

It is fine to dismiss dust theory because it simply isn't very helpful and because it has no predictions, testable or otherwise. I suppose it is also fine never to question the nature of consciousness as the answers don't seem to lead anywhere helpful either; though the question of it will continue to vex some instances of these configuration states.
Rebasing Ethics
by Shalmanese — 2009-12-15T13:56:09.689Z

Lets start with the following accepted as a given:

There exists no supernatural forces in the world and there is no objective morality imposed from above.

Our current moral codes are currently based on some mix of sociobiological influences & cultural forces

Of the current figures who accept these premises, most espouse some form of secular humanism which argues that humans are genetically programed not to lie, murder or steal, therefore this is both the right morality & the one they practice. This, to my mind, is committing the naturalistic fallacy.

Instead, I want to offer an analogy: Humans have an innate preference for certain foods which evolved in an environment radically different from modern society. In a modern society, it is widely regarded as virtuous to be actively working against our innate, genetic impulses through the practice of dieting. Similarly, our ethical landscape is radically different from the one we evolved in. Thus what we should be doing is actively working against our innate moral sense to be more in line with the modern world. In the same way that there is junk food that tastes good but is ultimately unhealthy for you, I believe there is ethical junk food which fills us with a feeling of virtue that is underserved.

Let me start off with a mild example: Tipping.

Altruism evolved in an era of small tribes where individual altruistic acts could be remembered & paid back. Now that we live in a large, anonymous society, there are many times when altruism doesn't pay. Unless you are with friends or a frequent diner at a restaurant or bar, the correct moral move is to stiff the waiter on the tip. If you're traveling somewhere alone, you should universally fail to tip as you're not likely to ever return there.

If this fills you with immediate moral revulsion, you're not alone. I'm so skeeved out by this that I've never yet worked up the nerve to do it. My empathetic system simulates how the waiter must feel to not get a tip and I get queasy in the pit of my stomach. But this empathetic system isn't based on any moral fact, it's simply an evolutionary shortcut that helps us survive in small groups. To rebase your ethics is to start actively fighting against that feeling of moral revulsion, just as the first step of a diet is to fight against the desire for fatty, sweet, salty foods.

In fact, I would go so far as to say that if yo...
Man-with-a-hammer syndrome
by Shalmanese — 2009-12-14T11:31:48.237Z

What gummed up Skinner’s reputation is that he developed a case of what I always call man-with-a-hammer syndrome: to the man with a hammer, every problem tends to look pretty much like a nail.

The Psychology of Human Misjudgment is an brilliant talk given by Charlie Munger that I still return to and read every year to gain a fresh perspective. There’s a lot of wisdom to be distilled from that piece but the one thing I want to talk about today is the man-with-a-hammer syndrome.

Man-with-a-hammer syndrome is pretty simple: you think of an idea and then, pretty soon, it becomes THE idea. You start seeing how THE idea can apply to anything and everything, it’s the universal explanation for how the universe works. Suddenly, everything you’ve ever thought of before must be reinterpreted through the lens of THE idea and you’re on an intellectual high. Utilitarianism is a good example of this. Once you independently discover Utilitarianism you start to believe that an entire moral framework can be constructed around a system of pleasures and pains and, what’s more, that this moral system is both objective and platonic. Suddenly, everything from the war in the middle east to taking your mid-morning dump at work because you need that 15 minutes of reflective time alone with yourself before you can face the onslaught of meaningless drivel that is part of corporate America but feeling guilty about it because you were raised to be a good Randian and you are not providing value from your employers so you’re committing and act of theft can be fit under the Utilitarian framework. And then, hopefully, a few days later, you’re over it and Utilitarianism is just another interesting concept and you’re slightly embarrassed about your behavior a few days prior. Unfortunately, some people never get over it and they become those annoying people write long screeds on the internet about their THE idea.

The most important thing to realize about man-with-a-hammer syndrome is that there’s no possible way to avoid having it happen to you. You can be a well seasoned rationalist who’s well aware of how man-with-a-hammer syndrome works and what the various symptoms are but it’s still going to hit you fresh with each new idea. The best you can do is mitigate the fallout that occurs.

Once you recognize that you’ve bee...
Previous Post Revised
by komponisto — 2009-12-14T06:56:50.528Z

Followup to: The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom

See also: The Importance of Saying "Oops"

I'm posting this to call attention to the fact that I've now reconsidered the highly confident probability estimates in my post from yesterday on the Knox/Sollecito case. I haven't retracted my arguments; I just now think the level of confidence in them that I specified was too high. I've added the following paragraph to the concluding section:

[EDIT: After reading comments on this post, I have done some updating of my own. I now think I failed to adequately consider the possibility of my own overconfidence. This was pretty stupid of me, since it meant that the focus was taken away from the actual arguments in this post, and basically toward the issue of whether 0.001 can possibly be a rational estimate for anything you read about on the Internet. The qualitative reasoning of this post, of course, stands. Also, the focus of my accusations of irrationality was not primarily the LW community as reflected in my previous post; I actually think we did a pretty good job of coming to the right conclusion given the information provided -- and as others have noted, the levelheadedness with which we did so was impressive.]

While object-level comments on the case and on my reasoning about it should probably continue to be confined to that thread, I'd be interested in hearing in comments here what people think about the following:

How much of a distraction did you find my extremely confident probabilities to be from the substance of my arguments?

How much did those confident estimates make it seem like I was disagreeing, rather than agreeing, with the LW survey consensus? (It seemed to me that I had provoked people into trumpeting pro-guilt arguments more than they otherwise would have if I had initally given more "reasonable" numbers.)

To what sorts of propositions, if any, do you yourself assign probabilities on the order of 0.999 or 0.001?
Against picking up pennies
by  — 2009-12-13T06:07:55.566Z

The eternally curious Tailsteak has written about how he always picks up pennies off the sidewalk. He's run a cost-benefit analysis and determined that it's better on average to pick up a penny than to pass it by. His mistake lies nowhere in the analysis itself; it's pretty much correct. His mistake is performing the analysis in the first place.

Pennies, you see, are easily the subject of scope insensitivity. When we come across a penny, we don't think, "Hey, that's something worth 0.05% of what I wish I had come across. I could buy a 25th of a gumball, a mouthful of an unhealthy carbonated beverage, a couple of seconds of footage on DVD, or enough gasoline to go a tenth of a mile." We think, "Hey, that's money," and we grab it.

The thing is, it's difficult to comprehend how little a penny is worth—we don't really have a separate concept for "mild happiness for a couple of seconds"—and we're likely to take risks that far outweigh the benefits. We don't think of bending over to pick up a penny as being a risky endeavour, but it's a penny. How much risk does it take to outweigh a penny? Surely the risk of "something unforeseen" easily does the job. Are you 99.999999% sure that picking up that penny won't kill you? You need a reason for every 9 (if you're ambivalent between using seven 9s and using nine 9s, you should use seven; the number of 9s is never arbitrary), and by the time you come up with eight reasons to pick up the penny, you'll have wasted several cents' worth of time. If you can reduce the probability of harm that far, I applaud you.

Of course, penny-grabbing doesn't have to involve actual pennies. Suppose that President Kodos of the Unified States of Somewhere (population 300 million) uses the word "idiot" in an important speech, causing the average citizen to scowl and ponder for one minute. Now, if a penny can buy you five seconds of happiness, and scowling and pondering brings the same amount of unhappiness, then that's twelve cents for every citizen, or 36 million dollars, of damage that Kodos just caused. Arguably, that's the value of a couple of human lives. As you can see, Kodos' decisions are extremely important. In this case, penny-grabbing would consist of anything less than trading precious seconds for precious human lives—if Kodos finds that he can save one life simply by going a few minutes out of his way, he sh...
The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom
by komponisto — 2009-12-13T04:16:20.840Z

Note: The quantitative elements of this post have now been revised significantly.

Followup to: You Be the Jury: Survey on a Current Event

All three of them clearly killed her. The jury clearly believed so as well which strengthens my argument. They spent months examining the case, so the idea that a few minutes of internet research makes [other commenters] certain they're wrong seems laughable

- lordweiner27, commenting on my previous post

The short answer: it's very much like how a few minutes of philosophical reflection trump a few millennia of human cultural tradition.

Wielding the Sword of Bayes -- or for that matter the Razor of Occam -- requires courage and a certain kind of ruthlessness. You have to be willing to cut your way through vast quantities of noise and focus in like a laser on the signal.

But the tools of rationality are extremely powerful if you know how to use them.

Rationality is not easy for humans. Our brains were optimized to arrive at correct conclusions about the world only insofar as that was a necessary byproduct of being optimized to pass the genetic material that made them on to the next generation. If you've been reading Less Wrong for any significant length of time, you probably know this by now. In fact, around here this is almost a banality -- a cached thought. "We get it," you may be tempted to say. "So stop signaling your tribal allegiance to this website and move on to some new, nontrivial meta-insight."

But this is one of those things that truly do bear repeating, over and over again, almost at every opportunity. You really can't hear it enough. It has consequences, you see. The most important of which is: if you only do what feels epistemically "natural" all the time, you're going to be, well, wrong. And probably not just "sooner or later", either. Chances are, you're going to be wrong quite a lot.

To borrow a Yudkowskian turn of phrase: if you don't ever -- or indeed often -- find yourself needing to zig when, not only other people, but all kinds of internal "voices" in your mind are loudly shouting for you to zag, then you're either a native rationalist -- a born Bayesian, who should perhaps be deducing general relativity from the fall of an apple any minute now -- or else you're simply not trying hard enough.

Oh, and another one of those consequences of humans' not being instinctively ...
A question of rationality
by mormon2 — 2009-12-13T02:37:41.722Z

Thank you For Your Participation

I would like to thank you all for your unwitting and unwilling participation in my little social experiment. If I do say so myself you all performed as I had hoped. I found some of the responses interesting, many them are goofy. I was honestly hoping that a budding rationalist community like this one would have stopped this experiment midway but I thank you all for not being that rational. I really did appreciate all the mormon2 bashing it was quite amusing and some of the attempts to discredit me were humorous though unsuccessful. In terms of the questions I asked I was curious about the answers though I did not expect to get any nor do I really need them; since I have a good idea of what the answers are just from simple deductive reasoning. I really do hope EY is working on FAI and actually is able to do it though I certainly will not stake my hopes or money on it.

Less there be any suspicion I am being sincere here.

Response

Because I can I am going to make one final response to this thread I started:

Since none of you understand what I am doing I will spell it out for you. My posts are formatted, written and styled intentionally for the response I desire. The point is to give you guys easy ways to avoid answering my questions (things like tone of the post, spelling, grammar, being "hostile (not really)" etc.). I just wanted to see if anyone here could actually look past that, specifically EY, and post some honest answers to the questions (real answers again from EY not pawns on LW). Obviously this was to much to ask, since the general responses, not completely, but for the most part were copouts. I am well aware that EY probably would never answer any challenge to what he thinks, people like EY typically won't (I have dealt with many people like EY). I think the responses here speak volumes about LW and the people who post here (If you can't look past the way the content is posted then you are going to have a hard time in life since not everyone is going to meet your standards for how they speak or write). You guys may not be trying to form a cult but the way you respond to a post like this screams cultish and even a some circle-jerk mentality mixed in there.

Post

I would like to float an argument and a series of questions. Now before you guys vote me down please do me the curtsey of reading the post. I a...
The persuasive power of false confessions
by matt — 2009-12-11T01:54:23.739Z

First paragraph from a Mind Hacks post:

The APS Observer magazine has a fantastic article on the power of false confessions to warp our perception of other evidence in a criminal case to the point where expert witnesses will change their judgements of unrelated evidence to make it fit the false admission of guilt.

The post and linked article are worth reading… and I don't have much to add.
What Are Probabilities, Anyway?
by Wei Dai — 2009-12-11T00:25:33.177Z

In Probability Space & Aumann Agreement, I wrote that probabilities can be thought of as weights that we assign to possible world-histories. But what are these weights supposed to mean? Here I’ll give a few interpretations that I've considered and held at one point or another, and their problems. (Note that in the previous post, I implicitly used the first interpretation in the following list, since that seems to be the mainstream view.)

Only one possible world is real, and probabilities represent beliefs about which one is real.

Which world gets to be real seems arbitrary.

Most possible worlds are lifeless, so we’d have to be really lucky to be alive.

We have no information about the process that determines which world gets to be real, so how can we decide what the probability mass function p should be?

All possible worlds are real, and probabilities represent beliefs about which one I’m in.

Before I’ve observed anything, there seems to be no reason to believe that I’m more likely to be in one world than another, but we can’t let all their weights be equal.

Not all possible worlds are equally real, and probabilities represent “how real” each world is. (This is also sometimes called the “measure” or “reality fluid” view.)

Which worlds get to be “more real” seems arbitrary.

Before we observe anything, we don't have any information about the process that determines the amount of “reality fluid” in each world, so how can we decide what the probability mass function p should be?

All possible worlds are real, and probabilities represent how much I care about each world. (To make sense of this, recall that these probabilities are ultimately multiplied with utilities to form expected utilities in standard decision theories.)

Which worlds I care more or less about seems arbitrary. But perhaps this is less of a problem because I’m “allowed” to have arbitrary values.

Or, from another perspective, this drops another another hard problem on top of the pile of problems called “values”, where it may never be solved.

As you can see, I think the main problem with all of these interpretations is arbitrariness. The unconditioned probability mass function is supposed to represent my beliefs before I have ob...