trentmkelly/LessWrong-43k huggingface dataset:
Waterfall Ethics
by calef — 2012-01-30T21:14:28.774Z

I recently read Scott Aaronson's "Why Philosophers Should Care About Computational Complexity" (http://arxiv.org/abs/1108.1791), which has a wealth of interesting thought-food. Having chewed on it for a while, I've been thinking through some of the implications and commitments of a computationalist worldview, which I don't think is terribly controversial around here (there's a brief discussion in the paper about the Waterfall Argument, and its worth reading if you're unfamiliar with either it or the Chinese room thought experiment).

That said, suppose we ascribe to a computationalist worldview. Further suppose that we have a simulation of a human running on some machine. Even further suppose that this simulation is torturing the human through some grisly means.

By our supposed worldview, our torture simulation is reducible to some finite state machine, say a one tape turing machine. This one tape turing machine representation, then, must have some initial state.

My first question: Is more 'harm' done in actually carrying out the computation of the torture simulation on our one tape turing machine than simply writing out the initial state of the torture simulation on the turing machine's tape?

The computation, and thus the simulation itself, are uniquely specified by that initial encoding. My gut feeling here is that no, no more harm is done in actually carrying out the computation, because the 'torture' that occurs is a structural property of the encoding. This might lead to perhaps ill-formed questions like "But when does the 'torture' actually 'occur'?" for some definition of those words. But, like I said, I don't think that question makes sense, and is more indicative of the difficulty in thinking about something like our subjective experience as something reducible to deterministic processes than it is a criticism of my answer.

If one thinks more harm is done in carrying out the simulation, then is twice as much harm done by carrying out the simulation twice? Does the representation of the simulation matter? If I go out to the beach and arrange sea shells in a way that mimics the computation of the torture, has the torture 'occurred'?

My second question: If the 'harm' occurring in the simulation is uniquely specified by the initial state of the Turing machine, how are we to assign mo...
Terminal Bias
by  — 2012-01-30T21:03:21.945Z

I've seen of people on Lesswrong taking cognitive structures that I consider to be biases as terminal values. Take risk aversion for example:

Risk Aversion

For a rational agent with goals that don't include "being averse to risk", risk aversion is a bias. The correct decision theory acts on expected utility, with utility of outcomes and probability of outcomes factored apart and calculated separately. Risk aversion does not factor them.

EDIT: There is some contention on this. Just substitute "that thing minimax algorithms do" for "risk aversion" in my writing. /EDIT

A while ago, I was working through the derivation of A* and minimax planning algorithms from a Bayesian and decision-theoretic base. When I was trying to understand the relationship between them, I realized that strong risk aversion, aka minimax, saves huge amounts of computation compared to the correct decision theory, and actually becomes more optimal as the environment becomes more influenced by rational opponents. The best way win is to deny the opponents any opportunity to weaken you. That's why minimax is a good algorithm for chess.

Current theories about the origin of our intelligence say that we became smart to outsmart our opponents in complex social games. If our intelligence was built for adversarial games, I am not surprised at risk aversion.

A better theoretical replacement, and a plausible causal history for why we have the bias instead of the correct algorithm are convincing to me as an argument against risk aversion as a value the way a rectangular 13x7 pebble heap is convincing to a pebble sorter as an argument against the correctness of a heap of 91 pebbles; it seems undeniable, but I don't have access to the hidden values that would say for sure.

And yet I've seen people on LW state that their "utility function" includes risk aversion. Because I don't understand the values involved, all I can do is state the argument above and see if other people are as convinced as me.

It may seem silly to take a bias as terminal, but there are examples with similar arguments that are less clear-cut, and some that we take as uncontroversially terminal:

Responsibility and Identity

The feeling that you are responsible for some things and not others, like say, the safety of your family, but not people being tortured in Syria, seems noble and practical. But I take it to be a bias.

I'm no evolutionary...
Meetup : LessWrong Megameetup
by atucker — 2012-01-30T19:33:02.904Z

Discussion article for the meetup : LessWrong Megameetup

WHEN: 04 February 2012 12:00:00PM (-0500)

WHERE: Leverage House, New York City (contact for address)

We're holding the second East Coast Megameetup this weekend at the Leverage House in NYC, kindly provided by Geoff and the Leverage Research team.

Contact me at aarondtucker <at> gmail <dot> com for the address, since Geoff doesn't want to post it on the internet.

We'll have semi-focused talks/discussions starting late Saturday afternoon on:

Nutrition

Community-building

What Leverage is doing

Group Intelligence

Psychological Frameworks

I'm really looking forward to this, and I hope to see you there! It's a great opportunity to meet the other LWers on your coast.

Discussion article for the meetup : LessWrong Megameetup
Hacking Less Wrong made easy: Vagrant edition
by Morendil — 2012-01-30T18:51:05.377Z

The Less Wrong Public Goods Team has already brought you an easy-to use virtual machine for hacking Less Wrong.

But virtual boxes can cut both ways: on the one hand, you don't have to worry about setting things up yourself; on the other hand, not knowing how things were put together, having to deal with a "black box" that doesn't let you use your own source code editor or pick an OS - these can be offputting. To me at least, these were trivial inconveniences that might stand in the way of updating my copy of the source and making some useful tweaks.

Enter Vagrant - and a little work I've done today for LW hackers and would-be hackers. Vagrant is a recent tool that allows you to treat virtual machine configurations as source code.

Instead of being something that someone possessed of arcane knowledge has put together, a virtual machine under Vagrant results from executing a series of source code instructions - and this source code is available for you to read, review, understand or change. (Software development should be a process of knowledge capture, not some hermetic discipline where you rely on the intransmissible wisdom of remote elders.)

Preliminary (but tested) results are up on my Github repo - it's a fork of the offiical LW code base, not the real thing. (One this is tested by someone else, and if it works well, I intend to submit a pull request so that these improvements end up in the main codebase.) The following assumes you have a Unix or Mac system, or if you're using Windows, that you're command-line competent.

Hacking on LW is now done as follows (compared to using the VM):

The following prerequisites are unchanged: git, Virtualbox

Install the following prerequisites: Ruby, rubygems, Vagrant

Download the Less Wrong source code as follows: git clone git@github.com:Morendil/lesswrong.git

Enter the "lesswrong" directory, then build the VM with: vagrant up (may take a while)

Log into the virtual box with: vagrant ssh

Go to the "/vagrant/r2" directory, and copy example.ini to development.ini

Change all instances of "password" in development.ini to "reddit"

You can now start the LW server with: paster serve --reload development.ini port=8080

Browse the URL http://localhost:8080/

The cool part is that the "/vagrant" directory on the VM is mapped to where you checked out the LW source code on your own machine: it's a shared di

...
Repost: Efficient Charity: Cheap Utilons via bone marrow registration
by atorm — 2012-01-30T17:42:48.022Z

DISCLAIMER: This article was posted a few days ago in Main. If you read it there, you probably don't need to read it again, although there is an additional paragraph at the bottom that might interest you. I am posting it again because I didn't quite understand how posts tend to show up on the site and because I want to get as many eyes on this article as possible.

This topic is not really related to the things normally discussed here, but I think it's really important, and it might interest Less Wrongers, especially since many of us are interested in ethics and utility calculations that are essentially cost-benefit analyses. Bone marrow donation in the United States is managed by the National Marrow Donor Program. Because typing donors for matching purposes can be costly, they often require people signing up to donate to pay a registration fee, which probably prevents a lot of people from signing up. These costs are being covered until the end of the month by a corporate sponsor, which means that right now, all you need to do if you live in the US is go tohttp://marrow.org/Join/Join_Now/Join_Now.aspx and fill out a simple questionnaire. You will be sent a kit to collect a cheek swab, and then you will be entered into the donor database. Doing this does not require you to donate if a match comes up.

The reason I think this might interest Less Wrongers is that this is a really cheap way to improve the world. According to their website, about 1 in 540 potential donors are actually asked to donate, so registering doesn't actually make it all that likely that you will be asked to do anything more. If you ARE a match for someone who needs a donation, the cost to you is at most the temporary pain of marrow extraction (many donors are asked only for blood cells), whereas the other person’s chance to live is much improved. This looks like a huge net positive.

Unfortunately I only found out about this a few days ago, and it only occurred to me [Saturday] that this might be a forum of people who would respond to the argument "you can make the world better at little cost to yourself." However, I ask that you go to the website and spend a few minutes signing up. This is like buying a 1 in 540 lottery ticket that SAVES SOMEONE’S LIFE. If the Singularity hits and an FAI can generate perfectly matched marrow for anyone who needs it from totipotency-

...
Working Through the Controlled Demolition Conspiracy
by rysade — 2012-01-30T16:11:42.191Z

I hope I'm not breaking any taboos here. It's been a while since I've come onto the discussion section and I admit I'm not too up to date on the topics.

I'm having difficulty responding to someone who is convinced that 7WTC was brought down by controlled demolition on September 11th, 2001. They're referencing the controlled-looking destruction of 7WTC and various other incriminating looking things. Thermite and thermite waste products seem to come up a lot.

Now, I have definitely noticed I'm confused here. While I hold the opinion that the towers went down because of the planes/fires (i.e. the standard explanation) I have difficulty seeing how the falsity of controlled demolition is the slam-dunk folks seem to think it is. Could somebody walk me through this?

[EDIT: About a million edits later, I have finally worked through the problem with my link: I needed it to be in HTML and not in the comment format.]
Charity when time isn't convertible to money?
by DataPacRat — 2012-01-29T19:38:42.217Z

Having just re-read "Money: The Unit of Caring", I noticed that the general methods proposed therein make some assumptions which don't seem to apply to me, and I'm trying to figure out how the conclusions therein change therefrom.

Avoiding certain personal details, I'm on a fixed income; I get a monthly deposit in my bank account every month. I don't expect this to change in the foreseeable future; and at least in the general sense of 'job', it's unlikely I'll be able to acquire one. In sum - I don't have any easy way to convert my time into additional money.

However, I still want to get the occasional warm fuzzy from causing the most possible good from what I can do - even if that involves my volunteering to spend some hours of my life doing things that would be inefficient for someone else. For example, donating blood, or taking an overnight shift keeping an eye on things at the local 'out of the cold' program; and using givewell.org as a guide for what money I am able to funnel into direct donating.

So - does anyone have any advice? (Or questions that would help better advice be given?)
Cataloging my skills - good idea?
by hamnox — 2012-01-29T14:07:19.889Z

tl,dr: Discuss utility of writing down a portfolio of bayesian evidence for/against your skills, and keeping it for personal reference.

Last night, I applied for the Write/Design Rationality Curriculum Position. I am the sort who sincerely needs the continuous requests to err on the side of applying to actually apply, and some part of me is still gobsmacked that I went ahead and did it. I do not regret applying in the smallest amount, because raising the sanity waterline is a dear little obsession of mine. At the moment, all that drive is going in circles for lack of clear direction. Although I feel supremely unqualified rationality-wise, I think this would be a fantastic opportunity to GET qualified and improve the world at the same time. By the time I finished the application I was much more confident in my own ability to be helpful.

But judging my application a net positive doesn't mean I can't analyze the heck out it.

I woke up this morning and realized that I'd listed a lot of conclusions about myself while barely even hinting at the evidence behind *why I thought those things*. That's something one shouldn't do on a regular job application, let alone when it's going to be read by professional rationality enthusiasts.

So I'm making an informal skills reference catalog for myself, because this needs to REALLY not happen with the next opportunity I sign up for. Like a resume, except focusing on quantity and bayesian evidence influencing your belief rather than what sort of self-presentation is most likely to get you hired. It differs significantly from a resume in that it should also include -negative- evidences against having certain skills, for evenness and self-honesty's sake. I suspect caching more evidence in a readily available format will be helpful for mitigating availability heuristic errors, and writing the reasoning process down instead of (only) resolving to keep a more accurate self-image in my head is useful because thoughts are slippery. [anecdote]I notice I get a lot more useful reasoning done when I write down my thoughts approximately as my inner monologue voices them. It prevents backtracking, as I can go back to quickly and confidently determine whether or not I've already thought of a point, and makes it substantially easier to notice when I've retro-actively edited a thought, either because I've contradicted myself or beca...
There's learned philosophers but not philosophical experts
by JamesCole — 2012-01-29T10:28:38.514Z

It seems to me that the notion of expertise can only apply to fields in which there is an established body of knowledge. By that I mean fields in which we have (empirical) grounds for believing our knowledge is at least an approximation or heading in the right direction. Physics or genetics or how to fix cars are examples of such fields. You can be an expert in physics.

Philosophy seems different. What makes philosophy interesting is that it's about things we don't understand well. In philosophy we're not even sure that existing approaches to problems are heading in the right direction.

Philosophy is pretty much by definition about things we don't understand well. Once a philosophical topic is understood it ceases to be part of philosophy, and becomes part of another field like physics, biology, economics, etc. (or alternatively, the problem may be dissolved and seen as a kind of misunderstanding.)

I would say the kind of knowledge that exists in the field of philosophy is more of ways of describing problems, or particular arguments for or against a view of problems. It's more like a discussion.

You can be an expert in the different positions about a philosophical problem, but I would distinguish this from the idea that someone can be an expert on a philosophical subject.

For example, someone can be an expert on the various problems and arguments associated with consciousness, but I don't think anyone can claim to be an expert on consciousness (at least the hard problem of consciousness) because we just don't understand it.

So rather than saying there are experts in philosophy I would say that there are people who are very learned in philosophy.

Why does this distinction matter?

When there isn't established knowledge, we're less certain that existing approaches are correct. The fact that an existing approach hasn't been able to solve a problem for long time may mean that it's the wrong approach. It is more likely in philosophy that someone who comes from outside of the field, who isn't well versed in the existing approaches, can add something of use to the table. The fact that they aren't familiar with existing arguments may even be a virtue.

If there aren't philosophical experts, then there aren't experts to challenge.

Yet it seems to me that philosophy seems to hold greater reverence for 'experts' than most other fields.

What do you think?

[I originally&

...
Personal research update
by Mitchell_Porter — 2012-01-29T09:32:30.423Z

Synopsis: The brain is a quantum computer and the self is a tensor factor in it - or at least, the truth lies more in that direction than in the classical direction - and we won't get Friendly AI right unless we get the ontology of consciousness right.

Followed by: Does functionalism imply dualism?

Sixteen months ago, I made a post seeking funding for personal research. There was no separate Discussion forum then, and the post was comprehensively downvoted. I did manage to keep going at it, full-time, for the next sixteen months. Perhaps I'll get to continue; it's for the sake of that possibility that I'll risk another breach of etiquette. You never know who's reading these words and what resources they have. Also, there has been progress.

I think the best place to start is with what orthonormal said in response to the original post: "I don't think anyone should be funding a Penrose-esque qualia mysterian to study string theory." If I now took my full agenda to someone out in the real world, they might say: "I don't think it's worth funding a study of 'the ontological problem of consciousness in the context of Friendly AI'." That's my dilemma. The pure scientists who might be interested in basic conceptual progress are not engaged with the race towards technological singularity, and the apocalyptic AI activists gathered in this place are trying to fit consciousness into an ontology that doesn't have room for it. In the end, if I have to choose between working on conventional topics in Friendly AI, and on the ontology of quantum mind theories, then I have to choose the latter, because we need to get the ontology of consciousness right, and it's possible that a breakthrough could occur in the world outside the FAI-aware subculture and filter through; but as things stand, the truth about consciousness would never be discovered by employing the methods and assumptions that prevail inside the FAI subculture.

Perhaps I should pause to spell out why the nature of consciousness matters for Friendly AI. The reason is that the value system of a Friendly AI must make reference to certain states of conscious beings - e.g. "pain is bad" - so, in order to make correct judgments in real life, at a minimum it must be able to tell which entities are people and which are not. Is an AI a person? Is a digital copy of a human person, itself a person? Is a human body with a complet...
The utility of information should almost never be negative
by Solvent — 2012-01-29T05:43:30.930Z

As humans, finding out facts that we would rather not be true is unpleasant. For example, I would dislike finding out that my girlfriend were cheating on me, or finding out that my parent had died, or that my bank account had been hacked and I had lost all my savings.

However, this is a consequence of the dodgily designed human brain. We don't operate with a utility function. Instead, we have separate neural circuitry for wanting and liking things, and behave according to those. If my girlfriend is cheating on me, I may want to know, but I wouldn't like knowing. In some cases, we'd rather not learn things: if I'm dying in hospital with only a few hours to live, I might rather be ignorant of another friend's death for the short remainder of my life.

However, a rational being, say an AI, would never rather not learn something, except for contrived cases like Omega offering you $100 if you can avoid learning the square of 156 for the next minute.

As far as I understand, an AI with a set of options decides by using approximately the following algorithm. This algorithm uses causal decision theory for simplicity.

"For each option, guess what will happen if you do it, and calculate the average utility. Choose the option with the highest utility."

So say Clippy is using that algorithm with his utility function of utility = number of paperclips in world.

Now imagine Clippy is on a planet making paperclips. He is considering listening to the Galactic Paperclip News radio broadcast. If he does so, there is a chance he might hear about a disaster leading to the destruction of thousands of paperclips. Would he decide in the following manner?

"If I listen to the radio show, there's maybe a 10% chance I will learn that 1000 paperclips were destroyed. My utility in from that decision would be on average reduced by 100. If I don't listen, there is no chance that I will learn about the destruction of paperclips. That is no utility reduction for me. Therefore, I won't listen to the broadcast. In fact, I'd pay up to 100 paperclips not to hear it."

Try and figure out the flaw in that reasoning. It took me a while to spot it, but perhaps I'm just slow.

* thinking space *

For Clippy to believe "If I listen to the radio show, there's maybe a 10% chance I will learn that 1000 paperclips were destroye...
Faustian bargains and discounting
by RolfAndreassen — 2012-01-29T05:10:43.655Z

I was reading TV Tropes on Hell, and it occurred to me: If your discounting was sufficiently hyperbolic, or indeed plain exponential with a low enough time preference, it would in some sense be rational to take a literal Faustian bargain. The integral to infinite time of some constant amount of torture per unit time, discounted exponentially or hyperbolically, is finite; enough worldly power and pleasure would outweight it.

But this clashes rather strongly with my intuition. Notice that the argument doesn't depend on hyperbolic discounting; no preference pumping is involved. It works just fine with exponentials and a high decay constant. Or, if the worldly pleasures were strong enough, a low decay constant, that is, a high time preference, such as (I assume) most LWers have. For example, would you take eternal torture for a guarantee of living until the heat-death of the universe, 10^130 years from now, with all the refinements of Fun Theory along the way? Intuition says no, infinity being infinity, but then again intuition is notoriously bad at dealing with very large and very small numbers. If I calculate the thing in time-discounted utilons, it seems to me that my decay constant has to be very tiny indeed for me to care about what happens at the end of *10^130* years.

So should I discard my intuition, and take such a bargain if Mephistopheles should suddenly turn up? (Noting that in 10^130 years, I might learn a thing or two about getting out of such difficulties...) Or alternatively, should I stop discounting future utilons?