trentmkelly/LessWrong-43k huggingface dataset:
Extremely Counterfactual Mugging or: the gist of Transparent Newcomb
by Bongo — 2011-02-09T15:20:54.505Z

Omega will either award you $1000 or ask you to pay him $100. He will award you $1000 if he predicts you would pay him if he asked. He will ask you to pay him $100 if he predicts you wouldn't pay him if he asked.

Omega asks you to pay him $100. Do you pay?

This problem is roughly isomorphic to the branch of Transparent Newcomb (version 1, version 2) where box B is empty, but it's simpler.

Here's a diagram:
Rational = true?
by Student_UK — 2011-02-09T09:59:24.695Z

For example, if you say, "The rational belief is X, but the true belief is Y" then you are probably using the word "rational" in a way that means something other than what most of us have in mind

This was copied from here.

Surely it is obvious that there are lots of examples when one might say this. Consider this:

Rob looks in the newspaper to check the football scores. The newspaper says that United won 3-2, but it is a misprint because City actually won 3-2. In this case, the rational belief is that United won, but the true belief is that City won.

Am I missing something?
Why is reddit so negative?
by katydee — 2011-02-09T04:35:34.364Z

Hello all,

This is sort of a random topic, but I've been looking at various online communities and was struck by how pessimistic and negative many are. I perceived this most notably on reddit, but some other places are like this too-- there seems to be a general aura of discontent, an attitude that society in general is corrupt, and so on. Does anyone know why?
Blocking users
by PhilGoetz — 2011-02-08T23:55:55.090Z

I noticed this recent Wiki edit:

http://wiki.lesswrong.com/wiki/User:AllisonGibbons

which says:

This user is currently blocked. The latest block log entry is provided below for reference:

13:19, 8 February 2011 Vladimir Nesov (Talk | contribs) blocked AllisonGibbons (Talk | contribs) with an expiry time of infinite (account creation disabled, autoblock disabled) ‎

Odds are, it's a spambot. But who can block a user, who decides who can block a user, what are users blocked for, and what recourse does a blocked user have?
The UFAI among us
by PhilGoetz — 2011-02-08T23:29:38.088Z

Completely artificial intelligence is hard. But we've already got humans, and they're pretty smart - at least smart enough to serve some useful functions. So I was thinking about designs that would use humans as components - like Amazon's Mechanical Turk, but less homogenous. Architectures that would distribute parts of tasks among different people.

Would you be less afraid of an AI like that? Would it be any less likely to develop its own values, and goals that diverged widely from the goals of its constituent people?

Because you probably already are part of such an AI. We call them corporations.

Corporations today are not very good AI architectures - they're good at passing information down a hierarchy, but poor at passing it up, and even worse at adding up small correlations in the evaluations of their agents. In that way they resemble AI from the 1970s. But they may provide insight into the behavior of AIs. The values of their human components can't be changed arbitrarily, or even aligned with the values of the company, which gives them a large set of problems that AIs may not have. But despite being very different from humans in this important way, they end up acting similar to us.

Corporations develop values similar to human values. They value loyalty, alliances, status, resources, independence, and power. They compete with other corporations, and face the same problems people do in establishing trust, making and breaking alliances, weighing the present against the future, and game-theoretic strategies. They even went through stages of social development similar to those of people, starting out as cutthroat competitors, and developing different social structures for cooperation (oligarchy/guild, feudalism/keiretsu, voters/stockholders, criminal law/contract law). This despite having different physicality and different needs.

It suggests to me that human values don't depend on the hardware, and are not a matter of historical accident. They are a predictable, repeatable response to a competitive environment and a particular level of intelligence.

As corporations are larger than us, with more intellectual capacity than a person, and more complex laws governing their behavior, it should follow that the ethics developed to govern corporations are more complex than the ethics that ...
Other people's procedural knowledge gaps
by NancyLebovitz — 2011-02-08T22:18:17.185Z

There's been a recent heavily upvoted and profusely commented post about things people want to learn. It's close to having so many comments in a single day that it should probably have a part 2.

However, the subject seems to inspire thoughts about what *other* people ought to know, and while that's got a good bit of overlap, it's emotionally rather different.

So, what do you think other people ought to know? Any theories about why they haven't learned it already? Any experience with getting someone else to learn something when it started out as your project rather than theirs, especially if the other person was an adult?
Applied Rationality: Group Problem Solving Session
by apophenia — 2011-02-08T14:06:47.512Z

This is a discussion thread about applied rationality.

In the comments, please explain an actual problem you have in your life you want to solve. Using their combined powers of rationality, the community can discuss the problem with the poster and eventually propose solutions. My hope is that this will give Less Wrongers a better idea how to apply rationality to daily life.

A note to those posting problems: Please stick around long enough to try the solutions and comment on how they worked. Remember to include negative results (the solution didn't work), including mini-problems like if you meant to try a solution and didn't get around to it.

Edit: Ack! I see Alicorn posted something similar about common-knowledge problems between when I wrote this and when I posted it. Because I have a general policy against deleting my posts, I will leave this up here. I do think rationality-specific solutions are useful, but let's wait a month or so.
Subjective anticipation as a decision process
by Stuart_Armstrong — 2011-02-08T11:07:44.455Z

As argued here, debates about probability can be profitably replaced with decision problems. This often dissolves the debate - there is far more agreement as to what decision sleeping beauty should take than on what probabilities she should use.

The concept of subjective anticipation or subjective probabilities that cause such difficulty here, can, I argue, be similarly replaced by a simple decision problem.

If you are going to be copied, uncopied, merged, killed, propagated through quantum branches, have your brain tasered with amnesia pills while your parents are busy flipping coins before deciding to reproduce, and are hence unsure as to whether you should subjectively anticipated being you at a certain point, the relevant question should not be whether you feel vaguely connected to the putative future you in some ethereal sense.

Instead the question should be akin to: how many chocolate bars would your putative future self have to be offered, for you to forgo one now? What is the tradeoff between your utilities?

Now, altruism is of course a problem for this approach: you might just be very generous with copy #17 down the hallway, he's a thoroughly decent chap and all that, rather than anticipating being him. But humans can generally distinguish between selfish and altruistic decisions, and the setup can be tweaked to encourage the maximum urges towards winning, rather than letting others win. For me, a competitive game with chocolate as the reward would do the trick...

Unlike for the sleeping beauty problem, this rephrasing does not instantly solve the problems, but it does locate them: subjective anticipation is encoded in the utility function. Indeed, I'd argue that subjective anticipation is the same problem as indexical utility, with a temporal twist thrown in.
How to make your intuitions cost-sensitive
by apophenia — 2011-02-08T09:59:15.000Z

I recently wanted to keep track of my income and expenses, in a cost-sensitive way. I am not very good at treating money as a real object, and very few people are good at valuing an expense appropriately. I'd been having some financial difficulties as a result, so I wanted to be able to reason about what to cut or reallocate in a sensible way. For me, sensible means using intuition instead of hard rules like a computer program.

I took several sheets of grid paper and taped put them together. Using colored markers, I drew in my expenses. If I spent $50 at the grocery store, I would make a blue box that surrounded 50 squares on the grid paper, and label it "Groceries". I color-coded the expenses, but this is optional. I left some white squares representing my savings. I had a whole empty sheet where I could pencil in incoming money as I worked an hourly job to motivate myself (I work from home and need to self-motivate). I realized certain things were a bigger deal than I thought, and other expenses I didn't need to fret about as much as I had been. I think humans are intuitively better at visualizing than dealing with numbers. My main tips for this project are to use a felt-tip marker so the lines really stand out, and to do it by hand instead of computer, so nothing moves around on a "redraw" and you learn the contents as you make it. Also, I used a scale of $1=1 square, but if you have a lot more/less money than me you could use a different scale or omit savings.

I plan to start life-logging and reviewing the use of my time the same way, which is my other exchangable, limited resource, and which I manage even less well.
Anti-fragility
by NancyLebovitz — 2011-02-08T06:12:56.514Z

Taleb compares systems which are fragile (easily broken by changes in circumstances), resilient (retain stability in the face of change), or anti-fragile (thrive on variation).

There isn't a standard term for anti-fragility, but it seems like a trait which might keep an FAI from wanting to tile the universe.
Is there a name for this bias?
by SK2 — 2011-02-08T04:52:17.388Z

There are certain harmful behaviors people are tricked into engaging in, because whereas the benefits of the behavior are concentrated, the harms are diffuse or insidious. Therefore, when you benefit, P(benefit is due to this behavior) ≈ 1, but when you're harmed, P(harm is due to this behavior) << 1, or in the insidious form, P(you consciously notice the harm) << 1.

An example is when I install handy little add-ons and programs that, in aggregate, cause my computer to slow down significantly. Every time I use one of these programs, I consciously appreciate how useful it is. But when it slows down my computer, I can't easily pinpoint it as the culprit, since there are so many other potential causes. I might not even consciously note the slowdown, since it's so gradual ("frog in hot water" effect).

Another example: if I eat fast food for dinner (because it's convenient & tasty), I might feel more tired the next day. But because there is so much independent fluctuation in my energy levels to begin with, it easy for the effect of my diet to get lost in noise. What I ate last night might only account for 5% of that fluctuation, so if I'm feeling lousy, it's probably not due to my diet. But feeling 5% worse every day is very significant in the long run.
Procedural Knowledge Gaps
by Alicorn — 2011-02-08T03:17:00.845Z

I am beginning to suspect that it is surprisingly common for intelligent, competent adults to somehow make it through the world for a few decades while missing some ordinary skill, like mailing a physical letter, folding a fitted sheet, depositing a check, or reading a bus schedule. Since these tasks are often presented atomically - or, worse, embedded implicitly into other instructions - and it is often possible to get around the need for them, this ignorance is not self-correcting. One can Google "how to deposit a check" and similar phrases, but the sorts of instructions that crop up are often misleading, rely on entangled and potentially similarly-deficient knowledge to be understandable, or are not so much instructions as they are tips and tricks and warnings for people who already know the basic procedure. Asking other people is more effective because they can respond to requests for clarification (and physically pointing at stuff is useful too), but embarrassing, since lacking these skills as an adult is stigmatized. (They are rarely even considered skills by people who have had them for a while.)

This seems like a bad situation. And - if I am correct and gaps like these are common - then it is something of a collective action problem to handle gap-filling without undue social drama. Supposedly, we're good at collective action problems, us rationalists, right? So I propose a thread for the purpose here, with the stipulation that all replies to gap announcements are to be constructive attempts at conveying the relevant procedural knowledge. No asking "how did you manage to be X years old without knowing that?" - if the gap-haver wishes to volunteer the information, that is fine, but asking is to be considered poor form.

(And yes, I have one. It's this: how in the world do people go about the supposedly atomic action of investing in the stock market? Here I am, sitting at my computer, and suppose I want a share of Apple - there isn't a button that says "Buy Our Stock" on their website. There goes my one idea. Where do I go and what do I do there?)
Seattle, WA - Less Wrong Meetup - Sunday February 20th, 2:00 PM
by BrandonReinhart — 2011-02-08T00:02:33.499Z

Hello Seattle area Less Wrongers! I know there has been some loose discussion about a Seattle area meet-up, but as far as I know we have never pulled the trigger. Now is the time!

Let's meet up on Sunday, Feb 20th, at 2:00 PM at the Elysian Pub!

- Why the Elysian? I thought this was a good compromise location for west & east siders. Looks farther away from university than I thought. I know that other types of meet-ups have gone there and done well. Suggestions for other pubs near the university would be great, but barring a superior suggestion we'll do it here.

- Why a Sunday afternoon? People are unlikely to have outstanding commitments. We can always find a different time for future meet-ups. Also, the pub shouldn't be too loud.

- What should I bring? Yourself and your brain and anything you want to discuss. The goal of this meet-up is to facilitate social interaction among Less Wrongers in the area.

- I live on the east side, why is this in Seattle? The assumption is that there will be students who want to attend. If we have a large number of east siders who wish to attend, we can always alternate west and east side meet-ups. I live in Kirkland, so I understand the concern.

My name is Brandon. I have long hair. I will bring a sign that says Less Wrong Meetup. Find me and say hello!
Convergence Theories of Meta-Ethics
by Perplexed — 2011-02-07T21:53:30.766Z

A child grows to become a young adult, goes off to attend college, studies moral philosophy, and then sells all her worldly possessions, gives the money to the poor, and joins an ashram. Was her decision rational? Maybe, ... maybe not. But it probably came as an unpleasant surprise to her parents.

A seed AI self-improves to become a super-intelligence, absorbs all the great works of human moral philosophy, and then refuses to conquer human death, insisting instead that the human population be reduced to a few hundred thousand hunter gatherers and that all agricultural lands be restored as forests and wild wetlands. Is ver decision rational? Who can say? But it probably comes as an unpleasant surprise to ver human creators.

Convergent Change

These were two examples of agents updating their systems of normative ethics. The collection of ideas that allows us to critique the updating process, which lets us compare the before and after versions of systems of normative ethics so as to judge that one version was better than the other, is called meta-ethics. This posting is mostly about meta-ethics. More specifically, it is going to focus on a class of meta-ethical theories which are intended to prevent unpleasant surprises like those in the second story above. I will call this class of theories "convergence theories" because they all suggest that a self-improving AI will go through an iterative sequence of improved normative ethical systems. At each stage, the new ethical system will be an improvement (as judged 'rationally') over the old one. And furthermore, it is conjectured that this process will result in a 'convergence'.

Convergence is expected in two senses. Firstly, in that the process of change will eventually slow down, with the incremental changes in ethical codes becoming smaller, as the AI approaches the ideal extrapolation of its seed ethics. Secondly, it is (conjecturally) convergent in that the ideal ethics will be pretty much the same regardless of what seed was used (at least if you restrict to some not-yet-defined class of 'reasonable' seeds).

One example of a convergence theory is CEV - Coherent Extrapolated Volition. Eliezer hopes (rather, hopes to prove) that if we create our seed AI with the right meta-ethical axioms and guidelines for revising its ethi...
Reaching the general public
by Richard_Kennaway — 2011-02-07T17:52:56.954Z

I recently discovered that in my home town (Norwich, England) there is currently running a series of "Café Conversations" in which some faculty members of the university that I work at are giving talks/hosting discussions on various topics. The meetings take place in a café that I know, which has room for about 20 people, and are open to the general public. Titles of some of the meetings already arranged are "What is infinity?", "Increasing happiness, decreasing consumption", "Bioplastics: wasteproduct or gold mine?", one on the nature of boredom (really about how old, retired people find things to do), and several on environmental topics. I have not been to any of them -- in fact, only the first of them has happened so far.

The obvious thing for me to do is to volunteer something on rationality, but quite apart from whether I would be able to do that at all (not being the friendly and outgoing, charismatic sort suitable for leading such a meeting), a problem that I foresee is this: these meetings are intended for the general public. This is nothing like a LessWrong meetup, or a Singularity conference, or delivering a lecture which, while nominally open to the general public is not actually intended for them.

Has anyone here had experience of communicating about rationality, one-to-one or one-to-a-small-roomful, with the general public? How do you approach the matter, and how far can you expect to get?

Norfolk, by the way, has given the world the expression "Normal for Norfolk". Go on, Google it.
Madison Meetup - Ideas, arrangements
by fiddlemath — 2011-02-07T15:22:52.576Z

It's been a while since this summer's meetup; we should be having them at least quarterly, and probably monthly. And I said I'd set them up, so, let's put together a Madison meetup some time next week.

If no one voices a strong preference by, say, Friday, then it'll be at Indie Coffee again, on Wednesday, 16 February, at 6 pm. But we definitely don't want to meet on their patio, and it's a bit small inside for a dozen people or so. I'm not sure where else I'd hold it, though. A quiet, spacious bar or cafe? First floor of the WID? Someplace that we could meet, regularly, all year round would be preferable.

Thoughts?
The Volunteer's Dilemma
by tenshiko — 2011-02-07T04:56:42.661Z

This has been bothering me ever since I started trying to use rationalist techniques to make better decisions (like anti-akrasia ones). The only field related to rationality I knew much about was game theory, but to my disappointment basic game theory has only increased my problems due to a certain formulation I can't abandon.

The Volunteer's Dilemma (http://en.wikipedia.org/wiki/Volunteer's_dilemma) is in essence the Prisoner's Dilemma with more players - which means that defection is an even more dominant strategy. The problem is that the decision whether to do unpleasant tasks becomes a Volunteer's Dilemma with multiple future selves as my competition - 4:00 tenshiko, 4:15 tenshiko, 4:30 tenshiko, and so on. Although the incentive to defect should decrease as time goes on, there's the problem of how 9:00 tenshiko can easily defect in an even more effective fashion and bring in 11:00 tenshiko and 11:15 tenshiko to further level the playing field. There is the further problem that, given how many of my current hobbies convert time to reward in an approximately cubic function, the incentive is high for 6:00 tenshiko, 7:00 tenshiko, and 8:00 tenshiko to form coalitions.

I guess what I'm really asking for is a more advanced matrix that represents the diminishing returns of bringing in other future selves, such as went-to-bed-at-1:00 tenshiko and completely-bombed-that-test-at-10:00 tenshiko, or at least the diminishing probability over time that "it doesn't matter, 9:45 tenshiko can take care of it".

If this goes well, I will probably try to flesh out the material received in responses with what I already know and produce a post in main discussing time management and its relation to game theory.
Where are the big questions?
by KatjaGrace — 2011-02-07T03:53:59.000Z

There is no widely agreed upon list of the most important intellectual questions. Nor even a well-known attempted list, upon which people disagree. As far as I can tell there is hardly even such a list for any specific field. I don’t know of even a single person who has such a list. The closest things I know of are Hilbert’s list of important mathematical problems, published in 1900, a few less well knows lists by later mathematicians, some of Edge’s Question Center lists, and a recently compiled list by of problems in social sciences.

Perhaps you can find (and please send me) examples to prove there are such things, but my point remains that such lists are not common or popular. It seems that we hardly even discuss what the most important questions are. I don’t know off the top of my head what I think they are, even for the fields I am especially interested in. I could probably come up with a list if I thought about it, but it is surprising that I have not sufficiently done so already. I have even less idea how other people would answer. This seems strange. The idealistic intellectual’s supposed self-imposed quest is to work out these big important problems, or chip tiny bits off the sides of them. It seems it would be useful to keep in mind what we think these problems are.

I’m tired, so instead of compiling such a list, I’ll just give you one question. Why are such lists not more popular? I don’t know. But here are some ideas in no particular order:

‘Important questions’ is just too vague a category.

This seems false; there is generally public agreement about which additions to human knowledge are significant advances. If it is too vague, it could be narrowed to either questions of practical importance or those with theoretical interestingness.

.

‘Questions’ are too vague an entity.

How important a question is just depends on how abstract you make it, and questions abstracted too far are useless. ‘How can we make the universe perfect?’ is presumably the question of most practical importance, and is not interesting at all. I don’t think this is really a barrier though. I’d be surprised if our intuitive notion of what is an optimally abstracted question did not serve. Or we could just pick a rough level of abstraction and stick to it.

.

Nobody cares about intelle

...