trentmkelly/LessWrong-43k huggingface dataset:
Prisoner's Dilemma (with visible source code) Tournament
by AlexMennen — 2013-06-07T08:30:20.271Z

After the iterated prisoner's dilemma tournament organized by prase two years ago, there was discussion of running tournaments for several variants, including one in which two players submit programs, each of which are given the source code of the other player's program, and outputs either “cooperate” or “defect”. However, as far as I know, no such tournament has been run until now.

Here's how it's going to work: Each player will submit a file containing a single Scheme lambda-function. The function should take one input. Your program will play exactly one round against each other program submitted (not including itself). In each round, two programs will be run, each given the source code of the other as input, and will be expected to return either of the symbols “C” or “D” (for "cooperate" and "defect", respectively). The programs will receive points based on the following payoff matrix:

“Other” includes any result other than returning “C” or “D”, including failing to terminate, throwing an exception, and even returning the string “Cooperate”. Notice that “Other” results in a worst-of-both-worlds scenario where you get the same payoff as you would have if you cooperated, but the other player gets the same payoff as if you had defected. This is an attempt to ensure that no one ever has incentive for their program to fail to run properly, or to trick another program into doing so.

Your score is the sum of the number of points you earn in each round. The player with the highest score wins the tournament. Edit: There is a 0.5 bitcoin prize being offered for the winner. Thanks, VincentYu!

Details:
All submissions must be emailed to wardenPD@gmail.com by July 5, at noon PDT (Edit: that's 19:00 UTC). Your email should also say how you would like to be identified when I announce the tournament results.
Each program will be allowed to run for 10 seconds. If it has not returned either “C” or “D” by then, it will be stopped, and treated as returning “Other”. For consistency, I will have Scheme collect garbage right before each run.
One submission per person or team. No person may contribute to more than one entry. Edit: This also means no copying from each others' source code. Describing the behavior of your program to othe...
Model Stability in Intervention Assessment
by Joanna Morningstar — 2013-06-06T23:24:24.981Z

In this post, I hope to examine the Bayesian Adjustment paradigm presented by Holden Karnofsky of Givewell from a mathematical viewpoint, in particular looking at how we can rigorously manage the notion of uncertainty in our models and the stability of an estimate. Several recent posts have touched on related issues.

In practise, we will need to have some substantive prior on the likely range of impacts that interventions can achieve, and I will look briefly at what kinds of log-ranges are supported in the literature, and the extent to which these can preclude extreme impact scenarios. I will then briefly look at less formal notions of confidence in a model, which may be more tractable either computationally or for heuristic purposes than a formal bayesian approach.

Bayesian Adjustment, and the Ap distribution

In the setting originally proposed, the BA framework takes a background prior on impacts and a noisy measurement of fixed variance of a fixed impact parameter. In this setting, the BA approach is provably correct. Unfortunately, the real world is not so accommodating; for general evidence about an intervention, the BA approach is not fully Bayesian. In this sense it unavoidably miscounts evidence. The general problem can be illustrated by working through the process formally. Consider propositions:

x := Has Impact x,
E := Background data,
C := there exists a given computation or argument to a given impact y.

We suppose for the framework that we have P(x|E), P(x|C) for each x. Since the set of propositions {x} are disjoint and exhaustive, these form distributions. For inference, what we actually want is P(x|EC). In the BA framework, we compute P(x|E)P(x|C) for each x, and normalise to get a distribution. Computing a bayesian update, we have:

P(x|EC) = P(xEC)/P(EC) = P(C|xE)P(x|E)/P(C|E).

So if the BA framework is to give the correct answer, we need to have P(x|EC) ∝ P(x|E)P(x|C), so that the normalisation in the BA framework fixes everything correctly. Since P(C|E) is also just a normalising factor, this proportionality occurs if and only if P(C|xE) ∝ P(x|C), which does not hold in general. In the precise setting that was originally proposed for the BA framework, there are two special features. Firstly, the estimate is a noisy measurement of x, and so P(C|x) = P(C|xE) because all dependence on the world factors through x. Secondly P(C|x) ∝ P(x...
Failure in the marketplace of ideas
by KatjaGrace — 2013-06-06T21:05:47.000Z

There is considerable disagreement over which things are roughly good or bad for the world. Encouraging same-sex marriage, speeding the cogs of ‘progress’, compelling would-be immigrants to live in third world shanty towns, and building self-replicating nanobots are considered by different parties to be either substantially positive or substantially negative. Such questions are sometimes loudly and fiercely debated.

There also seems to be much disagreement over how good different good interventions are. Is promoting sustainable agriculture a crucial last-ditch countermeasure in the face of impending societal destruction, or harmless feel-goodery? This type of disagreement seems very widespread, given the huge variety of endeavors people support with their money and time. However such disagreements are much quieter; for the most part people politely get on with their own causes. You are much more likely to be handed a brochure explaining the virtues of buying beads from women in the developing world than one earnestly arguing that this isn’t particularly valuable.

This seems like some evidence of a tragedy of the commons that one might expect prima facie. Even if you can hand out don’t-expend-effort-on-veganism brochures more cost-effectively than the vegans can hand out theirs, you can’t capture the efforts of the would-be vegans for your own preferred cause. The efforts you divert will probably be spread across many other causes.

Ultimately then, the public discussion of causes should become full of hooks to harvest goodwill and efforts, with few working to rescue people from bad causes and replenish the stock. The main demand for counterarguments should come from the people who wish to avoid joining causes themselves. But they don’t need strong cases, just some excuse to say as they decline to join. And indeed, the retorts used for such purposes seem rarely strong enough to change a person’s mind.

One big exception to this tragedy of the commons situation would be causes where people are quite divided – where leaving the pro-X side means joining the anti-X side, and where the anti-X supporters would be happy enough to diminish the X side anyway. These causes tend to be of the first sort I mentioned: where the disagreement is over whether X is excellent or hellish, rather than good or better. For these issues ther...
Tiling Agents for Self-Modifying AI (OPFAI #2)
by Eliezer Yudkowsky — 2013-06-06T20:24:25.936Z

An early draft of publication #2 in the Open Problems in Friendly AI series is now available: Tiling Agents for Self-Modifying AI, and the Lobian Obstacle. ~20,000 words, aimed at mathematicians or the highly mathematically literate. The research reported on was conducted by Yudkowsky and Herreshoff, substantially refined at the November 2012 MIRI Workshop with Mihaly Barasz and Paul Christiano, and refined further at the April 2013 MIRI Workshop.

Abstract:

We model self-modi cation in AI by introducing 'tiling' agents whose decision systems will approve the construction of highly similar agents, creating a repeating pattern (including similarity of the off spring's goals). Constructing a formalism in the most straightforward way produces a Godelian difficulty, the Lobian obstacle. By technical methods we demonstrate the possibility of avoiding this obstacle, but the underlying puzzles of rational coherence are thus only partially addressed. We extend the formalism to partially unknown deterministic environments, and show a very crude extension to probabilistic environments and expected utility; but the problem of finding a fundamental decision criterion for self-modifying probabilistic agents remains open.

Commenting here is the preferred venue for discussion of the paper. This is an early draft and has not been reviewed, so it may contain mathematical errors, and reporting of these will be much appreciated.

The overall agenda of the paper is introduce the conceptual notion of a self-reproducing decision pattern which includes reproduction of the goal or utility function, by exposing a particular possible problem with a tiling logical decision pattern and coming up with some partial technical solutions. This then makes it conceptually much clearer to point out the even deeper problems with "We can't yet describe a probabilistic way to do this because of non-monotonicity" and "We don't have a good bounded way to do this because maximization is impossible, satisficing is too weak and Schmidhuber's swapping criterion is underspecified." The paper uses first-order logic (FOL) because FOL has a lot of useful standard machinery for reflection which we can then invoke; in real life, FOL is of course a poor representational fit to most real-world environments outside a human-constructed computer chip with thermod...
Many Weak Arguments and the Typical Mind
by JonahS — 2013-06-06T18:52:32.590Z

In my previous post, I advanced the view that discovering and using many weak arguments generally produces better predictive models for answering questions about the human world than discovering and using a single relatively strong argument does.

My impression is that most high functioning people use the “many weak arguments” epistemic framework, and that this contrasts with people like my (past) self. I believe that people like me have misunderstood parts of the reasoning of most high functioning people due to typical mind fallacy, and that by extension, people like me have misunderstood parts of how society works.

I flesh out my thinking on this point below.

Simplistic generalizations and the typical mind

When we recognize that a member of a reference class has a given feature, we tend to generalize this feature to all members of the reference class. For example, when we encounter an immigrant from a given country, we reflexively assume that other people from the country have the same personality as this immigrant.

Because most people don’t have the resources or inclination to focus on improving their epistemic rationality, their beliefs are in part derived from uncritical simplistic generalizations of this type.

The reason that this sort of works in practice is that information about one member of a reference class is in fact evidence about other members of the reference class. This is a special case of Bayes’ theorem. But such simplistic generalization often yields bad epistemology, and the functionality of most people’s epistemology is often highly contingent on their use of the beliefs of those around them, which are functional by virtue of having survived natural selection.

The dangers of uncritical generalization

People responded to the Islamic terrorist attacks on September 11, 2001 by developing xenophobia toward Muslims in general, even though the terrorists represented a tiny fraction of Muslims. The xenophobia toward Muslims that followed the September 11th attacks did a great deal of harm, and if people had not been thinking in such sweeping terms, the harm may have been averted.

With such examples in mind, much of my past effort to improve my epistemic rationality has focused on refining the reflexive simplistic generalizations that I make. I’ve put a great deal of effort into appreci...
Meetup : [NYC] Self Improvement - Productivity Apps
by Raemon — 2013-06-06T18:28:28.946Z

Discussion article for the meetup : [NYC] Self Improvement - Productivity Apps

WHEN:
09 June 2013 03:30:59PM (-0400)

WHERE:
Winterfell House, 316 W 138th Street, New York NY 10030

There's a growing wealth of productivity apps, each designed to help solve some particular obstacle on one's path to productive power and glory. Many similar apps have slight differences, catering to different people who are motivated in different ways. Many apps doing something novel and interesting but people may not know about them at all, or have realized they have the problem the app is intended to solve.

This Sunday, those of us who have made use of productivity apps will report what we find useful (and what we found not useful).

NOTICE: We are rotating our schedule one week forward, because the current string of Fortnights was producing a lot of conflict for members. This is this upcoming Sunday.

The Less Wrong Self Improvement Group is a fortnightly meetup group for people looking to become more effective, at whatever it is they're trying to do, using modern rationality techniques.

Meetups are held every other Sunday, at Winterfell House (316 West 138th Street) at 3:30 PM. Goals are set on the public google doc.

Discussion article for the meetup : [NYC] Self Improvement - Productivity Apps
CFAR workshop, June 15th, Salt Lake City UT
by Julia_Galef — 2013-06-06T16:47:42.749Z

CFAR is experimenting with a mobile workshop, so we can bring our material to people who can't make it to Berkeley. So, next week, we're running a one-and-a-half day workshop in Salt Lake City, Utah!

Workshop Details

On Saturday June 15th, CFAR will be running a workshop in the Salt Lake City area. We’ll be presenting selected material from our four-day workshop and giving you the chance to consult with our instructors on how you can put these skills to work.

You’ll arrive for class at 10am on Saturday, and you and eleven other participants will spend the day learning highlights from our applied rationality curriculum: how to recognize and defuse a fight-or-flight response when it doesn't do you any good (you can’t outrun data you don’t like!), how to make sure your desire to complete a long-term project (say, writing a book) trickles down to motivate all the picayune steps along the way (doing a read-through to pick off unnecessary adjectives), and how to make the most of your intuitive judgments. Classes wrap up at 7pm, and then we’ll all go out for dinner, where you’ll have a chance to decompress and digest the day (along with your meal).

After dinner, if you’ve registered for the optional half-day, you’ll sleep over on site with the CFAR staff and play some fun, brain-teasing games. The evening is a time for unstructured conversation and collaboration. What are your pet projects and ambitions? Get feedback from classmates and instructors and start figuring out ways to make the most of your newfound skills.

The next morning, you’ll choose which of the previous day’s skills you really want to practice intensively. Catch any misunderstandings or sticking points while you’re still around to troubleshoot them with a CFAR instructor. At our four-day workshops, many participants report that our final-day review sessions are the point where they were finally able to internalize the material and start to use it instinctively.

After a half-day of review and reinforcement, we send you back out into the world, better prepared to make the most of your brain.

Application Details

The cost of the workshop will be $90 for the first day of instruction + $50 if you plan to stick around for the overnight and the second day of practice.

Registration is first come, f...
Meetup : Second Bristol meetup & mailing list for future meetups
by Benya — 2013-06-06T16:47:07.377Z

Discussion article for the meetup : Second Bristol meetup & mailing list for future meetups

WHEN:
16 June 2013 03:00:00PM (+0100)

WHERE:
Hodgkin House, 3 Meridian Place, Bristol BS8 1JG

At our lovely first meetup (four people came, if I count myself), I unfortunately forgot to take the opportunity to sort out when a good time for the next meetup would be. Sorry!

Since I'm about to be away for a while and I think others are leaving for the summer as well, I decided to just be bold once more and announce a time and hope that somebody else is free as well. But to make it easier to find good times in the future, please join the Google Group I've just created!

Last time, we ended up sitting in the cafe for hours without consuming much, so for this meetup I've booked the dining room at the student house where I live, which should be a quiet and comfortable place to talk. I'll also put up a LessWrong sign outside saying this, but please ring the buzzer marked "Basement", or you can call me at +43-660-1461996 (unfortunately I don't have a UK mobile yet, but if you ring just once, I'll come up and meet you).

Time & date is Sunday, the 16th of June, starting at 3pm. Hope that I didn't pick a terrible time and someone will be able to join me! :-)

Discussion article for the meetup : Second Bristol meetup & mailing list for future meetups
Ideas wanted: democracy in an Em world
by Stuart_Armstrong — 2013-06-06T14:05:01.730Z

One person, one vote - a fundamental principle of our democratic government. But what happens in a world where one person can be copied, again and again?

That is the world described by Robin Hanson's "Em economics". Ems, or uploads, are human minds instantiated inside software, and hence can be copied as needed. But what is the fate of democratic government in such a world of copies? Can it be preserved? Should it be preserved? How much of it should be preserved? Those are the questions we'll be analysing at the FHI, but we first wanted to turn to Less Wrong to see the ideas and comments you might have on this. Original thoughts especially welcome!

To start the conversation, here are some of the features of idealised democracy (the list isn't meant to be exhaustive or restrictive, or necessarily true about real world democracies). Which of these could exist in an Em world, and which should?

Democracy grants legitimacy to the government.

Democracy is fair and egalitarian - each person has a single vote.

Democracy aligns the interests of the rulers with that of the ruled.

Democracy is stable - powerful groups can generally seize power within the structure, rather than overthrowing it.

Democracy allows the competition of governing ideas.

Democracy often leads to market economies, which generate large wealth.

Democracy often lead to welfare states, which increase happiness.

Democracy doesn't need to use certain coercive methods, such as restrictions on free speech, that other systems require to remain stable.

Democracy stops a particular group from hanging on to power indefinitely, which can reduce corruption, inefficiency and excessive use of state power for private purposes.

EDIT: For clarification purposes, I am not claiming that democracies achieve these goals, or that these are all desirable. They are just ideas to start thinking about.
Mahatma Armstrong: CEVed to death.
by Stuart_Armstrong — 2013-06-06T12:50:11.711Z

My main objection to Coherent Extrapolated Volition (CEV) is the "Extrapolated" part. I don't see any reason to trust the extrapolated volition of humanity - but this isn't just for self centred reasons. I don't see any reason to trust my own extrapolated volition. I think it's perfectly possible that my extrapolated volition would follow some scenario like this:

It starts with me, Armstrong 1. I want to be more altruistic at the next level, valuing other humans more.

The altruistic Armstrong 2 wants to be even more altruistic. He makes himself into a perfectly altruistic utilitarian towards humans, and increases his altruism towards animals.

Armstrong 3 wonders about the difference between animals and humans, and why he should value one of them more. He decided to increase his altruism equally towards all sentient creatures.

Armstrong 4 is worried about the fact that sentience isn't clearly defined, and seems arbitrary anyway. He increase his altruism towards all living things.

Armstrong 5's problem is that the barrier between living and non-living things isn't clear either (e.g. viruses). He decides that he should solve this by valuing all worthwhile things - is not art and beauty worth something as well?

But what makes a thing worthwhile? Is there not art in everything, beauty in the eye of the right beholder? Armstrong 6 will make himself value everything.

Armstrong 7 is in turmoil: so many animals prey upon other animals, or destroy valuable rocks! To avoid this, he decides the most moral thing he can do is to try and destroy all life, and then create a world of stasis for the objects that remain.

There are many other ways this could go, maybe ending up as a negative utilitarian or completely indifferent, but that's enough to give the flavour. You might trust the person you want to be, to do the right things. But you can't trust them to want to be the right person - especially several levels in (compare with the argument in this post, and my very old chaining god idea). I'm not claiming that such a value drift is inevitable, just that it's possible - and so I'd want my initial values to dominate when there is a large conflict.

Nor do I give Armstrong 7's values any credit for having originated from mine. Under torture, I'm pretty sure I could be made to accept any system of values whatsoever; there are other ways that would provably alter my...
[link] The Economics of Social Status
by Kaj_Sotala — 2013-06-06T08:38:52.536Z

http://www.meltingasphalt.com/the-economics-of-social-status/

Discusses a number of aspects of social status, including the "social status as currency" concept that Morendil and I previously wrote about.

Now we get to the really interesting stuff: the economic properties of social status.

Let’s start with transactions, since they form the basis of an economy. Status is part of our system for competing over scarce resources, so it should be no surprise that it participates in so many of our daily transactions. Some examples:

We trade status for favors (and vice versa). This is so common you might not even realize it, but even the simple act of saying “please” and “thank you” accords a nominal amount of status to the person doing the favor. The fact that status is at stake in these transactions becomes clear when the pleasantries are withheld, which we often interpret as an insult (i.e., a threat to our status).

An apology is a ritual lowering of one’s status to compensate for a (real or perceived) affront. As with gratitude, withholding an apology is perceived as an insult.

We trade status for information (and vice versa). This is one component of “powertalk,” as illustrated in the Gervais Principle series.

We trade status for sex (and vice versa), which often goes by the name “seduction.” Sometimes even the institution of marriage functions as a sex-for-status transaction. Dowries illustrate this principle by working against it — they reinforce class/caste systems by making it harder for high-status men to marry low-status women.

We reward employees in the form of institutionalized status (titles, promotions, parking spots), which trade off against salary as a form of compensation.

We can turn money into status by means of conspicuous consumption, or status into money by means of endorsement (i.e., being paid to lend status to an endeavor).

But the part that I found the most interesting was the idea of defining communities via their status standards:

Previously we defined status with respect to a community, but we could also flip it around:

A community is a group of people who agree on how to measure status among their members.

In other words, it’s a group of people who share a common status currency. Silicon Valley, fo

...