trentmkelly/LessWrong-43k huggingface dataset:
Examples of AI's behaving badly
by Stuart_Armstrong — 2015-07-16T10:01:44.047Z

Some past examples to motivate thought on how AI's could misbehave:

An algorithm pauses the game to never lose at Tetris.

In "Learning to Drive a Bicycle using Reinforcement Learning and Shaping", Randlov and Alstrom, describes a system that learns to ride a simulated bicycle to a particular location. To speed up learning, they provided positive rewards whenever the agent made progress towards the goal. The agent learned to ride in tiny circles near the start state because no penalty was incurred from riding away from the goal.

A similar problem occurred with a soccer-playing robot being trained by David Andre and Astro Teller (personal communication to Stuart Russell). Because possession in soccer is important, they provided a reward for touching the ball. The agent learned a policy whereby it remained next to the ball and “vibrated,” touching the ball as frequently as possible.

Algorithms claiming credit in Eurisko: Sometimes a "mutant" heuristic appears that does little more than continually cause itself to be triggered, creating within the program an infinite loop. During one run, Lenat noticed that the number in the Worth slot of one newly discovered heuristic kept rising, indicating that had made a particularly valuable find. As it turned out the heuristic performed no useful function. It simply examined the pool of new concepts, located those with the highest Worth values, and inserted its name in their My Creator slots.

There was something else going on, though. The AI was crafting super weapons that the designers had never intended. Players would be pulled into fights against ships armed with ridiculous weapons that would cut them to pieces. "It appears that the unusual weapons attacks were caused by some form of networking issue which allowed the NPC AI to merge weapon stats and abilities," according to a post written by Frontier community manager Zac Antonaci. "Meaning that all new and never before seen (sometimes devastating) weapons were created, such as a rail gun with the fire rate of a pulse laser. These appear to have been compounded by the additional stats and abilities of the engineers weaponry."

Programs classifying gender based on photos of irises may have been artificially effective due to mascara in the photos.
Why you should attend EA Global and (some) other conferences
by Habryka — 2015-07-16T04:50:31.624Z

Many of you know about Effective Altruism and the associated community. It has a very significant overlap with LessWrong, and has been significantly influenced by the culture and ambitions of the community here.

One of the most important things happening in EA over the next few months is going to be EA Global, the so far biggest EA and Rationality community event to date, happening throughout the month of August in three different locations: Oxford, Melbourne and San Francisco (which is unfortunately already filled, despite us choosing the largest venue that Google had to offer).

The purpose of this post is to make a case for why it is a good idea for people to attend the event, and to serve as an information hub for information that might be more relevant to the LessWrong community (as well an additional place to ask questions). I am one of the main organizers and very happy to answer any questions that you have.

Is it a good idea to attend EA Global?

This is a difficult question, that obviously will not have a unique answer, but from the best of what I can tell, and for the majority of people reading this post, the answer seems to be "yes". The EA community has been quite successful at shaping the world to the better, and at building an epistemic community that seems to be effective at changing its mind and updating on evidence.

But there have been other people arguing in favor of supporting the EA movement, and I don't want to repeat everything that they said. Instead I want to focus on a more specific argument: "Given that I belief that EA is overall a promising movement, should I attend EA Global if I want to improve the world (according to my preferences)?"

The key question here is: Does attending the conference help the EA Movement succeed?

How attending EA Global helps the EA Movement succeed

It seems that the success of organizations is highly dependent on the interconnectedness of its members. In general a rule seems to hold: The better connected the social graph of your organization is, the more effective does it work.

In particular, any significant divide in an organization, any clustering of different groups that do not communicate much with each other, seems to significantly reduce the output the organization produces. I wish we had better studies on this, and that I could link to mor...
Rationality Reading Group: Part E: Overly Convenient Excuses
by Gram_Stone — 2015-07-16T03:38:55.977Z

This is part of a semi-monthly reading group on Eliezer Yudkowsky's ebook, Rationality: From AI to Zombies. For more information about the group, see the announcement post.

Welcome to the Rationality reading group. This fortnight we discuss Part E: Overly Convenient Excuses (pp. 211-252). This post summarizes each article of the sequence, linking to the original LessWrong post where available.

Essay: Rationality: An Introduction

E. Overly Convenient Excuses

46. The Proper Use of Humility - There are good and bad kinds of humility. Proper humility is not being selectively underconfident about uncomfortable truths. Proper humility is not the same as social modesty, which can be an excuse for not even trying to be right. Proper scientific humility means not just acknowledging one's uncertainty with words, but taking specific actions to plan for the case that one is wrong.

47. The Third Alternative - People justify Noble Lies by pointing out their benefits over doing nothing. But, if you really need these benefits, you can construct a Third Alternative for getting them. How? You have to search for one. Beware the temptation not to search or to search perfunctorily. Ask yourself, "Did I spend five minutes by the clock trying hard to think of a better alternative?"

48. Lotteries: A Waste of Hope - Some defend lottery-ticket buying as a rational purchase of fantasy. But you are occupying your valuable brain with a fantasy whose probability is nearly zero, wasting emotional energy. Without the lottery, people might fantasize about things that they can actually do, which might lead to thinking of ways to make the fantasy a reality. To work around a bias, you must first notice it, analyze it, and decide that it is bad. Lottery advocates are failing to complete the third step.

49. New Improved Lottery - If the opportunity to fantasize about winning justified the lottery, then a "new improved" lottery would be even better. You would buy a nearly-zero chance to become a millionaire at any moment over the next five years. You could spend every moment imagining that you might become a millionaire at that moment.

50. But There's Still A Chance, Right? - Sometimes, you calculate the probability of a certain event and find that the number is so unbelievably smal...
The Just-Be-Reasonable Predicament
by Satoshi_Nakamoto — 2015-07-16T03:17:52.926Z

If people don't see you as being “reasonable”, then you are likely to have troublesome interactions with them. Therefore, it is often valuable to be seen as “reasonable”. Reasonableness is a general perception that is determined by the social context and norms. It includes, but is not limited to, being seen as fair, sensible and socially cooperative. In summary, we can describe it as being noticeably rational in socially acceptable ways. What is “reasonable” and what is rational often converges, but it is important to note that they can also diverge and be different. For example, it was deemed “unreasonable” to free African-Americans from slavery because slavery was deemed necessary for the economy of the South.

The just-be-reasonable predicament occurs when you are chastised for doing something that you believe to be more rational and/or optimal than the norm or what is expected or desired. The chastiser has either: not considered, cannot fathom or does not care that what you are doing or want to do might be more rational and/or optimal than what is the default course of action. The predicament is similar to the one described in lonely dissent in that you must choose between making what you to believe to be the most rational and/or optimal course of action and the one that will be meet with the least amount of social disapproval.

An example of this predicament is when you are playing a game with a scrub (a player who is handicapped by self-imposed rules that the game knows nothing about). The scrub might criticise for continuing to use the best strategy that you are aware of, but that they thinks is cheap. If you try to argue that a strategy is a strategy, then the argument is likely to end with the scrub getting angry and saying the equivalent of “just be reasonable”, which basically means: “why can’t you just follow what I see as the rules and the way things should be done?” When you encounter this predicament, you need to weigh up the costs of leaving the way or choosing a non-optimal action vs. facing potential social disapproval. The way opposes being “reasonable” when it is not aligned with being rational. In the scrub situation, the main benefit of being “reasonable&rd...
Philosophy professors fail on basic philosophy problems
by Shmi — 2015-07-15T18:41:06.473Z

Imagine someone finding out that "Physics professors fail on basic physics problems". This, of course, would never happen. To become a physicist in academia, one has to (among million other things) demonstrate proficiency on far harder problems than that.

Philosophy professors, however, are a different story. Cosmologist Sean Carroll tweeted a link to a paper from the Harvard Moral Psychology Research Lab, which found that professional moral philosophers are no less subject to the effects of framing and order of presentation on the Trolley Problem than non-philosophers. This seems as basic an error as, say, confusing energy with momentum, or mixing up units on a physics test.

Abstract:

We examined the effects of framing and order of presentation on professional philosophers’ judgments about a moral puzzle case (the “trolley problem”) and a version of the Tversky & Kahneman “Asian disease” scenario. Professional philosophers exhibited substantial framing effects and order effects, and were no less subject to such effects than was a comparison group of non-philosopher academic participants. Framing and order effects were not reduced by a forced delay during which participants were encouraged to consider “different variants of the scenario or different ways of describing the case”. Nor were framing and order effects lower among participants reporting familiarity with the trolley problem or with loss-aversion framing effects, nor among those reporting having had a stable opinion on the issues before participating the experiment, nor among those reporting expertise on the very issues in question. Thus, for these scenario types, neither framing effects nor order effects appear to be reduced even by high levels of academic expertise.

Some quotes (emphasis mine):

When scenario pairs were presented in order AB, participants responded differently than when the same scenario pairs were presented in order BA, and the philosophers showed no less of a shift than did the comparison groups, across several types of scenario.

[...] we could find no level of philosophical expertise that reduced the size of the order effects or the framing effects on judgments of specific cases. Across the board, professional philosophers (94% with PhD’s) showed about the same size order and framing effects as similarly educat...
Recommended Reading for Evolution?
by Sable — 2015-07-15T18:04:11.852Z

I'll make this short and sweet.

I've been reading Dawkin's The Selfish Gene, and it's been really helpful filling in some of the gaps I have in my understanding of how evolution actually works.

The last biology class I took was in high school, and I don't think the mechanics of evolution is covered particularly well in American high schools.

I'm looking for recommendations - has anyone read any books that accurately describe the process of evolution for someone without specialized knowledge of biology? I've already checked LessWrong's recommended textbooks, and while it recommends some books on evolutionary psychology and on animal behavior from an evolutionary perspective, it doesn't appear to have anything that describes evolution itself in sufficient detail to model it.

I'm toying with the idea of trying to program an evolution simulator, and so I need a fairly detailed, accessible account.

Thanks for the help!
Meetup : Regular Moscow meetup: Löb's theorem, ways of mind improvement, DEL group, Zendo
by Slava Matyukhin — 2015-07-15T16:32:48.890Z

Discussion article for the meetup : Regular Moscow meetup: Löb's theorem, ways of mind improvement, DEL group, Zendo

WHEN:
19 July 2015 02:00:00PM (+0300)

WHERE:
Москва, Льва Толстого, 16

We're meeting at Yandex, at the Extropolis conference hall. Please fill this form OR join this FB event if you're planning to visit.

We have two one-hour talks planned:

Mikhail Dektyarev about MIRI's work on modal agents in prisoner's dilemma.

turchin on the roadmap of the ways of mind improvement.

We'll have our regular Dynamic Epistemic Logic group with Pion and Denis (read some links from project's pad if you want to participate).

Alexander230 will organize the game of Zendo.

Detailed schedule.

Information for the newcomers:

Here's the document explaining how to get to our meetups and what to expect. If you have any questions, or if you don't speak Russian and can't read the link, shoot me a PM.

Discussion article for the meetup : Regular Moscow meetup: Löb's theorem, ways of mind improvement, DEL group, Zendo
The Other Path - a poem
by Jacob Falkovich — 2015-07-15T13:40:23.740Z

Inspired by the call to rationalist poetry fans and informed by years of writing satire.

The Other Path

When you ask for truth and are offered illusion,

When senses deceive you and reasoning lies

I'll show you the path through the murky confusion,

Just follow and close your eyes.

On matters of fact there's no fact of the matter,

All moral and virtue are fashion and fad,

So dress in the creed that will fit you and flatter

No one can argue with that.

Some puzzles unyielding and mysteries ancient

No formula ever could hope to describe.

How proudly the scientist seeks explanations

How clearly in vain she strives.

Make cases like fortifications of metal,

No rival assertion shall ever go past.

Be carefree in choosing the side of the battle

But guard it until your last.

The sages declared that to know is to suffer,

Where wisdom is gained there is innocence lost

And learning is danger – best leave it to others,

Avoid it at any cost.

Some fools declare war on their very own nature

Their weapons are evidence, reason and math.

Don't offer compassion to those wretched creatures,

They've chosen the other path.
Biases and Fallacies Game Cards
by Gunnar_Zarncke — 2015-07-15T08:19:35.453Z

On the Stupid Questions Thread I asked

I need some list of biases for a game of Biased Pandemic for our Meet-Up. Do suitably prepared/formatted lists exist somewhere?

But none came forward.

Therefore I created a simple deck based on Wikipedia entries. I selected those that can be presumably be used easily in a game, summarized the description and added an illustrative quote.

The deck can be found in Dropbox here (PDF and ODT).

I'd be happy for corrections and further suggestions.

ADDED: We used these cards during the LW Hamburg Meetup. They attracted significant interest and even though we did use them during a board game we drew them and tried to act them out during a discussion round (which didn't work out that well but stimulated discussion nonetheless).
Waterfall Truth Predicates
by abramdemski — 2015-07-15T03:14:28.000Z

The waterfall-based approaches to the Löbian obstacle offer a way around finitely-terminating sequences of trust which we get by adding towers of soundness schemas to PA. This creates a kind of illusion of self-trust by way of a non-well-founded chain of trust.

Another familiar situation where we are normally faced with the ability to construct arbitrarily high towers but not a single self-referential system is that of truth predicates. Tarski's undefinability theorem blocks the existence of a full truth predicate within the same language as the one which it describes. Perhaps a similar waterfall construction can be applied, to get an infinite descending chain of languages.

Extend the language of PA with a family of truth predicates Trn. A Tarski-style approach would assert a T-schema Trn(┌ϕ┐)↔ϕ for ϕ which contain truth predicates indexed strictly lower than n. (┌ϕ┐ is the Gödel number of ϕ.) Here, we wish to flip this, and assert a T-schema which allows strictly higher n.

This brings to mind Yablo's Paradox. A contradiction can likely be worked out in a way resembling that, but instead I'll note that this theory implies the naive soundness waterfall in which we construct a sequence of theories Tn=Tn+1+Sound(Tn+1). This is because we can use the truth predicate Trn to carry out a proof of soundness for the axioms and inference rules, with the exception of instances of the T-schema involving Trm≤n. This gives us the naive soundness waterfall, which we know to be inconsistent. (Note that I have not checked this in detail, however.)

My idea for fixing the T-schema, then, is to introduce the same ψ(n) predicate which asserts that n is not the Gödel number of a proof of contradiction in ZFC. We make the new schema:

ψ(n)→[Trn(┌ϕ┐)↔ϕ], where ϕ contains only Trm>n.

Because we can prove any particular ψ(n), we can still apply the schema in specific cases. It seems likely that we can still carry out soundness arguments, as well, constructing the consistent version of the soundness waterfall. If so, the theory ends up being unsound as a result.

Here's a proof that the theory is unsound. Consider again Yablo's paradox. We construct an infinite sequence of statements, A0,A1,... each of which assert that the subsequent statements in the sequence are all false. Specifically: An↔∀m>n:¬Trn(┌Am┐)

Considering any particular An, we s

...
Meetup : Gen Con: Applied Game Theory
by Zubon — 2015-07-15T00:10:12.243Z

Discussion article for the meetup : Gen Con: Applied Game Theory

WHEN:
01 August 2015 02:00:00PM (-0400)

WHERE:
Indianapolis Convention Center

Tentative location: Hall F, the green tables just behind the CCG/TCG HQ. They say that space is generally available for open gaming, and failing that, the blue tables just across from HQ would be. If we have managed to pick peak CCG time, we will wait until 2:15 to gather folks and relocate (and will comment here to that effect). That spot is also just outside the exhibit hall, in case you decide you must own your own copy of whatever we play. Please bring your newest game or old favorite.

Meet up with Less Wrong friends and play games! Learn the newest releases, play classics, and otherwise have fun. A purely social event, running for as long as people want to stay and play, with potential continued discussion over dinner.

As at all events, there is no minimum degree, IQ, reading record, height, age, or neurotypicality to participate. Bring games if you like, learn new games if you like, and expect a range from "I am here for the championships" to "what are we Settling?"

We seemed like the kind of nerds who might go to Gen Con. You can also use the comments here to find others attending, arrange other connections at the event, discuss the con, etc.

Discussion article for the meetup : Gen Con: Applied Game Theory
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-07-14T21:52:13.932Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
19 July 2015 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

We'll be meeting to play games and/or not and just hang out and talk, whichever seems more fun (notice fun comes before games in the title). As usual, we will congregate in the courtyard from 3:00 to 3:30 p.m., with the hard start at 3:30.

If you want to get people together for a big game plan ahead! Post here looking for players and get an early start, because remember they'll be kicking us out a little before 7pm.

Please remember to bring games!

Upcoming Meetups:

Jul 26: Meta Meetup

Aug 2: Optical Illusions

Aug 9: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
Meetup : Moscow experimental LW group meetup
by Alexander230 — 2015-07-14T21:32:25.837Z

Discussion article for the meetup : Moscow experimental LW group meetup

WHEN:
25 July 2015 01:00:00PM (+0300)

WHERE:
Москва, Ростокинский проезд, д.3

Welcome to experimental meetup of Moscow LW group!

Our plan:

A talk about cognitive biases.

Fallacymania game ( https://lesswrong-ru.hackpad.com/Fallacymania--neGfMe9MFjH ).

Diplomacy board game on afterparty ( https://boardgamegeek.com/boardgame/483/diplomacy ).

Meetup schedule and directions are in hackpad: https://lw-msk-lect.hackpad.com/-25--NTK3dKs0ric

If you are lost, call Sasha at +7-905-527-30-82. Meetup begins at 13:00; the length of main part is 1.5 hours, the game on afterparty will take about 4 hours.

Discussion article for the meetup : Moscow experimental LW group meetup
Meetup : Warsaw: GO playing
by mkf — 2015-07-14T14:21:27.003Z

Discussion article for the meetup : Warsaw: GO playing

WHEN:
18 July 2015 02:00:00PM (+0200)

WHERE:
Warsaw

Warsaw, Kawiarnia Hoża 51

Let's play GO!

https://en.wikipedia.org/wiki/Go_%28game%29

Discussion article for the meetup : Warsaw: GO playing
Bragging Thread July 2015
by Viliam — 2015-07-13T22:01:03.320Z

Your job, should you choose to accept it, is to comment on this thread explaining the most awesome thing you've done this month. You may be as blatantly proud of yourself as you feel. You may unabashedly consider yourself the coolest freaking person ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that.

Remember, however, that this isn't any kind of progress thread. Nor is it any kind of proposal thread. This thread is solely for people to talk about the awesome things they have done. Not "will do". Not "are working on". Have already done. This is to cultivate an environment of object level productivity rather than meta-productivity methods.

So, what's the coolest thing you've done this month?

(Previous Bragging Thread)