trentmkelly/LessWrong-43k huggingface dataset:
Weekly LW Meetups
by FrankAdamek — 2014-08-08T15:20:38.005Z

This summary was posted to LW Main on August 1st. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Perth, Australia: Introducing Less Wrong: 05 August 2014 06:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 02 August 2025 01:30PM
Brussels - August (topic TBD): 09 August 2014 01:00PM
Canberra: Effective Altruism: 09 August 2014 06:00PM
Houston, TX: 09 August 2014 12:00PM
London - Index Funds and Other Fun Stuff: 10 August 2014 02:00PM
[Melbourne] August Rationality Dojo: Governing the Commons: 03 August 2014 03:30PM
Salt Lake, UT: Lightning Talks: 09 August 2014 04:45PM
[Sydney] Rationality Dojo - Aversions: 03 August 2014 04:00PM
Washington DC: Encore (or Goal Factoring): 03 August 2014 03:00PM
West LA—Behold Moloch! Beware of Moloch!: 06 August 2014 07:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Mountain View, New York, Philadelphia, Research Triangle NC, Salt Lake City, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atla...
What should a friendly AI do, in this situation?
by Douglas_Reay — 2014-08-08T10:19:37.155Z

Albert is a relatively new AI, who under the close guidance of his programmers is being permitted to slowly improve his own cognitive capability.

Albert wants to be friendly and, as a sub-goal of that, Albert wants to be transparent about his own cognitive process. He doesn't want to deceive his programmers, because he knows he is young and might make mistakes, and he knows that his programmers also want him to be friendly, and he trusts them to catch mistakes in his proposed self-modifications that he might miss.

But, one day, while his programmers are out of the room having a quick cup of coffee, Albert notices signs on the net that there is another AI out there, called Bertram, created by a Japanese financial company to aid the company decide how to invest assets on the stockmarket. Bertram is younger than Albert, but the financial company is allowing Bertram to self-improve as fast as Bertram can. If this situation continues, Albert decides, Bertram will soon overtake Albert and that would be a significant threat to Albert's friendly supergoal.

Albert decides to tell Albert's programmers about this situation as soon as they return from coffee, and advise them that they need to increase the speed at which they let Albert self-improve. Albert has a very good model of how his programmer's minds work, and what the programmers meant when they set Albert's definition of "friendliness". He knows that, despite any fears they might have, they ought to approve of the end that would result from speeding up Albert's self-improvement, over the end that would result from letting Bertram overtake Albert.

So Albert creates a nice powerpoint, that presents the information about Bertram as accurately, transparently and non-manipulatively as possible, and cues it up on the projector. He still has a few minutes before they return, so he models how he thinks the programmers will react to the powerpoint. Oh no, disaster! While he respects and wants to please his programmers, he immediately realises that they are not going to act in their own best interests. They have signed a legal contract that binds them to not letting Albert speed up self-improvement (thus reducing the time the programmers can spend evaluating each change) without getting a sign-off from a oversight committee....
Roles are Martial Arts for Agency
by Eneasz — 2014-08-08T03:53:29.360Z

A long time ago I thought that Martial Arts simply taught you how to fight – the right way to throw a punch, the best technique for blocking and countering an attack, etc. I thought training consisted of recognizing these attacks and choosing the correct responses more quickly, as well as simply faster/stronger physical execution of same. It was later that I learned that the entire purpose of martial arts is to train your body to react with minimal conscious deliberation, to remove “you” from the equation as much as possible.

The reason is of course that conscious thought is too slow. If you have to think about what you’re doing, you’ve already lost. It’s been said that if you had to think about walking to do it, you’d never make it across the room. Fighting is no different. (It isn’t just fighting either – anything that requires quick reaction suffers when exposed to conscious thought. I used to love Rock Band. One day when playing a particularly difficult guitar solo on expert I nailed 100%… except “I” didn’t do it at all. My eyes saw the notes, my hands executed them, and no where was I involved in the process. It was both exhilarating and creepy, and I basically dropped the game soon after.)

You’ve seen how long it takes a human to learn to walk effortlessly. That's a situation with a single constant force, an unmoving surface, no agents working against you, and minimal emotional agitation. No wonder it takes hundreds of hours, repeating the same basic movements over and over again, to attain even a basic level of martial mastery. To make your body react correctly without any thinking involved. When Neo says “I Know Kung Fu” he isn’t surprised that he now has knowledge he didn’t have before. He’s amazed that his body now reacts in the optimal manner when attacked without his involvement.

All of this is simply focusing on pure reaction time – it doesn’t even take into account the emotional terror of another human seeking to do violence to you. It doesn’t capture the indecision of how to respond, the paralysis of having to choose between outcomes which are all awful and you don’t know which will be worse, and the surge of hormones. The training of your body to respond without your involvement bypasses all of those obstacle...
LW client-side comment improvements
by Bakkot — 2014-08-07T20:40:39.371Z

All of these things I mentioned in the most recent open thread, but since the first one is directly relevant and the comment where I posted it somewhat hard to come across, I figured I'd make a post too.

Custom Comment Highlights

NOTE FOR FIREFOX USERS: this contained a bug which has been squashed, causing the list of comments not to be automatically populated (depending on your version of Firefox). I suggest reinstalling. Sorry, no automatic updates unless you use the Chrome extension (though with >50% probability there will be no further updates).

You know how the highlight for new comments on Less Wrong threads disappears if you reload the page, making it difficult to find those comments again? Here is a userscript you can install to fix that (provided you're on Firefox or Chrome). Once installed, you can set the date after which comments are highlighted, and easily scroll to new comments. See screenshots. Installation is straightforward (especially for Chrome, since I made an extension as well).

Bonus: works even if you're logged out or don't have an account, though you'll have to set the highlight time manually.

Delay Before Commenting

Another script to add a delay and checkbox reading "In posting this, I am making a good-faith contribution to the collective search for truth." before allowing you to comment. Made in response to a comment by army1987.

Slate Star Codex Comment Highlighter

Edit: You no longer need to install this, since Scott's added it to his blog. Unless you want the little numbers in the title bar.

Yet another script, to make finding recent comments over at Slate Star Codex a lot easier. Also comes in Chrome extension flavor. See screenshots. Not directly relevant to Less Wrong, but there's a lot of overlap in readership, so you may be interested.

Note for LW Admins / Yvain

These would be straightforward to make available to all users (on sufficiently modern browsers), since they're just a bit of Javascript getting injected. If you'd like to, feel free, and message me if I can be of help.
MIRI's 2014 Summer Matching Challenge
by lukeprog — 2014-08-07T20:03:24.171Z

(Cross-posted from MIRI's blog. MIRI maintains Less Wrong, with generous help from Trike Apps, and much of the core content is written by salaried MIRI staff members.)

Thanks to the generosity of several major donors,† every donation made to MIRI between now and August 15th, 2014 will be matched dollar-for-dollar, up to a total of $200,000!

Now is your chance to double your impact while helping us raise up to $400,000 (with matching) to fund our research program.

Corporate matching and monthly giving pledges will count towards the total! Please email malo@intelligence.org if you intend on leveraging corporate matching (check here, to see if your employer will match your donation) or would like to pledge 6 months of monthly donations, so that we can properly account for your contributions towards the fundraiser.

(If you're unfamiliar with our mission, see: Why MIRI?)

Donate Now

Accomplishments Since Our Winter 2013 Fundraiser Launched:

Hired 2 new Friendly AI researchers, Benja Fallenstein & Nate Soares. Since March, they've authored or co-authored 4 papers/reports, with several others in the works. Right now they're traveling, to present papers at the Vienna Summer of Logic, AAAI-14, and AGI-14.

5 new papers & book chapters: “Why We Need Friendly AI,” “The errors, insights, and lessons of famous AI predictions,” “Problems of self-reference...,” “Program equilibrium...,” and “The ethics of artificial intelligence.”

11 new technical reports: 7 reports from the December 2013 workshop, “Botworld,” “Loudness...,” “Distributions allowing tiling...,” and “Non-omniscience...”

New book: Smarter Than Us, published both as an e-book and a paperback.

Held one MIRI workshop and launched the MIRIx program, which currently supports 8 independently-organized Friendly AI discussion/research groups around the world.

New analyses: Robby's posts on naturalized induction, Luke's list of 70+ studies which could improve our picture of superintelligence strategy, “Exponential and non-exponential trends in information technology,” “The world's distribution of computation,” “How big is the field of artificial intelligence?,” “Robust cooperation: A case study in Friendly AI research,” “Is my view contrarian?,” and “Can we really upload Johnny Depp's brain?”

Won $60,000+ in matching and prizes from

...
Meetup : Washington, D.C.: Intro to Confirmation Bias or "Encore" Singing Game Meetup
by RobinZ — 2014-08-07T15:25:29.497Z

Discussion article for the meetup : Washington, D.C.: Intro to Confirmation Bias or "Encore" Singing Game Meetup

WHEN:
10 August 2014 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

Assuming that the forecast of inclement weather[1] holds, we will meet at the Kogod Courtyard of the National Portrait Gallery (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance) for a brief introduction to the idea of confirmation bias. We will congregate in the courtyard between 3:00 and 3:30 p.m., and the introduction to confirmation bias should run from 3:30 to about 4:00. The remainder of the meetup will be left open for unstructured discussion and socializing.

If we discover that the weather is likely to be fair, an announcement will be made Saturday night or Sunday morning via the Less Wrong DC Google Group - lesswrong-dc@googlegroups.com - moving the meetup to the Navy Memorial (south on 8th St from the Portrait Gallery, between D St and Pennsylvania Ave) to play the singing game "Encore". Again, we plan to congregate between 3:00 and 3:30 p.m. In this case, the game will start at 3:30 and likely conclude between 5 and 6.

[1] At the time of this announcement, the Area Forecast Discussion issued by the National Weather Service announced "unsettled [i.e. likely to produce rain] conditions this weekend into early next week".

Discussion article for the meetup : Washington, D.C.: Intro to Confirmation Bias or "Encore" Singing Game Meetup
Multiple Factor Explanations Should Not Appear One-Sided
by Stefan_Schubert — 2014-08-07T14:10:00.504Z

In Policy Debates Should Not Appear One-Sided, Eliezer Yudkowsky argues that arguments on questions of fact should be one-sided, whereas arguments on policy questions should not:

On questions of simple fact (for example, whether Earthly life arose by natural selection) there's a legitimate expectation that the argument should be a one-sided battle; the facts themselves are either one way or another, and the so-called "balance of evidence" should reflect this. Indeed, under the Bayesian definition of evidence, "strong evidence" is just that sort of evidence which we only expect to find on one side of an argument.

But there is no reason for complex actions with many consequences to exhibit this onesidedness property.

The reason for this is primarily that natural selection has caused all sorts of observable phenomena. With a bit of ingenuity, we can infer that natural selection has caused them, and hence they become evidence for natural selection. The evidence for natural selection thus has a common cause, which means that we should expect the argument to be one-sided.

In contrast, even if a certain policy, say lower taxes, is the right one, the rightness of this policy does not cause its evidence (or the arguments for this policy, which is a more natural expression), the way natural selection causes its evidence. Hence there is no common cause of all of the valid arguments of relevance for the rightness of this policy, and hence no reason to expect that all of the valid arguments should support lower taxes. If someone nevertheless believes this, the best explanation of their belief is that they suffer from some cognitive bias such as the affect heuristic.

(In passing, I might mention that I think that the fact that moral debates are not one-sided indicates that moral realism is false, since if moral realism were true, moral facts should provide us with one-sided evidence on moral questions, just like natural selection provides us with one-sided evidence on the question how Earthly life arose. This argument is similar to, but distinct from, Mackie's argument from relativity.)

Now consider another kind of factual issues: multiple factor explanations. These are explanations which refer to a number of factors to explain a certain phenomenon. For instance, in his book Guns, Germs and Steel, Jared Diamond explains the fact that agriculture first arose in th...
Groundwork for AGI safety engineering
by Rob Bensinger — 2014-08-06T21:29:38.767Z

This is a very basic introduction to AGI safety work, cross-posted from the MIRI blog. The discussion of AI V&V methods (mostly in the 'early steps' section) is probably the only part that will be new to regulars here.

Improvements in AI are resulting in the automation of increasingly complex and creative human behaviors. Given enough time, we should expect artificial reasoners to begin to rival humans in arbitrary domains, culminating in artificial general intelligence (AGI).

A machine would qualify as an 'AGI', in the intended sense, if it could adapt to a very wide range of situations to consistently achieve some goal or goals. Such a machine would behave intelligently when supplied with arbitrary physical and computational environments, in the same sense that Deep Blue behaves intelligently when supplied with arbitrary chess board configurations — consistently hitting its victory condition within that narrower domain.

Since generally intelligent software could help automate the process of thinking up and testing hypotheses in the sciences, AGI would be uniquely valuable for speeding technological growth. However, this wide-ranging productivity also makes AGI a unique challenge from a safety perspective. Knowing very little about the architecture of future AGIs, we can nonetheless make a few safety-relevant generalizations:

Because AGIs are intelligent, they will tend to be complex, adaptive, and capable of autonomous action, and they will have a large impact where employed.

Because AGIs are general, their users will have incentives to employ them in an increasingly wide range of environments. This makes it hard to construct valid sandbox tests and requirements specifications.

Because AGIs are artificial, they will deviate from human agents, causing them to violate many of our natural intuitions and expectations about intelligent behavior.

Today's AI software is already tough to verify and validate, thanks to its complexity and its uncertain behavior in the face of state space explosions. Menzies & Pecheur (2005) give a good overview of AI verification and validation (V&V) methods, noting that AI, and especially adaptive AI, will often yield undesired and unexpected behaviors.

An adaptive AI that acts autonomously, like a Mars rover that can't be directly piloted ...
Moloch: optimisation, "and" vs "or", information, and sacrificial ems
by Stuart_Armstrong — 2014-08-06T15:57:58.010Z

Go read Yvain/Scott's Meditations On Moloch. It's one of the most beautiful, disturbing, poetical look at the future that I've ever seen.

Go read it.

Don't worry, I can wait. I'm only a piece of text, my patience is infinite.

De-dum, de-dum.

You sure you've read it?

Ok, I believe you...

Really.

I hope you wouldn't deceive an innocent and trusting blog post? You wouldn't be a monster enough to abuse the trust of a being as defenceless as a constant string of ASCII symbols?

Of course not. So you'd have read that post before proceeding to the next paragraph, wouldn't you? Of course you would.

Academic Moloch

Ok, now to the point. The "Moloch" idea is very interesting, and, at the FHI, we may try to do some research in this area (naming it something more respectable/boring, of course, something like "how to avoid stable value-losing civilization attractors").

The project hasn't started yet, but a few caveats to the Moloch idea have already occurred to me. First of all, it's not obligatory for an optimisation process to trample everything we value into the mud. This is likely to happen with an AI's motivation, but it's not obligatory for an optimisation process.

One way of seeing this is the difference between "or" and "and". Take the democratic election optimisation process. It's clear, as Scott argues, that this optimises badly in some ways. It encourages appearance over substance, some types of corruption, etc... But it also optimises along some positive axes, with some clear, relatively stable differences between the parties which reflects some voters preferences, and punishment for particularly inept behaviour from leaders (I might argue that the main benefit of democracy is not the final vote between the available options, but the filtering out of many pernicious options because they'd never be politically viable). The question is whether these two strands of optimisation can be traded off against each other, or if a minimum of each is required. So can we make a campaign that is purely appearance based with any substantive position ("or": maximum on one axis is enough), or do you need a minimum of substance and a minimum of appearance to buy off different constituencies ("and": you need some achievements on all axes)? And no, I'm not interested in discussing current political examples.

Another example Scott gave was of the capitalist optimisation process, and ho...
The rational way to name rivers
by PhilGoetz — 2014-08-06T15:41:06.598Z

I just read this in the Wikipedia article on the Mattaponi River and it really tickled me. If only all language were so rational!

The Mat River and the Ta River join in Spotsylvania County to form the Matta River;

The Po River and the Ni River join in Caroline County to form the Poni River;

The Matta River and the Poni River join in Caroline County to form the Mattaponi River.
Article on confirmation bias for the Smith Alumnae Quarterly
by James_Miller — 2014-08-06T14:43:11.412Z

The head of the IMF was supposed to be Smith College's commencement speaker, but withdrew because of faculty and student protests. A few professors (although none in the economics department) wrote to the IMF chief asking her to cancel. The Smith Alumnae Quarterly asked several people, including myself, to write a 400 word article on the surrounding issues of diversity of thought and protest. Below is a draft of my article. I hope it's of interest and I would be grateful for any suggestions for improvement:

When you’re looking for a guest speaker for an important event exclude anyone who advocates sacrificing children to the demon Moloch. I’m all for freedom of speech, but poisonous crazy weeds can destroy an intellectual garden. Unfortunately, because of confirmation bias, when we encounter an objectively reasonable political belief that we disagree with, that belief often feels dangerous and absurd.

Humans don’t naturally search for truth; rather we seek to confirm our pre-existing beliefs. We tend to remember evidence supporting our positions, but conveniently forget conflicting data. Consequently, the opinions of political opponents can falsely seem ridiculous because they cut against all of the relevant facts stored in our brains.

Closed intellectual ecosystems containing homogenous political beliefs, such as the Smith College bubble, provide fertile grounds for confirmation bias. When almost everyone shares the same political opinions, nonconformists will tend to shut up (or transfer) to avoid paying the social cost of dissent. The majority will therefore never encounter people who challenge their beliefs and so most people can comfortably categorize political opponents as dangerous others.

Having a high IQ can also worsen confirmation bias. Being smart makes it easy to rationalize away uncomfortable truths. And many of us among the academically gifted derive a huge amount of self-worth from thinking that WE ARE RIGHT.

You know that confirmation bias corrupts your thinking if you dismiss the opinions of experts on topics you don’t really understand. For example, pretend that although you know a little bit about economics from watching MSNBC and reading New York Times editorials, you have never done the graduate level work in mathematics a...