trentmkelly/LessWrong-43k huggingface dataset:
Making Rationality General-Interest
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2013-07-24T22:02:55.576Z

Introduction

Less Wrong currently represents a tiny, tiny, tiny segment of the population. In its current form, it might only appeal to a tiny, tiny segment of the population. Basically, the people who have a strong need for cognition, who are INTx on the Myers-Briggs (65% of us as per 2012 survey data), etc.

Raising the sanity waterline seems like a generally good idea. Smart people who believe stupid things, and go on to invest resources in stupid ways because of it, are frustrating. Trying to learn rationality skills in my 20s, when a bunch of thought patterns are already overlearned, is even more frustrating.

I have an intuition that a better future would be one where the concept of rationality (maybe called something different, but the same idea) is normal. Where it's as obvious as the idea that you shouldn't spend more money than you earn, or that you should live a healthy lifestyle, etc. The point isn't that everyone currently lives debt-free, eats decently well and exercises; that isn't the case; but they are normal things to do if you're a minimally proactive person who cares a bit about your future. No one has ever told me that doing taekwondo to stay fit is weird and culty, or that keeping a budget will make me unhappy because I'm overthinking thing.

I think the questions of "whether we should try to do this" and "if so, how do we do it in practice?" are both valuable to discuss, and interesting.

Is making rationality general-interest a good goal?

My intuitions are far from 100% reliable. I can think of a few reasons why this might be a bad idea:

1. A little bit of rationality can be damaging; it might push people in the direction of too much contrarianism, or something else I haven't thought of. Since introspection is imperfect, knowing a bit about cognitive biases and the mistakes that other people make might make people actually less likely to change their mind–they see other people making those well-known mistakes, but not themselves. Likewise, rationality taught only as a tool or skill, without any kind of underlying philosophy of why you should want to believe true things, might cause problems of a similar nature to martial art skills taught without the traditional, often non-violent philosophies–it could result in people abusing the skill to win fights/debates, making the larger community worse ...
An argument against indirect normativity
by cousin_it — 2013-07-24T18:35:04.130Z

I think I've found a new argument, which I'll call X, against Paul Christiano's "indirect normativity" approach to FAI goals. I just discussed X with Paul, who agreed that it's serious.

This post won't describe X in detail because it's based on basilisks, which are a forbidden topic on LW, and I respect Eliezer's requests despite sometimes disagreeing with them. If you understand Paul's idea and understand basilisks, figuring out X should take you about five minutes (there's only one obvious way to combine the two ideas), so you might as well do it now. If you decide to discuss X here, please try to follow the spirit of LW policy.

In conclusion, I'd like to ask Eliezer to rethink his position on secrecy. If more LWers understood basilisks, somebody might have come up with X earlier.
Inferential credit history
by RyanCarey — 2013-07-24T14:12:52.190Z

Here’s an interview with Seth Baum. Seth is an expert in risk analysis and a founder of the Global Catastrophic Risk Institute. As expected, Bill O’Reilly caricatured Seth as extreme, and cut up his interview with dramatic and extreme events from alien films. As a professional provocateur, it is his job to lay the gauntlet down to his guests. Also as expected, Seth put on a calm and confident performance. Was the interview net-positive or negative? It’s hard to say, even in retrospect. Getting any publicity for catastrophic risk reduction is good, and difficult. Still, I’m not sure just how bad publicity has to be before it really is bad publicity…

Explaining catastrophic risks to the audience of Fox News is perhaps equally difficult to explaining the risk of artificial intelligence to anyone. This is a task that frustrated Eliezer Yudkowsky so deeply that he was driven to write the epic LessWrong sequences. In his view, the inferential distance was too large to be bridged by a single conversation. There were too many things that he knew that were prerequisites to understanding his current plan. So he wrote this sequence of online posts that set out everything he knew about cognitive science and probability theory, applied to help readers think more clearly and live out their scientific values. He had to write a thousand words per day for about two years before talking about AI explicitly. Perhaps surprisingly, and as an enormous credit to Eliezer’s brain, these sequences formed the founding manifesto of the quickly growing rationality movement, many of whom now share his concerns about AI. Since he wrote these, his Machine Intelligence Research Institute (formely the singularity Institute) has grown precipitously and spun off the Center for Applied Rationality, a teaching facility and monument to the promotion of public rationality.

Why have Seth and Eliezer had such a hard time? Inferential distance explains a lot, but I have a second explanation, Seth and Eliezer had to build an inferential credit history. By the time you get to the end of the sequences, you have seen Eliezer bridge many an inferential distance, and you trust him to span another! If each time I loan Eliezer some attention, and suspend my disbelief, he has paid me back (in the currency of interesting and useful i...
Business Insider: "They Finally Tested The 'Prisoner's Dilemma' On Actual Prisoners — And The Results Were Not What You Would Expect"
by chaosmage — 2013-07-24T12:44:05.763Z

Article at http://www.businessinsider.com/prisoners-dilemma-in-real-life-2013-7#ixzz2ZxwzT6nj, seems revelant to a lot of the discussion here.

There've been studies about people who consider themselves to be relatively successful are less cooperative than people who consider themselves relatively unsuccessful. The study referenced in that article seems to bear this out.

So if you want the other party to cooperate, should you attempt to give that party the impression it has been relatively unsuccessful, at least if that party is human?
Introducing Familiar, a quantified reasoning assistant (feedback sought!)
by jamesf — 2013-07-24T02:36:45.881Z

tl;dr: I'm making a thing that uses probabilistic graphical models to assist in drawing inferences from personal data. You should check it out, and share with me your wisdom/user experience.

I had this not-completely-original idea that there should be some kind of tool for easily performing statistical inference on Quantified Self-style data.

There are a lot of QS apps out there, but for the most part they seem to be designed for 1. a single domain and/or 2. recording things primarily to combat akrasia or (more often) sating curiosity/as a lifestyle accessory, rather than actively helping you discover correlations or determine causality between things-you-do and things-you-care-about. Quantified Mind stands out as a counterexample, but I can't come up with many others in that vein.

There are also commercial products and programming languages that allow one to use machine learning to perform inference on data, but they mostly seem to be proprietary and expensive software aimed at businesses, or free but intended to be used by scientists, engineers, etc.; nothing I've yet to find is really suitable for an individual without a background in statistics/machine learning who just wants to learn what they can by smashing together their Moodscope and their FitBit.

In our era of FOSS, APIs, QS, and ML, this seems like a seriously lacking state of affairs. Hence, Familiar.

Currently, it consists of a command line interface for storing variable definitions and data in a local database without too much fuss, building a naive Bayes classifier on those variables, and finding maximum likelihood estimates given the state of one variable for the states of all the other variables. This is unsophisticated and not extremely user-friendly, but those things will change in the near future. In the case where I keep working on this for a very long time, I want to automate away as much recording as possible (including things like mood and productivity), record everything with the highest reasonable time resolution, plug into every other app out there that might provide useful data, use more complex machine learning algorithms to identify causality and generate suggestions for personal experimentation, and generally have a piece of software that knows you so well it can help you think more like an ideal Bayesian reasoner and thereby assist you in living y...
Why Eat Less Meat?
by Peter Wildeford — 2013-07-23T21:30:21.234Z

Previously, I wrote on LessWrong about the preliminary evidence in favor of using leaflets to promote veganism as a way of cost-effectively reducing suffering. In response, there was a large discussion with 530+ comments. In this discussion, I found that a lot of people wanted me to write about why I think nonhuman animals deserve our concern anyway.

Therefore, I wrote this essay with an attempt to defend the view that if one cares about suffering, one should also care about nonhuman animals, since (1) they are capable of suffering, (2) they do suffer quite a lot, and (3) we can prevent their suffering. I hope that we can have a sober, non mind-killing discussion about this topic, since it’s possibly quite important.

Introduction

For the past two years, the only place I ate meat was at home with my family. As of October 2012, I've finally stopped eating meat altogether and can't see a reason why I would want to go back to eating meat. This kind of attitude toward eating is commonly classified as "vegetarianism" where one refrains from eating the flesh of all animals, including fish, but still will consume animal products like eggs and milk (though I try to avoid egg as best I can).

Why might I want to do this? And why might I see it as a serious issue? It's because I'm very concerned about the reality of suffering done to our "food animals" in the process of making them into meat, because I see vegetarianism as a way to reduce this suffering by stopping the harmful process, and because vegetarianism has not been hard at all for me to accomplish.

Animals Can Suffer

Back in the 1600s, Réné Descartes thought nonhuman animals were soulless automatons that could respond to their environment and react to stimuli, but could not feel anything — humans were the only species that were truly conscious. Descartes hit on an important point — since feelings are completely internal to the animal doing the feeling, it is impossible to demonstrate that anyone is truly conscious.

However, when it comes to humans, we don’t let that stop us from assuming other people feel pain. When we jab a person with a needle, no matter who they are, where they come from, or what they look like, they share a rather universal reaction of what we consider to be evidence of pain. We also extend this ...
low stress employment/ munchkin income thread
by  — 2013-07-23T21:22:27.583Z

TL;DR: this is a repository for discussing income generation strategies optimized for free time

I hope I'm not cluttering up LW but maybe enough people are also interested in this? I graduated high school about a year ago.

I have a lot in common with Will Newsome's self description in this post
http://lesswrong.com/lw/2qp/virtual_employment_open_thread/

But it's a dead thread, and there's been some interest in early retirement extreme, (http://earlyretirementextreme.com/) and having repositories for stuff.

The upshot of it is that I want to optimize for free time and mobility. Need about $2,000 to live (1600 expenses 400 savings/buffer) 2nd EDIT: no I don't, I must have screwed something up when I was adding this it's more like $1600. ($1300 to spend $300 buffer). A 20 hour workweek or even shorter is what I'm going for here. Right now I'm barely functional. Even that much is a bit of a stretch for me as I am now. Plenty of advice abounds on optimizing my health and squashing akrasia though, and I'm sure that if I implemented it I could get to the point of handling part time work. But I think I would always find being a 9 to 5er unappealing.

I'd value spending that time reading texbooks or walking around town or lazing around on the beach more than I'd value extra money. I'm also interested to hear about some more conventional part time jobs if they pay enough. I'm ok with doing somewhat boring work if the hours are light and I have time to think.

I've generated some candidate strategies if anyone here has experience at these. I don't have much knowledge of what they would entail or how to break into them. Or they might give someone some ideas I dunno but anyway:

4hww style dropship business (but success at that seems hard to set up and sustain)

freelance work at a site like odesk or elance

Own a popular app or forum

Push carts at wal mart part time (but I don't think that pays enough)

Self employment doing massage therapy (I can set my own hours but I'd need to invest time and money to get trained)

Tutoring (I might like this one. Do I need a college degree? Can I make enough with part time hours? Is it hard to find leads for clients? How would I do that?)

Online poker (but it seems kinda hard)

Does anyone here live in a yurt? And has anyone tried living in other countries to cut down expenses?

edited to add: Did I make a m...
Gains from trade: Slug versus Galaxy - how much would I give up to control you?
by Stuart_Armstrong — 2013-07-23T19:06:01.006Z

Edit: Moved to main at ThrustVectoring's suggestion.

A suggestion as to how to split the gains from trade in some situations.

The problem of Power

A year or so ago, people in the FHI embarked on a grand project: to try and find out if there was a single way of resolving negotiations, or a single way of merging competing moral theories. This project made a lot of progress in finding out how hard this was, but very little in terms of solving it. It seemed evident that the correct solution was to weigh the different utility functions, and then for everyone maximise the weighted sum, but all ways of weighting had their problems (the weighting with the most good properties was a very silly one: use the "min-max" weighting that sets your maximal attainable utility to 1 and your minimal to 0).

One thing that we didn't get close to addressing is the concept of power. If two partners in the negotiation have very different levels of power, then abstractly comparing their utilities seems the wrong solution (more to the point: it wouldn't be accepted by the powerful party).

The New Republic spans the Galaxy, with Jedi knights, battle fleets, armies, general coolness, and the manufacturing and human resources of countless systems at its command. The dull slug, ARthUrpHilIpDenu, moves very slowly around a plant, and possibly owns one leaf (or not - he can't produce the paperwork). Both these entities have preferences, but if they meet up, and their utilities are normalised abstractly, then ARthUrpHilIpDenu's preferences will weigh in far too much: a sizeable fraction of the galaxy's production will go towards satisfying the slug. Even if you think this is "fair", consider that the New Republic is the merging of countless individual preferences, so it doesn't make any sense that the two utilities get weighted equally.

The default point

After looking at various blackmail situations, it seems to me that it's the concept of default, or status quo, that most clearly differentiates between a threat and an offer. I wouldn't want you to make a credible threat, because this worsens the status quo, I would want you to make a credible offer, because this improves it. How this default is established is another matter - there may be some super-UDT approach that solves it from first principles. Maybe there is some deep way of distinguishing between threats and pro...
MIRI's 2013 Summer Matching Challenge
by lukeprog — 2013-07-23T19:05:56.873Z

(MIRI maintains Less Wrong, with generous help from Trike Apps, and much of the core content is written by salaried MIRI staff members.)

Update 09-15-2013: The fundraising drive has been completed! My thanks to everyone who contributed.

The original post follows below...

Thanks to the generosity of several major donors,† every donation to the Machine Intelligence Research Institute made from now until (the end of) August 15th, 2013 will be matched dollar-for-dollar, up to a total of $200,000!

Donate Now!

Now is your chance to double your impact while helping us raise up to $400,000 (with matching) to fund our research program.

This post is also a good place to ask your questions about our activities and plans — just post a comment!

If you have questions about what your dollars will do at MIRI, you can also schedule a quick call with MIRI Deputy Director Louie Helm: louie@intelligence.org (email), 510-717-1477 (phone), louiehelm (Skype).

Early this year we made a transition from movement-building to research, and we've hit the ground running with six major new research papers, six new strategic analyses on our blog, and much more. Give now to support our ongoing work on the future's most important problem.

Accomplishments in 2013 so far

Released six new research papers: (1) Definability of Truth in Probabilistic Logic, (2) Intelligence Explosion Microeconomics, (3) Tiling Agents for Self-Modifying AI, (4) Robust Cooperation in the Prisoner's Dilemma, (5) A Comparison of Decision Algorithms on Newcomblike Problems, and (6) Responses to Catastrophic AGI Risk: A Survey.

Held our 2nd and 3rd research workshops.

Published six new analyses to our blog: The Lean Nonprofit, AGI Impact Experts and Friendly AI Experts, Five Theses..., When Will AI Be Created?, Friendly AI Research as Effective Altruism, and What is Intelligence?

Published the Facing the Intelligence Explosion ebook.

Published several other substantial articles: Recommended Courses for MIRI Researchers, Decision Theory FAQ, A brief history of ethically concerned scientists, Bayesian Adjustment Does Not Defeat Existential Risk Charity, and others.

Published our first three expert interviews, with James Miller, Roman Yampolskiy, and Nick Beckstead.

Launched our new website at intelligence.or

...
Meetup : Cleveland Ohio Meetup
by Raelifin — 2013-07-23T18:51:00.380Z

Discussion article for the meetup : Cleveland Ohio Meetup

WHEN:
28 July 2013 02:00:00PM (-0400)

WHERE:
Gypsy Bean Coffee, Cleveland, OH

Time and place are tentative. Discussion is occurring on the cleveland mailing list: less-wrong-cleveland@googlegroups.com

Discussion article for the meetup : Cleveland Ohio Meetup
Meetup : [Boston] Becoming Stronger
by ModusPonies — 2013-07-23T14:54:28.700Z

Discussion article for the meetup : [Boston] Becoming Stronger

WHEN: 28 July 2013 02:00:00PM (-0400)

WHERE: 25 Ames St, Cambridge, MA

What are you currently doing to make yourself a better person? Are you overcoming mental hurdles, improving your skills, or trying new things? We'll break into small groups and talk about our current projects in as much detail as is comfortable. This is an opportunity to get feedback on your plans and take useful ideas from other people.

Cambridge/Boston-area Less Wrong meetups are every Sunday at 2pm in MIT's building 66 at 25 Ames St, room 156. Room number subject to change based on availability; signs will be posted with the actual room number.

Our default schedule is as follows:

—Phase 1: Arrival, greetings, unstructured conversation.

—Phase 2: The headline event. This starts promptly at 2:30, and lasts 30-60 minutes.

—Phase 3: Further discussion. We'll explore the ideas raised in phase 2, often in smaller groups.

—Phase 4: Dinner. It's about a ten minute walk to the usual restaurant.

Discussion article for the meetup : [Boston] Becoming Stronger
LINK: TCP and decision theory
by Stuart_Armstrong — 2013-07-23T13:53:58.797Z

Via Anders Sandberg

MIT is using genetic algorithms to optimise the TCP protocol:

http://web.mit.edu/remy/

http://web.mit.edu/remy/TCPexMachina.pdf

It seems they are trying to achieve Hofstadterian superrationality:

Why “Remy”?

In his Metamagical Themas columns in Scientific American, Douglas Hofstadter laid out a decision-theoretic framework of "superrationality." Traditional game theory looks for “rational” solutions where each player's decision is optimal after fixing that of the other players. When such a solution converges, it is a Nash equilibrium. But a Nash equilibrium may be an inefficient or unfair one. For example, in the case of the prisoner's dilemma, both prisoners are better off defecting no matter what the other prisoner does. Mutual defection is therefore the Nash equilibrium, even though each player would benefit if both cooperated instead.

Superrationality is an alternate framework where instead of fixing the decision of just the other players, each player assumes that the other players are as rational as he is, and those players make the same assumption in turn, etc. Because all players are superrational, if their positions are identical, each player can infer that the others will settle on the same strategy. The question for each player is: given that we are all going to reason similarly and end up with the same strategy, what should that strategy be?

Remy solves the problem of finding the superrational TCP for a given (uncertain or randomly-drawn) network, where all endpoints are assumed to run the same algorithm, and the question is simply what that algorithm should be to maximize each endpoint's expected value of the objective function.

In writing a program to search for a superrational TCP, we called the system Remy—the main character from the movie Ratatouille.
Low-hanging fruit: improving wikipedia entries
by LanceSBush — 2013-07-23T13:14:57.071Z

Many people are likely stumble across the Wikipedia entry for topics of interest relevant to those of us who frequent LessWrong: rationality, artificial intelligence, existential risks, decision theory, etc. These pages often shape one’s initial impressions of how interesting, important, or even credible a given topic is, and may have the potential to direct people towards productive resources (reading material, organizations like CFAR, notable figures such as Eliezer, etc.). As a result, ensuring that the Wikipedia entries on these topics are of better quality than some of them presently are presents an opportunity for investing relatively little effort in an activity with potentially substantial payoffs relative to the cost of time and effort put in.

I have already decided to improve some of the pages, beginning with the rather sloppy page that’s currently serving as the entry for existential risks, though of course others are welcome to contribute and may be more suited to the task than I am:

https://en.wikipedia.org/wiki/Risks_to_civilization,_humans,_and_planet_Earth

If you look at the section on risks posed by AI, for instance, it's notably inadequate, while the page includes a bizarre section referencing Mayan doomsday forecasts and Newton's predictions about the end of the world, neither of which seem adequately distinguished from rigorous attempts to actually assess legitimate existential risks.

I’m also constructing a list of other pages that are or are potentially in need of updating it and organizing it by my rough estimates of their relative importance (which I’m happy to share, modify, or discuss).

Turning this into a collaborative effort would be far more effective than doing it myself. If you think this is a worthwhile project and want to get involved I’d definitely like to hear from you and figure out a way to best coordinate our efforts.