trentmkelly/LessWrong-43k huggingface dataset:
Meetup : Denver Area LW June Meetup
by PonchoPal — 2017-05-17T00:14:54.128Z

Discussion article for the meetup : Denver Area LW June Meetup

WHEN:
06 June 2017 07:00:00PM (-0600)

WHERE:
4955 South Ulster Street #103, Denver, CO

Casual Meetup in the back room of Darcy's Pub or outside, if the weather is nice and we can have the space. You can find us by asking for "Steven Zuber's Group.

Discussion article for the meetup : Denver Area LW June Meetup
Acausal trade: conclusion: theory vs practice
by Stuart_Armstrong — 2017-05-16T19:33:51.000Z

When I started this dive into acausal trade, I expected to find subtle and interesting theoretical considerations. Instead, most of the issues are practical.

Theory

The big two theoretical questions are whether we model infinite worlds with infinitely many agents, and whether we should agree to some 'pre-existence' deal with all agents, including those that don't and cannot exist. We lay aside the infinite case for the time being; pre-existence deals simply lead to all agents maximising a single utility joint function. There are many issues with that - why would the agents accept a deal that gives them nothing at the moment they accept it, how can the agents share a common prior, how much effort are they required to make to not deal with logically impossible agents, and so on - but it's a possible option.

Practice

Without prexistence deals, then the situation is not hard to model, and though practical issues seems to dominate acausal trade. There is the perennial issue of how to divide gains from trade and how to avoid extortion. There is a "Double decrease": when an acausal trade network has fewer contributors, then those contributors also contribute less (since they derive lower advantage from doing so), compounding the decrease (and a converse result for larger trade networks).

There are many reasons an acausal trade network could be smaller. All agents could be unusual and distinct, making it almost impossible to figure out what agents actually exist. The different utilities could fail to be compatible in various ways. The agent's decision algorithms and concepts of fairness could be incompatible. And many agents could be deliberately designed to not engage in acausal trade.

Against that all, the number N of potential agents could be so absurdly high that a lot of acausal trade happens anyway. This is probably necessary, to compensate for the extreme guesswork that goes into acausal trade: all the other agents exist only in our heads. Trade is still possible with such agents, but we shouldn't forget our potential biases and errors when we attempt that estimation.

Scott's example

The only major detailed example I know of that illustrates acausal trade, is Scott's example here. There an AI that realises it's likely not the first AI, and attempts to surrender by simulating the reaction of a potential earlier AI.

Note that this is not acausal, it's an acausal-

...
Reaching out to people with the problems of friendly AI
by Val — 2017-05-16T19:30:49.689Z

There have been a few attempts to reach out to broader audiences in the past, but mostly in very politically/ideologically loaded topics.

After seeing several examples of how little understanding people have about the difficulties in creating a friendly AI, I'm horrified. And I'm not even talking about a farmer on some hidden ranch, but about people who should know about these things, researchers, software developers meddling with AI research, and so on.

What made me write this post, was a highly voted answer on stackexchange.com, which claims that the danger of superhuman AI is a non-issue, and that the only way for an AI to wipe out humanity is if "some insane human wanted that, and told the AI to find a way to do it". And the poster claims to be working in the AI field.

I've also seen a TEDx talk about AIs. The talker didn't even hear about the paperclip maximizer, and the talk was about the dangers presented by the AIs as depicted in the movies, like the Terminator, where an AI "rebels", but we can hope that AIs would not rebel as they cannot feel emotion, so we should hope the events depicted in such movies will not happen, and all we have to do is for ourselves to be ethical and not deliberately write malicious AI, and then everything will be OK.

The sheer and mind-boggling stupidity of this makes me want to scream.

We should find a way to increase public awareness of the difficulty of the problem. The paperclip maximizer should become part of public consciousness, a part of pop culture. Whenever there is a relevant discussion about the topic, we should mention it. We should increase awareness of old fairy tales with a jinn who misinterprets wishes. Whatever it takes to ingrain the importance of these problems into public consciousness.

There are many people graduating every year who've never heard about these problems. Or if they did, they dismiss it as a non-issue, a contradictory thought experiment which can be dismissed without a second though:

A nuclear bomb isn't smart enough to override its programming, either. If such an AI isn't smart enough to understand people do not want to be starved or killed, then it doesn't have a human level of intelligence at any point, does it? The thought experiment is contradictory.

We don't want our future AI researches to start working with such a mentality.

What can we do to raise awareness? We don't have th...
Acausal trade: being unusual
by Stuart_Armstrong — 2017-05-16T18:38:43.000Z

Other posts in the series: Introduction, Double decrease, Pre-existence deals, Full decision algorithms, Breaking acausal trade, Trade in different types of utility functions, Being unusual, and Summary.

It's important to remember that, in acausal trade, the other agents truly know nothing about you. They have some statistical image about the initial conditions throughout the universe, the probability of life evolving, they've maybe run some simulations: but they truly have nothing beyond that. Nor do you have any idea about them.

What that means is that if you are a particularly unusual entity, then no causally disconnected agent knows that. You can't update them on the fact that you exist, and that they should therefore trade more with you.

(Or maybe you're an agent of type A1 and agents of type A1 are either almost infinitely rare (with high probability) or somewhat more common, independently of other agent types. Based on the fact that you exist, you can update towards the fact that A1 are more common - but only A1-type agents will be able to do this update, so those are the only agents you'll have much trade with.)

So unusual agents are less likely to benefit from acausal trade, but they're also less pressured to follow it. Suppose that there are Ω humans, all maximising the same u, and one will be designated utility-dictator.

By shear coincidence, your decision theory is disconnected from that of the rest of the humans, and you've been designated utility dictator.

Any causally separated agent doesn't know this, and so you can reach your decision while making an 1/Ω difference to how they deal with humanity. In many cases, it can be rational to simply defect from acausal trade entirely.

If you have a different utility function from the other humans, all the more reason to defect, because you get almost no benefit from cooperation.

Of course, a sensible human community will resolve its internal differences before designating a utility-dictator. And the community can't plan to benefit from this loophole, since that would mean you being designated is not so unlikely after all.

Things might be different if you think you might be in a simulation, but unusual agents aren't very informative in simulations either.

Seeing all the other potential barriers and difficulties with acausal trade, it possible, if every existing agent is unusual enough and have distinct enough ut

...
The robust beauty of improper linear models
by Stuart_Armstrong — 2017-05-16T15:06:26.094Z

It should come as no surprise to people on this list that models often outperform experts. But these are generally finely calibrated models, integrating huge amounts of data, so this seems less surprising. How can the poor experts compete against that?

But sometimes the models are much simpler than that, and still perform better. For instance, the models could be linear, rather than having higher order complexities. These models can still outperform experts, because in practice, despite their beliefs that they are doing a non-linear task, expert decisions can often best be modelled as being entirely linear.

But surely the weights of the linear models are subtle and need to be set exactly? Not really. It seems that if you take a linear model, and weigh the variables by +1 or -1 depending on whether it has a positive or negative impact on the result, then you will get a model that still often outperforms experts. These models with ±1 weights are called improper linear models.

What's going on here? Well, there's been a bit of a dodge. I've been talking about "taking" a linear model, with "variables", and weighing the factors depending on a positive or negative "impact". And to do all that, you need experts. They are the ones that know which variables are important, and know the direction (positive or negative) in which they impact the result. They don't choose these variables by just taking random possibilities and then figuring out what the direction is. Instead, they understand the situation, to some extent, and choose important variables.

So that's the real role of the expert here: knowing what should go into the model, what really makes the underlying dependent variable change. Selecting and coding the variable information, in the terms that are often used.

But, just as experts can be very good at that task, they are human, and humans are terrible at integrating lots of information together. So, having selected the variables, they get regularly outperformed by proper linear models. And when you add the fact that the experts have selected variables of comparable importance, and that these variables are often correlated with each other, it's not surprising that they get outperformed by improper linear models as well.
Are causal decision theorists trying to outsmart conditional probabilities?
by Caspar Oesterheld — 2017-05-16T08:01:27.426Z

Presumably, this has been discussed somewhere in the past, but I wonder to which extent causal decision theorists (and many other non-evidential decision theorists, too) are trying to make better predictions than (what they think to be) their own conditional probabilities.

To state this question more clearly, let’s look at the generic Newcomb-like problem with two actions a1 and a2 (e.g., one-boxing and two-boxing, cooperating or defecting, not smoking or smoking) and two states s1 and s2 (specifying, e.g., whether there is money in both boxes, whether the other agent cooperates, whether one has cancer). The Newcomb-ness is the result of two properties:

No matter the state, it is better to take action a2, i.e. u(a2,s1)>u(a1,s1) and u(a2,s2)>u(a1,s2). (There are also problems without dominance where CDT and EDT nonetheless disagree. For simplicity I will assume dominance, here.)

The action cannot causally affect the state, but somehow taking a1 gives us evidence that we’re in the preferable state s1. That is, P(s1|a1)>P(s1|a2) and u(a1,s1)>u(a2,s2).

Then, if the latter two differences are large enough, it may be that

E[u|a1] > E[u|a2].

I.e.

P(s1|a1) * u(s1,a1) + P(s2|a1) * u(s2,a1) > P(s1|a2) * u(s1,a2) + P(s2|a2) * u(s2,a2),

despite the dominance.

Now, my question is: After having taken one of the two actions, say a1, but before having observed the state, do causal decision theorists really assign the probability P(s1|a1) (specified in the problem description) to being in state s1?

I used to think that this was the case. E.g., the way I learned about Newcomb’s problem is that causal decision theorists understand that, once they have said the words “both boxes for me, please”, they assign very low probability to getting the million. So, if there were a period between saying those words and receiving the payoff, they would bet at odds that reveal that they assign a low probability (namely P(s1,a2)) to money being under both boxes.

But now I think that some of the disagreement might implicitly be based on a belief that the conditional probabilities stated in the problem description are wrong, i.e. that you shouldn’t bet on them.

The first data point was the discussion of CDT in Pearl’s Causality. In sections 1.3.1 and 4.1.1 he emphasizes that he thinks his do-ca...
Repersonalizing the High Status
by squidious — 2017-05-16T07:00:00.000Z

In studying dissociative disorders, I came upon depersonalization, a symptom where someone feels detached from themselves, either mentally or physically. The world begins to feel dreamlike, or less real. For someone experiencing depersonalization, this may feel the case all the time, whereas a highly anxious person may feel familiar with this state if, for example, they have social anxiety and freeze up around people they don't know very well. But even less anxious people have spoken of seeing themselves in third person or feeling like time has gotten slower when anticipating embarrassment, especially in front of someone they want to impress. In these cases, we are not only feeling separate from ourselves, but we are also detaching the public face from the real person in front of us.
When I was in college, I was part of the local poetry scene. I remember standing in line waiting to get in to a show, when a hooded figure jumped out of the van parked out front, took a picture of my bag, and disappeared back into the van before I could say anything. After the show, I found out that this had been none other than featured poet Buddy Wakefield, world poetry slam champion, and one of my personal favorites in the world of performance poetry. He apologized for startling me earlier, and asked me a question about my bag, which had a squid on it. Turns out, he was a fan of squids.
I don't remember exactly what I said (some stupid comment, I think, about squids being like sea ninjas). What I do remember is feeling almost like I had left my body. All I could think about was how amazing it was that Buddy Wakefield liked my bag, that we were both into squids, that he seemed so down to earth, so humble, for a world champion. But in the process, I lost what would have been a good opportunity for connection. If a fellow show-goer had expressed interest in squids, I probably would have asked them what made them interested in squids, if they had a favorite species, or if they'd seen any at the local aquarium. Because Buddy Wakefield was someone I saw as famous, high status, I was less curious about his experience as a person, and in a sense, I depersonalized him.
It turns out, high status people have a lot of conversations with fans who see them as more of a figure than a person. Picture the host of a big event, surrounded by people commenting on the food and decor. Less often woul

...
Meetup : Sydney Rationality Dojo - June 2017
by luminosity — 2017-05-15T22:20:29.171Z

Discussion article for the meetup : Sydney Rationality Dojo - June 2017

WHEN:
04 June 2017 04:00:00PM (+1000)

WHERE:
10 Shepherd Street, Chippendale

Join us for another chance to polish your planning, motivational and other rationality skills. Social dinner afterwards.

Discussion article for the meetup : Sydney Rationality Dojo - June 2017
Value Learning for Irrational Toy Models
by orthonormal — 2017-05-15T20:55:05.000Z

(This is a half-formed idea from discussions within MIRI; if it's dumb, I take the full blame.)

In value learning contexts, it's useful to have a toy model of human psychology, to see (for example) if a certain approach would work to learn the values of an idealized rational agent, but might robustly fail when faced with a more realistically constructed agent.

For example, here is a toy model of an irrational agent: take a world where the deterministic mapping f from actions to outcomes is fully known, and take three different preference orderings on the set of outcomes. When the agent chooses among actions a1,…,an, we first check whether any f(ai) dominates the others according to at least two of the preference orderings; if so, we take that action with certainty. Otherwise, we select randomly among the available actions. (This agent is a moral democracy, and if two of the three subagents agree on a policy, that policy is taken; otherwise, the agent hits a deadlock and acts at random.)

It is easy to construct agents of this form which exhibit circular preferences in binary choices. We can therefore ask whether a particular value learning algorithm would satisfy sensible desiderata when learning from such an agent. (For instance, if outcome X strictly dominates outcome Y according to all three preference orderings, we might desire that our value learning algorithm not act so as to result in Y when it could instead have acted so as to result in X.)

The Hard Problem of Value Learning

But of course, a human brain is not even as simple as that toy model of irrationality. I've thought it might be useful to sketch out the level of generality that I actually believe is necessary, in order to show how hard the problem may be to get right.

Human brains do some amount of consequentialist reasoning [citation needed], so arguably at some point of cognition there exist heuristics for evaluating the overall desirability of various outcomes. We would like our value learning process to infer these heuristics and take them into account (this seems necessary, not sufficient).

We cannot assume that the human will take actions that effectively argmax these heuristics (though it will strongly correlate in some regime); we cannot assume whether these heuristics give us values for states, or for action-state pairs; we cannot assume that these heuristics make use of all the important in

...
Meetup : San Francisco Meetup: Reciprocity Circle
by rocurley — 2017-05-15T18:12:57.450Z

Discussion article for the meetup : San Francisco Meetup: Reciprocity Circle

WHEN:
15 May 2017 06:15:00PM (-0700)

WHERE:
1769 15th St, San Francisco, CA 94103-3333, United States

We’re trying a new-ish format this week - a variant on our usual topic of Group Debugging.

This time we’ll go around and have each person list 1-3 things they wish someone could help them with, and then have others (hopefully) volunteer to help them. Once we’ve gone around the circle, we break to discuss with the people we are helping or being helped by.

For help getting into the building, please call (or text, with a likely-somewhat-slower response rate): 301-458-0764.

Format:

We meet and start hanging out at 6:15, but don’t officially start doing the meetup topic until 6:45-7 to accommodate stragglers. Usually there is a food order that goes out before we start the meetup topic.

About these meetups:

The mission of the SF LessWrong meetup is to provide a fun, low-key social space with some structured interaction, where new and non-new community members can mingle and have interesting conversations. Everyone is welcome.

We explicitly encourage people to split off from the main conversation or diverge from the topic if that would be more fun for them (moving side conversations into a separate part of the space if appropriate). Meetup topics are here as a tool to facilitate fun interaction, and we certainly don’t want them to inhibit it.

Discussion article for the meetup : San Francisco Meetup: Reciprocity Circle
A Month's Worth of Rational Posts - Feedback on my Rationality Feed.
by sapphire — 2017-05-15T14:21:32.995Z

For the last two months I have been publishing a feed of rationalist articles. Oriignally the feed was only published on the SSC discord channel SSC Discord (Be charitable, kind and don't treat the place like 4chan). For the last few days I have also been publishing it on my blog deluks917.wordpress.com. I categorize the links and include a brief excerpt, review, and/or teaser. If you would like to see an exampel in practice just check today's post. The average number of links per day, in the last month, has been six. But this number has been higher recently. I have not missed a single day since I started, so I think its likely I will continue doing this. The list of blogs I check is located here: List of Blogs.

I am looking for some feedback. At the bottom of this post I am including a month's worth of posts categorized using the current system. Posts are not nescessarily in any particular order since my categorization system has not been constant over time. Lots of posts were moved around by hand.

1 - Should I share the feed the results somewhere other than SSC-discord + my blog? Mindlevelup suggested I write up a weekly roundup. I could share such a roundup via some on lesswrong and SSC. I would estimate the expected number of links in such a psot to be around 35. Links would be posted in chronoligcal order within categories. Alternatively I could share such a post every two weeks. Its also possible to have a mailing list but I currently find this less promising.

2 - Do the categories make a reasonable amount of sense? What tweaks would you make. I have ocnsidered mixing some of the smaller categories (Math and CS, Amusement into "misc").

3 - Are there any blogs I should include/drop from the feed. For example I have been considering dropping ribbonfarm. The highest priority is to get the content thats directly about instrumental/epsitemic rationality. The bar is higher for politics and culture_war. I should note I am not going to personally include any blog without an RSS feed.

4 - Is anyone willing to write a "Best of rationalist tumblr" post. If I write a weekly/bi-weekly round up I could combine it with an equivalent "best of tumblr" post. The tumblr post would not have to be daily, just weekly or every other week. We could take turns posting the resulting combination to lesswrong/SSC and collecting the juicy ...