trentmkelly/LessWrong-43k huggingface dataset:
The Pavlov Strategy
by sarahconstantin — 2018-12-20T16:20:00.542Z

Epistemic Status: Common knowledge, just not to me

The Evolution of Trust is a deceptively friendly little interactive game. Near the end, there’s a “sandbox” evolutionary game theory simulator. It’s pretty flexible. You can do quick experiments in it without writing code. I highly recommend playing around.

One of the things that surprised me was a strategy the game calls Simpleton, also known in the literature as Pavlov. In certain conditions, it works pretty well — even better than tit-for-tat or tit-for-tat with forgiveness.

Let’s set the framework first. You have a Prisoner’s dilemma type game.

If both parties cooperate, they each get +2 points.

If one cooperates and the other defects, the defector gets +3 points and the cooperator gets -1 point

If both defect, both get 0 points.

This game is iterated — you’re randomly assigned to a partner and you play many rounds. Longer rounds reward more cooperative strategies; shorter rounds reward more defection.

It’s also evolutionary — you have a proportion of bots each playing their strategies, and after each round, the bots with the most points replicate and the bots with the least points die out. Successful strategies will tend to reproduce while unsuccessful ones die out. In other words, this is the Darwin Game.

Finally, it’s stochastic — there’s a small probability that any bot will make a mistake and cooperate or defect at random.

Now, how does Pavlov work?

Pavlov starts off cooperating. If the other player cooperates with Pavlov, Pavlov keeps doing whatever it’s doing, even if it was a mistake; if the other player defects, Pavlov switches its behavior, even if it was a mistake.

In other words, Pavlov:

cooperates when you cooperate with it, except by mistake

“pushes boundaries” and keeps defecting when you cooperate, until you retaliate

“concedes when punished” and cooperates after a defect/defect result

“retaliates against unprovoked aggression”, defecting if you defect on it while it cooperates.

If there’s any randomness, Pavlov is better at cooperating with itself than Tit-For-Tat. One accidental defection and two Tit-For-Tats are stuck in an eternal defect cycle, while Pavlov’s forgive each other and wind up back in a cooperate/cooperate pattern.

Moreover, Pavlov can exploit CooperateBot (if it defects by accident, it will keep greedily defecting against the hapless CooperateBot, while Tit-For-Tat wil...
What self-help has helped you?
by Gordon Seidoh Worley — 2018-12-20T03:31:52.497Z

Simply, what self-help techniques have you tried and found to help you? Bonus points if you can say something about the context in which you did the self-help and any speculations you have about mechanism of action.

Please give one technique per answer, multiple answers accepted per person (at least, I hope LW allows that, but I'm guessing it will), please read existing answer to avoid duplication, and comment on existing answers about a technique if you have more to say about that technique.

I'll put a couple of my own answers in below as examples and because I have answers to my own question!
Defining Freedom
by pku — 2018-12-20T02:41:38.865Z

I’ve always found the concept of Freedom confusing.

There’s a level on which it makes sense. When William Wallace is talking about being free from the yoke of Edward Longshanks, it’s obvious what he means – the English king and his goons regularly come to town, order people around, take their stuff and beat them up. Being free of English rule means they don’t do that anymore, and William Wallace and his friends can just go around farming and living their peaceful lives without having to worry about anything worse than the soul-crushing depression of living in Scotland.

On the other hand, what the heck does the word free mean in the context of “the land of the free and the home of the brave”? Is there even a reasonable definition? The original song is about being free from English rule, but it’s been two and a half centuries since Edward Longshanks and his goons were ruling America, and not-being-ruled-by-the-King isn’t really a core property of most Americans’ identities these days.

I settle the first one by thinking of freedom as defined relative to a constraint. You can be free of something if you don’t have to worry about it when making decisions. This matches the common use of freedom – you’re free of having to worry about parking if you don’t have a car, free of a tyrant if you aren’t constrained by him telling you what to do, etc. This also explains why the second use seems so weird – it’s trying to use a fundamentally relative term without using it in relation to anything. So my response to the second use used to be to roll my eyes at people throw around empty deep-sounding terms.

But now I think we can resolve this. We start with the mathematical definition of degrees of freedom – your total freedom level at a given time is the number of options you have available to you at that time (if you want to sound all mathy, you can call this the local dimension of your options space or something)[1].

But there’s a fundamental problem here: At the end of the day you’ll only pick one of the options you had, because you can only pick one – once you eliminate the big constraints, you’ll just be subject to smaller and smaller constraints until you’re down to one option. Even if there’s no law forcing you to wear black socks, you’ll end up wearing black socks by the constraint that they were a dollar cheaper on Amazon and you were too lazy to scroll down. If you actu...
Reasons compute may not drive AI capabilities growth
by Tristan H — 2018-12-19T22:13:34.474Z

How long it will be before humanity is capable of creating general AI is an important factor in discussions of the importance of doing AI alignment research as well as discussions of which research avenues have the best chance of success. One frequently discussed model for estimating AI timelines is that AI capabilities progress is essentially driven by growing compute capabilities. For example, the OpenAI article on AI and Compute presents a compelling narrative, which shows a trend of well-known results in machine learning using exponentially more compute over time. This is an interesting model because if valid we can do some quantitative forecasting, due to somewhat smooth trends in compute metrics which can be extrapolated. However, I think there are a number of reasons to suspect AI progress to be driven more by engineer and researcher effort than compute.

I think there's a spectrum of models between:

We have an abundance of ideas that aren't worth the investment to try out yet. Advances in compute capability unlock progress by make researching more expensive techniques economically feasible. We'll be able to create general AI soon after we have enough compute to do it.

Research proceeds at its own pace and makes use of as much compute is convenient to save researcher time on optimization and achieve flashy results. We'll be able to create general AI once we come up with all the right ideas behind it, and either:

We'll already have enough compute to do it

We won't have enough compute and we'll start optimizing, invest more in compute, and possibly start truly being bottlenecked on compute progress.

My research hasn't pointed too solidly in either direction, but below I discuss a number of the reasons I've thought of that might point towards compute not being a significant driver of progress right now.

There's many ways to train more efficiently that aren't widely used

Starting October of 2017, the Stanford DAWNBench contest challenged teams to come up with the fastest and cheapest ways to train neural nets to solve certain tasks.

The most interesting was the ImageNet training time contest. The baseline entry took 10 days and cost $1112; less than one year later the best entries (all by the fast.ai team) were down to 18 minutes for $35, 19 minutes for $18 or 30 minutes for $14[^1]. This is ~800x faster and ~80x cheaper than the baseline.

Some of this w

...
Anthropic paradoxes transposed into Anthropic Decision Theory
by Stuart_Armstrong — 2018-12-19T18:07:42.251Z

Anthropic Decision Theory (ADT) replaces anthropic probabilities (SIA and SSA) with a decision theory that doesn't need anthropic probabilities to function. And, roughly speaking, ADT shows that total utilitarians will have a behaviour that looks as if it was using SIA, while average utilitarians look like they are using SSA.

That means that the various paradoxes of SIA and SSA can be translated into ADT format. This post will do that, and show how the paradoxes feel a lot less counter-intuitive under ADT. Some of these have been presented before, but I wanted to gather them in one location. The paradoxes examined are:

The Doomsday Argument.

The Adam and Eve problem.

The UN++ problem.

The Presumptuous Philosopher.

Katja Grace's SIA doomsday argument.

The first three are are paradoxes of SSA (which increases the probability of "small" universes with few observers), while the last three are paradoxes of SIA (which increases the probability of "large" universes with many observers).

No Doomsday, just a different weighting of rewards

The famous Doomsday Argument claims that, because of SSA's preferences for small numbers of observers, the end of the human species is closer than we might otherwise think.

How can we translate that into ADT? I've found it's generally harder to translate SSA paradoxes into ADT that SIA ones, because average utilitarianism is a bit more finicky to work with.

But here is a possible formulation: a disaster may happen 10 years from now, with 50% probability, and will end humanity with a total of ω humans. If humans survive the disaster, there will be Ω humans total.

The agent has the option of consuming X resources now, or consuming Y resources in 20 years time. If this were a narrow-minded selfish agent, then it will consume early if X>Y/2, and late if X<Y/2.

However, if the agent is an average utilitarian, the amount of expected utility they derive from from consuming early is (X/ω+X/Ω)/2 (the expected average utility of X, averaged over survival and doom), while the expected utility for consuming late is (Y/Ω)/2 (since consuming late means survival).

This means that the breakeven point for the ADT average utilitarian is when:

Y=X(1+Ω/ω).

If Ω is much larger than ω, then the ADT agent will only delay consumption if Y is similarly larger than X.

This looks like a narrow-minded selfish agent that is convinced that doom is almost

...
Solstice Album Crowdfunding
by Raemon — 2018-12-18T20:51:31.183Z

For several years, people have been singing songs at Solstice, and some of those songs have been recorded, but always as a live performance which means the quality isn't as good and the musicians are recording-in-one-take-that-includes-all-mistakes.

So, I'm finally putting together a professional Solstice Album. I recorded an initial 12 songs for $8,000 (paying for musicians and studio time), recently recorded another few songs for $2,000. Ideally I'd like at least a bit more studio time to layer in some more vocal tracks so that 2/3 of the songs aren't just me singing.

The IndieGogo campaign is here. Since I've already spent a bunch of money, it's a flexible fund campaign (however much we raise, the album will still come out, and I'll get reimbursed or whatever people put it). But the more we raise the higher quality the songs will be end up being and the more likely I'll be to take on similar projects in the future.

If you choose the Early Access tier and I know who you are (in particular, I have your email), I'll send you a link to the google drive folder with the rough cuts as soon as I can, rather than waiting till after the IndieGogo wraps up.

The Songs

These are the songs that are already recorded and guaranteed to be in the album. Some of them have links to snippets so that you can hear what the instrumentation is like:

Bold Orion
Bring the Light
Gather Round
Bitter Wind Lullaby
Chasing Patterns
Stardust
Time Wrote the Rocks
Do You Realize
Hymn to the Breaking Strain
Bitter Wind March
Brighter Than Today
Endless Light
Here and Now
Forever Young
Uplift
Five Thousand Years

There will be bonus tracks available at higher tiers that include rough recordings of some of the off-beat silly songs or songs that get suggested by people on the "suggest a song" tier.

The bonus tracks will also include instrumental versions of most songs so that people can sing along karaoke style, or use the backing-track for their own Solstice events if their community doesn't have musicians.

Suggest a Bonus Song

Higher level tiers include "Suggest a Bonus Song" and "Costly Signal a Bonus Song". These do not guarantee a particular bonus song, because things are expensive and the world is complicated and the campaign might not even recover the money to pay the existing expenses.

However, if there is a lot of support for a par...
18-month follow-up on my self-concept work
by Kaj_Sotala — 2018-12-18T17:40:03.941Z

About eighteen months ago, I found Steve Andreas’s book Transforming Your Self, and applied its techniques to fixing a number of issues in my self-concepts which had contributed to my depression and anxiety. Six weeks after those changes, I posted a report called “How I found & fixed the root problem behind my depression and anxiety after 20+ years”. I figured that by now it would be time for a follow-up on how those effects have lasted.

Overall summary and general considerations

Looking back, this was definitely a major milestone in improving my mental health. I feel like since 2014, I have been ongoing a process of completely transforming myself from the depression- and anxiety-ridden person who was convinced that he had no other option than becoming a total failure, to someone calmly confident who has the option of constructing his life to his taste. I don’t claim to be there yet, but I feel like I’m constantly getting closer. I feel like the self-concept work discussed in my post, was one of the largest engines powering this transition. (Other major ones being me getting antidepressants, changing how I thought about ethics, and learning a new mindset from CFAR in 2014, properly learning Focusing and Core Transformation as well as starting to meditate according to The Mind Illuminated system in 2017, and starting to apply Internal Family Systems this year.)

There are two difficulties evaluating my self-concept post afterwards. First is that I have a poor emotional memory, so it’s a little hard for me to remember what I felt before these changes. The second is that after doing self-concept work, I’ve also done plenty of other things, such as meditation and moving together with some housemates, which have also had a definite impact on my mental health. I can’t know how well the self-concept work would have stuck around, if I hadn’t also implemented those other changes. It’s possible and even likely that some of my current results are because of those other changes instead.

At the same time, the self-concept work is also not independent from everything that I’ve done later. For instance, I think that being able to eliminate the feelings of pointless shame has been a major reason why I’ve been able to live with housemates and find them a definite net positive. Previously the feelings of shame would have made it too draining to have to engage in social inte...
Good arguments against "cultural appropriation"
by Tyrrell_McAllister — 2018-12-18T17:23:52.900Z

[Originally posted to Facebook.]

I'm collecting steel-man arguments that the concept of "cultural appropriation" describes a real problem. Below are three arguments that seem somewhat reasonable to me in some cases. They seem to point to plausibly real costs of cross-cultural sharing and re-interpretation.

(Just to lay my cards on the table: I currently think that the benefits of cross-cultural sharing so often outweigh these cost that cultural appropriation as such should not be stigmatized.)

I'll abbreviate cultural appropriation as CA from here on out.

1. Some CA is taken to be a kind of mockery. Such CA is thought to result in diminished status and power for people in the "appropriated" culture. Alleged examples are team mascots, Halloween costumes, and Charlie Chan.

On the one hand, such arguments should be scrutinized skeptically and accepted only tentatively, because they rest on claims about difficult-to-measure effects on the vaguely defined status of amorphous social groups.

On the other hand, and for the same reason, you can't be certain that such claims are false. And you shouldn't just trust your own intuition on this kind of thing, because you don't have a bird's eye view of the entire complicated network of power and status that makes up our super-culture and all of its various subcultures. So it does make sense to listen to how other people think so-called CA affects their social standing.

2. Some CA amounts to diluting a piece of the cultural commons from which people in that culture were benefiting.

For example, people choose their clothes based on how they want to be seen by others. Tie-dye, for example, has a certain meaning in our culture. You wear tie-dye if you want people to see you in a certain way. But suppose that our culture found itself immersed in some larger surrounding culture, and people in the larger culture started wearing tie-dye without any knowledge of the whole system of sartorial signification within which tie-dye is embedded in our own culture. Now there's a bunch of people walking around wearing tie-dye who don't mean to signal what tie-dye signaled for us. As a result, tie-dye loses its signaling value for us. Having lost this signaling tool, we are that much poorer.

In some cultures, such markers of meaning are much more potent than they are in ours. So the loss of...
Experiences of Self-deception
by Bucky — 2018-12-18T11:10:26.965Z

It seems to me that self-deception can describe two different things - conscious and unconscious self-deception.

Sometimes the elephant believes something untrue all by itself without the rider ever getting a look in. The claims of elephant in the brain seem to focus on this type of unconscious self-deception.

At other times the rider is complicit in endorsing a particular known untrue belief. The elephant analyses a situation, determines what it is beneficial to believe and motivates the rider to believe this. The rider has access to information which indicates that this isn't true. If the rider brings this information to full attention then it is one of those rare occasions where he can override the elephant's desires. However the rider also has the option to push the information to the side and believe a beneficial lie. It is possible to do this well enough that the information is forgotten or completely overridden with new, inaccurate, information.

In pushing the information to the side, the rider can sometimes just never bring the information to full attention. Failing that, it can drown the information out by presenting other information (which agrees with its favoured interpretation) as loudly as possible in order to doubt/ignore/forget the information which it doesn't like.

At least, this is something I experience but I don't know whether other people do. I have a few examples where this has happened and have even experimented with allowing myself to start down the route of conscious self-deception to see what it feels like. To me it feels like cognitive dissonance (feeling hot, brain feeling "fuzzy", adrenaline kicking in) whilst the rider works on counteracting the information. I guess this would be followed by the relief of resolving said dissonance when the rider starts to believe the lie but I haven't experimented that far!

The literature appears to be understandably non-committal on whether the subjects are consciously aware of their self-deception - I guess that would be pretty hard to determine.

So my question is - do other people recognise this as something which happens to them? How would you describe the experience? Is it something which you've trained yourself to recognise when it starts?
2018 AI Alignment Literature Review and Charity Comparison
by Larks — 2018-12-18T04:46:55.445Z

Cross-posted to the EA forum.

Introduction

Like last year

and

the year before

, I’ve attempted to review the research that has been produced by various organisations working on AI safety, to help potential donors gain a better understanding of the landscape. This is a similar role to that which GiveWell performs for global health charities, and somewhat similar to an securities analyst with regards to possible investments. It appears that once again no-one else has attempted to do this, to my knowledge, so I've once again undertaken the task.

This year I have included several groups not covered in previous years, and read more widely in the literature.

My aim is basically to judge the output of each organisation in 2018 and compare it to their budget. This should give a sense for the organisations' average cost-effectiveness. We can also compare their financial reserves to their 2019 budgets to get a sense of urgency.

Note that this document is quite long, so I encourage you to just read the sections that seem most relevant to your interests, probably the sections about the individual organisations. I do not recommend you skip to the conclusions!

I’d like to apologize in advance to everyone doing useful AI Safety work whose contributions I may have overlooked or misconstrued.

Methodological Considerations

Track Records

Judging organisations on their historical output is naturally going to favour more mature organisations. A new startup, whose value all lies in the future, will be disadvantaged. However, I think that this is correct. The newer the organisation, the more funding should come from people with close knowledge. As organisations mature, and have more easily verifiable signals of quality, their funding sources can transition to larger pools of less expert money. This is how it works for startups turning into public companies and I think the same model applies here.

This judgement involves analysing a large number papers relating to Xrisk that were produced during 2018. Hopefully the year-to-year volatility of output is sufficiently low that this is a reasonable metric. I also attempted to include papers during December 2017, to take into account the fact that I'm missing the last month's worth of output from 2017, but I can't be sure I did this successfully.

This article focuses on AI risk work. If you think other causes are important...
Equivalence of State Machines and Coroutines
by Martin Sustrik — 2018-12-18T04:40:00.750Z

In the past I often referred to the equivalence between state machines and coroutines as a kind of obvious fact that doesn't need any additional explanation. It was brought to my attention, however, that that may not always be the case.

This article therefore doesn't attempt to express and deep and ground-breaking truth, rather, it illustrates the equivalence of finite state machines and coroutines using a practical example.

The example is stolen from Wikipedia's article on finite state machines:

A turnstile, used to control access to subways and amusement park rides, is a gate with three rotating arms at waist height, one across the entryway. Initially the arms are locked, blocking the entry, preventing patrons from passing through. Depositing a coin or token in a slot on the turnstile unlocks the arms, allowing a single customer to push through. After the customer passes through, the arms are locked again until another coin is inserted.

Wikipedia presents the state machine in question in the following manner:

And here's my attempt to rewrite the state machine as a coroutine in Go: