trentmkelly/LessWrong-43k huggingface dataset:
Two labyrinths - where would you rather be? 
by KyriakosCH — 2019-06-08T17:48:51.069Z

Two Kings and two Labyrinths is a very short story, written by J.L. Borges. It barely manages to fill the space of a single page; and yet there is enough in it to allow for an interesting dissertation.

The actual story is about a rivalry between two kings: The King of Babylon had once invited the King of Arabia to his capital, and there got him to enter a labyrinth made of intricate passages, surrounded by tall walls. The King of Arabia only managed to find his way out after imploring his God for help. The experience terrified him, and he swore that in the future he would repay the Babylonian in kind, by introducing him to another labyrinth; one particular to his native and desolate Arabian realm...

After his victory in war, the King of Arabia takes the King of Babylon hostage. He brings him to the desert, where, at the end of a three-day journey, he is abandoned. The desert is another kind of labyrinth. It has neither passages nor walls, but still finding one's way out of it is virtually impossible.

A Labyrinth is More Than Just a Prison Cell

A labyrinth isn’t just a structure which confines; it is one which serves the purpose of getting one disoriented. While a prison cell – regardless if it is nameless and obscure or one as famous as the stone vault in Sophocles’ play, Antigone, which was used to imprison the heroine and slowly drain her of the will to live – is just a simple room, enough to enclose, limit, and cause desperation, an actual labyrinth functions by allowing the person inside to still hope there is a chance of finding a way out... The labyrinth is different from a group of interconnecting cells, in that somewhere in it one may still discover a passage which will lead to liberation...

The possibility of finding the exit may be so small that, in practice, one wouldn't ever succeed in this quest... It's not important, though, because the very form of the labyrinth forces its prisoner to accept that there are always new routes to explore, or another idea to test; the progression from each part of the labyrinth to the next one may be quite monotonous, and almost reveal no change, but the prisoner inside is actually moving, is still progressing – and this allows for hope.

The Babylonian Labyrinth

The first of the labyrinths presented in the story is the one the reader would readily identify as a typical laby...
Asymmetric Weapons Aren't Always on Your Side
by Davis_Kingsley — 2019-06-08T08:47:49.675Z

Some time ago, Scott Alexander wrote about asymmetric weapons, and now he writes again about them. During these posts, Scott repeatedly characterizes asymmetric weapons as inherently stronger for the "good guys" than they are for the "bad guys". Here is a quote from his first post:

Logical debate has one advantage over narrative, rhetoric, and violence: it’s an asymmetric weapon. That is, it’s a weapon which is stronger in the hands of the good guys than in the hands of the bad guys.

And here is a quote from his more recent one:

A symmetric weapon is one that works just as well for the bad guys as for the good guys. For example, violence – your morality doesn’t determine how hard you can punch; they can buy guns from the same places we can.

An asymmetric weapon is one that works better for the good guys than the bad guys. The example I gave was Reason. If everyone tries to solve their problems through figuring out what the right thing to do is, the good guys (who are right) will have an easier time proving themselves to be right than the bad guys (who are wrong). Finding and using asymmetric weapons is the only non-coincidence way to make sustained moral progress.

One problem with this concept is that just because something is asymmetric doesn't mean that it's asymmetric in a good direction.

Scott talks about weapons that are asymmetric towards those who are right. However, there are many more types of asymmetries than just right vs. wrong - physical violence is asymmetric towards the strong, shouting people down is asymmetric towards the loud, and airing TV commercials is asymmetric towards people with more money. Violence isn't merely symmetric - it's asymmetric in a bad direction, since fascists are better than violence than you.

This in turn means that various sides will all be trying to pull things in directions that are asymmetric to their advantage. Indeed, a basic principle in strategy is to try to shift conflicts into areas where you are strong and your opponent is weak.

For instance, people who are good at violence benefit from things getting violent. People who are locally popular benefit from popularity contests. People who have lots of free time benefit from time-consuming processes. People who are better at keeping their composure benefit from discourse norms that punish displays of emotion.

Developing asymmetric ...
The Schelling Choice is "Rabbit", not "Stag"
by Raemon — 2019-06-08T00:24:53.568Z

Followup/distillation/alternate-take on Duncan Sabien's Dragon Army Retrospective and Open Problems in Group Rationality.

There's a particular failure mode I've witnessed, and fallen into myself:

I see a problem. I see, what seems to me, to be an obvious solution to the problem. If only everyone Took Action X, we could Fix Problem Z. So I start X-ing, and maybe talking about how other people should start X-ing. Action X takes some effort on my part but it's obviously worth it.

And yet... nobody does. Or not enough people do. And a few months later, here I'm still taking Action X and feeling burned and frustrated.

Or –

– the problem is that everyone is taking Action Y, which directly causes Problem Z. If only everyone would stop Y-ing, Problem Z would go away. Action Y seems obviously bad, clearly we should be on the same page about this. So I start noting to people when they're doing Action Y, and expect them to stop.

They don't stop.

So I start subtly socially punishing them for it.

They don't stop. What's more... now they seem to be punishing me.

I find myself getting frustrated, perhaps angry. What's going on? Are people wrong-and-bad? Do they have wrong-and-bad beliefs?

Alas. So far in my experience it hasn't been that simple.

A recap of 'Rabbit' vs 'Stag'

I'd been planning to write this post for years. Duncan Sabien went ahead and wrote it before I got around to it. But, Dragon Army Retrospective and Open Problems in Group Rationality are both lengthy posts with a lot of points, and it still seemed worth highlighting this particular failure mode in a single post.

I used to think a lot in terms of Prisoner's Dilemma, and "Cooperate"/"Defect." I'd see problems that could easily be solved if everyone just put a bit of effort in, which would benefit everyone. And people didn't put the effort in, and this felt like a frustrating, obvious coordination failure. Why do people defect so much?

Eventually Duncan shifted towards using Stag Hunt rather than Prisoner's Dilemma as the model here. If you haven't read it before, it's worth reading the description in full. If you're familiar you can skip to my current thoughts below.

My new favorite tool for modeling this is stag hunts, which are similar to prisoner’s dilemm

...
AGI will drastically increase economies of scale
by Wei Dai — 2019-06-07T23:17:38.694Z

In Strategic implications of AIs’ ability to coordinate at low cost, I talked about the possibility that different AGIs can coordinate with each other much more easily than humans can, by doing something like merging their utility functions together. It now occurs to me that another way for AGIs to greatly reduce coordination costs in an economy is by having each AGI or copies of each AGI profitably take over much larger chunks of the economy (than companies currently own), and this can be done with AGIs that don't even have explicit utility functions, such as copies of an AGI that are all corrigible/intent-aligned to a single person.

Today, there are many industries with large economies of scale, due to things like fixed costs, network effects, and reduced deadweight loss when monopolies in different industries merge (because they can internally charge each other prices that equal marginal costs), but because coordination costs among humans increase super-linearly with the number of people involved (see Moral Mazes and Short Termism for a related recent discussion), that creates diseconomies of scale which counterbalance the economies of scale, so companies tend to grow to a certain size and then stop. But an AGI-operated company, where for example all the workers are AGIs that are intent-aligned to the CEO, would eliminate almost all of the internal coordination costs (i.e., all of the coordination costs that are caused by value differences, such as all the things described in Moral Mazes, "market for lemons" or lost opportunities for trade due to asymmetric information, principal-agent problems, monitoring/auditing costs, costly signaling, and suboptimal Nash equilibria in general), allowing such companies to grow much bigger. In fact, from purely the perspective of maximizing the efficiency/output of an economy, I don't see why it wouldn't be best to have (copies of) one AGI control everything.

If I'm right about this, it seems quite plausible that some countries will foresee it too, and as soon as it can feasibly be done, nationalize all of their productive resources and place them under the control of one AGI (perhaps intent-aligned to a supreme leader or to a small, highly coordinated group of humans), which would allow them to out-compete any other countries that are not willing to do this (and don't have some other competitive advantage to compensate ...
Risks from Learned Optimization: Conclusion and Related Work
by evhub — 2019-06-07T19:53:51.660Z

This is the fifth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper.

Related work

Meta-learning. As described in the first post, meta-learning can often be thought of as meta-optimization when the meta-optimizer's objective is explicitly designed to accomplish some base objective. However, it is also possible to do meta-learning by attempting to make use of mesa-optimization instead. For example, in Wang et al.'s “Learning to Reinforcement Learn,” the authors claim to have produced a neural network that implements its own optimization procedure.(28) Specifically, the authors argue that the ability of their network to solve extremely varied environments without explicit retraining for each one means that their network must be implementing its own internal learning procedure. Another example is Duan et al.'s “RL2: Fast Reinforcement Learning via Slow Reinforcement Learning,” in which the authors train a reinforcement learning algorithm which they claim is itself doing reinforcement learning.(5) This sort of meta-learning research seems the closest to producing mesa-optimizers of any existing machine learning research.

Robustness. A system is robust to distributional shift if it continues to perform well on the objective function for which it was optimized even when off the training environment.(29) In the context of mesa-optimization, pseudo-alignment is a particular way in which a learned system can fail to be robust to distributional shift: in a new environment, a pseudo-aligned mesa-optimizer might still competently optimize for the mesa-objective but fail to be robust due to the difference between the base and mesa- objectives.

The particular type of robustness problem that mesa-optimization falls into is the reward-result gap, the gap between the reward for which the system was trained (the base objective) and the reward that can be reconstructed from it using inverse reinforcement learning (the behavioral objective).(8) In the context of mesa-optimization, pseudo-alignment leads to a reward-result gap because the system's behavior outside the training environment is determined by its mesa-...
Personal musings on Individualism and Empathy
by dragohole — 2019-06-07T17:07:37.283Z

One type of the more common accusations that people throw at me can be described like this: "You're an egoist. You don't think about others." To my teenage self this sounded amusing, and I would respond with a smug "Why yes, I am an egoist, thank you very much!"

Here I have to make a confession: back then I was pretty much obsessed with Ayn Rand's Objectivism. For those who don't know much about her philosophy, it's a flawed attempt to create a consistent philosophical system that covers just about everything, from ontology to aesthetics. It has very stark individualistic, libertarian-ish vibes. Being a naive yet curious kid, I was enamored with what turned out to be my first expose to (relatively) rigorous argumentation, logic, and philosophy/critical thought in general. I can't thank Rand enough for that, because through her I encountered the concepts of reason and rationality, which in turn led me to LW, The Sequences and everything else which formed the basis of my mature thought.

Anyway, after a while I moved away from objectivist ideas. I clearly remember the "final nail in the coffin" moment. I was rereading THE SEMINAL WORK THAT SYSTEMATICALLY EXPLORES AND DEVELOPS THE IDEAS OF AYN RAND, particularly the part about the consciousness axiom. I remember thinking: "Wait, the assumptions that the argument lies on are completely unfounded, what the hell is this epistemology LOL". That was it. No dramatic catharsis-like moment, no nothing, just the feeling of having finally checked off an item from my bucket list.

Despite my going through the rejection ritual, a lot of moral intuitions that I formed in that period carried over into my new, semi-adult life. I shunned social norms and laughed at those who had to conform. I prided myself on being socially brave, freely asking people for favors and being able to take a rejection and reject myself, giving compliments left and right whenever I felt like others deserved them, expressing myself authentically, without fear of judgement or public shaming.

Most of this was connected to the classical-liberal-ish notion of "anything's OK as long as it's not initiation of physical force" and its corollaries, coupled with intuitions like "nobody owes nothing to no one, thus e.g. asking for favors is totally OK and even good (cuz self-interest), but only if you're able to take a 'no' for an answer; we all are free people after

...
For the past, in some ways only, we are moral degenerates
by Stuart_Armstrong — 2019-06-07T15:57:10.962Z

Have human values improved over the last few centuries? Or is it just that current human values are naturally closer to our (current) human values and so we think that there's been moral progress towards us?

If we project out in the future, the first scenario posits continuing increased moral improvements (as the "improvement trend" continues) and the second posits moral degeneration (as the values drift away from our own). So what is it?

I'll make the case that both trends are happening. We have a lot less slavery, racism, ethnic conflicts, and endorsements of slavery, racism, and ethnic conflicts. In an uneven way, poorer people have more effective rights than they did before, so it's somewhat less easy to abuse them.

Notice something interesting about the previous examples? They can all be summarised as "some people who were treated badly are now treated better". Many people throughout time would agree that these people are actually being treated better. On the issue of slavery, consider the following question:

"If X would benefit from being a non-slave more than being a slave, and there were no costs to society, would it be better for X not to be a slave?"

Almost everyone would agree to that throughout history, barring a few examples of extremely motivated reasoning. So most defences of slavery rest on the idea that some classes of people are better off as slaves (almost always a factual error, and generally motivated reasoning), or that some morally relevant group of people benefited from slavery enough to make it worthwhile.

So most clear examples of moral progress are giving benefits to people, such that anyone who knew all the facts would agree it was beneficial for those people.

That trend we might expect to continue; as we gain greater knowledge how to benefit people, and as we gain greater resources, we can expect more people to be benefited.

Values that we have degenerated on

But I'll argue that there are a second class of values that have less of a "direction" to, and where we could plausibly be argued to have "degenerated". And, hence, where we might expect our descendants to "degenerate" more (ie move further away from us).

Community and extended family values, for example, are areas where much of the past would be horrified by the present. Why are people not (generally) meeting up with their second cousins every two weeks, and why do people waste

...
Map of (old) MIRI's Research Agendas
by Jsevillamol — 2019-06-07T07:22:42.002Z

In late 2016 I independently curated a visual map of the areas of research MIRI was working on at the time, together with descriptions of each area and arrows explaining the relationships between areas.

While MIRI's research has evolved since, I have foud myself going back to this map as a reference while doing research, so I though I would share it with the rest of you in case it might be useful for some.

Map of Research Areas in AI Alignment

The map is based on the Agent Foundations agenda by Soares and Fallenstein of 2014 and the Alignment for Advanced ML Systems agenda by Taylor et al in 2016.

Let me know if this was useful for you! It will be useful evidence on whether spending time curating more maps of the same style is a good use of my time.
Fractional Reserve Charity
by Alicorn — 2019-06-07T05:14:28.688Z

On the EA Corner discord server, various participants, most notably @wolframhead, tossed around the following idea: Some people of an EA bent may want to donate vast amounts of money, huge percentages of high salaries, but be held back in case they turn out to need that money as savings later. A health crisis, their job market failing them, or some other unforeseen but expensive eventuality could make them regret prior profligate giving once they need a runway for a few months with no paychecks, run into a pay cut, or have to contend with huge surprise bills. But of course not all of these people will actually experience such a crisis even though all of them run some risk of doing so. If there was a way for them to get their money back from the charity they donated to conditional on an emergency cropping up, the charity would be able to keep most such conditional funds even not taking into account the possibility that they could make interest on the capital in the time between donation and refund. Overall charity revenue would probably go up if this "fractional reserve" model were feasible. Here's a writeup of my understanding of the topic; the EA Corner has open membership for anyone who wants to go view the conversation in its original form.

1. Possible Structures

A - Charities could directly provide this service. This would require them to deal with the overhead of assessing emergency needs in their donors, which is probably not their core competency. The charity would operate under uncertainty about not only its future budget but also its current budget, as the rug could be pulled out from under them on sizable donations at any time. It'd also probably affect how EA money flowed to charities that managed this versus ones that did not, as a charity that has a refund possibility is "cheaper" to donate to than one that does not. The concept of "room for more funding" could possibly limit the strangling effect on charities that didn't participate to a point, but only to a point, and making lots of charities spend on overhead and uncertainty-tolerant budget forecasts just to remain competitive donation targets seems like a deadweight loss.

B - A middleman organization could form (or it could grow out of an existing meta-charity like Givewell). Such an organization would mostly just pass on donations to the ...
Visiting the Bay Area from 17-30 June
by Chris_Leong — 2019-06-07T02:40:03.668Z

I'll be in the Bay Area from the 17-30 June for training as a TripleByte Interviewer. My interests include:

Effective altruism - particularly movement building
Agent foundations - particularly updateless decision theory and anthropics
Infinite ethics
More broadly, the boundary of philosophy of maths, but also philosophy in general

PM me if you want to meet up or if you have any recommendations.

Update: I'll be training during weekdays, so I'll only be free during evenings and weekends
Arbital scrape
by emmab — 2019-06-06T23:11:17.943Z

Update: Arbital Scrape V2

I've scraped http://arbital.com as the site is unusably slow and hard to search for me.

The scrape is locally browsable and plain HTML save for MathJax.

https://drive.google.com/open?id=1b7dKhOzfMpFwngAeI8efeOzv147Lv_mx

https://emma-borhanian.github.io/arbital-scrape/

If there's interest let me know as I may tidy up and open source my code. edit: working on this

Mirror: www.obormot.net/arbital
Circle Games
by sarahconstantin — 2019-06-06T16:40:00.596Z

I may be reinventing a known thing in child development or psychology here, but bear with me.

The simplest games I see babies play — games simple enough that cats and dogs can play them too — are what I’d call “circle games.”

Think of the game of “fetch”. I throw the ball, Rover runs and brings it back, and then we repeat, ad infinitum. (Or, the baby version: baby throws the item out of the stroller, I pick it up, and then we repeat.)

Or, “peek-a-boo.” I hide, I re-emerge, baby laughs, repeat.

My son is also fond of “open the door, close the door, repeat”, or “open the drawer, close the drawer, repeat”, which are solo circle games, and “together/apart”, where he pushes my hands together and apart and repeats, and of course being picked up and put down repeatedly.

A lot of toys are effectively solo circle games in physical form. The jack-in-the-box: “push a button, out pops something! close the box, start again.” Fidget toys with buttons and switches to flip: “push the button, get a satisfying click, repeat.”

It’s obvious, observing a small child, that the purpose of these “games” is learning. And, in particular, learning cause and effect. What do you learn by opening and closing a door? Why, how to open and close doors; or, phrased a different way, “when I pull the door this way, it opens; when I push it that way, it closes.” Playing fetch or catch teaches you about how objects move when dropped or thrown. Playing with button-pushing or latch-turning toys teaches you how to handle the buttons, keys, switches, and handles that are ubiquitous in our built environment.

But what about peek-a-boo? What are you “learning” from that? (It’s a myth that babies enjoy it because they don’t have object permanence; babies get object permanence by 3 months, but enjoy peek-a-boo long after that.) My guess is that peek-a-boo trains something like “when I make eye contact I get smiles and positive attention” or “grownups go away and then come back and are happy to see me.” It’s social learning.

It’s important for children to learn, generally, “when I act, the people around me react.” This gives them social efficacy (“I can achieve goals through interaction with other people”), access to social incentives (“people respond positively when I do this, and negativey when I do that”), and a sense of social significance (“people care enough about me to respond to my actions.”) Attach...