Harley-ml/lesswrong huggingface dataset:
# Disincentives for participating on LW/AF

I was in a research retreat recently with many AI alignment researchers, and found that the vast majority of them do not participate (post or comment) on LW/AF or participate to a much lesser extent than I would prefer. It seems important to bring this up and talk about whether we can do something about it. Unfortunately I didn't get a chance to ask them about why that is the case as there were other things to talk about, so I'm going to have to speculate here based on my personal experiences and folk psychology. (Perhaps the LW team could conduct a survey and get a better picture than this.)

* Criticism that feels overly harsh or is otherwise psychologically unpleasant
* Downvotes
* Not getting enough upvotes as one feels deserved
* Not getting enough engagement
* More adversarial (zero-sum) nature of public discussion / preferring private discussions for more collaborative nature
* Feeling like "losing" a debate when the other person gets more upvotes than you
* More effort needed to write comments than to talk to people IRL
* Not real-time / time lag between replies
* Feeling ignored when someone stops responding
* Others?
* ETA: Potentially leaving a record of being wrong in public

One meta problem is that different people have different sensitivities to these disincentives, and having enough disincentives to filter out low-quality content from people with low sensitivities just necessarily means some potential high-quality content from people with high sensitivities are also kept out. But it seems like there are still some things worth doing or experimenting with. For example:

* Support for real-time collaborative discussion which subsequently get posted and voted upon as one unit (with votes going equally to both participants)
* Disabling downvotes on AF
* Having more indications/reminders of how much posting to LW/AF benefits the individual posters and the wider community, in terms of making intellectual progress and spreading good ideas. I'm not sure what form this could take, but maybe things like an indication of how many times a post is read.
* My previous feature suggestion to help with the "feeling ignored" problem
* Being less critical of new users and engaging more positively with them

There's a separate issue that some people don't *read* LW/AF as much as I would prefer but I have much less idea what is going on there.

On a tangentially related topic, is LW making any preparations (such as thinking about what to do) for a seemingly not very far future where automated opinion influencers are widely available as hirable services? I'm imagining some AI that you can hire to scan internet discussion forums and make posts or replies in order to change readers' beliefs/values in some specified direction. This might be very crude at the beginning but could already greatly degrade the experience of participating on public discussion forums.
# Rob B's Shortform Feed

This is a repository for miscellaneous short things I want to post. Other people are welcome to make top-level comments here if they want. (E.g., questions for me you'd rather discuss publicly than via PM; links you think will be interesting to people in this comment section but not to LW as a whole; etc.)
# The Athena Rationality Workshop - June 7th-10th at EA Hotel

For the past few months, Matt Goldenberg has been in charge of teaching applied rationality at the EA hotel. And honestly, we’ve been quite impressed by the quality. Matt has been a great teacher, and some of us felt like we wanted to delve deeper into his material. So let us delve into it together!

During the first weekend of June, the [EA Hotel](http://eahotel.org) will host its first rationality workshop. Open to anyone interested.

We will delve into the [Ease process](https://www.lesswrong.com/posts/mFvuQTzHQiBCDEKw6/a-framework-for-internal-debugging), which is step by step process for overcoming internal blocks and creating psychological alignment towards your goals.

Matt has been working intensely on self-improvement for the past 15 years, was a professional coach for 5 years and has been running applied rationality group workshops for the past 18 months. Some guest teachers will also be chiming in with their techniques, including Toon Alfrink with some lessons he learned living at a Zen Monastery, and possibly others.

The workshop will be held June 7th-10th, Friday to Monday, starting at 12:00 and ending at 19:00. Travelers from afar can stay the night on Thursday. There will be an optional chill-out day on Tuesday, June 11th, and you’re welcome to stay longer. Payment will be at the end of the workshop for whatever you think it was worth for you (cost price is £40). All proceeds will go to fund the EA hotel.

If you want to join: [Sign up here](https://docs.google.com/forms/d/e/1FAIpQLScRXtBkwIQElvEfenxZJ53dlvWRJhfKW4GNZy_tcOTwYBdvVw/viewform).
# Why books don't work

A look at how books are not optimal for conveying information. Analogies to lectures, with an interesting take on cognitive models, i.e. the assumptions you make about how learning happens. Also some interesting citations on average reading time and attention span.

I really like this piece because it ties together lots of thoughts I've previously tried to express, but didn't find the right words for. I think it does a very good job of pointing out how the default mediums are not optimal.

Also interesting to note the author has collaborated with Michael Nielsen (who wrote a [fantastic online book on neural networks](http://neuralnetworksanddeeplearning.com/)) on another online book on [quantum computing](https://quantum.country/qcvc/) which incorporates spaced repetition, and he has also done prior work for Khan Academy.
# Narcissism vs. social signalling

The main thrust of [Robin Hanson's](https://www.overcomingbias.com/author/robin-hanson) work is that so much of human behaviour is the result of social signalling; that is, the attempt to convince others that we possess good qualities. On the other hand, [The Last Psychiatrist](https://thelastpsychiatrist.com/) presents narcissism as an alternative theory; that is, much of our behaviour is an attempt to convince ourselves that we possess good qualities. I haven't read enough of The Last Psychiatrist to be able to provide a good summary of their ideas, so this will just be a short comment to raise awareness that an alternative theory exists and to discuss how these hypotheses relate.

The separation between these two theories isn't as clear as it may first appear. After all, we may attempt to convince other people of our goodness in order to ultimately convince ourselves or we may attempt to convince ourselves of our goodness so that we can more persuasively convince other people. For the later, imagine someone prepping for a job interview teaching when they are aware that their knowledge of the material isn't as strong as they'd like, but believing they might get hired if they seems confident.

Trying disambiguate using revealed preferences may be misleading - someone may spend most of their time trying to impress other people, but that may simply be the strategy they've adopted for convincing themselves that they are worthy and they may drop it as soon as they learn other strategies. Alternatively, trying to use people's ultimate goals to disambiguate this is tricky when, according to both theories, we often don't know why we do what we do. And indeed, when we talk about ultimate goals, are we referring to the ultimate goal that is represented in the brain or are we allowed to reference evolutionary psychology reasons for behaviour? Then of course these the issue that most of the time, both theories will be correct to a certain extent. I'll admit the the practical consequences of adopting one theory over another aren't always immediately clear, but I expect that adopting one model instead of the other would necessarily result in some differences in predictions.
# The Geography of the Room — impact of position, orientation, and visibility on how often extraverts start conversations

_I write long-form articles about social interactions based on ~6 years of observational data. This particular piece is part of a longer sequence exploring why some people are frequently approached by strangers whereas others are mostly ignored._
# What are some "Communities and Cultures Different From Our Own?"

"[Legal Systems Different From Our Own](http://www.daviddfriedman.com/Academic/Course_Pages/legal_systems_very_different_12/LegalSystemsDraft.html)" is a fantastic work that explores how many different cultures approached legality. The diversity of ideas there has given me both concrete ideas of how to incentivize people, and helped shake me out of some default frames I was holding.

I'd be interested in other works that do the same thing, but for different aspects of community building (including but not limited how to help people find connection, how to help them gain skills, how to ensure the community has a good mixture of growth and stability, etc).

Three useful answers to this question would include:

1) if there literally is a book that is an overview of different communities or cultures and interesting elements of how they work, that'd be useful

2) if you have read a book that deals with one particular community or type of community, writing up a summary of that book

3) writing up experiences from a given community, especially if there were particular social tools that the community used that have neat implications.
# Epistea Summer Experiment

ESE is a week-long experimental summer workshop combining elements of applied rationality and experiential education happening in Europe.

We will play, explore, introspect, practice - all in order to boost our learning. The program is focused on _group rationality_, or in other words: how we can have accurate beliefs and act in accordance with them not only as individuals but together with other people.

*   **When:** 28.7. – 3.8. 2019 (6,5 days of program)
*   **Where:** in beautiful Czech countryside in Loutí – near Prague, Czech Republic
*   **For whom**: All rationalists who are excited about the mixture of playful summer camp, experimenting with old and new rationality techniques, and learning how to be rational _together_.
*   **Price**: 600 Euro (In case you would hesitate to join because of financial issues, let us know.)
*   **Application deadline**: 19th of May

For more detailed info, check the [website](https://experience.epistea.org/).
# How to improve at critical thinking on science/medical literature?

I'm a medical student, and I will often read articles that are critical of scientific literature ([Scott Alexander on Pharmacogenomics](https://slatestarcodex.com/2017/03/06/antidepressant-pharmacogenomics-much-more-than-you-wanted-to-know/); [EMCrit on thrombolysis in ischemic stroke](https://emcrit.org/emnerd/the-case-of-the-magicians-sleight/), etc.) with some awe at the authors' ability to evaluate evidence.

I'm sure that part of this is practice. If I spend more time critically reading scientific literature, and less time taking experts at face value, I will likely become better able to think independently.

However, part of it strikes me as a lack of technical skills. I'm often unsure how to critique study designs when I don't understand the statistical methods being used.

Any recommendations for how I might get the skills I need to think independently about scientific/medical literature?

\[Edit: Changed formatting of links after a comment\]
# Feature Request: Self-imposed Time Restrictions

Hacker News has a feature called "noprocrast". Here's how they explain it in the [FAQ](https://news.ycombinator.com/newsfaq.html):

> **In my profile, what is noprocrast?**

> It's a way to help you prevent yourself from spending too much time on HN. If you turn it on you'll only be allowed to visit the site for maxvisit minutes at a time, with gaps of minaway minutes in between. The defaults are 20 and 180, which would let you view the site for 20 minutes at a time, and then not allow you back in for 3 hours.

If you try to use HN when you precommitted to not using it, you'll get the following message from them:

> **Get back to work!**

> Sorry, you can't see this page. Based on the anti-procrastination parameters you set in your profile, you'll be able to use the site again in 43 minutes.

I was thinking that something like this would be awesome for LessWrong. Personally, I have a rather large problem browsing the web - which includes browsing LessWrong - when I should be doing other things. After reading [Digital Minimalism](http://www.calnewport.com/books/digital-minimalism/), I get the impression that such struggles are moreso the norm than the exception.
# How do you know it's time for a break?

My original question was something like "how much productive work can people do in a day?", but even if I got a signed piece of paper from God telling me the exact average, that wouldn't be very useful to individuals except as cover for stopping once they'd reached the average. Some people are clearly capable of doing more than others, the same person has different capacities for different kinds of work.

So instead I'm asking "How do you know when you need a break? When you're done for the day?". My hope is that reading answers will inspire people to recognize tells in themselves they previously hadn't, and this will let them better assess when to push through and when to break.
# Kevin Simler's "Going Critical"

An interactive blogpost by Kevin Simler on network dynamics, with a final section on academia and intellectual progress. I generally think careful exploration of small-scale simulations like this can help quite well with understanding difficult topics, and this post seems like a quite good execution of that approach.

Also some interesting comments on intellectual progress and academia (though I recommend reading the whole post):

  

> For years I've been fairly dismissive of academia. A short stint as a PhD student left a bad taste in my mouth. But now, when I step back and think about it (and abstract away all my personal issues), I have to conclude that academia is still _extremely_important.

> Academic social networks (e.g., scientific research communities) are some of the most refined and valuable structures our civilization has produced. Nowhere have we amassed a greater concentration of specialists focused full-time on knowledge production. Nowhere have people developed a greater ability to understand and critique each other's ideas. This is the beating heart of progress. It's in these networks that the fire of the Enlightenment burns hottest.

> But we can't take progress for granted. If the reproducibility crisis has taught us anything, it's that science can have systemic problems. And one way to look at those problems is network degradation.

> Suppose we distinguish two ways of practicing science: _Real Science_ vs. _careerist science_. Real Science is whatever habits and practices reliably produce knowledge. It's motivated by curiosity and characterized by honesty. (Feynman: "I just have to understand the world, you see.") Careerist science, in contrast, is motivated by professional ambition, and characterized by playing politics and taking scientific shortcuts. It may look and act like science, but it _doesn't_ produce reliable knowledge.

> (Yes this is an exaggerated dichotomy. It's a thought exercise. Bear with me.)

> Point is, when careerists take up space in a Real Science research community, they gum up the works. They angle to promote themselves while the rest of the community is trying to learn and share what's true. Instead of striving for clarity, they complicate and obfuscate in order to sound more impressive. They engage in (what Harry Frankfurt might call) scientific bullshit. And consequently, we might model them as dead nodes, immune to the good-faith information exchanges necessary for the growth of knowledge:
# European Community Weekend 2019

From Friday August 30th to Monday 2nd September aspiring rationalists from across Europe and further afield will gather for 4 days of socializing, fun and intellectual exploration. There will be scheduled talks, but the majority of the content will be unconference style and participant driven.

On Friday afternoon we put up four wall-sized daily planners and by Saturday morning the attendees fill them up with +50 workshops, talks and activities of their own devising, such as icebreaker games, rationality techniques, EA community building discussions, comfort zone expansion workshop, polyamory and relationships workshops, morning meditation sessions in the winter garden and many more.

This is our 6th year and we feel that the atmosphere and sense of community at these weekends is something that’s really special. If that sounds like something you would enjoy and you have some exciting ideas and skills to contribute do come along and get involved. This year is the biggest one yet and it’s an entire day longer than previous years!

We have a very limited amount of spots. If you would like to be among the first selected in the beginning of May, sign up today: www.tiny.cc/lwcw2019_signup and make sure to let us know what experience and ideas you may contribute to this event: www.tiny.cc/lwcw2019_contribution.


TL;DR:

When? 30th August – 2nd September 2019

Where? jh-wannsee.de

Tickets? Regular Ticket (200€), Supporter Ticket (300/400€): www.tiny.cc/lwcw2019_signup

If you have any question, please email us at lwcw.europe@gmail.com.

Looking forward to a great weekend,
The Community Weekend organizers and LessWrong Deutschland e.V.
# Which scientific discovery was most ahead of its time?

Looking into the history of science, I've been struck by how continuous scientific progress seems. Although there are many examples of great intellectual breakthroughs, most of them build heavily on existing ideas which were floating around immediately beforehand - and quite a few were discovered independently at roughly the same time (see https://en.m.wikipedia.org/wiki/List_of_multiple_discoveries).

So the question is: which scientific advances were most ahead of their time, in the sense that if they hadn't been made by their particular discoverer, they wouldn't have been found for a long time afterwards? (Ideally taking into account the overall rate of scientific progress: speeding things up by a decade in the 20th century seems about as impressive a feat as speeding things up by half a century in ancient Greece).
# Simulation Typology and Termination Risks

The main idea of the article is that we likely live in a “Fermi simulation”, that is, a simulation which is created by aliens to solve Fermi paradox via simulating possible global risks, or in a "Singularity simulation", – a simulation where future AI models its own origin (gaming simulations will also play with our period of history as a most interesting one). It means that our simulation will be likely turned off soon, as at least one of three conditions will be reached:

\- Our simulation will model a global catastrophe, which is subjectively equal to the termination.  
\- Our simulation will reach unknown to us goal (but likely related to our AI's origin conditions) and will be terminated.  
\- Our simulation will reach a nested simulation level shortly after strong AI creation, which will drastically increase computational resources demand and thus the simulation will be terminated.

We also suggested a classification of different types of simulations, patched the Simulation Argument and suggested Universal Simulation Argument which now has to take into account all possible civilisations in the universe. This makes SA's branches where we are not in the simulations ("extinction soon" or "ethical future") much less likely, as it would require that all possible civilisations will go extinct or all possible non-human civilisations will be ethical.
# Comment section from 05/19/2019

I moved the big meta-level comment thread from ["Yes Requires the Possibility of No"](https://www.lesswrong.com/posts/G5TwJ9BGxcgh5DsmQ/yes-requires-the-possibility-of-no#ijE36ZJAyTj8t7jfy) over to here, since it seemed mostly unrelated to that top-level post. This not being on frontpage also makes it easier for people to just directly discuss the moderation and meta-level norms.
# Would an option to publish to AF users only be a useful feature?

Right now there are quite a few private safety docs floating around. There's evidently demand for a privacy setting lower than "only people I personally approve", but higher than "anyone on the internet gets to see it". But this means that safety researchers might not see relevant arguments and information. And as the field grows, passing on access to such documents on a personal basis will become even less efficient.

My guess is that in most cases, the authors of these documents don't have a problem with other safety researchers seeing them, as long as everyone agrees not to distribute them more widely. One solution could be to have a checkbox for new posts which makes them only visible to verified Alignment Forum users. Would people use this?
# Go Do Something

There's this notion going around in the community sometimes that holds that the best way to progress on one's rationality skills is to make lots and lots of theoretical study by yourself or to go off and "level up" a bunch on your own prior to getting involved in object-level projects or efforts.

I think that's false and in fact that it's not only false but it's almost the opposite of what needs to be done. In point of fact, much of the time I've seen people go off to meditate in the darkness for a year and "level up" a bunch, this has not only not helped much but in some cases has seemed to actually harm, as people have a tendency to go in strange and unsound directions when doing this sort of thing on their own without feedback.

Instead, I think that if you want to make progress on your rationality skills, the best way to do that is to get involved with object-level projects and use those as testing grounds for your practice. Not only will this help you test skills in a more realistic and practical setting, but it will also provide demonstrations that you can later refer to to show how things worked (or didn't), and it will quite possibly help you build a [secret identity](https://www.lesswrong.com/posts/gBewgmzcEiks2XdoQ/mandatory-secret-identities) as well.

So, yeah. If you want to be an advanced rationalist, don't just theorize - go out there and do stuff.
# TAISU - Technical AI Safety Unconference

**Start:** Thursday, August 22, 10am  
**End:** Sunday, August 25, 7pm  
**Location:** [EA Hotel](http://eahotel.org/wiki/#travel), 36 York Street, Blackpool

It is an unconference, which means that it will be what we make of it. There will be an empty schedule wich you, the participants, will fill up with talks, discussions and more.

To be able to have high level discussion during the unconference, we require that all participants have some prior involvement with AI Safety. Here is a non complete list of things that are sufficient:

*   Have participated in one of the following: [AI Safety Camp](https://aisafetycamp.com/), [MSFP/AISFP](https://rationality.org/workshops/apply-msfp), [Human-aligned AI Summer School](http://humanaligned.ai/), [Learning-by-doing AI Safety workshop](https://www.lesswrong.com/events/z9peEBfuiPB7L2Edb/learning-by-doing-ai-safety-workshop)
*   Are currently or have previously worked for or interned at an established AI Safety organization
*   Have published papers or sufficiently high quality blogpost about AI Safety

You can participate in the unconference as may or as few days as you want to. You are also welcome to stay longer at at EA Hotel before or after the unconference. However, be aware that there is another event, [Learning-by-doing AI Safety workshop](https://www.lesswrong.com/events/z9peEBfuiPB7L2Edb/learning-by-doing-ai-safety-workshop), the weekend before. If you want to join this workshop, you should [apply](https://docs.google.com/forms/d/e/1FAIpQLScFTxfmon-qpcHpJl9ff7RuVeSzqaxjNlAS0MfuyEac3m1wJg/viewform) for this separately.

If you are staying more than a few days extra, we ask you to book your stay though the EA Hotel [booking system](http://eahotel.org/booking/).

**Price:** Pay what you want (cost price is £10/person/day).  
**Food:** All meals will be provided by EA Hotel. All food will be vegan.  
**Lodging:** The EA Hotel has two dorm rooms that have been reserved for TAISU participants, and more rooms will be booked at nearby hotels if necessary. However, if you want a private room you might be asked to pay for it yourself.

**_If you want to join: [Sign up here](https://docs.google.com/forms/d/e/1FAIpQLSfB9q6_HbxIEY-Azq3RK_vJdUtd_c8rGJKQfkdY7ldrLbosOg/viewform)_**
# Where are people thinking and talking about global coordination for AI safety?

Many AI safety researchers these days are not aiming for a full solution to AI safety (e.g., the classic Friendly AI), but just trying to find good enough partial solutions that would buy time for or otherwise help improve global coordination on AI research (which in turn would buy more time for AI safety work), or trying to obtain partial solutions that would only make a difference if the world had a higher level of global coordination than it does today.

My question is, who is thinking directly about how to achieve such coordination (aside from FHI's [Center for the Governance of AI](https://www.fhi.ox.ac.uk/GovAI), which I'm aware of) and where are they talking about it? I personally have a bunch of questions related to this topic (see below) and I'm not sure what's a good place to ask them. If there's not an existing online forum, it seems a good idea to start thinking about building one (which could perhaps be modeled after the AI Alignment Forum, or follow some other model).

1. What are the implications of the current US-China trade war?
2. Human coordination ability seems within an order of magnitude of what's needed for AI safety. Why the coincidence? (Why isn’t it much higher or lower?)
3. When humans made advances in coordination ability in the past, how was that accomplished? What are the best places to apply leverage today?
4. Information technology has massively increased certain kinds of coordination (e.g., email, eBay, Facebook, Uber), but at the international relations level, IT seems to have made very little impact. Why?
5. Certain kinds of AI safety work could seemingly make global coordination harder, by reducing perceived risks or increasing perceived gains from non-cooperation. Is this a realistic concern?
6. What are the best intellectual tools for thinking about this stuff? Just study massive amounts of history and let one's brain's learning algorithms build what models it can?
# What is your personal experience with "having a meaningful life"?

I hear a lot of different stories about how meaning should fit into one's life

> "What's all this meaning bullshit? Just focus on doing your job well and providing for your family."

^my grandparents

> "Wanting meaning is wanting a simple narrative to your life, no simple narrative can possibly be true which means you should forgo the impulse for meaning in favor of the truth."

^some rationalists I know now

> "Sure you can have meaning, but base it off of something real like 'pushing the bounds of human knowledge' instead of some ancient conception of a deity."

^some other rationalists I know now

> "Without meaning you might still be able to have an okay life, but you're missing out on one of the most important/enjoyable/most-human parts of being a human."

^my parents

> "Without meaning, you and your society will slowly degrade and fall apart and it is imperative that you find a narrative that works, otherwise game over."

^Jordan Peterson maybe(?)

**Question:** Do you personally feel a need/desire/impulse to have something like meaning in your life? How do you feel when you have it? How do your feel when you don't? If you do experience a need for meaning, how do you feel about having that need?

If you feel a need for meaning, what sorts of things feel meaningful? If you don't feel a need for meaning, what is that like? If you feel a need for meaning but don't endorse it, why is that the case?

**Alternative Question** (If your answer is something along the lines of "I don't really care about meaning much, I mostly just try to spend time on things I care about"): Are you readily able to discern what you care about? Do you think and decide what to care about? Have you ever had the experience of caring about ABC for a long time, then some event cause you to no longer care about ABC? What does life feel like when you're doing stuff you care about? What does it feel like when you aren't?

This is an open ended and fuzzy topic, and I'm am looking for any and all personal experience data points you can provide.

_Edit/Clarification_: My main motive for asking is that I was planning to write some posts about meaning, and wanted to check if the views I was responding to were ones other people held, or just straw men. Again, I'm interested in either personal details of how you experience meaning, but also find useful hearing about people's explicit models of "what is the thing in the brain that is meaning?" (I would conceivable ask for models as a related question, but I'm not sure how that feature works and don't know what it looks like when used)
# And the AI would have got away with it too, if...

Paul Christiano presented some low-key [AI catastrophe scenarios](https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/more-realistic-tales-of-doom); in response, Robin Hanson [argued](http://www.overcomingbias.com/2019/04/agency-failure-ai-apocalypse.html) that Paul's scenarios were not consistent with the "large (mostly economic) literature on agency failures".

He concluded with:

>For concreteness, imagine a twelve year old rich kid, perhaps a king or queen, seeking agents to help manage their wealth or kingdom. It is far from obvious that this child is on average worse off when they choose a smarter more capable agent, or when the overall pool of agents from which they can choose becomes smarter and more capable. And its even less obvious that the kid becomes maximally worse off as their agents get maximally smart and capable. In fact, I suspect the opposite.

Thinking on that example, my mind went to Edward the Vth of England (one of the "[Princes in the Tower](https://en.wikipedia.org/wiki/Princes_in_the_Tower)"), deposed then likely killed by his "protector" [Richard III](https://en.wikipedia.org/wiki/Richard_III_of_England). Or of the [Guangxu Emperor](https://en.wikipedia.org/wiki/Guangxu_Emperor) of China, put under house arrest by the Regent [Empress Dowager Cixi](https://en.wikipedia.org/wiki/Empress_Dowager_Cixi). Or maybe the ten year-old [Athitayawong, king of Ayutthaya](https://en.wikipedia.org/wiki/Athittayawong), deposed by his main administrator after only 36 days of reign. More examples can be dug out from some of Wikipedia's list of [rulers deposed as children](https://en.wikipedia.org/wiki/Category:Rulers_deposed_as_children).

We have no reason to restrict to child-monarchs - so many Emperors, Kings, and Tsars have been deposed by their advisers or "agents". So yes, there are many cases where agency fails catastrophically for the [principal](https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem) and where having a smarter or more rational agent was a disastrous move.

By restricting attention to agency problems in economics, rather than in politics, Robin restricts attention to situations where institutions are strong and behaviour is punished if it gets too egregious. Though even today, there is plenty of betrayal by "agents" in politics, even if the results are less lethal than in times gone by. In economics, too, we have fraudulent investors, some of which escape punishment. Agents betray their principals to the utmost - when they can get away with it.

So Robin's argument is entirely dependent on the assumption that institutions or rivals will prevent AIs from being able to abuse their agency power. Absent that assumption, most of the "large (mostly economic) literature on agency failures" becomes irrelevant.

So, would institutions be able to detect and punish abuses by future powerful AI agents? I'd argue we can't count on it, but it's a question that needs its own exploration, and is very different from what Robin's economic point seemed to be.
# What should rationalists think about the recent claims that air force pilots observed UFOs?

NY times has [a new material](https://www.nytimes.com/2019/05/26/us/politics/ufo-sightings-navy-pilots.html) about possible observations of UFOs by air force pilots. Should rationalists use their arsenal of new ideas about how the world works – simulation, superintelligent AIs etc – to explain these things, or should they be of more sceptical side and explain them via some combination of biases and hoaxes?

For example, one could speculate that an alien AI have terminated long before now, but some of its self-replicating robots colonised the universe. Or may be we live in a simulation which is full of glitches and viruses?

It is possible to create many hypothesis like these with infinite explanatory powers.
# A shift in arguments for AI risk

_The linked post is work done by Tom Adamczewski while at FHI. I think this sort of expository and analytic work is very valuable, so I'm cross-posting it here (with his permission). Below is an extended summary; for the full document, see his linked blog post._

Many people now work on ensuring that advanced AI has beneficial consequences. But members of this community have made several quite different arguments for prioritising AI.

Early arguments, and in particular _Superintelligence_, identified the “alignment problem” as the key source of AI risk. In addition, the book relies on the assumption that superintelligent AI is likely to emerge through a discontinuous jump in the capabilities of an AI system, rather than through gradual progress. This assumption is crucial to the argument that a single AI system could gain a “decisive strategic advantage”, that the alignment problem cannot be solved through trial and error, and that there is likely to be a “treacherous turn”. Hence, the discontinuity assumption underlies the book’s conclusion that existential catastrophe is a likely outcome.

The argument in _Superintelligence_ combines three features: (i) a focus on the alignment problem, (ii) the discontinuity assumption, and (iii) the resulting conclusion that an existential catastrophe is likely.

Arguments that abandon some of these features have recently become prominent. They also generally tend to have been made in less detail than the early arguments.

One line of argument, promoted by Paul Christiano and Katja Grace, drops the discontinuity assumption, but continues to view the alignment problem as the source of AI risk. Even under more gradual scenarios, they argue that, unless we solve the alignment problem before advanced AIs are widely deployed in the economy, these AIs will cause human values to eventually fade from prominence. They appear to be agonistic about whether these harms would warrant the label “existential risk”.

Moreover, others have proposed AI risks that are unrelated to the alignment problem. I discuss three of these: (i) the risk that AI might be misused, (ii) that it could make war between great powers more likely, and (iii) that it might lead to value erosion from competition. These arguments don’t crucially rely on a discontinuity, and the risks are rarely existential in scale.

It’s not always clear which of the arguments actually motivates members of the beneficial AI community. It would be useful to clarify which of these arguments (or yet other arguments) are crucial for which people. This could help with evaluating the strength of the case for prioritising AI, deciding which strategies to pursue within AI, and avoiding costly misunderstanding with sympathetic outsiders or sceptics.
# Evidence for Connection Theory

Connection Theory (CT) is the original philosophy underpinning [Leverage Research](http://leverageresearch.org/), a research think tank focused that has worked with the effective altruism movement, and the rationality community, in the past on community-building, and existential risk reduction. CT was developed by Leverage's executive director, Geoff Anders. Since there are few if any other publicly available online resources for understanding or evaluating CT, I thought I would share this document.
# What is required to run a psychology study?

I periodically find myself wishing someone had run an experiment on a particular topic. But they haven't.

Often, it seems like there's a relatively easy experiment you can run, which would give me some evidence about it. Maybe it wouldn't be perfect evidence, but it would be better than me making stuff up from my armchair.

The two clusters of things-I-can-imagine-being-quite-hard are:

*   Legal requirements
*   Actually doing the science right (avoiding sampling biases, actually testing the right variable, etc)

**Legal Stuff**

Scott's [IRB Nightmare](https://slatestarcodex.com/2017/08/29/my-irb-nightmare/) suggests that the legal requirements can be quite awful. I'm unclear on what those requirements are if I don't want to publish anywhere, don't plan to interface with any hospital bureaucracies, I just want to ask a bunch of people to try something and see how it goes (and maybe make a LessWrong blogpost so people on LessWrong can know too)

Am I allowed to just go out and ask a whole bunch of people stuff? If I want to know things about 8 year olds or 16 year olds, if their parents sign a form and I'm not doing anything especially weird or traumatic, would that be fine or would terrible things happen to me? (Could I replicate the Marshmallow Test?)

**Doing Science Good Enough**

I have a sense that there's a lot of ways to deceive yourself if you have a pet psychology theory. But... I dunno it also seems like LessWrong collectively should be pretty good at this. The thing I might want to do (or get others to do sometimes) is post a plan for a study here, and get critiqued until it seems like an actually good plan, do the plan, write up the results.

**Particular Things I was interested in:**

(not meant to be exhaustive, important, or particularly achievable plans, just the things that generated this question)

*   How do most people relate to their job? (There's a survey study that asks if people consider it a job, a career, or a calling, but the study had a very narrow range of participants – people who worked at one particular place, and I'm curious how a wider variety of people would respond)
*   How do most people relate to ambition? (where by ambition I mean "form plans to create or change things that will impact large numbers of people.")
*   How many 8 year olds can learn to program? Can they implement FizzBuzz? How many 16 year olds? How many 30 year olds? I've heard that people either have-or-don't-have a "programmer trait" that's hard to learn, and I'm not sure how much of that is true at all, and if so if it's more about nature or nurture.
# Seeking suggestions for EA cash-prize contest

  

I have grown interested in the cash-prize EA essay contest format that I've seen on LessWrong. I'm interested in sponsoring a 3-essay series of contests, and would appreciate exploratory thoughts and critique. My goal is to make this essay series produce genuinely useful new information for the community and lead to positive changes in the way we organize these contests. Any strategic advice would be helpful.

The first essay would be on the prompt "What is the most effective way to run a cash-prize EA essay contest? Make sure to specify how winners should be selected and prize money distributed."

The second would implement suggestions from the first contest, and be on the prompt "what is the most compelling argument for not running cash-prize EA essay contests?"

The third would be on the prompt "How could we design a useful scientific experiment to determine the ideal size of essay cash prizes?"

My plan is to offer $50-$100 as cash prizes for each essay. I don't earn a lot by American standards, which is why these prizes are relatively small. I'm considering crowdfunding for higher prizes. I'm not sure if I should allocate it all to the winner or divide it among top posts.

For the second essay, I can determine the winner through the method suggested by the first essay. But for judging the first essay, how should I go about it?
# How to find a lost phone with dead battery, using Google Location History Takeout

Or, wow, it's crazy how much Google knows about me.

(I'm writing this in part because I tried to Google variations of "how to find a lost phone with dead battery" and among all the articles in the search results that I clicked on, didn't see anyone write up this technique that I later figured out myself.)

I lost a brand new Android phone recently inside my own home. I knew it was inside my home because I remembered using it after I got home. Normally in that situation I would call it or use Google's Find Device feature to make the phone ring, but the phone's battery had died before I realized that it was lost. I spent a couple of hours looking for it in all the obvious places, like behind sofa cushions, in the garbage can, everywhere in the house I had been (or might have been but forgot) since I last used the phone, but failed to find it. Find Device gives the last known location of a phone, but the error circle encompassed the whole house so that wasn't very helpful. 

Eventually I decided to download all of the data Google has on the history of the phone's location using a feature called [Takeout](https://takeout.google.com), just in case it offered any further clues, and that's when I realized that in Google's databases there's an entry for every time one of my phones moves even a little bit, with a timestamp, location estimate, velocity estimate, and a guess of whether I was walking, running, biking, in a car, etc. (presumably inferred using the phone's accelerometer). I [converted](https://github.com/Scarygami/location-history-json-converter) the data from JSON to CSV and then looked at the last time the phone moved. (BTW the timestamps are in UTC time, which confused me for a while until I realized that.) Then I looked at my Google [Activity History](https://myactivity.google.com/) to find out what I was doing at that time. The Activity History told me I did a particular Google search around that time, and I remembered I was sitting in a particular armchair while doing that search. (Google Activity History also has a whole bunch of other information, like every time I launch an app on my phone, or visit a website using Chrome.)

Of course I had searched that armchair already but this knowledge incentivized me to do a much more thorough search, and it turned out that the phone had fallen down a very narrow crack in the armchair to a pocket underneath. I ended up retrieving the phone by cutting open the fabric covering the bottom of the armchair.
# Archive of all LW essay contests

[I've compiled all the LW writing contests I could find to date.](https://docs.google.com/spreadsheets/d/11Zm2YSgb-U-QOXKNvkariT-eXh_ztoi3pm_ftsoxAlI/edit?usp=sharing) There are 15 in all, plus a few more from other websites. Quite a few of the articles have disappeared from the web, and a couple contests are either still running or were never completed. To help future contest-runners, I added basic info about the structure of each contest. Feel free to add links to other LW posts about essay bounties you think would be useful. This is a document that anyone can edit. I have a copy saved just in case.
# How would you advise a peer-supported virtues-oriented self-help group?

The "virtues" are the classical name for those habitual characteristics of acting and thinking that tend toward a flourishing, beneficial life. An influential theory of virtues holds that they are much like many other human skills in that you can become fluent in them through sustained practice. I'm helping to create a peer-supported self-help group in which each individual will identify a virtue they want to work on, and then work with a peer (who is doing the same) to come up with a practice curriculum and a way of mutual support / accountability so as to keep at it, eventually moving on to a second virtue, and so on. We're "alpha testing" it now. I'm hoping the LW community can give me some advice, and/or some pointers to other groups doing similar projects.
# How to determine if my sympathetic or my parasympathetic nervous system is currently dominant?

I've been reading CFAR Handbook's chapter about Againstness. The chapter's idea is that when your sympathetic nervous system (SNS) is dominant over parasympathetic nervous system (PSNS), your introspection is impaired and you tend to be less rational, hence you should learn how to know which system is currently dominant and also learn to switch to PSNS dominance.

They provide the following table of how your body, mind, and behaviour change depending on which system is dominant

![table sns vs psns](http://dump.bitcheese.net/files/makisaj/Screenshot_20190529_113135.png)

In order to learn to know where you are on SNS-PSNS spectrum they recommend observing yourself in different situations, determine which system is dominant, and try to find patterns (e.g., maybe after doing physical exercise your SNS is usually dominant).

Well, I tried observing myself and figuring out which nervous system is active, and about 70% of the times I can't determine it. I ask myself questions like "Do I feel joy?", "Am I at peace?", "Are my muscles tense?", "Does my skin feel like flushed?", "Is my breathing shallow, or slow and in belly?". More often than not I can't answer these questions. And it doesn't help that there's no easy way to calibrate by observing the ground truth.

If there are people here who can determine which of your two nervous systems is dominant, how do you do it? Any tips for me?

Also, sometimes I get signals from my body that don't fit in this framework. When it's time for me to go to sleep but I don't do it, after a while my awareness and introspective clarity become impaired, I feel both energized and lacking energy at the same time, I fill like my system 2 (as in Kahneman's two systems) doesn't turn on a lot, and I get impatient. What happens with SNS and PSNS at this time? I don't know.
# When Observation Beats Experiment

Suppose we have a strain of lab rats which are colored purple, and we want to know why. We suspect that chemical X is responsible, so we run an experiment:

*   We genetically modify our purple rats to repress X production, and find that their purple coloration disappears.
*   We genetically modify ordinary rats to produce X, and find that their coats turn purple.

We conclude that chemical X is both necessary and sufficient to turn rats’ coats purple. Case closed!

… or maybe not.

Suppose that rats are purple-colored if-and-only-if they express Purple Pigment (PP) above some threshold level. Purple Pigment, in turn, is chemically produced from X and Y:

 X+Y⇌PP

High levels of PP could result from high levels of X, or from high levels of Y. Either way, increasing X enough will always turn a rat purple, and decreasing X enough will always turn a rat not-purple. So our experiment doesn’t tell us whether our particular rats are purple due to high X or high Y - it could be either. In order to tell the difference, we need to go measure X and Y levels in our rats - not an experiment, but an observation.

(Warning: technical details not relevant to the main point were brushed under the rug there.)

Generalizing: experiments are really good for figuring out the _structure_ of the underlying causal graph. How can we tell that Purple Pigment is produced from X and Y in the first place? Experiment: we try various levels of X and Y and see which rats are purple.

But if we want to know the _state_ of the causal graph, in some real-world system, then observation beats experiment. To find out whether our particular rats are purple because of high X or high Y, we should measure their X and Y levels, without any experimental intervention. Of course, this only works if we’ve already done the experiments to figure out the structure of the system.