trentmkelly/LessWrong-43k huggingface dataset:
Writing Down Conversations
by Raemon — 2017-12-28T22:08:54.031Z

Epistemic Status: Didn't think through exactly how I worded things.

tldr: When you have insightful conversation, write it down and share it so people can build on it (instead of just sharing in person). Most of humanity's power comes from being able to build complex thoughts out of other thoughts and transmit them across the world.

This is a rehash/re-examining post. Related:

Write Down Your Process (Zvi Mowshowitz)
Some Thoughts on Public Discourse (Holden Karnofsky)
Why and How to Name Things (Conor Moreton)
Single Locus of Discussion (Anna Salamon)
Return to Discussion (Sarah Constantin)
[Edit: Turning Discussions into Blogposts by Brian Tomasik apparently covers this very topic. Still needed to be said again]

This is part 2 of N of my "Ray writes down conversations he had with people" series. It's also the most adorably meta of them.

A month ago I was talking with Oliver Habryka about why Less Wrong was important. (Call us biased if you will). One thing we both noted: in the days of yore, it seemed like a lot of prominent scholars/thinkers wrote down their insights and research on Less Wrong. Then, eventually they turned professional and joined official organizations whose job was to think fulltime.

Also over the past few years, those organizations (including but not limited to MIRI, CFAR, Givewell/OpenPhil) shifted from being younger-with-nothing-to-lose to older-with-reputations-to-safeguard, and their public facing tone seems to have shifted from "earnestly sharing thoughts as they come up" to "carefully crafted PR statements."

Thirdly, a lot of people moved to major geographic hubs, where it became easier to have in person conversations than to communicate via written blogpost. So... that's what people have tended to do.

I sympathize with the notion that people are busy and writing things up is a) time consuming and b) potentially risky. But I think the consequences of this are at least underweighted.

At least, I think people having informal conversations should make more of an effort to write those up in accessible form when there aren't reasons to be cautious.

Consequences of not writing stuff down include:

If you're not plugged into the personal-conversation-network, it's hard to keep up with a lot of collective insights relating to both rationality, effective altruism and x-risk. (This results in

...
Maps vs Buttons; Nerds vs Normies
by Bound_up — 2017-12-28T21:11:40.577Z

It took me the longest time to see through the illusion that was "rational" discussion.

The setting: A friend and I exchange inquiries about each other's beliefs about X.

The result: the friend would give a number of answers that allowed me to piece together their view of X. An astounding percentage of the time, even a majority, perhaps, one of their answers would contradict another. So, I point this out. My friend, after hearing an explanation, agrees it is a contradiction. We move on, me convinced that I had contributed to their work (the work I assumed that others, not just myself, engaged in: namely, piecing together as accurate a model of reality as possible), only to find, weeks or months later, that on discussing X again, they still gave the same contradictory answers as before, with apparently no memory of our past discussion.

Or discussions. This pattern has repeated up to 4 times (that I've bothered to keep track), my interlocutor agreeing with my corrections, and then showing no sign of having even heard of them down the road.

My whole above description of "the result" is faulty in a subtle way, one that undermines the entirety of it. Here's what was really happening, expressed from my friend's perspective.

The (real) result: A friend and I were discussing X. They showed respect for me by asking me to express myself on it several times. At some point, they dared to point out, rather like a pedantic schoolboy, that there was a sort of inconsistency between my expressions and the forms we studied in textbooks on logic. They were not rude, though, so I graciously acknowledged the rather dry, uninteresting observation, after which we happily continued our conversation. I had thought for a moment that they wished to challenge me, but all seemed to be rather well-resolved, so I forgot the particulars of the incident, only taking with me the general, updated state of our relationship, including, for example, the points that my expressions were generally respected, but that my friend was willing to make some small challenges if they felt like it, though also willing to move on after I was gracious in response.

Or something like this, perhaps. And this key difference I call the map vs the button idea.

I, and other nerds, are conceptual cartographers, attempting to piece together a map of reality. I've often wondered ...
Dan Dennett on Stances 
by Charlie Steiner — 2017-12-27T08:15:53.124Z
Link: https://ase.tufts.edu/cogstud/dennett/papers/intentionalsystems.pdf

Part of a series. Index is here.

I'm writing up a post on consciousness, and I realized that this is worth linking separately. The gist of it is that we can think of a person as having feelings and intentions, and we can think of a person as a colleciton of atoms obeying the laws of physics, and both are valid ways of thinking. The person is the same thing out in reality either way - it's just the model we're using that changes, and, specifically, what kind of predictions our model is good at making.

You can really see where Dennett was influenced by Less Wrong. Especially in his work from the `70s.

Anyhow, I think this review is a real Dennett classic, up there with Eliminate The Middletoad. If you have a favorite in a similar vein, I'd love to hear about it.
TSR #7: Universal Principles
by Hazard — 2017-12-27T01:54:39.974Z

**This is part of a series of posts where I call out some ideas from the latest edition of The Strategic Review (written by Sebastian Marshall), and give some prompts and questions that I think people might find useful to answer. I include a summary of the most recent edition, but it's not a replacement for reading the actual article. Sebastian is an excellent writer, and your life will be full of sadness if you don't read his piece. The link is below.

Background Ops #7: UniversaL Principles

SUMMARY

Don’t forget about soft technologies.
Examples of Places with principle oriented decision making

Bridgewater (Ray Dalio’s hedge fund)

Principles

(Ray Dalio’s book on how he thinks about principles)

Book In a Box (unorthodox publishing company)

The Book In a Box Culture Doc

Rules Concerning principles

Principles must state action.
Principles must have an antithesis.
Principles must overrule power.

Hundreds of all the little decisions you have to make are automatically made by having strong principles.
Guidance

Search for and define universal principles
Diagnose before prescription
Operationalize

Pulling it all together

Recognize that universal principles can be distilled, and that it’s worth doing.
Schedule enough time to explore, codify, create, test, and operationalize them.
Start by diagnosing the problem first, before jumping into solutions.
Codify the problem; write it down.
Explore different solutions until you find a set of solutions that are the food fit.
Codify the solutions; write those down.
Operationalize it — build tools and practices around ensuring the principle is put into action.
Entrain it — practice until it’s consistently happening in your life and your organization.

I notice that often I feel a tinsy bit confused when thinking about principles. I have accepted that having explicit principles and operating on them is something that has a good shot at making me more effective, yet I’ve always felt a bit hazy about what exactly principles are. Should they outline the sort of world states I'm striving for? Should they be strict policies that inform me on how to make decisions? Do I decide them or discover them?

I’d really recommend you read the BIAB (Book In a Box) Culture document that Sebastian uses examples from. Seeing a list of 10 well thought out, well described principles has done wonders to my understanding of how to usefully think about principles....
AlphaGo variant reaches superhuman play in multiple games
by Commander Zander — 2017-12-26T16:19:35.804Z
Link: https://www.theguardian.com/technology/2017/dec/07/alphazero-google-deepmind-ai-beats-champion-program-teaching-itself-to-play-four-hours

https://www.theguardian.com/technology/2017/dec/07/alphazero-google-deepmind-ai-beats-champion-program-teaching-itself-to-play-four-hours

https://arxiv.org/abs/1712.01815

I'm posting this slightly late; the paper is from December 5.

I'd be interested to learn if AlphaZero could be applied to other closed-environment tasks, such as designing hardware in a simulator.
The Story CFAR
by Zvi — 2017-12-25T15:10:00.320Z

In addition to to my donation to MIRI, I am giving $4000 to CFAR, the Center for Applied Rationality, as part of their annual fundraiser. I believe that CFAR does excellent and important work, and that this fundraiser comes at a key point where an investment now can pay large returns in increased capacity.

I am splitting my donation and giving to both organizations for three reasons. I want to meaningfully share my private information and endorse both causes. I want to highlight this time as especially high leverage due to the opportunity to purchase a permanent home. And importantly, CFAR and its principles have provided and in the future will provide direct personal benefits, so it’s good and right to give my share of support to the enterprise.

As with MIRI, you should do your own work and make your own decision on whether a donation is a good idea. You need to decide if the cause of teaching rationality is worthy, either in the name of AI safety or for its own sake, and whether CFAR is an effective way to advance that goal. I will share my private information and experiences, to better aid others in deciding whether to donate and whether to consider attending a workshop, which I also encourage.

Here are links to CFAR’s 2017 retrospective, impact estimate, and plans for 2018.

I

My experience with CFAR starts with its founding. I was part of the discussions on whether it would be worthwhile to create an organization dedicated to teaching rationality, how such an organization would be structured and what strategies it would use. We decided that the project was valuable enough to move forward, despite the large opportunity costs of doing so and high uncertainty about whether the project would succeed.

I attended an early CFAR workshop, partly to teach a class but mostly as a student. Things were still rough around the edges and in need of iterative improvement, but it was clear that the product was already valuable. There were many concepts I hadn’t encountered, or hadn’t previously understood or appreciated. In addition, spending a few days in an atmosphere dedicated to thinking about rationality skills and techniques, and socializing with others attending for that purpose that had been selected to attend, was wonderful and valuable as well. Such benefits should not be underestimated.

In the years since then, many of my friends in the commu...
2017 AI Safety Literature Review and Charity Comparison 
by Larks — 2017-12-24T18:52:31.816Z

Summary: I review a significant amount of 2017 research related to AI Safety and offer some comments about where I am going to donate this year. Cross-posted from

here

upon request.

Contents

Contents

Introduction

The Machine Intelligence Research Institute (MIRI)

The Future of Humanity Institute (FHI)

Global Catastrophic Risks Institute (GCRI)

The Center for the Study of Existential Risk (CSER)

AI Impacts

Center for Human-Compatible AI (CFHCA)

Other related organisations

Related Work by other parties

Other major developments this year

Conclusion

Disclosures

Bibliography

Introduction

Like last year

, I’ve attempted to review the research that has been produced by various organisations working on AI safety, to help potential donors gain a better understanding of the landscape. This is a similar role to that which GiveWell performs for global health charities, and somewhat similar to an securities analyst with regards to possible investments. It appears that once again no-one else has attempted to do this, to my knowledge, so I've once again undertaken the task. While I've been able to work significantly more efficiently on this than last year, I have been unfortunately very busy with my day job, which has dramarically reduced the amount of time I’ve been able to dedicate.

My aim is basically to judge the output of each organisation in 2017 and compare it to their budget. This should give a sense for the organisations' average cost-effectiveness. Then we can consider factors that might increase or decrease the marginal cost-effectiveness going forward. We focus on organisations, not researchers.

Judging organisations on their historical output is naturally going to favour more mature organisations. A new startup, whose value all lies in the future, will be disadvantaged. However, I think that this is correct. The newer the organisation, the more funding should come from people with close knowledge. As organisations mature, and have more easily verifiable signals of quality, their funding sources can transition to larger pools of less expert money. This is how it works for startups turning into public companies and I think the same model applies here.

This judgement involves analysing a large number papers relating to Xrisk that were produced during 2017. Hopefully the year-to-year volatility of output is sufficiently low that this is a reasonable metric. I also at...
Idea: Monthly Community Thread
by Chris_Leong — 2017-12-24T12:43:01.328Z

The FrontPage Posting Guidelines currently discourage community focused discussion and suggest including the community tag once a tagging system has been implemented. This system hasn't been implemented yet, but I believe it is still important to be able to have some level of community discussion in the meantime.

One suggestion I have is the idea of a monthly community thread. This would avoid this discussion from being overwhelming, but it would also allow necessary discussion to take place. I would be happy to create these. Would anyone object to me doing this?
I Am not a Tax Lawyer
by Jacob Falkovich — 2017-12-24T03:20:00.459Z

I am really not a tax lawyer. This will become obvious soon enough, but it’s important for my readers to keep in mind from the very top. Still, this spoke to my heart:

I sometimes think I missed my calling as a tax lawyer. Tax lawyers, if they are any good, approach the world with a childlike sense of wonder. Someone (Congress) has given them a magical toy (the tax code) to play with, a toy whose possibilities are limited only by their imagination. And their job is to come up with new games to play with it, new combinations to try, new solutions to the riddles hidden in the code.
– Matt Levine

Well, Congress gave Americans a new toy, and so far all rationalists / effective altruists have done is whine about it. I actually think that Scott’s thinking about the tax plan is very confused. The tax code is not a purchase of $100 billion, it’s better thought of as enacting a transfer and establishing a set of incentives. The transfer is mostly from people who will pay taxes in the future (young Americans) to people who owned American stocks in 2017 (older and richer Americans). This part isn’t great. The incentive is for people and corporations to make money in the US. This part isn’t terrible.

We can compare this to Bernie Sanders’ free college plan, which Scott thinks is better. Subsidizing college would transfer money from taxpayers to colleges, which is pretty bad. It will also incentivize marginal students to waste more precious years in colleges while making the zero-sum game of credentialism and signaling ever more destructive, this part is really fucking bad.

The tax plan is poorly thought out, unfair to blue states, and inefficient. But it’s not a catastrophe – it’s not even as bad as free college. But it doesn’t matter how bad we think the tax plan is, the plan is here to stay (for at leat a few years). The important thing to focus on now is how do we exploit it?

If we’re playing a board game and you munchkin the rules, I can complain that you’re following the letter of the law but not the spirit. But the government can’t complain – it writes the rules, and it does so with the full expectation that every citizen will do their damnedest to exploit them as they are written. If we’re worried that greedy plutocrats get most of the gains from gaming the system, that just means that good folk like us need to step up and do our fair share of cheating optimizing.

Cont...
The Behavioral Economics of Welfare
by Vanessa Kosoy — 2017-12-22T11:35:09.617Z

Crossposted from Tumblr.

Epistemic Status: I offer some speculation, don’t really understand what is going on, would be glad to receive more insight.

You sometime hear libertarians say that welfare is unnecessary since private charity can replace it. IMO, it looks like the evidence doesn’t support this: only few people donate sizable amounts to charity, and historically, charity didn’t substantially ameliorate poverty during the early Industrial revolution (to the best of my knowledge). In fact, welfare developed precisely because the systems of poverty relief that existed in small rural communities became obsolete with mass industrialization and urbanization.

This seems somewhat paradoxical. All Western-style democracies have welfare systems, which means that presumably the majority is interested in having a welfare system, despite that it is a financial net negative for most people (that is, most people pay more money in taxes for welfare than receive welfare, I think?) So, apparently most people care about the poor. In this case, why can’t charity replace the welfare system?

Let’s consider several theories. These theories are not mutually exclusive, and the real explanation might be a combination of several (or something else altogether.)

Theory 1: People don’t trust charity funds. Normally, when you buy a commodity or a service, you can test its quality directly, and if it is found wanting you will not return to the same supplier. On the other hand, it is difficult to observe the effects of charity if you are not acquainted with the recipients. Therefore, verifying the trustworthiness of a charity fund is much more costly than verifying the trustworthiness of other suppliers, and most people aren’t sufficient altruistic to pay this cost.

This theory is somewhat supported by many explanations I heard people give for why they don’t donate to charity, although many times it sounds a bit like an excuse.

Theory 2: People indeed care about the poor, but only about the poor in their neighborhood or town. Therefore, acting against poverty requires either very local charities or coordination. Coordination is difficult (tragedy of the commons) and very local charities would face even greater trust issues (This theory is an extension of Theory 1.)

I liked this theory initially, but upon deliberation I find it difficult to believe. Most people who speak out in favor of h...
The essay "Interstellar Communication Using Microbes: Implications for SETI" has implications for The Great Filter.
by MakerOfErrors — 2017-12-22T06:05:22.671Z

Apparently I can add body text to a link post, instead of having to give a TL;DR as a comment? Excellent. Here are some excerpts from the essay:

Let us consider our own Sun as a case in point. At 1 AU, the sun attracts objects to it with a gravitational acceleration of 0.006 m/s2. It also repels objects from it via its light pressure with a force of 0.000009 N/m2. If these two forces are equal, and object will feel no attraction from the Sun, and fly out of the solar system in a straight line with the Earth’s velocity of 30 km/s. (Both of these forces change with the inverse square of their distance from the Sun, so if they are equal at 1 AU, they remain equal at any distance.)

...

So, depending on assumed conditions, the diameter of the objects could range from 1 to 10 microns, and be readily projectible across interstellar distances using no other mechanism than the pressure of the star’s light. This is precisely the size range of many typical bacteria.

...

So the question is, what kind of information is really worth broadcasting – that is distributed as widely as possible to people who we don’t know and are not likely to hear back from? If human experience is any guide, the answer is propaganda.

...

The key to propaganda is in the root of the word itself: propagate. Through propaganda we seek to propagate ourselves across both space and time. This can be in spirit, as in the cases described above, or in the flesh, through physical reproduction.

...

It is a striking fact that, despite several centuries of microbial research by thousands of competent investigators, no free-living organisms have been found on Earth that are simpler than bacteria. This is truly remarkable because, as simple as bacteria may be compared to more complex organisms, they are certainly not simple in any absolute sense, incorporating as they do, among other things, the entire elegant double-helix scripted language of DNA. Believing that bacteria were the first life forms to emerge from chemistry is like believing that the iPhone was the first human-invented machine. This is incredible. Just as the development of the iPhone had to be preceded by the development of computers, radio, telephones, electricity, glassware, metallurgy, and written and spoken language, to name just a few necessary technological predecessors, so the creation of the first bacterium had to

...
Rationalist Politicians
by Bound_up — 2017-12-21T23:29:44.984Z

The following ideas significantly overlap in my mind:

Humans as political animals
Normies
Social Thinkers
Non-nerds
Subcommunicators
Homo Hypocriticus (the kind of hypocrisy everyone's secretly okay with)
Ribbonfarm "Losers"
Tribalists

The following contrasting ideas also overlap:

Nerds
Literal/propositional thinkers
Straightforward, forthright, open communicators
Ribbonfarm "Clueless"

I'm trying to hint at a dichotomy, or more likely, a spectrum that I've been thinking about. Most of us on LW are probably on one side; normal people are on the other. Politicians are also on the other side.

Nerds think in terms of propositions about reality. Is X true or not? Does it correspond to reality or not? This question often overrides other instincts like "Is this okay to say or not?" "Is this rude to express or not?" "Will people understand what I mean by this or not?"

This nerdy kind of propositional thinking is so natural to me, that it hardly needs explanation. It means exactly what it sounds like. At least, the way it sounds to other nerds is very clear. Now we're going to review the non-nerdy "social thinkers," to whom this may sound like a series of jumbled signals, poorly constructed, revealing both an socially inept mind, and a very rude one to suggest such faux pas so openly.

Social thinkers, in contrast, hear a bunch of signals because that's their vocabulary, their conceptual palette, the visible spectrum of their mental landscape. Social thinkers see a social reality. Their idea of "true" is strongly tied to their sense of "in-group acceptable to profess." If something affords status to them, it's well on the way to seeming "true" to them in as strong a sense of the word as they ever use.

(On some deeper level, they know which things they'd bet their survival on if they had to, but the word "truth" does not evoke such ideas to them. The word "truth" activates in their concept-space a cluster of ingroup-specific concepts that they need to profess)

Now, I'm getting to a point here. Politicians are overwhelmingly social thinkers. They have a natural sense of social status, how to get it, and how to use it that all rests on a foundation of perceiving with clarity the "social truth" of a statement, ie, the effects on soc...