trentmkelly/LessWrong-43k huggingface dataset:
The rationalist community's location problem
by mingyuan — 2020-09-23T18:39:26.278Z

The Problem

Basically ever since the first rationalists settled in Berkeley, people have been saying, “Why do you live in Berkeley, Berkeley sucks! You should all move to Location X instead, it’s so much better.” The problem has always been that no one agrees on what Location X is. Some common candidates for Location X:

A smaller, cheaper, friendlier US city
NYC
Australia
Canada
Somewhere with very low cost of living (often in Southeast Asia or Latin America)
London
Oxford
Blackpool
Prague
A castle
A private island

and of course

Wherever the speaker is from

In the past I've brushed off all such suggestions, because it was just too hard a coordination problem to get multiple hundreds of rationalists to leave Berkeley, where they've gotten jobs, rented or even bought houses, established organizations, enrolled in schools, and established social circles.

But we're in a unique time! Due to the pandemic, there's far less reason to stay in any one place - work and school are remote, expensive leases can be terminated, and you can't see your friends anyway. Most of the rationalist houses I know have moved or dissolved, and the former Berkeley rationalists are flung across all corners of the globe (yeah I know globes don't have corners). A fair number of us have stayed, but I think for most of us it's just because our friends are here, we're hoping that someday the rest of our friends come back, and we're not sure where else to go.

So, if ever there were a time when we actually had the chance to move the physical locus of the rationalist community, it's now. Below, I'll lay out what I believe to be some of the most important general considerations for deciding on a new location. I encourage people to make their case for a specific location, either in comments or in their own posts. (Looking at you, Mikk!)

Considerations for Location X

Potential dealbreakers

Visas

In order to settle in a location, you have to be able to legally live there long-term. Most Berkeley rationalists are US citizens, and those who aren't have already paid the steep cost of acquiring US visas and learning US immigration law. This feels like a strong argument in favor of staying in the US somewhere, although it's possible there are places where this wouldn't actually be that much of an issue. In any case, it's certainly an argument against countries with strict immigration laws, like Switzerland....
[AN #118]: Risks, solutions, and prioritization in a world with many AI systems
by Rohin Shah — 2020-09-23T18:20:04.779Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

AI Governance: Opportunity and Theory of Impact (Allan Dafoe) (summarized by Rohin): What is the theory of change for work on AI governance? Since the world is going to be vastly complicated by the broad deployment of AI systems in a wide variety of contexts, several structural risks will arise. AI governance research can produce “assets” (e.g. policy expertise, strategic insights, important networking connections, etc) that help humanity make better decisions around these risks. Let’s go into more detail.

A common perspective about powerful AI is the “superintelligence” perspective, in which we assume there is a single very cognitively powerful AI agent. This leads people to primarily consider “accident” and “misuse” risks, in which either the AI agent itself “wants” to harm us, or some bad actor uses the AI agent to harm us.

However, it seems likely that we should think of an ecology of AI agents, or AI as a general purpose technology (GPT), as in e.g. CAIS (AN #40) or Age of Em. In this case, we can examine the ways in which narrow AI could transform social, military, economic, and political systems, and the structural risks that may arise from that. Concrete examples of potential existential structural risks induced by AI include nuclear instability, geopolitical turbulence, authoritarianism, and value erosion through competition.

A key point about the examples above is that the relevant factors for each are different. For example, for nuclear instability, it is important to understand nuclear deterrence, first strike vulnerability and how it could change with AI processing of satellite imagery, undersea sensors, cyber surveillance and weapons, etc. In contrast, for authoritarianism, relevant processes include global winner-take-all-markets, technological displacement of labor, and authoritarian surveillance and control.

This illustrates a general principle: unlike in the superintelligence perspective, the scope of both risks and solutions in the ecology / GPT perspectiv...
Dehumanisation *errors*
by Stuart_Armstrong — 2020-09-23T09:51:53.091Z

In response to my post contrasting value learning with anthropomorphisation, steve2152 brought up the fact that dehumanisation can be seen as the opposite of anthropomorphisation.

I agree with this insight, but only when dehumanisation causes errors of interpretation. I was using empathy in the sense of "insight into the other agent", rather than "sympathy with the other agent".

In practice, dehumanisation does tend to cause errors. We see outgroups as more homogeneous, coherent, and organised than they actually are. Despite the suave psychopaths depicted in movies, psychopaths tend to be less effective at achieving their goals (as evidenced by the large number of psychopaths in prison). Torturers are less effective at extracting true information than classical interrogators.

Now, it's not a universal law by any means, but it does seem that dehumanisation can often lead to errors, and from that perspective can be seen as a failure of value learning.

The meaning of errors

"Objection! Hold on just a minute!" screams the convenient strawman I have just constructed.

"You've claimed that 'agent's goals' are interpretations by the outside observer; that you can model a human as perfectly rational, without being wrong. You've claimed that this is 'structured white box knowledge', which can't be deduced from the agent's policy or its algorithm."

"Given that, how can you claim that anyone 'fails' at interpreting the goals of others, or that they make 'errors'?"

This a very valid point, strawman, but I've also pointed out that human theory of mind/empathy is very similar from human to human, and tends to agree with how we interpret our own goals. Because of this, there is a rough "universal human theory of mind", ie a universal way of going from human policy to human preferences.

When I'm talking about errors, I'm talking about deviations from this ideal[1].

Because human theories of mind do not agree perfectly, there will always be an irreducible level of uncertainty in this ideal, but there is agreement on the broad strokes of it. ↩︎
The new Editor
by habryka — 2020-09-23T02:25:53.914Z

Look	at	this	glorious	table

Celebrations! The new editor is finally here!

Starting from today, all desktop users will by-default use the new visual editor that we've been testing for a while. While the primary goal of this is to have a better foundation on which to build future editor features, here are a number of things you can do starting from today:

Insert tables! A heavily requested feature.
Copy-paste LaTeX without everything breaking!
Nest bullet lists and other block elements in blockquotes! (still no nested blockquotes though, though if enough people want that, it would be easy to change)
Image Uploads! (Just drag-and-drop images into the editor, and things should work out naturally. You can also copy-paste, though beware that copy-pasting from other websites means we link to the copies of those images from other websites, and won't reupload them.)
Much less jankyness and brokenness!

Let us know what you think about the new editor. We've been testing it for a while and have been pretty happy with it (and users who had opted into beta features also had predominantly positive feedback). You can also use the old editor if you run into any problems by checking the "Restore the previous WYSIWYG editor" checkbox in your user settings.
New Tagging Power Tools: Dashboard + Upgraded Tag Editing Experience
by Ruby — 2020-09-23T01:49:14.600Z

Following the strong response of taggers to calls for help, plus lessons learned from our earlier designs, we are launching new and improved power tools for maintaining and improving the tag/wiki (twiki?) corpus.

We are first applying these tools to completing the underway Wiki Import campaign, and when that's done we'll reconfigure them for long-term use.

There are several connected parts to the new tools:

Tag Flags
New Twiki Dashboard (www.lesswrong.com/tags/dashboard)
Upgraded Tag Editing
Tag Discussion Sections

Tag Flags

At the heart of the new Tagging Power Tools are tag flags. These are a limited and generally fixed set of flags that people can set on tags when they require work, and then remove when the work is done.

The Special Tag Flags for the Wiki Import
The numbers indicate how many tags have this flag set.

Temporarily, the tag flags are designed specifically for the import (those from the deprecated spreadsheet). Long-term they'll be general things like "needs a description", "should have more posts tagged", "is out of date". The tag flags will be managed by admins, of course, with input from taggers. This is largely because we want to keep the number to something 5 main ones and maybe another 5-10 minor ones. Keep it manageable.

We are deprecating the Tag Grading Scheme.

New Twiki Dashboard

You can get lists of tags with each kind of tag-flag on the new dashboard. By default, you get a list of all tags with any flag applied. If you click on a flag in the header list, the page will be filtered for only tags with that tag. The selected Tag Filter displays in black. You can remove the filter with the Reset Filter button.

If you hover of the flag, you'll see a description describing what it means and how to address it.

Hover-Over explains the Tag Flag

Editing Tags

Once you've found tags you want to edit, you have two options for how to do that.

Twiki Dashboard On-Page Editing
Full-Edit on Tag Page

On the Twiki Dashboard page, if you click the Edit button for a tag, it will open it up for editing on-page.

On-page editing.

As you see, here you can toggle the flags. Don't forget to click submit! If a removed flag was part of your filter, the tag probably won't disappear from the list until you refresh.

Upgraded Tag Editing

Continuing from the above, you can alternatively start editing tags in full-edit mode by clicking on the text body on the dashboard (rath...
AI Advantages [Gems from the Wiki]
by habryka — 2020-09-22T22:44:36.671Z
Link: https://www.lesswrong.com/tag/ai-advantages

During the LessWrong 1.0 Wiki Import we (the LessWrong team) discovered a number of great articles that most of the LessWrong team hadn't read before. Since we expect many others to also not have have read these, we are creating a series of the best posts from the Wiki to help give those hidden gems some more time to shine.

The original wiki article was fully written by Kaj Sotala, who I've added as a coauthor to this post. Thank you for your work on the wiki!

AI advantages are various factors that might favor AIs in case there was ever a conflict between them and humans. These can be classified as hardware advantages, self-improvement capabilities, co-operative advantages, and human handicaps.

Hardware advantages

Superior processing power: Having more serial processing power would let an AI think faster than humans, while having more parallel processing power and more memory would let it think about more things at once.

Self-improvement capabilities

An AI with access to its source code may directly modify the way it thinks, or create a modified version of itself. An AI can intentionally be built in a manner that is easy to understand and modify, and may even read its own design documents. Self-improvement capabilities may enable recursive self-improvement to occur, thereby triggering an intelligence explosion.

Improving algorithms: An AI may modify its existing algorithms, e.g. making them faster, to consume less memory, or to rely on fewer assumptions.
Designing new mental modules: A mental module is a part of a mind that specializes in processing a certain kind of information. An AI could create entirely new kinds of modules, custom-tailored for specific problems.
Modifiable motivation systems: Humans frequently suffer from problems such as procrastination, boredom, mental fatigue, and burnout. A mind which did not become bored or tired with its work would have a clear advantage over humans.

Co-operative advantages

Copyability: A digital mind can be copied very quickly, and doing so has no cost other than access to the hardware required to run it.
Perfect co-operation: Minds might be constructed to lack any self-interest. Such entities minds could share the same goal system and co-operate perfectly with one another.
Superior communication: AIs could communicate with each other at much higher bandwidths than humans, and modify themselves to understand each other better

...
Sunday September 27, 12:00PM (PT) — talks by Alex Flint, Alex Zhu and more
by habryka — 2020-09-22T21:59:56.546Z

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts).

For the first hour, we will have a series of lightning talks each lasting about 5 minutes followed by discussion. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
From 1PM to 2PM, we will hangout in Gather.town. Link will be sent out closer to the event.
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

Alex Flint: "Why go on retreat?"

Curated posts: The Ground of Optimization, Reply to Paul Christiano on Inaccessible Information, Our take on CHAI's research agenda in under 1500 words

Other speakers: TBA

Details

When? Sunday September 27, 12:00PM (PT)

Where? https://us02web.zoom.us/j/87259855821
Comparative Advantage is Not About Trade
by johnswentworth — 2020-09-22T18:43:11.496Z

Braudel

is probably the most impressive historian I have read. His quantitative estimates of premodern populations and crop yields are exactly the sort of foundation you’d think any understanding of history would be based upon. Yet reading his magnum opus, it became steadily clearer as the books progressed that Braudel was missing some fairly fundamental economic concepts. I couldn’t quite put my finger on what was missing until a section early in book 3:

... these deliberately simple tautologies make more sense to my mind than the so-called ‘irrefutable’ pseudo-theorem of David Ricardo (1817), whose terms are well known: that the relations between two given countries depend on the “comparative costs” obtaining in them at the point of production

Braudel, apparently, is not convinced by the principle of

comparative advantage

. What is his objection?

The division of labor on a world scale (or on world-economy-scale) cannot be described as a concerted agreement made between equal parties and always open to review… Unequal exchange, the origin of the inequality in the world, and, by the same token, the inequality of the world, the invariable generator of trade, are longstanding realities. In the economic poker game, some people have always held better cards than others…

It seems Braudel is under the impression that comparative advantage is only relevant in the context of “equal” exchange or “free” trade or something along those lines.

If an otherwise impressive economic historian is that deeply confused about comparative advantage, then I expect other people are similarly confused. This post is intended to clarify.

The principle of comparative advantage does not require that trade be “free” or “equal” or anything of the sort. When the Portugese or the British seized monopolies on trade with India in the early modern era, those trades were certainly not free or equal. Yet the monopolists would not have made any profit whatsoever unless there were some underlying comparative advantage.

For example, consider an oversimplified model of the salt trade. People historically needed lots of salt to preserve food, yet many inland areas lack local sources, so salt imports were necessary for survival.

Transport by ship was historically orders of magnitude more efficient than overland

, so a government in control of a major river could grab a monopoly on the salt trade. Since the peo...
The Haters Gonna Hate Fallacy
by Kaj_Sotala — 2020-09-22T12:20:06.050Z

Occasionally I see people doing what I think of as the “Haters Gonna Hate Fallacy”.

The HGHF says something like: “People are going to misinterpret you no matter how carefully you word things. Therefore, there’s no point wasting time wording things carefully.”

An example:

“I think [term X] in your post is going to cause misunderstandings, I’d suggest phrasing it differently.”
“Oh, haters are gonna hate, there’s no amount of rephrasing that’s going to prevent this from being misinterpreted if people want to.”

Now there’s obviously a grain of truth in this. It is impossible to phrase something in a way that would always be interpreted correctly, and for pretty much any message there are people who are hostile to it and who will twist it in the most uncharitable possible way.

The fallacy is in assuming that if you cannot avoid all misunderstandings, there is no point in avoiding any misunderstandings. Maybe 5% of your audience will dismiss the message no matter what, but 30% will dismiss the old phrasing while being receptive to the new phrasing.

This is most obvious if you take it to an extreme:

“Hey maybe you shouldn’t start your essay by saying that all of your readers are idiots who deserve to be shot.”
“Eh, if that upsets them then they wouldn’t like me explaining the theory of general relativity anyway.”

Communication is hard and – importantly – contextual. Most of your readers will be reasonable people and assume you to use words to mean things they’re used to them meaning. If they’re used to word X being used differently than how you mean it, that doesn’t make them haters.

When I’ve fallen into something like the fallacy myself, it has often been motivated by an unwillingness to put in work. Other people should just understand me right away! “It’s beneath me to waste my time on doing other people’s interpretative work for them!” It’s dangerous to psychoanalyze others, but I have seen at least one person communicate unclearly, have that pointed out to them, then argue for why it was right for them to be unclear… only to later on admit that they were enjoying the frustration of being misunderstood.

Now avoiding misunderstandings is a lot of work, and i...
Anthropomorphisation vs value learning: type 1 vs type 2 errors
by Stuart_Armstrong — 2020-09-22T10:46:48.807Z

The Occam's razor paper showed that one cannot deduce an agent H's reward function (RH - using the notation from that paper) or their level of rationality (pH) by observing their behaviour or even by knowing their policy (πH). Subsequently, in a LessWrong post, it was demonstrated that even knowing the agent's full algorithm (call this aH) would not be enough to deduce either RH or pH individually.

In an online video, I argued that the reason humans can do this when assessing other humans, is because we have an empathy module/theory of mind EH, that allows us to model the rationality and motives of other humans. These EH are, crucially, quite similar from human to human, and when we turn them on ourselves, the results are similar to what happens when others assess us. So, roughly speaking, there is an approximate 'what humans want', at least in typical environments[1], that most humans can agree on.

I struggled to convince people that, without this module, we would fail to deduce the motives of other humans. It is hard to imagine what we would be like if we were fundamentally different.

But there is an opposite error that people know very well: anthropomorphisation. In this situation, humans attribute motives to the behaviour of the wind, the weather, the stars, the stock market, cute animals, uncute animals...

So the same module that allows us to, somewhat correctly, deduce the motivations of other humans, also sets us up to fail for many other potential agents. If we started 'weakening' EH, then we would reduce the number of anthropomorphisation errors we made, but we'd start making more errors about actual humans.

So our EH can radically fail at assessing the motivations of non-humans, and also sometimes fails at assessing the motivations of humans. Therefore I'm relatively confident in arguing that EH is not some "a priori" object, coming from pure logic, but is contingent and dependent on human evolution. If we met an alien race, they we would likely assess their motives in ways they would find incorrect - and they'd assess our motives in ways we would find incorrect, no matter how much information either of us had.

See these posts for how we can and do extend this beyond typical environments. ↩︎
Emptiness and Form
by lsusr — 2020-09-22T04:20:49.986Z

Translation note: There is no English equivalent to the Sanskrit words शून्यता śūnyatā and रूप rūpa. By convention, शून्यता is translated "emptiness" and रूप is translated "form". I follow this convention. My use of the words "emptiness" and "form" in this post have little to do with the English words "emptiness" and "form"; they are placeholders for Sanskrit.

Consider a cat. From the perspective of fundamental physics, the cat is a collection of particles no more special than any other collection of particles. There is no clear line between "cat" and "non-cat". Everything is quantum fields. The "cat" is a representation created by the human mind. It is a trick of human perspective. From the perspective of an omniscient unbiased observer, the cat is just a scoop of water in a limitless ocean.

Cats are real.

The perspective "cats are real" is called "form". The perspective "cats are an arbitrary ontology with no well-defined meaning amongst the fundamental laws of the universe" is called "emptiness". There is no conflict between form and emptiness just as there is no conflict between quantum mechanics and classical mechanics. They are different ways interpreting the same thing at different scales.

Classical mechanics can be more practical than quantum mechanics even though quantum mechanics is more fundamental than classical mechanics. Similarly, emptiness is more fundamental than form yet form is a more useful model of the world than emptiness. Emptiness and form are neither equally true nor equally practical.

Maps ≠ Form & Emptiness ≠ Territory

You could say "form" roughly corresponds to "maps" and "emptiness" roughly corresponds to "territory". That would constitute a better translation from the original Sanskrit than "form" and "emptiness". But the form-emptiness dichotomy draws its line in a slightly different place than the map-territory dichotomy.

The map-territory dichotomy draws the line between reality and models of reality. The map-territory dichotomy distinguishes between reality and one's simplified models of reality. In this way, the map-territory dichotomy is a materialist perspective.

The form-emptiness dichotomy is an informatic perspective. If there is no difference between a map and a territory then—mathematically—the map and the territory are isomorphic respresentations of the same group.

Ontologies

"Emptiness" describes a shared quality b...