trentmkelly/LessWrong-43k huggingface dataset:
Not all communication is manipulation: Chaperones don't manipulate proteins
by ChristianKl — 2020-09-27T16:45:52.172Z

Epistemic status: Original work, explanation of a mental model that I developed for a few years that brings together knowledge from existing fields.

Is all communication manipulation? I hear this sentiment frequently expressed and want to explain in this article that there’s nonmanipulative communication by using protein folding as an intuition pump.

It is common knowledge within molecular biology that proteins fold into their native state. That native state is the folded shape that possesses a minimum of free energy. Finding global minima is however a hard problem. For bigger proteins, it's at the time of writing - still impossible to calculate the shape.

Even in vivo protein folding is a hard problem. Cells are densely packed with many different molecules that push against each other. Frequently, resources are wasted when a protein misfolds into a shape that's not its native state.

Nature is clever and developed a way to help proteins fold into their native state. Cells produce chaperones. A chaperone surrounds an unfolded protein to protect it from outside influences to help the protein to fold into its native state. A chaperone doesn't need to know the native state of a protein to help the protein fold into that state. Instead of manipulating the protein like a sculpture, it holds space for a protein to be safe from outside influences, while it folds into its native form.

This allows a chaperone that works in an uncomplicated way to achieve a result that very complex machine learning algorithms currently don't achieve. The machine learning algorithm tries to figure out the best way for the protein to fold while the chaperone just lets the protein find this way by itself.

The psychologist Carl Rogers advocated that good psychologists act in the same way nonmanipulative with their patients. In his view, it's not the job of the therapist to solve the problem of their patient by manipulating the patient into a healthy form. A good therapist isn’t like ta sculptor sculpts a sculpture. The job of the therapist is rather to hold a space for the patient in which the patient is safe from certain forces that prevent the patient from finding their healthy authentic native state.

I don't intend to argue for nonmanipulative communication from a moral perspective. In cases where you know how to fix the problem of the person you are talking with and are conf...
Numeracy neglect - A personal postmortem
by vlad.proex — 2020-09-27T15:12:45.307Z

My failed enlightenment

I've been thinking about my intellectual education, and what I wish had gone differently.

I am 26 years old. I’ve been reading books and going to school since I was 8. This puts my career as a learner at about 19 years. Honestly? I feel a bit disappointed. I've had a predominantly "humanistic" education, which is a nice way of saying that my gaps in scientific subjects are embarrassing. Meanwhile, I ended up interacting with people who've invested their formative years in getting a solid foundation in mathy and sciency subjects. Inevitably, I found myself envying their skills and wondering where my study time has gone, and what do I have to show for it.

In particular, I have diagnosed myself with a condition I call numeracy neglect. When I reflect on my education, I find that: (1) I was a bright and precocious kid. (2) I was always very curious and had a strong motivation to understand the world. (3) Despite this, and despite all the resources that society invested in me, I managed to go at least 15 years without learning much about mathematics, physics, chemistry, and computer science (to mention just the basics).

This contradiction pains me, but it also makes me curious. How does something like this happen?

Numeracy neglect

I will focus on mathematics, since it's a subject that most people are taught, but it's typically misunderstood and unappreciated.

Reflecting on my experience, I can identify two problems.

1. Aesthetic insensitivity. The inability to experience the beauty of mathematics, and to apply one’s general curiosity to it.

2. Epistemic ignorance. The inability to see or accept the fact that mathematics is the language of science, and if you don't understand mathematics, you won't understand most science. In general, the inability to understand mathematics' relevance and usefulness in life.

Another way to put it is that the first is a failure to grasp the intrinsic value of mathematics [1], while the second is a failure to understand its instrumental value.

How does this apply to my experience?

Aesthetic insensitivity

I was not born with a natural aversion for mathematics. I remember enjoying arithmetic and geometry in elementary school. Today, I feel a deep curiosity for mathematical subjects; I’ve also developed, or perhaps rediscovered, an aesthetic appre...
Sander Verhaegh on Quine's Naturalism
by Chris_Leong — 2020-09-27T10:58:59.845Z
Link: http://www.sanderverhaegh.nl/Interview%203AM%20Magazine.pdf

Quine is famous for his naturalistic stance on philosophy which aligns heavily with the stance of people on Less Wrong. In fact, I've seen him mentioned a few times, but I wanted to learn more about his views. This interview gives a pretty good overview.
The whirlpool of reality
by Gordon Seidoh Worley — 2020-09-27T02:36:34.276Z

This post is mostly to share this quote that presents a metaphor describing the dance between map and territory (and the key insight of embeddedness) that was new to me, so let me share it first:

Consider a whirlpool. We can observe it, we can admire it, and we can talk about it. But we don't readily appreciate that what we glibly call "the whirlpool" is not a particular thing. The whirlpool, shifting in the lake or on the river, doesn't hold its form. It becomes shallower; the molecules of water are in continuous change. In no two moments do we find the same configuration of water molecules defining the surface or shape of the whirlpool. Yes we view "the whirlpool" as a thing[.]

We imagine enduring things, even as we think of "them" as fleeting. We hold to this notion at the expense of realizing that there's no actual thing there at all.

[...]

If you attend very carefully, it becomes apparent that everything is like the whirlpool—not a thing at all, but change itself. If we could speed up our perspective enough, even a mass of stone would appear, over the course of millennia, to change and disappear, just like a whirlpool. (And if we alter our perspective in the opposite way, we'd see the stone's constant molecular and atomic changes.)

But there's a far more subtle point here. It's only out of convention that I used the terms stone and whirlpool. Because, in fact, there is only change[.] There's no abiding thing called "stone" or "whirlpool" that changes. There's really no particular thing that's stone or whirlpool or you or me at all.

This is from chapter 40, "Ice Forming in Fire", of Buddhism Is Not What You Think: Finding Freedom Beyond Beliefs by Steve Hagen. The title of the chapter is from a perhaps less easily relatable metaphor from Dogen that Hagen liberally translates as "What is Reality? An icicle forming in fire".

I like the whirlpool metaphor because it makes clear how it is our minds bringing things into existence by finding useful patterns in the soup of reality and reifying those patterns into more persistent-seeming forms. Our minds readily trick us into thinking our maps, models, and ideas of what reality is like are more solid than they actually are. It's a quick and easy jump from observation to reification and then from reification to detachment of the map from the territory, paying more attention to our beliefs about what reality is than real...
Blog posts as epistemic trust builders
by Adam Zerner — 2020-09-27T01:47:07.830Z

I've really been enjoying Zvi's weekly posts on the coronavirus. Keeping up with what's going on is something I want to do, but not badly enough to put in the time myself. I'm not even sure how capable I would be even if I wanted to put in the time myself; it seems difficult to sift through all of the information out there.

Reading Zvi's posts works out perfectly for me though. 20-30 minutes a week and I get just what I need.

But all of this only works because I trust Zvi.

Like most people nowadays, I spend a lot of time online. In particular, reading blog posts. LessWrong, Overcoming Bias, Slate Star Codex, Hacker News, FiveThirtyEight, etc. When I need a break, I have my little routine of websites that I click through.

Sometimes I reflect on how much value I get out of reading all of these blog posts. Nothing against the authors, but when I finish an article, I usually am not left with the feeling that I've gained much. I see it as a numbers game: most of the time I don't gain much, but once in a while I come across something that really influences me.

But even when I'm not left off feeling particularly inspired by a post, I think that there is something more subtle that I gain by reading it: epistemic trust.

By reading the same authors over and over again, I start to get a feel for how much I can trust their reasoning ability. The more I trust them, the more I update in response to what they say. And when I reflect on the updates I perform, a surprisingly large proportion of them are of the (rough) form "I'll take your word for it".

The ultimate example of this is probably with respect to AI safety. I think AI safety is a huge deal, but the reason why I think so largely comes from me saying "I'll take your word for it". I have a very amateurish understanding of it all and wouldn't really be able to come to the conclusion "this is by far the most important thing in the world" via gears level reasoning.

But fortunately, I have a very high level of epistemic trust for the rationalist community. I've been able to cash in on this trust and update my beliefs about something that is very, very important.

I want to be careful about what I'm implying here. I'm not trying to imply that epistemic trust building is the main purpose of blog posts. I'm not even trying to be at all precise about how important I think that function is. My point is just that I think it's a functio...
Distributed public goods provision
by paulfchristiano — 2020-09-26T21:20:05.352Z

Most people benefit significantly from privately funded public goods (e.g. Wikipedia).

If we all contribute to such public goods, then we can all end up better off. But as an individual it’s almost never a good return on investment. I think of supporting such public goods as being a good citizen, but that leaves open the question what is a good amount to contribute? I can make most decisions by balancing costs and benefits, but I think that basically never leads to making small contributions to public goods in the name of being a “good citizen.”

This post doesn’t aim to answer that full question, but it sets up one simple formal model for the situation and a very natural “wish list” for public goods funding norms. It then proposes the following norm that meets this wish list:

My contribution to public good X should be 100 times larger than the amount I’d personally benefit if public good X received 1% more funding.

For example, suppose I’m considering how much to contribute to Wikipedia. Maybe I get $1000/year of value from Wikipedia, and I’d get $1000.10 of value if Wikipedia had 1% more funding. This rule says I should contribute $10/year (= 10 cents x 100).

(In practice I think that labor is often a more important contribution than money, but a similar principle can apply. I’d also prefer that most donors to public goods use something like a donor lottery so that they can make a more informed decision about a larger amount of money when they do decide to give. This post is trying to bite off a tiny bit of the conceptual problem.)

Formal model

Assume there are n people and k projects. Person i makes a contribution x_ij to public good j. Let y_j = sum x_ij be the total funding for public good j.

Each person has a utility function U_i(y_1, …, y_k) expressing how much they benefit if the public goods are funded at levels y_1, …, y_k. The total utility for person i is U_i(y_1, …, y_k) – sum_j x_ij.

For simplicity I’ll assume that everyone’s utility is monotone in public goods: I’m never unhappy when one of the projects receives more funding.

I’ll also assume that the utility functions U are convex: increases in funding only ever decrease the marginal value of more funding. Convexity is often realistic and it makes it much easier to prove theorems or do computations, so it seems like a good starting point. Relaxing it seems important but not something I’m going to ge...
Notebook for generating forecasting bets
by Amandango — 2020-09-26T20:36:05.878Z

We made a colab notebook that lets you generate a bet from two people's Elicit distributions. You can edit the notebook to generate your bet (the changes won't be saved). Here's an example of a suggested bet between Ben Pace and SDM on AI timelines:

Comparison of predictions:

Snapshot link

What the notebook outputs:

If the event occurs between 24 Oct 2051 and 17 Sep 2059, Ben Pace should pay SDM $87. Otherwise, SDM should pay Ben Pace $13.

Disagreement: You disagree most between 24 Oct 2051 and 17 Sep 2059.

Probabilities: Ben Pace thinks the probability the event occurs in this range is 8%, while SDM thinks it's 18%.

Odds: You should make a bet with 87:13 odds.

Expected Value: Given your beliefs, you each win $5 in expectation

What this is actually doing:

We search across the question range to find the interval (with a width of 10% of the total range) where your probabilities differ the most, including the probabilities outside of the bounds
We determine odds that create equal and positive expected value for each person using the method outlined in this blog post
Surviving Petrov Day
by Mati_Roy — 2020-09-26T16:40:03.169Z

Sorry, but I can't let this metaphorical world be destroyed.

In Honoring Petrov Day on LessWrong, in 2020, Ben Pace explains that during Petrov's Day (ie. today), 270 people will have received an email giving them a launch code they could use to bring down LessWrong for a day. Here's how the website looked like:

At 7:37 EST, Neel Nanda posted that the site was already down.

Chris Leong writes:

Sorry, I got tricked:

petrov_day_admin_account September 26, 2020 11:26 AM Hello Chris_Leong,

You are part of a smaller group of 30 users who has been selected for the second part of this experiment. In order for the website not to go down, at least 5 of these selected users must enter their codes within 30 minutes of receiving this message, and at least 20 of these users must enter their codes within 6 hours of receiving the message. To keep the site up, please enter your codes as soon as possible. You will be asked to complete a short survey afterwards.

We failed at preventing the launch, but it's not over.

Stopping the threat

There's an image preventing access to the main page. To remove it, create a bookmark with any name, and put as the bookmarked URL:

javascript:var element = document.getElementsByClassName("PetrovDayLossScreen-root")[0]; element.parentNode.removeChild(element);

Then go on https://lesswrong.com and click the bookmark to remove the image.

Metaphor for anti-ballistic missiles.

Living through the winter

Here are alternative places where LessWrongers gather:

Effective Altruism Forum

AI Alignment Forum

Effective Altruism Group Directory

LessWrong

TheMotte

RadicalxChange Forum

I would say SlateStarCodex, but it already got its real world Petrov Day caused by the New York Times. People from SlateStarCodex now hangout at https://www.reddit.com/r/slatestarcodex/.

Metaphor for ALLFED researching food security.

Recovering from a backup

Use https://www.greaterwrong.com/, a mirror of LessWrong. If this also gets down, use the Wayback Machine to retrieve URLs from the GreaterWrong. If it also gets down, use WebCitation. If it also gets down, email me at contact@matiroy.com for a local backup (if this post gets taken down, I'll post it on https://matiroy.com). If I get offline, ask other rationalists that have made a local backup.

Here's how Gwern archives websites: https://www.gwern.net/Archiving-URLs. I used to use http://www.webcitation.org/comb to massively a...
Petrov Day is not about unilateral action
by sen — 2020-09-26T13:41:02.481Z

It's about the dangers of unrestrained escalation. It's a failure of humanity that we even got to the point where a few people's small and difficult decisions decided the fate of so much. This post is a friendly reminder to...

Please avoid escalating socio-economic disagreements to nuclear levels.
Please avoid escalating market competitions to Moloch levels.
Please avoid escalating paperclippers to superintelligent levels.

Learn to recognize signs of unrestrained escalation. You may be participating in unrestrained escalation...

If your information landscape becomes increasingly narrowed by a centralized source or by recommender algorithms.
If you attempt to fight over-escalated issues in a way that encourages opposing parties to fight back. For example, if you try to overtly starve the object of escalation of its resources and ability to act.
If your actions become increasingly coordinated with large and powerful groups.
If your judgment becomes increasingly dictated by stories told from a single point of view or single optimization criterion.

When you do recognize signs of unrestrained escalation, please take actions to de-escalate. You may do this...

By learning and teaching others to recognize unrestrained escalations.
By refusing to participate in group actions that encourage further escalation.
By starving the participants of their motivation (rather than of their resources and ability to act).
By using every escalation as a teaching opportunity for how to recognize and reduce future escalations.

And, of course, when considering whether a contentious situation warrants escalation, please do not divorce your opinions from the nuances of the current situation. In high-energy, high-pressure scenarios, small-scale effects often have large-scale consequences, and dually large-scale consequences are often driven by small-scale effects.

Thank you for your attention. Escalate responsibly.
What is complexity science? (Not computational complexity theory) How useful is it? What areas is it related to?
by philip_b — 2020-09-26T09:15:50.446Z

I've been reading "Playing with movement - Hargrove 2019". This book told me about "complexity science", which is a field which supposedly studies complex systems and it does it not by reductionism, but by looking at the system as a whole. It seems that some key concepts used in complexity science are: complex system, emergence, adaptivity, nonlinearity, self-organization, constraints, attractors, feedback loops. This book pitched an idea that with many complex adaptive systems, if you want the system to achieve a certain goal, it's bad to specify a specific plan for it and instead it's better to specify or build constraints under which the system, being adaptive and all, will achieve the goal on its own, supposedly in a more optimal way. Examples:

System: my body. Goal: get healthy by physical exercise. Specific plan: do strength training by doing specified sets of reps, using specific instructions on how to position my body. Alternative (constraints): live in such an environment that a lot of physical activity will happen naturally.

System: economics of a country. Goal: make it productive. Specific plan: central planning of what to produce, in what quantities and how. Alternative: free market economy, the organizations will behave however they want, and hopefully the whole system will act in an optimal way.

System: my body. Goal: eat healthily. Specific plan: write down a schedule of specific meals for the next year and eat them. A potential problem is I might miss something and that might lead to a lack of some nutrients. If instead I eat whatever I want, I will probably crave those other nutrients and hence I'll go and eat them. Obviously, this has downsides.

This idea seems plausible and very important. However, I've never heard of complexity science before. I've been following all kinds of links about complexity science trying to figure out what it is, what fields it's related to and what subareas it has. Also, I want to know how much of it is correct and how to apply it in the real world. Please help me figure out these questions. So, I guess I want a primer. Except I know that if I find a primer written by a complexity scientist, it'll claim that complexity science is the greatest invention ever. Instead, I want a sceptical primer which specifies the domain of applicability of complexity science and the domain of applicability of theory of complex ...
Honoring Petrov Day on LessWrong, in 2020
by Ben Pace — 2020-09-26T08:01:36.838Z

The Petrov Day Red Button.

Just after midnight last night, 270 LessWrong users received the following email.

Subject Line: Honoring Petrov Day: I am trusting you with the launch codes

Hello {username},

On Petrov Day, we celebrate and practice not destroying the world.

It's difficult to know who can be trusted, but today I have selected a group of (270) LessWrong users who I think I can rely on in this way. You've all been given the opportunity to not destroy LessWrong.

This Petrov Day, if you, {username}, enter the launch codes below on LessWrong, the Frontpage will go down for 24 hours, removing a resource thousands of people view every day. Each entrusted user has personalised launch codes, so that it will be clear who nuked the site.

Your personalised codes are: {codes}

I hope to see you in the dawn of tomorrow, with our honor still intact.

–Ben Pace & the LessWrong Team

P.S. Here is the on-site announcement.

Not Destroying the World

Stanislav Petrov once chose not to destroy the world.

As a Lieutenant Colonel of the Soviet Army, Petrov manned the system built to detect whether the US government had fired nuclear weapons on Russia. On September 26th, 1983, the system reported five incoming missiles. Petrov’s job was to report this as an attack to his superiors, who would launch a retaliative nuclear response. But instead, contrary to the evidence the systems were giving him, he called it in as a false alarm, for he did not wish to instigate nuclear armageddon. (He later turned out to be correct.)

During the Cold War, many other people had the ability to end the world – presidents, generals, commanders of nuclear subs from many countries, and so on. Fortunately, none of them did. As humanity progresses, the number of people with the ability to end the world increases, and so too does the standard to which we must hold ourselves. We lived up to our responsibilities in the cold war, but barely. (The Global Catastrophic Risks Institute has compiled this list of 60 close calls.)

In 2007, Eliezer named September 26th Petrov Day, and the rationality community has celebrated the holiday ever since. We celebrate Petrov's decision, and we ourselves practice not destroying things, even if it is pleasantly simple to do so.

The Big Red Button

Raymond Arnold has suggested many ways of observing Petrov Day.

You can discuss it with your friends.

You can hold a quiet, dignif...