trentmkelly/LessWrong-43k huggingface dataset:
The Wiki is Dead, Long Live the Wiki! [help wanted]
by Ruby — 2020-09-12T03:34:50.622Z

That's just a few of them. We imported like 5x as many as these.

With the goal of eventually archiving it fully, we have imported 573 pages and 266,000 words of content from the old LessWrong wiki to LessWrong 2.0

The old wiki is a great store of knowledge and still gets two thousand pageviews each day. Incorporating it into the new site gets us at least the following benefits:

Pages imported from the old wiki now appear in search results on LessWrong proper.
Pages imported from the old wiki benefit from all the features of new LessWrong such as hover-preview, subscriptions, commenting, and functioning as tags on posts.
Since LessWrong proper is an active site, hopefully, the wiki content continues to get updated.
People who land on the old wiki content will more easily find the rest of the awesome content/activity/community on LessWrong proper.
I like us being the kind of community that when people have spent hundreds (thousands?) of hours generating valuable content, we commit to preserving it.

Quick Links

List of all pages imported from the LW 1.0 Wiki
The Tagging/Wiki Dashboard

Guide to the New Tagging Dashboard
For extra detail on helping, see this section below.

Join the Tagger Slack.

The Three Import Types

Pages have been imported in one of three ways:

76 are imported as new tags that can be applied to posts.
111 are merged with existing tag pages, which is currently in progress and could use some help (see below).
386 are imported as "wiki-only" pages . These pages cannot be applied to posts and do not currently appear on the Concepts page.

The list of imported of all 573 wiki pages

To be honest, it would be more accurate to say that we are part-way through the import. We have completed the programmatic part, and now there remains some manual work to do, hence the help needed.

First, there is some general clean-up of links and other elements that didn't import correctly. Second, and more importantly, a manual text merge is required for the 111 pages are being merged into existing tags. This means taking the text of the existing tag (if it has any) and combining it appropriately with the old wiki page.

Right now, "merged pages" have the old pages' revision history (click History on the tag), but the current text is unchanged.

You can help us out fixing up the wiki import and follow along on completed/incomplete work you can find on the Tagging/Wiki Dashboard. Mo...
on “learning to summarize”
by nostalgebraist — 2020-09-12T03:20:08.333Z

This post is a much extended version of an LW comment I made about OpenAI’s new paper, “Learning to summarize from human feedback.”

Context: this paper is a direct extension of the work OpenAI published last year about fine-tuning GPT-2 with human preference data. I hadn’t actually read that one closely at the time, but went back and did so now, so this is really a commentary on both.

—-

IMO there are two almost unrelated ideas going on in OpenAI’s preference learning work.

First, the idea of collecting binary preference annotations on LM samples, and (in some way) tuning the LM so its samples are better aligned with the preferences.
Second, a specific method for tuning the sampling behavior of LMs to maximize an (arbitrary) score function defined over entire samples.

It may help explain this to go into detail about what they do. Concretely:

They feed a bunch of prompts to a language model (LM) like GPT-2/3, and for each one, save several different samples. They hire annotators to rank the samples in order of perceived quality.
They use the annotation dataset to fine-tune a copy of the original model. The fine-tuning task is not text generation, but something very different: predicting how “good” a sample is, i.e. how likely the annotators are to prefer it to other candidates. They call this a “reward model.”
The reward model assigns a single score to an entire sample of N tokens. They want to fine-tune another copy of the model so that its samples maximize these scores.
But LM training is usually done with an objective that specifies the quality of the model’s predictions for every single token. Knowing how good a full sequence of (say) 20 words is does not tell you how good each individual word is.
To bridge this gap, they use reinforcement learning. Now, the task is not “choose the next word correctly,” but “choose the next word so as to maximize your expected score at the end, after choosing all the later ones as well.”
Their RL method requires two separate copies of the LM, in addition to the one they tuned as the reward model: a “policy model” and a “value model.” (In this paper they show that sharing param between these 2 is worse than making them separate.) I’ll just call these two “the final model” below for simplicity.
Samples from the final model are still, technically, generated one token at a time. They treat this like the usual RL setup in which

...
What's Wrong with Social Science and How to Fix It: Reflections After Reading 2578 Papers
by habryka — 2020-09-12T01:46:07.349Z
Link: https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/

Really interesting analysis of social science papers and replication markets. Some excerpts:

Over the past year, I have skimmed through 2578 social science papers, spending about 2.5 minutes on each one. This was due to my participation in Replication Markets, a part of DARPA's SCORE program, whose goal is to evaluate the reliability of social science research. 3000 studies were split up into 10 rounds of ~300 studies each. Starting in August 2019, each round consisted of one week of surveys followed by two weeks of market trading. I finished in first place in 3 out 10 survey rounds and 6 out of 10 market rounds. In total, about $200,000 in prize money will be awarded.

The studies were sourced from all social sciences disciplines (economics, psychology, sociology, management, etc.) and were published between 2009 and 2018 (in other words, most of the sample came from the post-replication crisis era).

The average replication probability in the market was 54%; while the replication results are not out yet (175 of the 3000 papers will be replicated), previous experiments have shown that prediction markets work well.1

This is what the distribution of my own predictions looks like:2

[...]

Check out this crazy chart from Yang et al. (2020):

Yes, you're reading that right: studies that replicate are cited at the same rate as studies that do not. Publishing your own weak papers is one thing, but citing other people's weak papers? This seemed implausible, so I decided to do my own analysis with a sample of 250 articles from the Replication Markets project. The correlation between citations per year and (market-estimated) probability of replication was -0.05!

You might hypothesize that the citations of non-replicating papers are negative, but negative citations are extremely rare.5 One study puts the rate at 2.4%. Astonishingly, even after retraction the vast majority of citations are positive, and those positive citations continue for decades after retraction.6

As in all affairs of man, it once again comes down to Hanlon's Razor. Either:

Malice: they know which results are likely false but cite them anyway.
or, Stupidity: they can't tell which papers will replicate even though it's quite easy.

Accepting the first option would require a level of cynicism that even I struggle to muster. But the alternative doesn't seem much better: how can they not know? I, an idiot with no

...
Zen and Rationality: Map and Territory
by Gordon Seidoh Worley — 2020-09-12T00:45:40.323Z

This is post 3/? about the intersection of my decades of LW-style rationality practice and my several years of Zen practice.

In today's installment, I look at form and emptiness from a rationalist perspective.

Rationalists have a few key ideas or memes (in the memetic sense), and one of them is "the map is not the territory". Lots has been written about this idea on LessWrong, but it's an idea with a history that stretches back for thousands of years, so it's perhaps not surprising that it's also one of the ideas at the core of Zen.

But in Zen we don't use the words "map" and "territory", instead preferring numerous other metaphors to point at this distinction. Let's explore a few of them, because each elucidates a different aspect of the truth pointed at by these duals.

Before Zen was Zen, Nagarjuna formalized this idea that there's a duality between map and territory in the two truths doctrine. He called these two pairs form and emptiness, pointing at the way our minds put our experiences together into forms or objects that are fixed, at least in our minds, yet ultimately reality is empty of these forms or any other kind of inherent distinctions, essences, or ultimate and timeless truths. Everything we know is provisional, taking a skeptical epistemic stance similar to Pyrrhonism.

Form and emptiness have their place in Zen, but more common is to make a distinction between the relative and the absolute. The relative is that which changes, which exists in our minds, which comes and goes. The absolute is that which exists prior to our perception of it; it's the space in which the relative arises. But Zen doesn't stop there. Form is emptiness and emptiness is form, as the Heart Sutra says, and the relative and the absolute can be thought of as dancing reality into existence, simultaneously unified and separate. Dongshan (Japanese: Tozan) explored this in his poem on the Five Ranks, a subtle teaching that can take some effort to penetrate but is worth the effort.

Talking about relative and absolute can get a bit abstract, as can talking about form and emptiness, so there's another pair that's been used extensively in Zen teaching that, alas, holds little currency for us Westerners: guest and host, or alternatively vassal and lord. I don't have much to say on these because they mostly make sense in the context of the pre-colonial Sinosphere, but I mention them in case...
‘Ugh fields’, or why you can’t even bear to think about that task (Rob Wiblin)
by habryka — 2020-09-11T20:31:00.990Z
Link: https://medium.com/@robertwiblin/ugh-fields-or-why-you-can-t-even-bear-to-think-about-that-task-5941837dac62

Rob Wiblin with more accessible explanation of the Ugh Field concept on Medium. Some quotes:

The problem

Have you ever had a long-overdue task to do, a task which isn’t so bad in itself, but which you can barely bring yourself to think about without feeling awful?
Most people experience this from time to time. Here’s how things get to such a strange and dire state.

The first day the task is on your to-do list, you don’t end up starting, because the short-term reward isn’t large enough to overcome the psychological cost of doing so.

Maybe you feel low energy. Maybe you have more urgent priorities. Maybe you’re insecure about whether you can do a good job. Maybe the task involves a bit of social awkwardness. It doesn’t matter the reason — you delay.

Unfortunately, this task is one that only gets more unpleasant over time.

For instance, maybe now you’re going to have to rush it and do a bad job, and you fear everyone is going to judge you negatively.

[...]

Limiting the damage

I don’t have a perfect way to escape this mental flytrap but here are some things that might help:

1. Urgh Fields happen to basically everyone, even very conscientious people, so it’s worth trying to see the humour in this absurd design flaw in the human brain. There’s no more reason to feel ashamed about it than there is to feel ashamed of e.g. enjoying eating food.

It’s just how people are built and sadly there are no brain engineers around to roll out a patch to the human race. We have to find practical work-arounds instead.

2. Just recognising and labelling the Ugh Field phenomenon can make it less bad, because it’s an accurate systemic explanation for what’s going on, rather than a misleading personal one like “I’m hopeless and never get things done”.

3. Because you’ve been avoiding thinking about the problem, if you do think about it for a bit while keeping an open mind, you might quickly strike on a way to get out of the task, or a way to do a much shorter version of it.

For instance perhaps you could just email back something like: “Thanks for your patience on this. Unfortunately I don’t see how I’m going to be able to fit it into my schedule just now, is there anyone else who can take it on?”

4. If you think about it calmly, you may well find that the task actually isn’t as important as it has come to feel. The person you imagine is disgusted by your failure may only be 2/10 annoyed, or per

...
The Short Case for Verificationism
by ike — 2020-09-11T18:48:00.372Z

Follow-up to: https://www.lesswrong.com/posts/PSichw8wqmbood6fj/this-territory-does-not-exist

Here's a simple and direct argument for my version of verificationism.

Note that the argument uses ontological terms that are meaningless on my views. It functions as a reductio - either one must accept the conclusion, or accept that some of the premises are meaningless, which amounts to the same thing.

Premise 1: The level IV multiverse is possible.

Premise 2: If the level IV multiverse is possible, then we cannot know that we are not in it.

Lemma 1: We cannot know that we are not in the level IV multiverse.

Premise 3: If we are in the level IV multiverse, then ontological claims about our world are meaningless, because we simultaneously exist in worlds where they are true and worlds where they are not true.

Lemma 2: If we can know that ontological claims are meaningful, then we can know we're not in the level IV multiverse.

Conclusion: We cannot know that ontological claims about our world are meaningful.

Edited to add two lemmas. Premises and conclusion unchanged.
Should some variant of longtermism identify as a religion?
by mako yass — 2020-09-11T05:02:43.740Z

The material is already there, it wouldn't involve any lies at all. There are many advantages that need to be weighed.

How could it plausibly identify as a religion?

There are many reasons you might think that it couldn't. We do not lie to our friends, even if the lie is pleasant and locally adaptive. If there were a god, we would respect and admire it but there is no circumstance in which we would worship it. Technology's pace makes blindly inherited culture always maladapted, there isn't enough time to evolve, any more, we must instead design, we must imagine that we can build something better than what we had before, we must be lucid and objective about the underpinnings of our culture, none of our myths can be sacred enough that no evidence could allow us to find them to be false and reject them. Those claims, in combination, distinguish us from everything that has been called a religion before. I believe this to be true.

But there's a lot of other stuff weighing on the other hand, arguably, more.

I have not seen anyone around me compellingly disagree with the simulation hypothesis. We can put forward some new work in decision theory that literally inserts a non-physical entity into our world models that represents an ideal agent that links us all. It provides a moral framework for approaching coordination problems that all robust agents, human or not, can be expected to adhere to, which exceeds the requirements of Yuval Harari's operational definition of religion. I've been wrestling with an acausal protocol for trade with simulators for 7 years and though the first version had been given had some accounting problems I still have not been able to dismiss its biggest most religion-flavored claim. We all seem to agree that humans evolved, and we seem to believe it to an extent that most western secular liberals do not, and that lets us approach a greater depth of understanding of human psychology, it gives us a shared creation story and a notion of the sacredness of nature as it pertains to us, a direct relationship with the process that created us, it is profound and it is useful. We behave as if these things are not important, but they are important. They add up to something.

Why?

Shared worldviews and moral resolutions build communities that are strengthened by trust.

It is good to write about nature in a way that lets you feel it viscerally, and if you...
Choose simplicity and structure.
by DirectedEvolution — 2020-09-10T21:45:13.770Z

Complexity of your life = (number of choices) * (difficulty of choices).

People hate complexity. Maybe more than anything. When people hate something besides complexity itself, it's because what they hate is making their lives more complex. It's forcing them to make more choices. Or it's making the choices more difficult.

People love simplicity. Anything that reduces the number of decisions we have to make, or makes the choice easier, is more valuable than gold. This explains the success of major corporations that have standardized and simplified our world.

Yet simplicity is not the only virtue. The other is quality.

We want our lives to be high-quality, yet simple. Unfortunately, people often must sacrifice one to get the other. Globally, more options yield better quality but more complexity. Is there a way to achieve greater simplicity, without sacrificing quality?

Yes.

The way forward is to master simplification and structure. You can achieve this through these methods:

Focus on simple tasks.
Spend money to simplify.
Adopt structure freely.
Follow your impulse.
Slow is smooth, smooth is fast.
"In the Dust of This Planet," by Eugene Thacker
by Callmesalticidae — 2020-09-10T19:50:56.864Z

Salticidae Philosophiae is a series of abstracts, commentaries, and reviews on philosophical articles and books.

Eugene Thacker suggests that we look to the genre of horror as offering a way of thinking about the unthinkable world. To confront this idea is to confront the limit of our ability to understand the world in which we live – a central motif of the horror genre.

Highlights

There is a limit to our ability to comprehend the world.
The genre of supernatural horror may be the most effective means for exploring and discussing this limitation.
There may be order to the universe but it is indifferent to us and we cannot be fully aware of it.
We should make a distinction between the parts of the world that are knowable by us and the parts of the world that we cannot uncover.
Extinction means the nonexistence of thought and the end of experience.
If we are going to talk about the parts of the world that we cannot uncover, then we can only do so by describing what these parts are not.
We should, at last, recognize that there is no clear distinction between ourselves and the world; we are but a part of the latter.

New or uncommon terminology

Thacker describes three typical modes of understanding the manifested world: the mythological, which acknowledges the world as being not totally under human control, but anthropomorphizes it; the theological, which likewise personifies the world and understands it through the framework of sin, debt, and redemption; and the existential, which presently tends to ground itself on the fruits of science and constricts the world to the experience of the individual.
The world, or perhaps the cosmos, is likewise divided into three parts: the world-for-us, which is the world as we directly interact with it, which we interpret and to which we grant meaning, and which is defined chiefly in in terms of our relationship with it; the world-in-itself, which is described in occasionally contradictory terms but coexists with the world-for-us, resists or ignores our attempts to mold it, and is chiefly accessed (and then transmuted into the world-for-us, if possible) through scientific inquiry and technological intervention; and the world-without-us, which does not and cannot coexist with the world-for-us because it is the subtraction of the human element from the world, and is therefore (at present) spectral and speculative. Thacker also refer

...
Sunday September 13, 12:00PM (PT) — talks by John Wentworth, Liron and more
by habryka — 2020-09-10T19:49:06.325Z

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts).

Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

John Wentworth: Talk topic "How to detect unknown unknowns"

Previously curated posts:

Alignment As A Bottleneck To Usefulness Of GPT-3
Everyday Lessons from High-Dimensional Optimization
Transportation as a Constraint

Liron Shapira: Bloated MVPs: Startups that raised money for logically-impossible ideas

Previously curated post: How Specificity Works

Ruby Bloom: Unprecedentedly Accessible Expertise

Previously curated post: Conversational Cultures: Combat vs Nurture

Details

When? Sunday September 13, 12:00PM (PT)

Where? https://us02web.zoom.us/j/89420485417
Covid 9/10: Vitamin D
by Zvi — 2020-09-10T19:00:01.664Z

Last week: Covid 9/3: Meet the New CDC

Imagine there is a simple, cheap, safe and effective solution for Covid-19.

The solution is something known to be safe. It is widely available for reasonable prices. Any patents have long expired. It is something that people need and benefit from anyway. It’s probably worth doing without the pandemic. It just happens to also have a dramatic effect on Covid-19.

You might think that once the solution was discovered, everyone would shout it from the rooftops. There would rapidly be studies to confirm the solution if it was even considered ethical to not give the solution to everyone. Production would kick into high gear. The pandemic would soon be over.

Or, if you’ve been paying attention, you might think that our civilization is so dysfunctional, so inadequate, that none of that would happen. That for no particular reason, or for reasons we’ll get into later, the whole thing would end up mostly being ignored. We’d carry on with all the same arguments, all the same deaths, all the same economic devastation, putting all of our lives on hold.

That the world you would see would not look much different from our own.

That cynical view looks right.

The solution has quite possibly been found. We were talking about it, including in the rationalist community, back in February.

Everyone’s mostly ignoring it.

The solution we’re talking about, of course, is Vitamin D.

Are we certain or even highly confident this is the whole ballgame? No. Of course not.

We’re not a functional enough civilization to figure this one out in half a year. But we are exactly functional enough of a civilization to start to notice this as a potential solution, and to have run one tiny study that showed dramatic results. If it’s not a dramatic real effect, it’s either taxes or fraud, and I don’t think it’s taxes.

So that’s the headline this week.

I don’t want to oversell this – it’s still possible this is all a false alarm and there’s nothing to see here, because we dropped this ball so utterly that the first study just came in and it’s tiny. But at this point I’d be very surprised if this isn’t, at a bare minimum, a gigantic piece of low hanging fruit.

Here’s the thing.

I think about the world in which Vitamin D is a huge deal, and eliminating Vitamin D deficiency would make the pandemic harmless enough that we cou...