trentmkelly/LessWrong-43k huggingface dataset:
How We Failed COVID-19
by LukeOnline — 2020-12-02T21:29:50.532Z

Since the start of the year, the West has been in the grip of an unprecedented public health crisis. Hundreds of thousands have died, daily life has changed severely and the economy has been dramatically disrupted. It's plausible to call this the West's biggest crisis since World War II.

COVID-19 originated in China. Despite Asia being the epicenter of the outbreak, the consequences of the virus have been a lot less disastrous there. Taiwan, Vietnam and South Korea respectively suffered 7, 35 and 526 deaths, at the moment of writing.

In contrast, the Netherlands, Germany and the United States suffered 9518, 17177 and 270642 deaths. Why do Western countries have so much more casualties? It's tempting to say we chose 'the economy' over public health, but that couldn't be less true. Letting the virus spread through entire countries for a long time has been highly disruptive to our habits and thus to our businesses. Compared to for example South Korea, we've had way more casualties, harsher measures for a much longer time, and more economic decline. We're doing worse in every metric. What mistakes are we making?

The Public Doesn't Know How The Virus Spreads

Source: https://english.elpais.com/society/2020-10-28/a-room-a-bar-and-a-class-how-the-coronavirus-is-spread-through-the-air.html

At the start of the crisis, we focused on two main modes of transmitting the virus: via 'big droplets', emitted by for example coughing, and via surfaces. To prevent being infected by big droplets, you've got to keep distance from other people: somewhere between 1 and 2 meters, with different countries recommending different distances. To prevent infection via surfaces, you've got to wash your hands and disinfect commonly touched objects.

In many Western countries, these are still the main recommended strategies to avoid being infected by COVID. Yet in highly successful countries like Japan, different recommendations have been made from the start. It seems possible to transmit COVID via aerosols, small particles that can float in the air for a long time, and that can fill a room way beyond the recommended safe distance of 1 or 2 meters. Proper ventilation and things like maintaining the right humidity and temperature can help prevent this, but it's highly plausible to be infected while maintaining a safe distance, disinfecting surfaces and wearing face masks.

I liv...
Follow Up: Predictions as a Substitute For Reviews
by Elizabeth — 2020-12-02T19:40:09.010Z

In August I wrote about using PredictionBook as a substitute for weekly reviews. Several months later, I am rather predictably not using it, for the same reason weekly reviews stop working: it reminded me of something I endorsed doing but didn’t finish, and that put an ugh field around the whole thing.

Weirdly, I re-picked-up that thing a week ago. This was completely independent: I haven’t looked at PredictionBook since September. I don’t think this has any bearing on the viability of predictions substituting for reviews, it’s just a cute coincidence.
[AN #127]: Rethinking agency: Cartesian frames as a formalization of ways to carve up the world into an agent and its environment
by Rohin Shah — 2020-12-02T18:20:05.196Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

This newsletter is an extended summary of the recently released Cartesian frames sequence.

Cartesian Frames(Scott Garrabrant) (summarized by Rohin): The embedded agency sequence (AN #31) hammered in the fact that there is no clean, sharp dividing line between an agent and its environment. This sequence proposes an alternate formalism: Cartesian frames. Note this is a paradigm that helps us think about agency: you should not be expecting some novel result that, say, tells us how to look at a neural net and find agents within it.

The core idea is that rather than assuming the existence of a Cartesian dividing line, we consider how such a dividing line could be constructed. For example, when we think of a sports team as an agent, the environment consists of the playing field and the other team; but we could also consider a specific player as an agent, in which case the environment consists of the rest of the players (on both teams) and the playing field. Each of these are valid ways of carving up what actually happens into an “agent” and an “environment”, they are frames by which we can more easily understand what’s going on, hence the name “Cartesian frames”.

A Cartesian frame takes choice as fundamental: the agent is modeled as a set of options that it can freely choose between. This means that the formulation cannot be directly applied to deterministic physical laws. It instead models what agency looks like “from the inside”. If you are modeling a part of the world as capable of making choices, then a Cartesian frame is appropriate to use to understand the perspective of that choice-making entity.

Formally, a Cartesian frame consists of a set of agent options A, a set of environment options E, a set of possible worlds W, and an interaction function that, given an agent option and an environment option, specifies which world results. Intuitively, the agent can “choose” ...
Recursive Quantilizers II
by abramdemski — 2020-12-02T15:26:30.138Z

I originally introduced the recursive quantilizers idea here, but didn't provide a formal model until my recent Learning Normativity post. That formal model had some problems. I'll correct some of those problems here. My new model is closer to HCH+IDA, and so, is even closer to Paul Christiano style systems than my previous.

However, I'm also beginning to suspect that quantilizers aren't the right starting point. I'll state several problems with quantilizers at the end of this post.

First, let's reiterate the design criteria, and why the model in Learning Normativity wasn't great.

Criteria

Here are the criteria from Learning Normativity, with slight revisions. See the earlier post for further justifications/intuitions behind these criteria.

No Perfect Feedback: we want to be able to learn with the possibility that any one piece of data is corrupt.

Uncertain Feedback: data can be given in an uncertain form, allowing 100% certain feedback to be given (if there ever is such a thing), but also allowing the system to learn significant things in the absence of any certainty.
Reinterpretable Feedback: ideally, we want rich hypotheses about the meaning of feedback, which help the system to identify corrupt feedback, and interpret the information in imperfect feedback. To this criterion, I add two clarifying criteria:

Robust Listening: in some sense, we don't want the system to be able to "entirely ignore" humans. If the system goes off-course, we want to be able to correct that.
Arbitrary Reinterpretation: at the same time, we want the AI to be able to entirely reinterpret feedback based on a rich model of what humans mean. This criterion stands in tension with Robust Listening. However, the proposal in the present post is, I think, a plausible way to achieve both.

No Perfect Loss Function: we don't expect to perfectly define the utility function, or what it means to correctly learn the utility function, or what it means to learn to learn, and so on. At no level do we expect to be able to provide a single function we're happy to optimize. This is largely due to a combination of Goodhart and corrupt-feedback concerns.

Learning at All Levels: Although we don't have perfect information at any level, we do get meaningful benefit with each level we step back and say "we're learning this level rather than keeping it fixed", because we can provide meaningful approximate loss function

...
Prizes for Last Year's 2018 Review
by Raemon — 2020-12-02T11:21:16.211Z

About a year ago, LessWrong held it's first annual review, where we looked over the best posts from 2018. The LessWrong team offered $2000 in prizes for the top post authors, and (up to) $2000 in prizes for the best reviews of those posts.

For our top post authors, we have decided to award.... *drumroll*

Abram Demski & Scott Garrabrant are each awarded $200 for Embedded Agents
Eliezer Yudkowsky receives $200 for The Rocket Alignment Problem, and another $200 for Local Validity as the Key to Sanity and Civilization
Paul Christiano is awarded $200 for Arguments about Fast Takeoff
Abram Demski receives an additional $200 for Towards a New Technical Explanation of Technical Explanation.
Martin Sustrik receives $200 for Anti-Social Punishment
Scott Alexander receives $200 for The Tails Coming Apart as a Metaphor for Life
Alkjash receives $200 for Babble
Patrick LaVictoire $200 for The Loudest Alarm is Usually False

For Reviews, there are three tiers of prize ($300, $200, $100):

Vanessa Kosoy receives $300 for her reviews of Realism about rationality, Coherence arguments do not imply goal-directed behavior, and Clarifying “AI Alignment”.
Zack M. Davis receives $200 for commentary on Meta-Honesty, and Decoupling vs Contextualising Norms
Bucky receives $200 for his critique of Unknown Knowns
Abram Demski receives $100 for his response to Realism about Rationality
Daniel Filan receives $100 for a variety of reviews of Coherence arguments, Explain enlightenment in non-mysterious terms, Realism about Rationality, Towards a New Impact Measure
Mingyuan receives $100 for a good critique of Give Praise.
Jacobian receives $100 for his review of Intelligent Social Web.
Zvi and Jameson Quinn both receive $100 for lots of good short reviews.
Val receives $100 for taking the time to reflect on and rewrite his own post, The Intelligent Social Web.

Not for reviews, but for discussion in the review, $50 apiece goes to Richard Ngo and Rohin Shah.

Prizewinners, we'll reach out to you in a week or so to give you your prize-money.

Congratulations to all the winners!
The LessWrong 2019 Review
by habryka — 2020-12-02T11:21:11.533Z

Today is the start of the 2019 Review, continuing our tradition of checking which things that were written on LessWrong still hold up a year later, and to help build an ongoing canon of the most important insights developed here on LessWrong.

The whole process will span 8 weeks, starting on December 1st:

From December 1st to the 14th, any user that was registered before January 1st 2019 can nominate any post written in 2019 to be considered for the review.
From December 14th to January 11th, any user can leave reviews on any posts with at least two nominations, ask questions of other users and the author, and make arguments for how a post should be voted on in the review.
From January 11th to January 25th any LessWrong user registered before 2019 can vote on the nominated posts, using a voting system based on quadratic voting. (There will be two votes, one for 1000+ karma users, and one for all users)

But before I get more into the details of the process, let's go up a level.

Why run a review like this?

The Review has three primary goals:

Improve our incentives, feedback, and rewards for contributing to LessWrong.
Create a highly curated "Best of 2019" sequence and physical book
Create common knowledge about the LW community's collective epistemic state about the most important posts of 2019

Improving our incentives and rewards

Comments and upvotes are a really valuable tool for allocating attention on LessWrong, but they are ephemeral and frequently news-driven, with far-from-perfect correlation to the ultimate importance of an idea or an explanation.

I want LessWrong to be a place for Long Content. A place where we can build on ideas over decades, and an archive that helps us collectively navigate the jungle of infinite content that spews forward on LessWrong every year.

One way to do that is to take some time between when you first see a post and when you evaluate it. That's why today we are starting the 2019 review, not the 2020 review. A year is probably enough time to no longer be swept away in the news or excitement of the day, but recent enough that we can still remember and write down how an idea or explanation has affected us.

I also want LessWrong to not be overwhelmed by research debt:

Research debt is the accumulation of missing interpretive labor. It’s extremely natural for young ideas to go through a stage of debt, like ear

...
Should I take glucosamine?
by gilch — 2020-12-02T05:20:04.734Z

This paper came up on my news feed today: https://www.jabfm.org/content/jabfp/33/6/842.full.pdf

According to the research, daily intake of glucosamine/chondroitin supplements for at least a year is associated with a 39% reduction in all-cause mortality.

This is a cohort study, not a randomized trial, so the evidence is not conclusive. Individuals taking supplements were self-selected. Perhaps there is another factor not controlled for in the paper. The researchers are calling for a more thorough study.

However a 39% reduction appears to be a pretty large effect. And glucosamine/condroitin supplements are fairly inexpensive and available over-the-counter, and seem likely to be relatively harmless even if the effect is illusory. Maybe the kind of person who takes this kind of bet comes out ahead, even if they're wrong sometimes.

I'm not looking for mere social permission or a gut-feel answer here. I want to understand your reasoning. How should a rationalist reason about this? Do we plug numbers into a Bayesian net? Should I take the bet and start supplementing daily? Is the effect too small to bother with? Should I wait for more evidence?
Eth2 Staking explained as a Financial Instrument
by Annapurna — 2020-12-02T03:33:19.776Z

A major cryptocurrency can now be applied as an income generating asset.

If you don’t keep up with cryptocurrency news, you probably don’t know that today marks the beginning of one of the most important updates in one of the largest cryptocurrencies out there.

Ethereum

, the largest cryptocurrency by

transactions per day

, and the second largest cryptocurrency by

market capitalization

, has started transitioning its consensus protocol from

Proof of Work

to

Proof of Stake

. This marks the beginning of Eth2 (also known as Ethereum 2.0 or Serenity).

I am not going to explain what Ethereum is or the main differences between Proof of Work and Proof of Stake.

There

are

plenty

of

resources

out there that explain what Ethereum is far better than I ever will. Instead, I am going to explain why Eth2 Staking is a financial instrument using finance terminology.

Eth2 Staking is very similar to purchasing a bond.

Staking (Depositing) Eth2 in the

Beacon Chain

is the equivalent of calling your broker and buying a corporate or government bond. That being said, today it is far easier to buy a bond today than it is to deposit Eth2. To buy a bond today, you would call your broker/financial advisor and they would ask a trading desk to source the bond. After agreeing on the price, the trade is executed and you will see the bond in your account the next day. To stake Eth2 today,

the process is far more complex.

That being said,

some intermediaries have started to provide staking services

and I would not be surprised if in a couple of years major financial institutions also offer staking services, which would make the Eth2 Staking process just as easy as buying a bond.

There is no end date to Eth2 Staking. This is similar to perpetual bonds.

To stake Eth2, you need 32 ETH ($19,611 USD). This is the equivalent of the Face Value (FV) Minimum Piece when buying a bond. Most FV minimum pieces for bonds are either 1,000, 2,000, 10,000 or 200,000.

If you want to stake more Eth2, you need to do it in 32 ETH increments. This is the equivalent of the FV Minimum Increment when buying a bond. Most bonds have minimum increments of 1,000, although rarely some bonds have higher increments. The

Santander 5.179% 2025

bond is one of those rare bonds that has a minimum piece of 200,000 and a minimum increment of 200,000.

Today, you only can stake (deposit) Eth2. Once the Eth2 protocol reaches phase 1.5, you woul...
Poll: Any interest in an editing buddy system?
by dynomight — 2020-12-02T02:18:40.443Z

There was a popular post today linked from Hacker News on the benefits of hiring a freelance editor. Looking into it, I think that what's described is really "Developmental editing" and "structural" editing as described here.

Maybe there's no replacement for a professional editor. But for a community like this with (I think) such a high ratio of contributors, I wondered if it might be valuable to try to arrange a kind of "buddy system" where we give feedback to each other to improve our writing.

For now, please don't worry about how that might work. At the moment I'd just like to gauge if there is enough interest that this is something that might be worth trying to organize.

Would you be interested in participating in this? If so, could you leave a short comment with answers to two questions?

Question 1: Would you participate in this ?

Answer A: No

Answer B: I'd volunteer to edit but wouldn't submit.

Answer C: I'd edit and submit.

Question 2: How good do you think you would be at editing? Take the average front-page post at Lesswrong. If you took 30 minutes to give comments, and the author used your feedback, how much better would that post get? Measure quality via upvotes if you want.

Answer A: Less than 1.5x

Answer B: 1.5x-2x

Answer C: 2x-4x

Answer D: 4x or more

(Apologies if "real" polls are possible / better. The documentation for those seems to be really out of date.)
In a multipolar scenario, how do people expect systems to be trained to interact with systems developed by other labs?  
by JesseClifton — 2020-12-01T20:04:18.197Z

I haven't seen much discussion of this, but it seems like an important factor in how well AI systems deployed by actors with different goals manage to avoid conflict (cf. my discussion of equilibrium and prior selection problems here).

For instance, would systems be trained

Against copies of agents developed by other labs (possibly with measures to mask private information)?

Simultaneously with other agents in a simulator that each developer has access to?

Against copies of themselves?

Against distributions of counterpart policies engineered to have certain properties? What would those properties be?
Open & Welcome Thread - December 2020
by Pattern — 2020-12-01T17:03:48.263Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Forecasting Newsletter: November 2020
by NunoSempere — 2020-12-01T17:00:58.898Z

Highlights

DeepMind claims a major breakthrough in protein folding.

OPEC forecasts slower growth

Gnosis announces futarchy experiment

Index

Highlights

In The News

Prediction Markets & Forecasting Platforms

United States Presidential Elections Post-mortems

Hard To Categorize

Long Content

Sign up here or browse past newsletters here.

In the News

DeepMind claims a major breakthrough in protein folding (press release, secondary source)

DeepMind has developed a piece of AI software called AlphaFold that can accurately predict the structure that proteins will fold into in a matter of days.

This computational work represents a stunning advance on the protein-folding problem, a 50-year-old grand challenge in biology. It has occurred decades before many people in the field would have predicted. It will be exciting to see the many ways in which it will fundamentally change biological research.

Figuring out what shapes proteins fold into is known as the "protein folding problem", and has stood as a grand challenge in biology for the past 50 years. In a major scientific advance, the latest version of our AI system AlphaFold has been recognised as a solution to this grand challenge by the organisers of the biennial Critical Assessment of protein Structure Prediction (CASP). This breakthrough demonstrates the impact AI can have on scientific discovery and its potential to dramatically accelerate progress in some of the most fundamental fields that explain and shape our world.

In the results from the 14th CASP assessment, released today, our latest AlphaFold system achieves a median score of 92.4 GDT overall across all targets. This means that our predictions have an average error (RMSD) of approximately 1.6 Angstroms, which is comparable to the width of an atom (or 0.1 of a nanometer). Even for the very hardest protein targets, those in the most challenging free-modelling category, AlphaFold achieves a median score of 87.0 GDT.

Crucially, CASP chooses protein structures that have only very recently been experimentally determined (some were still awaiting determination at the time of the assessment) to be targets for teams to test their structure prediction methods against; they are not published in advance. Participants must blindly predict the structure of the proteins.

The Organization of the Petroleum Exporting Countries (OPEC) forecasts slower growth...