trentmkelly/LessWrong-43k huggingface dataset:
Omicron Post #3
by Zvi — 2021-12-02T15:10:00.632Z

I have split off the Omicron post from the weekly post, which covers other aspects of Covid. I made judgment calls on where to put a few things.

Omicron continues to spread quickly, but the news surrounding Omicron is clearly slowing down, so these updates can hopefully space out a bit more.

Past Omicron posts: #1, #2.

Post on Polymarket’s Covid prediction markets, and potential future markets discussion. Direct link to market on whether Omicron will be >1% of cases by the end of the year, which is currently a moderate favorite.

Biden on Omicron

A few hours after my second Omicron update, Biden went on TV to talk about the new variant.

Here are some highlights:

(1:00) He says that he told us he would always be honest with us, so he’s going to talk about the new Omnicron (which I do admit sounds cooler) variant. Damn that campaign promise, guess we’re going to have to be honest.

(1:00) He says that South Africa’s transparency in telling us about Omnicron is to be applauded, for it allows us to take action, so he took action right away to cut off travel to South Africa. What even are incentives, how do they work, not a metaphor for entire legislative agenda?

(1:30) We can’t stop Omnicron, we can only slow it down, giving us time to take other actions. Therefore get vaccinated. ‘Cause for concern, not cause for panic.’ Because we’re the best, you see, and also science.

(2:30) ‘We have more tools than ever to fight the variant. From vaccines to boosters, to vaccines for children.’ So… vaccines? We have vaccines? ‘And much more.’

(3:00) Did you know ‘deaths from Delta are coming down’? Checks notes.

(3:30) Booster. Vaccinated. Vaccinated. Booster.

(4:30) Mask.

(4:45) This, which I wish I fully believed:

We will now find out what counts as ‘every available tool.’ Also, why aren’t we using ‘every available tool’ with regard to, let’s say, Paxlovid?

(5:30) I will tell the CDC and FDA to use ‘the fastest process available without cutting any corners.’

Wait, if there’s a faster process available that doesn’t cut any corners, then… then…

(5:45) Same thing for ‘modifications to current treatments for those who get ill with the Covid virus.’

(5:50) “Look, I’m removing all roadblocks to keeping the American people safe.”

Paxlovid remains illegal.

(6:05) “If you are vaccinated and worried about this virus, go get your booster.”

Logically this implies that if you have gott...
Built-in Measuring Spoons
by jefftk — 2021-12-02T13:20:06.597Z

In making the kids breakfast each morning, which has moved on from

crepes and

waffles to pancakes, I want a consistent
amount of baking powder. I realized that the container has a built-in
"measuring spoon". If you shake it with the lid on, powder gets up
above the leveling shelf:

Take the lid off, and it remains:

Then you can dump it in the batter:

It's a bit awkward to do this with a full container, since you can't
tip it very much, but it's quite easy once you've used a bit first.

With the particular brand of baking powder we've been buying it's
consistently 2/3 of a teaspoon.

I wonder whether anyone has tried designing a baking powder container
to make this easier? The current shape is not ideal for measuring,
and it would make much more sense for it to be one teaspoon instead of
2/3.

Comment via: facebook
Covid Prediction Markets at Polymarket
by Zvi — 2021-12-02T12:50:00.750Z

We now have some useful prediction markets up on Covid issues, so it’s worth looking at what they say and thinking about what other markets we could generate. I encourage you to suggest additional markets in the comments, with as much detail as possible.

As I wrote a while ago, if you want prediction markets to be successful, you need five elements:

Well Defined. The resolution mechanisms must be clear and transparent.
Quick Resolution. The longer it takes, the less people will be interested.
Probable Resolution. There has to be a definitive outcome most of the time.
Limited Hidden Information. If others know what I don’t, I’m the sucker.
Sources of Disagreement and Interest. Suckers at the table.

To these, of course, we can add an implied sixth, which is

Real Money. Money talks, bullshit walks. Are we doing this or not?

Thus, now that we have Polymarket posting real Covid-19 markets, we have the potential to learn things that matter, with a level of resolution we can use, and I’m glad I’ve been able to help them figure out what markets to offer.

We need real money prediction markets if we want to rely on their information. I’m glad Metaculus exists and puts up some markets, and it’s good for getting some idea at all, but it is not remotely the same thing. Their main advantage is that, exactly because they don’t use real money, they can violate the five pillars and those who are inclined to participate at all will still participate because there is no cost of capital, need to invest tons of attention or worry about being the sucker when it’s for internet points.

Whereas at Polymarket, a prime motivation of liquidity providers is to use the markets to generate information, providing motivation in turn for others to come participate.

There are three new markets up, let’s check them out.

Will Omicron be >1% of all USA cases by the end of the year?

Here’s what it would look like betting $1000 or $10,000 on this respectively.

It was my suggestion to use a relatively low threshold like 1% of cases, in order to allow the market to resolve faster. There is an interesting and important question of whether Omicron would then fully displace Delta, but if Omicron goes from 0% to 1% in a month, going from 1% to 50% is a lot less exponential growth than that and should happen that much faster. The game is very much over at that point.

If the cha...
Sydney AI Safety Fellowship
by Chris_Leong — 2021-12-02T07:34:28.500Z

I’m excited to announce the launch of the Sydney AI Safety Fellowship which will provide fellows from Australia and New Zealand the opportunity to pursue projects in AI Safety or spend time upskilling. These projects may be technical projects, projects related to policy, or movement building projects.
The fellowship will take place at WeWork in Sydney which provides fantastic views, plus free coffee, beer and energy drinks. It will take place during the 7 weeks 10th of January to the 25th of February, but there will be the option to start a week earlier if so desired.

The fellowship would include:
• Coworking membership
• Talks and/or Q&A sessions from people involved in AI Safety
• Connection with others in the AI Safety community who may have similar interests or be able to provide support for your project
• Mentorship (may be someone relatively junior)
• A welcome dinner, final dinner, and social events (ie. watching a movie, rock-climbing, or whatever the fellows are interested in) - costs covered
• Free lunch once per week
• Whatever further activities the fellows decide to self-organise

We have funding for up to two (non-local) fellows to receive $1000 to help offset flights and accommodation. I know that this isn’t much compared to the costs, but it’s what we can commit to at the moment.

Fellows would be expected to spend four days a week at the coworking space working on projects or participating in other activities, although in some cases we might allow fellows to spend only three days a week if they needed to work to support themselves. There would be one day a week where we’d expect all of the fellows to attend so that we would be able to organise talks + have a meeting where fellows could share their ideas and ask for feedback.

There will also be up to two working fellowships available for people who are working as AI programmers or who otherwise have significant AI knowledge. These fellows will be able to pursue their normal work at the coworking space whilst also participating in the activities and sharing their knowledge with the fellows. They would have the option of pursuing a project part-time, which would provide an advantage during the application process, but this isn’t a requirement. Working fellows aren’t eligible for the non-local subsidy. They would receive half-subsidisation of a WeWork coworking membership and free access to the activities...
Morality is Scary
by Wei Dai — 2021-12-02T06:35:06.736Z

I'm worried that many AI alignment researchers and other LWers have a view of how human morality works, that really only applies to a small fraction of all humans (notably moral philosophers and themselves). In this view, people know or at least suspect that they are confused about morality, and are eager or willing to apply reason and deliberation to find out what their real values are, or to correct their moral beliefs. Here's an example of someone who fits this view:

I’ve written, in the past, about a “ghost” version of myself — that is, one that can float free from my body; which travel anywhere in all space and time, with unlimited time, energy, and patience; and which can also make changes to different variables, and play forward/rewind different counterfactual timelines (the ghost’s activity somehow doesn’t have any moral significance).

I sometimes treat such a ghost kind of like an idealized self. It can see much that I cannot. It can see directly what a small part of the world I truly am; what my actions truly mean. The lives of others are real and vivid for it, even when hazy and out of mind for me. I trust such a perspective a lot. If the ghost would say “don’t,” I’d be inclined to listen.

I'm currently reading The Status Game by Will Storr (highly recommended BTW), and found in it the following description of how morality works in most people, which matches my own understanding of history and my observations of humans around me:

The moral reality we live in is a virtue game. We use our displays of morality to manufacture status. It’s good that we do this. It’s functional. It’s why billionaires fund libraries, university scholarships and scientific endeavours; it’s why a study of 11,672 organ donations in the USA found only thirty-one were made anonymously. It’s why we feel good when we commit moral acts and thoughts privately and enjoy the approval of our imaginary audience. Virtue status is the bribe that nudges us into putting the interests of other people – principally our co-players – before our own.

We treat moral beliefs as if they’re universal and absolute: one study found people were more likely to believe God could change physical laws of the universe than he could moral ‘facts’. Such facts can seem to belong to the same category as objects in nature, as if they could be observed under microscopes or proven by mathematical formulae. If

...
AXRP Episode 12 - AI Existential Risk with Paul Christiano
by DanielFilan — 2021-12-02T02:20:17.041Z

YouTube link

This podcast is called AXRP, pronounced axe-urp and short for the AI X-risk Research Podcast. Here, I (Daniel Filan) have conversations with researchers about their papers. We discuss the paper and hopefully get a sense of why it’s been written and how it might reduce the risk of artificial intelligence causing an existential catastrophe: that is, permanently and drastically curtailing humanity’s future potential.

Why would advanced AI systems pose an existential risk, and what would it look like to develop safer systems? In this episode, I interview Paul Christiano about his views of how AI could be so dangerous, what bad AI scenarios could look like, and what he thinks about various techniques to reduce this risk.

Topics we discuss:

How AI may pose an existential threat

AI timelines

Why we might build risky AI

Takeoff speeds

Why AI could have bad motivations

Lessons from our current world

“Superintelligence”

Technical causes of AI x-risk

Intent alignment

Outer and inner alignment

Thoughts on agent foundations

Possible technical solutions to AI x-risk

Imitation learning, inverse reinforcement learning, and ease of evaluation

Paul’s favorite outer alignment solutions

Solutions researched by others

Decoupling planning from knowledge

Factored cognition

Possible solutions to inner alignment

About Paul

Paul’s research style

Disagreements and uncertainties

Some favorite organizations

Following Paul’s work

Daniel Filan:
Hello everybody. Today, I’ll be speaking with Paul Christiano. Paul is a researcher at the Alignment Research Center, where he works on developing means to align future machine learning systems with human interests. After graduating from a PhD in learning theory in 2017, he went onto research AI alignment at OpenAI, eventually running their language model alignment team. He’s also a research associate at the Future of Humanity Institute in Oxford, a board member at the research non-profit Ought, a technical advisor for Open Philanthropy, and the co-founder of the Summer Program on Applied Rationality and Cognition, a high school math camp. For links to what we’re discussing, you can check the description of this episode and you can ...
Common Probability Distributions
by jsteinhardt — 2021-12-02T01:50:17.115Z

When we output a forecast, we're either explicitly or implicitly outputting a probability distribution.

For example, if we forecast the AQI in Berkeley tomorrow to be "around" 30, plus or minus 10, we implicitly mean some distribution that has most of its probability mass between 20 and 40. If we were forced to be explicit, we might say we have a normal distribution with mean 30 and standard deviation 10 in mind.

There are many different types of probability distributions, so it's helpful to know what shapes distributions tend to have and what factors influence this.

From your math and probability classes, you're probability used to the Gaussian or normal distribution as the "canonical" example of a probability distribution. However, in practice other distributions are much more common. While normal distributions do show up, it's more common to see distributions such as log-normal or power law distributions.

In the remainder of these notes, I'll discuss each of these in turn. The following table summarizes these distributions, what typically causes them to occur, and several examples of data that follow the distribution:

Distribution
Gaussian
Log-normal
Power Law

Causes
Independent additive factors
Independent multiplicative factors
Rich get richer, scale invariance

Tails
Thin tails
Heavy tails
Heavier tails

Examples
-heights
-US GDP in 2030
-city population

-temperature
-price of Tesla stock in 2030
-twitter followers

-measurement errors

-word frequencies

Normal Distribution

The normal (or Gaussian) distribution is the familiar "bell-shaped" curve seen in many textbooks. Its probability density is given by $p(x) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\Big(-\frac{(x-\mu)^2}{2\sigma^2}\Big)$, where $\mu$ is the mean and $\sigma$ is the standard deviation.

Normal distributions occurs when there are many independent factors that combine additively, and no single one of those factors "dominates" the sum. Mathematically, this intuition is formalized through the central limit theorem.

Example 1: temperature. As one example, the temperature in a given city (at a given time of year) is normally distributed, since many factors (wind, ocean currents, cloud cover, pollution) affect it, mostly independently.

Example 2: heights. Similarly, height is normally distributed, since many different genes have some effect on height, as do other factors such as ...
The 2020 Review
by Raemon — 2021-12-02T00:39:19.191Z

Today is the first day of the LessWrong 2020 Review. At the end of each year, we take a look at the best posts from the previous year, and reflect on which of them have stood the test of time.

As we navigate the 21st century, a key issue is that we’re pretty confused about which intellectual work is valuable. We have very sparse reward signals when it comes to “did we actually figure out things that matter?”

The Review has a few goals. It improves our incentives, feedback, and rewards for contributing to LessWrong. It creates common knowledge about the LW community's collective epistemic state about the most important posts of 2020. And it turns all of that into a highly curated sequence that people can read. You can read more about the philosophy behind the Review in

last year’s announcement post

.

A few important announcements about this year’s review:

We’ve replaced the nomination process with Preliminary Voting.
Winning posts will get Donation Buttons.
There’s a new

View My Past Upvotes

page to help you find posts to vote on.
2019 Books will be shipping in a couple weeks. We’re still evaluating whether and how to do books for 2020.

How does the review work?

The review has three phases:

Preliminary Voting Phase (Dec 1- 14)
Discussion Phase (Dec 14 - Jan 11)
Final Voting (Jan 11 to Jan 25)

Users who registered before January 1st 2020 can vote. The LessWrong moderation team will take the results of the vote as input for a curated sequence of posts, and award prizes. We’ll be giving more weight to the votes of users with 1000+ karma.

Preliminary Voting

The first big change this year is changing the Nomination Phase to the Preliminary Vote Phase. Eligible voters voters will see this UI:

If you think a post was an important intellectual contribution, you can cast a vote indicating roughly how important it was. A vote of 1 means “it was good.” A vote of 4 means “it was quite important”, and is weighted 4x a vote of 1. A vote of 9x means it was a crucial piece of intellectual progress.

You can vote at the top of a post, or anywhere the post appears in a list (like the

All Posts page

, or the new

View Your Past Upvotes

page).

Posts that get at least one positive vote go to the Voting Dashboard, where other users can vote on it. You’re encouraged to give at least a rough vote based on what you remember from last year.

If you feel a post was important, you’re also encourage...
Biology-Inspired AGI Timelines: The Trick That Never Works
by Eliezer Yudkowsky — 2021-12-01T22:35:28.379Z

- 1988 -

Hans Moravec: Behold my book Mind Children. Within, I project that, in 2010 or thereabouts, we shall achieve strong AI. I am not calling it "Artificial General Intelligence" because this term will not be coined for another 15 years or so.

Eliezer (who is not actually on the record as saying this, because the real Eliezer is, in this scenario, 8 years old; this version of Eliezer has all the meta-heuristics of Eliezer from 2021, but none of that Eliezer's anachronistic knowledge): Really? That sounds like a very difficult prediction to make correctly, since it is about the future, which is famously hard to predict.

Imaginary Moravec: Sounds like a fully general counterargument to me.

Eliezer: Well, it is, indeed, a fully general counterargument against futurism. Successfully predicting the unimaginably far future - that is, more than 2 or 3 years out, or sometimes less - is something that human beings seem to be quite bad at, by and large.

Moravec: I predict that, 4 years from this day, in 1992, the Sun will rise in the east.

Eliezer: Okay, let me qualify that. Humans seem to be quite bad at predicting the future whenever we need to predict anything at all new and unfamiliar, rather than the Sun continuing to rise every morning until it finally gets eaten. I'm not saying it's impossible to ever validly predict something novel! Why, even if that was impossible, how could I know it for sure? By extrapolating from my own personal inability to make predictions like that? Maybe I'm just bad at it myself. But any time somebody claims that some particular novel aspect of the far future is predictable, they justly have a significant burden of prior skepticism to overcome.

More broadly, we should not expect a good futurist to give us a generally good picture of the future. We should expect a great futurist to single out a few rare narrow aspects of the future which are, somehow, exceptions to the usual rule about the future not being very predictable.

I do agree with you, for example, that we shall at some point see Artificial General Intelligence. This seems like a rare predictable fact about the future, even though it is about a novel thing which has not happened before: we keep trying to crack this problem, we make progress albeit slowly, the problem must be solva...
Open & Welcome Thread December 2021
by Pattern — 2021-12-01T19:57:32.520Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Viral Mutation, Pandemics and Social Response
by jmh — 2021-12-01T18:36:59.409Z

It seems that both historically and currently the first order response is quarantine and travel restrictions to prevent potential transmission. Pretty sensible as we have plenty of reasons and plenty of evidence supporting the travel and trade spreads diseases to new locations.

But is that really the same when we're talking about mutations of an existing virus that is widespread? I'm not entirely sure, hence the post. Hence the rather speculative post I should probably say.

I would imaging the starting point to such a question just might be "So just what is the mutation process?" That seems to be somewhat poorly understood -- but I'm hardly well informed here.

Mutations come in many forms but we might put them into one of three buckets: "advantage" gained, no effect (neutral mutation?) and "this just broke everything". The first is really what most are concerned with here and what the Delta and Omicron mutations are really about.

In general it seems these are largely taken as just random events. I think that view is what underlies the idea that one can (in theory I suppose) trace these variants back to an original host from which the new dominant strain emerged.

The remarkable capacity of some viruses to adapt to new hosts and environments is highly dependent on their ability to generate de novo diversity in a short period of time. Rates of spontaneous mutation vary amply among viruses. RNA viruses mutate faster than DNA viruses, single-stranded viruses mutate faster than double-strand virus, and genome size appears to correlate negatively with mutation rate. Viral mutation rates are modulated at different levels, including polymerase fidelity, sequence context, template secondary structure, cellular microenvironment, replication mechanisms, proofreading, and access to post-replicative repair. Additionally, massive numbers of mutations can be introduced by some virus-encoded diversity-generating elements, as well as by host-encoded cytidine/adenine deaminases. Our current knowledge of viral mutation rates indicates that viral genetic diversity is determined by multiple virus- and host-dependent processes, and that viral mutation rates can evolve in response to specific selective pressures.

( https://link.springer.com/article/10.1007/s00018-016-2299-6)

Emphasis added in the above quote. Huh? Virus may actually have evolved structures that seek to produce mutat...