trentmkelly/LessWrong-43k huggingface dataset:
Information Charts
by Rafael Harth — 2020-11-13T16:12:27.969Z

Edit 2020/12/13: I think a lot of people have misunderstood my intent with this post. I talk about the 2020 election because it seemed like the perfect example to illustrate the point, but it was still only an example. The goal of this post is to establish a philosophically sound ground truth for probabilities.

1. Motivation

It seems to me that:

Predictions can be better or worse.

This is true even with a sample size of 1.

The quality of a prediction is not purely a function of its probability. (A 90% prediction for a false outcome may still be good, and two 70% predictions for the same outcome need not be equally good.)

A lot of people have a hard time wrapping their heads around how 1-3 can simultaneously be true

In this post, I work out a framework in which there is a ground truth for probabilities, which justifies 1-3.

2. The Framework

Consider the following game:

Throw a fair coin 100 times, counting the number of heads. You win the game if this number is at most 52 and lose if it is 53 or higher.

Using the binomial formula, we can compute that the probability of winning this game is around 0.69135. (In mathy notation, that's P(X≤52) with X∼B(100,12).)

Suppose we play this game, writing down each coin flip as it occurs, and put the progression into a chart:

Here, the red line denotes the number of heads we have after n flips, and the blue line denotes the 'baseline', which is the number of heads such that continuing at this pace would end up at precisely 52 heads after 100 flips. We end the game above the baseline, with 54 heads, which means that we lose.

Since we know exactly how this game works, it's possible to compute the current probability of winning at every point during the game.[1] Here is a chart of these probabilities, given the flips from the chart above:

Note that the y-axis shows the probability of winning the game after observing the n-th flip. Thus, there are precisely 101 values here, going from the probability after observing the 0-th flip (i.e., without having seen any flips), which is ≈0.69135, to the probability after observing the 100-th flip, which is 0. By comparing both graphs visually, you can verify that they fit together.

Each of these 101 y-values is a prediction for the outcome of the game. However, they're evidently not equally well-informed predictions: the ones further to the right are based on more information ...
Reading/listening list for the US failing or other significant shifts?
by Nicole Ross — 2020-11-13T15:34:49.319Z

I have a tentative hypothesis that the US is quite precarious. Although I believe we have likely avoided the worst outcomes this year, I think the next 10-30 yrs have some reasonable chance of going very badly. I don't know what that means, nor what the implications are for top cause areas.

I want to create a reading/listening list for exploring all this. Things I'm interested in:

history, particularly broad arcs/analysis of history (e.g., maybe I should read some Peter Turchin stuff)
history, particularly significant changes in global power dynamics (where power also involves technological advancement or similar)
....other things I'm not thinking of that feel relevant

I have no background here at all. So even very basic recommendations would be appreciated.
Misalignment and misuse: whose values are manifest?
by KatjaGrace — 2020-11-13T10:10:00.759Z

Crossposted from world spirit sock puppet.

AI related disasters are often categorized as involving misaligned AI, or misuse, or accident. Where:

misuse means the bad outcomes were wanted by the people involved,

misalignment means the bad outcomes were wanted by AI (and not by its human creators), and

accident means that the bad outcomes were not wanted by those in power but happened anyway due to error.

In thinking about specific scenarios, these concepts seem less helpful.

I think a likely scenario leading to bad outcomes is that AI can be made which gives a set of people things they want, at the expense of future or distant resources that the relevant people do not care about or do not own.

For example, consider autonomous business strategizing AI systems that are profitable additions to many companies, but in the long run accrue resources and influence and really just want certain businesses to nominally succeed, resulting in a worthless future. Suppose Bob is considering whether to get a business strategizing AI for his business. It will make the difference between his business thriving and struggling, which will change his life. He suspects that within several hundred years, if this sort of thing continues, the AI systems will control everything. Bob probably doesn’t hesitate, in the way that businesses don’t hesitate to use gas vehicles even if the people involved genuinely think that climate change will be a massive catastrophe in hundreds of years.

When the business strategizing AI systems finally plough all of the resources in the universe into a host of thriving 21st Century businesses, was this misuse or misalignment or accident? The strange new values that were satisfied were those of the AI systems, but the entire outcome only happened because people like Bob chose it knowingly (let’s say). Bob liked it more than the long glorious human future where his business was less good. That sounds like misuse. Yet also in a system of many people, letting this decision fall to Bob may well have been an accident on the part of others, such as the technology’s makers or legislators.

Outcomes are the result of the interplay of choices, driven by different values. Thus it isn’t necessarily sensical to think of them as flowing from one entity’s values or another’s. Here, AI technology created a better option for both Bob and some newly-minted misaligned AI values...
Sunday November 15th, 12:00PM (PT) — talks by Abram Demski, Daniel Kokotajlo and (maybe) more!
by habryka — 2020-11-13T00:53:17.126Z

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts). Again fully in Gather Town.

For the first hour, we will have a series of lightning talks each lasting about 5 minutes followed by discussion. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
From 1PM to 2PM, we'll just be casually hanging out in Gather Town. If you are not interested in the talks, feel free to just show up for this part (or the other way around).
We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

Speakers

Abram Demski: "Maybe something about a potential direction for raising the sanity waterline which I've been pondering"
Daniel Kokotajlo: Why we should build an AI Alignment Hub in Singapore

Details

When? Sunday November 15, 12:00PM (PT)

Where? http://garden.lesswrong.com?code=LeBk&event=curated-talks (this link will start working at the event start time. The event will be taking place in the Bacon hall.)
Notes on Respect-for-Others
by David Gross — 2020-11-12T23:33:15.845Z

This post examines the virtue of respect-for-others. It explores what other people have learned about this virtue more than my own opinions about it, though I’ve been selective about what I found interesting or credible, according to my own inclinations. I wrote this not as an expert, but as someone who wants to learn. I hope it helps people who want to know more about this virtue and how to nurture it.

What is this virtue?

The word “respect” is ambiguous; it covers several things. For example: You can respect a person’s position or rank by granting them authority. You can respect a person’s reputation or skills or character or taste. You can respect the threat a potentially dangerous person or thing poses to you. You can show respect for someone as a form of showing submission to them.

The virtue of respect-for-others I cover in this post is different. It has to do with understanding that other people have lives just as subjectively rich as yours, that they have their own perspectives, goals, desires, and priorities, and so forth, and that yours do not have objective priority over theirs.

This virtue is well summed up by the version of Kant’s Categorical Imperative that goes: “So act that you treat humanity… always at the same time as an end, never merely as a means.”[1]

There are a couple of ways people tend to describe how this variety of respect works. These are not mutually-exclusive, but people may emphasize one more than the other:

“I give every person some minimum baseline of respect that everyone deserves just by virtue of being a member of the human family, no matter who they are or what they’ve done.”
“I give everyone I meet a certain default amount of respect, and then adjust that amount up or down as I get to know them better.”

Related virtues

It seems odd to me that there isn’t a word in English that precisely encapsulates this virtue. Some related virtues that touch on respect-for-others include:

concern, consideration, thoughtfulness, compassion
sympathy, empathy
civility, politeness, tact
acknowledgement, recognition, appreciation, gratitude
liberality, tolerance
humility (in the sense of not overvaluing oneself compared with others)

It also harmonizes with “humanism” and “philanthropy” in the sense of valuing human beings highly, relative to institutions, ambitions, or other-worldly values.

In human development

Children develop respect for others slowly, in ...
What risks from vaccines?
by Bucky — 2020-11-12T23:32:30.278Z

I was chatting with friends about the Covid vaccine and they were concerned about side effects. I don’t expect significant side effects but don’t really have a principled analysis as to why not.

What kind of probability should one assign to significant side effects from a vaccine, particularly long term side effects which are less likely to be picked up in shorter testing regimes?
Final Babble Challenge (for now): 100 ways to light a candle
by Bird Concept — 2020-11-12T23:17:07.790Z

It is time. The final challenge in the 7-week babble challenge series.

Let’s

become stronger

. Let’s go out with a bang.

On the table in front of you is a candle.

This candle will burn as a metaphor for the light of Science, a little beacon of rationality. It will represent the will to keep practicing and honing our Art.

Your task is simple.

Light it.

You have 1 hour to come up with 100 ways.

Looking back

Here are the rankings before the final round. (You gain a star for completing a challenge, and lose one for missing a week. I’m not including myself since I’m the gamemaster.)

Fantastic work, everyone.

★★★★★★ gjm

★★★★★ Yonge

★★★★ Slider

★★★ Bucky

★★ Tetraspace Grouping, supposedlyfun

★ NunoSempere, Elizabeth, Mark Xu

Overall, since starting on September 30th, there’s now been more than 100 completed babble challenges.

As a result, we have babbled over 5000 ideas.

I haven’t counted how many unique users joined, but plausibly more than 70. For many of them, the babble challenge was one of their first comments ever on LessWrong. Welcome to you all.

I want to thank everyone who joined this quest, it’s been an honor practicing creativity with you.

There were really too many good submissions over the weeks to list them all. What’s more, much of the value comes in just being able to think of many different ones, rather than a single idea being excellent. Nonetheless, to celebrate and inspire you for the final challenge, I gathered some great ones from previous weeks:

Ways of going to the moon

I leave it on Earth ; eventually, in 4 billion years, the sun will have absorbed both the thing and the Moon and hopefully some parts of both will mix. (Vanilla_cabs)

Break out of the simulation, then reprogram myself to be on the moon. (mr-hire)

Use CRISPR to make myself smarter. Do whatever plan smarter Neel comes up with (Neel Nanda)

Send spaceships out to the asteroid belt to collect asteroids and bring them to earth. Not to extract valuable minerals, just to make the earth bigger and heavier. Both the increased radius and the increased gravity will bring the moon closer. Eventually it will be close enough that I can just reach out and put my object on the moon. (gjm)

…and more than 1000 more ideas!

Ways of escaping a locked room

Metal bars on the windows? Pee on them, take apart the phone, then connect one terminal of the phone batter

...
Communication Prior as Alignment Strategy
by johnswentworth — 2020-11-12T22:06:14.758Z

Alice has one of three objects:

A red triangle
A blue square
A red circle

She wants Bob to learn which object she has. However, Alice may only send one of three messages:

“My object is round”
“My object is red”
“This is not a pipe”

The rules of the game (i.e. the available messages) are common knowledge before the game starts. What message should Alice send for each object, and what object should Bob deduce from each message?

Let’s think it through from Bob’s standpoint. A clever human might reason like this:

“My object is round” implies it’s the red circle, because that’s the only round object.
“My object is red” implies it’s the red triangle, because only the triangle and circle are red, and Alice could have perfectly conveyed the information with “My object is round” if it were the circle.
“This is not a pipe” implies it’s the blue square, because Alice could have perfectly conveyed the information with one of the other two messages otherwise.

If you’ve played the game

CodeNames

, then this sort of reasoning might look familiar: "well, 'blue' seems like a good hint for both sky+sapphire and sky+water, but if it were sky+water they would have said 'weather' or something like that instead, so it's probably sky+sapphire...".

Intuitively, this sort of reasoning follows from a communication prior - a prior that someone is choosing their words in order to communicate. In everyday life, this comes up in e.g. the difference between connotation and denotation: when someone uses a connotation-heavy word, the fact that they used that word rather than some more neutral synonym is itself important information. More generally: the implication of words is not the same as their literal content. A communication prior contains a model of how-and-why-the-words-were-chosen, so we can update on the words to figure out their implications, not just their literal meanings.

Communication priors suggest an approach to certain problems in AI alignment. Intuitively, rather than saying “I want X” and the AI taking that completely literally (as computers generally do), the AI instead updates on the fact that I said “I want X”, and tries to figure out what those words imply about what I actually want. It’s like pushing the “do what I mean” button - the AI would try to figure out what we mean, rather than just doing what we say. Indeed, we could even have the AI treat its own source code as a signal ab...
Tips on organizing online meetups
by JoshuaFox — 2020-11-12T20:02:35.793Z

I have been co-organizing the SlateStarCodex online meetup series with ~40-140 participants in each meetup. We've experimented to get it to work smoothly and have some tips.

This advice is for meetups which have a talk and QA and post-talk socializing. I will appreciate similar tips from meetup organizers.

For socializing, we use breakout rooms as mentioned below; but for pure socializing, Icebreaker works even better, putting people together for short conversations (like "speed dating"). This worked well in some Effective Altruism Icebreaker sessions.

Another good format is Rump Sessions: Four-minute Lightning talks, given by on-the-spot volunteers (no need to pre-apply); and for each, the audience votes for a three-minute extension if the speaker wants. That worked well for the preliminary session in the series, but all the rest had invited speakers.

Here's how we do it.

We advertise using a regular (non-event) post on LessWrong, as LessWrong does not support non-geographical events. We also make an event post under LessWrong Tel Aviv, allowing people to get a notification or see this in the event page.
We also post to /r/slatestarcodex
We post to Facebook. Few Facebook groups in the rationality-sphere are really active, so we do not share to any of those. (I do post to my local LessWrong Israel group.)
We have a MailChimp mailing list, which has worked very well to get out the word. People can sign up when they register for a meetup.
For each session, we ask people to register; then we send them an invitation. So, we do not publicize a direct video-meet link. This approach gives us partial visibility into who will join.
We use Google Meet because the paid version allows more than 100 participants, and the paid version of Zoom does not. Usually someone can offer a paid version from their work or school, but note that the owner of the account will need to administer the call, so the owner cannot just hand off to the meeting organizers.
I suggest in the invitation and in my introduction that -- both for the main event and the after-talk socializing -- participants turn on video, and connect with their real name, for social connectivity. However, this is not mandatory.
We do it Sunday 18:30 UTC because this accommodates the US, Europe, and Israel times of day and workdays. (This leaves out are

...
Covid 11/12: The Winds of Winter
by Zvi — 2020-11-12T14:30:01.387Z

Spoiler Alert below the fold, can’t be helped.

We have perhaps the best possible news. Pfizer’s vaccine is over 90% effective, and they are going to ask for emergency use authorization in a few weeks. Woo-hoo!

We also have very bad news. It was clear the numbers were going to get worse, and they are substantially worse than I expected. Things are about to get quite bad out there. Also the attempts to overturn the results of the election aren’t great.

There’s a thought I can’t shake, though.

You ever think a writer is getting a bit lazy when it comes time to wrap things up?

An inhuman threat that grows more deadly as it gets colder, that no ordinary sword can slay and that cares nothing for politics, grows exponentially in power by the day.

There remain two who would rule the land.

One, who is currently in the seat of power, claims they and their family are the best at business but their longtime revenue sources are bankrupt and they rely heavily on loans from banks to stay afloat, while their true accounting method is being vindictive as hell. Infighting has caused the most competent people in the administration to abandon it. Having seized control in the crucial moment amidst divided opposition and maintained it against claims of their illegitimacy and worse, they put up a surprisingly strong fight. When provided with evidence of the inhuman threat right in front of their face, menacing them directly, they briefly acknowledge it, but mostly ignore it and are happy to let others deal with it. It’s not their problem. They warn that their rival would destroy all if allowed to rule. They refuse to admit defeat and keep only the counsel of very close family, even after the verdict is clear.

The other used to be high up the line of succession before power changed hands, and has returned with allies of varied ethnic origins. They have assembled a broad coalition, many with pasts they are ashamed about, and some of whom have proven difficult to control and prone to rioting. Many are there purely because they would welcome almost any change. They have suffered great tragedy in their life, losing two children and a spouse. They arrive to restore the old norms, for honor and dignity, and to free the people. They pledge to fight systemic injustice. They claim they will rule for all the people, especially those who need help the most.

As the...
What are Examples of Great Distillers?
by adamShimi — 2020-11-12T14:09:59.128Z

A distiller (name based on https://distill.pub/) is a writer who can explain a complex topic, one with a lot of research debt, so that knowledgeable undergraduates in the subject can get a good intuition and picture of the subject. I'm looking for examples of people you consider great distillers in a field in which you have advanced knowledge. For example, Terrence Tao is a great distiller of Mathematics, and Scott Aaronson is a great distiller of Complexity Theory and Quantum Computing. What are your favorite distillers, and why?

I'm asking because I'm trying to improve my own distilling skills, and studying/stealing from the masters is a great way to get better. Only issue is that you need to know the masters.
Any work on honeypots (to detect treacherous turn attempts)?
by David Scott Krueger (formerly: capybaralet) — 2020-11-12T05:41:56.371Z

I know the idea of making a "honeypot" to detect when an AI system would attempt a treacherous turn if given the opportunity has been discussed (e.g. IIRC, in Superintelligence). But is there anyone actually working on this? Or any work that's been published?
Does there exist a detailed Bayesian COVID tracker?
by Optimization Process — 2020-11-12T05:06:25.014Z

[bounty: $100 for recommending a tool that I use for more than two weeks]

I would love (and happily pay for) a piece of software that I could tell (a) a bunch of recent interactions between people ("Alice, masked, spent 3 hours, indoors, distanced, with Bob, unmasked, on Nov 1"), and (b) a bunch of evidence about their health over time ("Dolores tested negative on Nov 3"), and make queries about how likely various people are to have COVID.

I'd want it to be capable of complicated inferences like "Alice met with Bob yesterday; Bob met with Charlie 4d ago; Charlie separately met with Dolores that same day. Dolores just tested negative; given that, Alice is now less likely to be incubating COVID."

Ideally, it would take into account things like contagiousness-over-time profiles, and incubation periods, and asymptomatic cases, and how all those things differ between people, and tests' false negative rates -- but I realize that's a lot to ask.

Some non-solutions:

https://microcovid.org is great at what it does, but what it does is analyze individual activities, not make inferences between people and across time.

^ The associated MicroCOVID spreadsheet does better on this front, but (AFAICT) doesn't capture correlations between people's risk levels, or make inferences like "Zelda tested negative, therefore all the microcovids she inflicted over the last couple weeks should be somewhat discounted."

Privacy-conscious COVID tracking apps can't offer the level of sophistication I want. I want to be able to account for masked-ness and ventilation, which flatly isn't captured by "How many pings did Alice's phone hear from Bob's?"