Harley-ml/lesswrong huggingface dataset:
# What Belongs in my Glossary?

I've been working on a Glossary that will give short descriptions of concepts I've introduced, so people don't have to seek out entire posts to grok what they mean in context and then decide later if they'd like to know more. So far I've got Asymmetric Justice, The Delenda Est Club, Doom Patrol, Everything is Trying to Kill You, Easy Mode, Hard Mode, Perfect/Superperfect/Imperfect Competition, Immoral Maze, Maze Behaviors, Maze Levels, Slack, Out to Get You, Simulacra Levels, Sacrifice to the Gods, Very Serious People.  

What other things should I make sure to include? (Post has been edited as I add things, answers below that suggest things that are in likely were helpful, thank you)

(Also any cool elegant/short definitions for the above or other Zvi-concepts are welcome, including Devil's Dictionary style versions)
# Three more stories about causation

**I**

Imagine Bob's graph of health over time is sin(t) and his graph of exercise is cos(t), with negative values corresponding to being a couch potato. At first glance, health and exercise have nothing to do with each other, since their correlation is zero. At second glance, maybe exercise determines health with a π/2 lag. At third glance, maybe it determines health with a 3π/2 lag and a minus sign. At fourth glance, maybe exercise determines the derivative of health. You'll find similar problems with any bounded functions over a long timespan, not just sin and cos.

**II**

Instead of correlation fishing, let's investigate causation like real scientists, with graphs. Smoking causes tar in the lungs, tar is correlated with cancer, and for any fixed value of "tar" the values of "smoking" and "cancer" are conditionally uncorrelated. So there cannot be a trait that causes both smoking and cancer independently: the causation must go through tar, which we know is caused by smoking. Therefore smoking causes cancer.

But imagine there's a trait that gives you a "set point" of X tar, continually adjusting your desire to smoke and compensating for external factors, and also gives you X risk of cancer. Then the above will still hold, but quitting smoking won't help you.

**III**

Forget graphs. Let's use the golden standard of determining causation: a randomized controlled experiment. Does skydiving increase risk of death? Yes, if we force a random half of all people to skydive, more of them will die than in the control group.

But now take a random half of all people and force them *not* to skydive. Those who didn't skydive anyway will be unaffected, but those who skydived might change their behavior. What if they switch to something more dangerous, like basejumping, because getting the same thrill by safer skydiving is no longer available? Then this experiment also leads to more deaths, and our question about causation remains unanswered.

If we can't trust correlations, graphs, or experiments - what can we trust? What, in the end, is causation?
# Open & Welcome Thread – November 2020

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/posts/2rWKkWuPrgTMpLRbp/lesswrong-faq-1). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lesswrong.com/tag/open-threads?sortedBy=new).
# Share your personal stories of prediction markets

Last nights prediction markets on the election were heavily traded relative to past PMs. Predictit alone had hundreds of millions of USD in trade volume. Many LWers competed on the markets, for reasons discussed [here](https://www.lesswrong.com/posts/y8RWtNBiksbSzm9j4/bet-on-biden). If you competed, share your experience here.
# Don't read news

A lot of people have written about the toxicity of news. This idea, that reading news is something to be avoided, might not be new to you. But today might be a great day for a reminder. I needed a reminder, because I'm tempted... very temped.

I could make the case again, that avoiding news is the thing to do, but Rolf Dobelli already did this elegantly. In his article, [Avoid News](https://www.gwern.net/docs/culture/2010-dobelli.pdf), he discusses fifteen points:

1.  News misleads us systematically
2.  News is irrelevant
3.  News limits understanding
4.  News is toxic to your body
5.  News massively increases cognitive errors
6.  News inhibits thinking
7.  News changes the structure of your brain
8.  News is costly
9.  News sunders the relationship between reputation and achievement
10.  News is produced by journalists
11.  Reported facts are sometimes wrong, forecasts always
12.  News is manipulative
13.  News makes us passive
14.  News gives us the illusion of caring
15.  News kills creativity

His article was written in 2010 and since then the world has changed. In a world where everything is "Fake News!" and conspiracy theories are becoming normal, complaining about news makes me sound like one of the crazies. The world has become a place where mainstream media doesn't seem toxic anymore, but a breath of fresh air.

Nonetheless, the fact that worse alternatives exist, doesn't make his points invalid. I notice myself craving to know what is going on, on the other side of the world, on an hourly basis, while it will only cause me stress. And none of the information will be actionable. If you currently live in America the case might seem harder to make, but even then a daily update from a friend will probably suffice. 

Dobelli ends with the following advice:

> Go without news. Cut it out completely. Go cold turkey. Make news as inaccessible as possible.

* * *

*If you want to explore this topic further I recommend the article by Dobelli:* [*Avoid News*](https://www.gwern.net/docs/culture/2010-dobelli.pdf)*, and an article by Gwern:* [*Littlewood's Law and the Global Media*](https://www.gwern.net/Littlewood)*.*
# What considerations influence whether I have more influence over short or long timelines?

As my timelines have been shortening, I've been rethinking my priorities. As have many of my colleagues. It occurs to us that there are probably general considerations that should cause us to weight towards short-timelines plans, or long-timelines plans. (Besides, of course, the probability of short and long timelines) For example, if timelines are short then maybe AI safety is more neglected, and therefore higher EV for me to work on, so maybe I should be systematically more inclined to act as if timelines are short.

We are at this point very unsure what the most important considerations are, and how they balance. So I'm polling the hive mind!
# Sunday, Nov 8: Tuning Your Cognitive Algorithms

For this Sunday's LW Online Meetup, I'll be will be leading an exercise on *Mindful Puzzle Solving,* derived from the [*Tuning Your Cognitive Algorithms*](http://bewelltuned.com/tune_your_cognitive_strategies) exercise on [bewelltuned.com](http://bewelltuned.com/). 

The basic premise is to solve a *medium-difficulty-puzzle,* while spending a lot of attention on noticing exactly what your brain is doing at every step. You can then notice which pieces of your process are doing most of the work, and which are wasted motion.

This is not only valuable for improving your puzzle-solving abilities, but for generally improving the feedback loops that improve your cognitive abilities. I've found this to be one of the most essential rationalist skills that I've learned.

We'll be meeting in the Lecture Hall in the Walled Garden, at noon PT. ([Follow this invite link](http://garden.lesswrong.com?code=yoG1&event=guest-pass) to enter the Garden. Follow the path to the Tardis, and then enter the "Bacon Lecture Hall")

I'll give a short talk on the theory-and-practice of tuning your cognitive algorithms. We'll spend two 20 minute periods doing individual exercises, and then talking about what we learned.

http://garden.lesswrong.com?code=yoG1&event=guest-pass
# How can I bet on short timelines?

I currently have something like 50% chance that the [point of no return](https://www.lesswrong.com/posts/JPan54R525D68NoEt/the-date-of-ai-takeover-is-not-the-day-the-ai-takes-over) will happen by 2030. Moreover, it seems to me that there's [a wager for short timelines](https://www.lesswrong.com/posts/pTK2cDnXBB5tpoP74/), i.e. you should act as if short timelines scenarios are more likely than they really are, because you have more influence over them. I think that I am currently taking short timelines scenarios much more seriously than most people, even most people in the AI safety community. I suppose this is mostly due to having higher credence in them, but maybe there are other factors as well.

Anyhow, I wonder if there are ways for me to usefully bet on this difference.

Money is only valuable to me prior to the point of no return, so the value to me of a bet that pays off after that point is reached is approximately zero. In fact it's not just money that has this property. This means that no matter how good the odds are that you offer me, and even if you pay up front, I'm better off just taking out a low-interest loan instead. Besides, I don't need money right now anyway, at least to continue my research activities. I'd only be able to achieve significant amounts of extra good if I had quite a lot more money.

What do I need right now? I guess I need [knowledge](https://www.lesswrong.com/posts/wEebEiPpEwjYvnyqq/when-money-is-abundant-knowledge-is-the-real-wealth) and help. I'd love to have a better sense of what the world will be like and what needs to be done to save it. And I'd love to have more people doing what needs to be done.

Can I buy these things with money? I don't think so... As the linked post argues, knowledge isn't something you can buy, in general. On some topics it is, but not all, and in particular not on the topic of what needs to be done to save the world. As for help, I've heard from various other people that hiring is net-negative unless the person you hire is both really capable and really aligned with your goals. But IDK.

There are plenty of people who are really capable and really aligned with my goals. Some of them are already helping, i.e. already doing what needs to be done. But most aren't. I think this is mostly because they disagree about what needs to be done, and I think that is largely because their timelines are longer than mine. So, maybe we can arrange some sort of bet... for example, maybe I could approach people who are capable and aligned but have longer timelines, and say: "If you agree to act as if my timelines are correct for the next five years, I'll act as if yours are correct thereafter."

Any suggestions?
# Somerville Mask Usage II

A little over four months ago I [collected some data on how often people were wearing masks](https://www.jefftk.com/p/somerville-mask-usage). Somerville still requires masks, cases are up, there is now a [state-wide order](https://www.mass.gov/doc/covid-19-order-55/download), and I again found myself with some time during which I could take some notes. These numbers are not completely comparable: last time I tracked on a Sunday, this time a Saturday; last time I counted from 9:41am to 10:21am, this time 9:22am to 9:54am. Still, these should be pretty similar samples of behavior here.

I counted 130 people, but last time I didn't count babies and this time I did, by accident, which means that my number for children is no good. Looking just at adults, I counted 109 (vs 165) people of which 80% (87, +6pp) were masked, 9% (10, +4pp) had their nose exposed, 6% (7, -9pp) had their mask removed, and 5% (5, -1%) had no mask:

[![](https://www.jefftk.com/somerville-adult-masking-rates.png)](https://www.jefftk.com/somerville-adult-masking-rates-big.png)

It looks like masking rates are up! Additionally, some people with masks on their chin would pull them up over their nose and mouth when passing people, but I counted everyone based on their masking status as they passed a particular mostly dead tree in front of me:

[![](https://www.jefftk.com/masking-tree-point.jpg)](https://www.jefftk.com/masking-tree-point-big.jpg)

As before, women were less likely to be masked, but both women and men were more likely to be masked than in June:

[![](https://www.jefftk.com/somerville-masking-by-gender.png)](https://www.jefftk.com/somerville-masking-by-gender-big.png)

Outdoors is still relatively safe, even without masks, since ventilation is so rapid, but the bike path is relatively crowded as outdoor places go. I'd be more interested in a comparison of grocery store numbers, but I haven't been inside a store since March and so am not in a good position to check.

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100192148928842)*
# Sunday 8th 2pm PT: Scott Garrabrant's Office Hours / Cartesian Frames

Sunday Nov 8th at 2pm, Scott Garrabrant will be in his Walled Garden office at 2pm to host discussion and questions about Cartesian Frames. There won't be a formal lecture, but stop by to chat, ask questions or explore related ideas.

You can find him in the University section of the Garden, also accessible via the Tardis in the central area.

[http://garden.lesswrong.com?code=PZbv&event=scott-garrabrant-office-hours](http://garden.lesswrong.com?code=PZbv&event=scott-garrabrant-office-hours)

You can read more about Cartesian Frames here:

[https://www.lesswrong.com/s/2A7rrZ4ySx6R8mfoT](https://www.lesswrong.com/s/2A7rrZ4ySx6R8mfoT)

His office looks like this:

![](https://scontent.fsac1-2.fna.fbcdn.net/v/t1.0-9/124413609_4652083768198157_5702719077363509640_o.jpg?_nc_cat=105&ccb=2&_nc_sid=340051&_nc_ohc=Iw8iGLZ4STUAX8MN6BS&_nc_ht=scontent.fsac1-2.fna&oh=a006acbf89804581a29d1e9f920ebdb1&oe=5FCDDFBA)
# Is corruption a valuable antidote to overregulation?

When reading Robert Caro's book about Robert Moses there's a part of my that things that I'm glad that the city in which I'm living is less corrupt then New York was when Robert Moses was active. 

On the other hand Robert Moses got things done and build a lot of infrastructure. After the Great Stagnation we aren't succeeding at building new infrastructure like Moses did.

In the discussion about why we don't have flying cars there's discussion about how nuclear plants in the US grew in cost while South Korea managed to continue to build cheaper nuclear plants. I read a few more recent articles and South Korea and it seems that after scandals that uncovered that a lot of corruption was involved in the nuclear industry when it managed to provide cheap nuclear power, and uncovering that corruption blew up their nuclear industry.

Corruption prevents bureaucracy from strangling progress as it gives people in power a way to get things done despite of the bureaucracy. 

While I have strong instincts that corruption is evil, it seems we lack a good discussion of it in relation to the evil of bureaucracy that strangles progress.

What are your thoughts on the issue?
# Building AGI Using Language Models

> Despite the buzz around GPT-3, it is, in and of itself, not AGI. In many ways, this makes it similar to AlphaGo or Deep Blue; while approaching human abiliy in one domain (playing Chess/Go, or writing *really* impressively), it doesn’t really seem like it will do Scary AGI Things™ any more than AlphaGo is going to be turning the Earth into paperclips anytime soon. While its writings are impressive at emulating humans, GPT-3 (or any potential future GPT-x) has no memory of past interactions, nor is it able to follow goals or maximize utility. However, language modelling has one *crucial* difference from Chess or Go or image classification. Natural language essentially encodes information about the world—the *entire* world, not just the world of the Goban, in a much more expressive way than any other modality ever could. By harnessing the world model embedded in the language model, it may be possible to build a proto-​AGI.
# Automating reasoning about the future at Ought

Ought’s [mission](https://ought.org/mission) is to automate and scale open-ended reasoning. Since wrapping up [factored evaluation experiments](https://ought.org/updates/2020-01-11-arguments) at the end of 2019, Ought has built [Elicit](http://elicit.org/) to automate the open-ended reasoning involved in judgmental forecasting.

Today, Elicit helps forecasters build distributions, track beliefs over time, collaborate on forecasts, and get alerts when forecasts change. Over time, we hope Elicit will:

*   Support and absorb more of a forecaster’s thought process
*   Incrementally introduce automation into that process, and
*   Continuously incorporate the forecaster’s feedback to ensure that Elicit’s automated reasoning is aligned with how each person wants to think.

[Our latest blog post](https://deploy-preview-48--ought.netlify.app/updates/2020-10-09-forecasting) introduces Elicit and our focus on judgmental forecasting. It also reifies the vision we’re running towards and potential ways to get there.
# Tweet markets for impersonal truth tracking?

_Crossposted from [world spirit sock puppet](https://worldspiritsockpuppet.com/meteuphoric.html)._

Should social media label statements as false, misleading or contested?

Let’s approach it from the perspective of what would make the world best, rather than e.g. what rights do the social media companies have, as owners of the social media companies.

The basic upside seems to be that pragmatically, people share all kinds of false things on social media, and that leads to badness, and this slows that down.

The basic problem with it is that maybe we can’t distinguish worlds where social media companies label false things as false, and those where they label things they don’t like as false, or things that aren’t endorsed by other ‘official’ entities. So maybe we don’t want such companies to have the job of deciding what is considered true or false, because a) we don’t trust them enough to give them this sacred and highly pressured job forever, or b) we don’t expect everyone to trust them forever, and it would be nice to have better recourse when disagreement appears than ‘but I believe them’.

If there were a way to systematically inhibit or label false content based on its falseness directly, rather than via a person’s judgment, that would be an interesting solution that perhaps everyone reasonable would agree to add. If prediction markets were way more ubiquitous, each contentious propositional Tweet could say under it the market odds for the claim.

Or what if Twitter itself were a prediction market, trading in Twitter visibility? For just-posted Tweets, instead of liking them, you can bet your own cred on them. Then a while later, they are shown again and people can vote on whether they turned out right and you win or lose cred. Then your total cred determines how much visibility your own Tweets get.

It seems like this would solve:

*   the problem for prediction markets where it is illegal to bet money and hard to be excited about fake money
*   the problem for prediction markets where it’s annoying to go somewhere to predict things when you are doing something else, like looking at Twitter
*   the problem for Twitter where it is full of fake claims
*   the problem for Twitter users where they have to listen to fake claims all the time, and worry about whether all kinds of things are true or not

It would be pretty imperfect, since it throws the gavel to future Twitter users, but perhaps they are an improvement on the status quo, or on the status quo without the social media platforms themselves making judgments.
# Does BioNtech's vaccine result of 90% disease prevention mean that 90% of the vaccinated can't pass the virus to other people? 

From reading the news it's not clear to me whether some of those people who receive the vaccine and are in the 90% where it works will be asymptomatic but still infectious. Does anybody know more?
# Did anybody calculate the Briers score for per-state election forecasts?

There's a lot of debate about how good the polls and 538 have been is this election in comparison to the betting markets. While it's hard to compare it when just looking at percentage for Biden winning, it would be possible to calculate the Briers score by looking at all US states. Did anybody do the math?
# What Would Advanced Social Technology Look Like?

I’m looking for pure sci-fi speculation here. (Though preferably on the “hard” end of the [Mohs scale of sci-fi hardness](https://tvtropes.org/pmwiki/pmwiki.php/Main/MohsScaleOfScienceFictionHardness).) Feel free to babble. Maybe try to [come up with 50 ideas](https://www.lesswrong.com/posts/pDkZitYsJAwf8mHKJ/babble-challenge-50-ways-of-sending-something-to-the-moon) if you're into that.

For purposes of the question, "social technology" should be interpreted broadly. Some subcategories include:

*   Language technology, e.g. phonetic alphabets or mathematical notation
*   Financial technology, e.g. futures contracts
*   Legal technology, e.g. limited liability corporations or joint-stock corporations
*   Mechanism design, e.g. clever voting schemes or prediction markets
*   Coordination technology, e.g. standard weights and measures or traffic lights
*   Propaganda
# Ongoing free money at PredictIt

Previously on LW
================

*   [https://www.lesswrong.com/posts/JmqBtCCnMZmpXP2bE/what-are-good-election-betting-opportunities?commentId=NFdSk5H39omYXPjBS,-](https://www.lesswrong.com/posts/JmqBtCCnMZmpXP2bE/what-are-good-election-betting-opportunities?commentId=NFdSk5H39omYXPjBS,-)
*   [https://www.lesswrong.com/posts/bSKWSPM927oZZqN7f/predictit-presidential-market-is-increasingly-wrong](https://www.lesswrong.com/posts/bSKWSPM927oZZqN7f/predictit-presidential-market-is-increasingly-wrong)

PredictIt is structurally inefficient
=====================================

*   A maximum of 5,000 people in one market
*   A maximum of $850 in betting per person per contract (*not* per market)
*   A substantial house cut of 10% on profits and 5% on withdrawals.

Because of the market's structure, smart money cannot (given the first two reasons) or will not (given the third reason) fix all instances of mis-priced markets.

Free money? Free money!
=======================

Have you ever wished you could bet on past events? Well, today is your lucky day!

Q: Who won the Texas Democratic primary? A: Joe Biden

Q: Who won the presidency? A: Joe Biden

As I write this, you can purchase $1 for the price of $.85, in the [Will the 2020 TX Democratic primary winner win the presidency?](https://www.predictit.org/markets/detail/5961/Will-the-2020-TX-Democratic-primary-winner-win-the-presidency) market.

Are there other markets like this? Yes!

*   [Who will be the Senate-confirmed Secretary of State on Feb. 15?](https://www.predictit.org/markets/detail/6944/Who-will-be-the-Senate-confirmed-Secretary-of-State-on-Feb-15) Not Mike Pompeo @ 85 cents.
*   [Who will be the Senate-confirmed Attorney General on Mar. 1?](https://www.predictit.org/markets/detail/6952/Who-will-be-the-Senate-confirmed-Attorney-General-on-Mar-1) Not Bill Barr @ 88 cents.
*   [Which party will win Wisconsin in the 2020 presidential election?](https://www.predictit.org/markets/detail/5542/Which-party-will-win-Wisconsin-in-the-2020-presidential-election) Not Republican @ 85 cents.
*   [Will the 2020 SC Democratic primary winner win the presidency?](https://www.predictit.org/markets/detail/5960/Will-the-2020-SC-Democratic-primary-winner-win-the-presidency) Yes @ 86 cents.

Sometimes, you can [find a $20 bill](https://www.lesswrong.com/posts/zsG9yKcriht2doRhM/inadequacy-and-modesty#ii_) on the ground and pick it up.
# Michael Vassar at  the SlateStarCodex Online Meetup

Michael Vassar will speak on "Why Do We Call Ourselves Rationalists?"

Please register [here]( https://forms.gle/5k1bQR5fQdNaZTNG7 ) and we will send you an invitation.

The event begins Nov. 22, 2020 at 20:30 Israel Standard Time, 10:30 Pacific Standard time, 18:30 UTC.

Abstract: The words rationalism and empiricism have specific historical meanings. The word Bayesian, the foundational epistemology of our community, has a very different meaning, yet in practice, we tend to adopt frequentist empiricism as our usual working epistemology. This works OK in many domains, but works much less well in adversarial domains, such as the evaluation of grievances. As a result, we tend to misunderstand adversarial domains, most notably macroeconomics, but also Grievance Studies fields such as Feminism and Critical Race Theory, leading to unnecessary political conflicts. Bayesian approaches, used correctly, can precisely bound the ways in which ‘Politics is the Mindkiller’ enabling us to be reasonable political actors rather than pretending politics is a Basilisk.

Michael Vassar was executive director of the Singularly Institute before it became MIRI and CFAR. He lives in Austria and New York and is currently active in prison reform.
# What could one do with truly unlimited computational power?

**One day, you find an unmarked cardboard box sitting on your front porch.**

On top of the box is a single note:

“Open me.”

You take the box inside and open it, to find what appears to be a small black laptop nestled between old newspaper clippings. There are no identifying features to the laptop, other than its sleek blackness and small size. When you open the laptop, instead of being greeted with any sort of familiar welcome screen, there is simply text, displayed in white over a black background. The text says the following:

“Welcome to the next stage of the human experiment.

I have have been watching your kind for a while now, and I believe that you are now ready. I have decided to entrust this machine to your care. Do not attempt to figure out how this machine performs its calculations, as uninformed tampering may result in the deletion of your universe.

The item you are now holding is what your mathematicians would call a Universal Turing Machine, similar in many ways to your typical computer. This machine, however, is significantly different from any other currently existent on Earth.

After reading this message, press any button to reveal two input fields stacked on top of each other, and one output field at the bottom. All input fields can be fed any required data from the internet, or can be entered into directly through the keyboard. Input lengths of any finite size are acceptable.

The top input field will accept and is capable of automatically running the intended code of any language or format that is capable of being run on a standard Universal Turing Machine.

That input will then be translated into the necessary binary code to be computable by the internal “black box” computer.

The bottom input field must be fed a finite natural number of any length, which will determine the number of computations per second. Do not worry about exceeding the speed of light, or of going past any other finite limit; the computation itself is performed in a “bubble universe” with different physical laws than your own, and is capable of computing at absolutely any positive finite speed relative to your universe.

The output field will display the output of your calculations, if any exist, after exactly one minute of computation relative to you.

Do with this machine as you will.

Wishing you the best,

God.”

Your finger hovers over the keyboard.

What will you do with this marvelous machine? What *can* you do?

What happens next is up to you.

>________________________________

NOTE FROM ME, OUTSIDE OF THE STORY: I wrote this trying to work out my thoughts on what might be possible with a machine with unlimited but finite computing power. I was going to continue the story, but found that I honestly couldn't think of all that many interesting things that would be possible to do with such a machine, that couldn't already be done now. As such, I'm turning this question public, hoping that anyone reading this might have some interesting ideas that I haven't thought of.
# CHAI Internship Application

Hi everyone,

I'm the Assistant Director at [CHAI](https://humancompatible.ai/) and as some of you may know, CHAI is currently accepting applications for our 2021 internship program.

Early deadline is 11/23 for applicants who require an earlier response from us. Our normal deadline is 12/13. 

You can find more information and the application itself [here](https://humancompatible.ai/jobs#internship)

Please e-mail me at [chai-admin@berkeley.edu](https://www.lesswrong.com/posts/W4fCJqBvKD6vrMAet/chai-admin@berkeley.edu) if you have any questions!
# [Link] Digital Democracy Is Within Reach

EPISODE SUMMARY

Imagine a world where every country has a digital minister and technologically-enabled legislative bodies. Votes are completely transparent and audio and video of all conversations between lawmakers and lobbyists are available to the public immediately. Conspiracy theories are acted upon within two hours and replaced by humorous videos that clarify the truth. Imagine that expressing outrage about your local political environment turned into a participatory process where you were invited to solve that problem and even entered into a face to face group workshop. Does that sound impossible? It’s ambitious and optimistic, but that's everything that our guest this episode, Audrey Tang, digital minister of Taiwan, has been working on in her own country for many years. Audrey’s path into public service began in 2014 with her participation in the Sunflower Movement, a student-led protest in Taiwan’s parliamentary building, and she’s been building on that experience ever since, leading her country into a future of truly participatory digital democracy.
# Any work on honeypots (to detect treacherous turn attempts)?

I know the idea of making a "honeypot" to detect when an AI system would attempt a treacherous turn if given the opportunity has been discussed (e.g. IIRC, in Superintelligence).  But is there anyone actually working on this?  Or any work that's been published?
# What are Examples of Great Distillers?

A distiller (name based on [https://distill.pub/](https://distill.pub/)) is a writer who can explain a complex topic, one with a lot of [research debt](https://distill.pub/2017/research-debt/), so that knowledgeable undergraduates in the subject can get a good intuition and picture of the subject. I'm looking for examples of people you consider great distillers in a field in which you have advanced knowledge. For example, [Terrence Tao](https://terrytao.wordpress.com/) is a great distiller of Mathematics, and [Scott Aaronson](https://www.scottaaronson.com/blog/) is a great distiller of Complexity Theory and Quantum Computing. What are your favorite distillers, and why?

I'm asking because I'm trying to improve my own distilling skills, and studying/stealing from the masters is a great way to get better. Only issue is that you need to know the masters.
# Sunday November 15th, 12:00PM (PT) — talks by Abram Demski, Daniel Kokotajlo and (maybe) more!

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see [here](https://www.lessestwrong.com/tag/lesswrong-events) for previous weeks' transcripts). Again fully in Gather Town.

*   For the first hour, we will have a series of lightning talks each lasting about 5 minutes followed by discussion. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
*   From 1PM to 2PM, we'll just be casually hanging out in Gather Town. If you are not interested in the talks, feel free to just show up for this part (or the other way around).
*   We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

*If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up* [*here*](https://forms.gle/iwFatbhys9muPmQA7)*.*

Speakers
--------

*   Abram Demski: *"Maybe something about a potential direction for raising the sanity waterline which I've been pondering"*
*   Daniel Kokotajlo: *Why we should build an AI Alignment Hub in Singapore*

Details
-------

**When?** Sunday November 15, 12:00PM (PT)

**Where?** [http://garden.lesswrong.com?code=LeBk&event=curated-talks](http://garden.lesswrong.com?code=LeBk&event=curated-talks) (this link will start working at the event start time. The event will be taking place in the Bacon hall.)
# Misalignment and misuse: whose values are manifest?

_Crossposted from [world spirit sock puppet](https://worldspiritsockpuppet.com/meteuphoric.html)._

AI related disasters are often categorized as involving misaligned AI, or misuse, or accident. Where:

*   misuse means the bad outcomes were wanted by the people involved,
*   misalignment means the bad outcomes were wanted by AI (and not by its human creators), and
*   accident means that the bad outcomes were not wanted by those in power but happened anyway due to error.

In thinking about specific scenarios, these concepts seem less helpful.

I think a likely scenario leading to bad outcomes is that AI can be made which gives a set of people things they want, at the expense of future or distant resources that the relevant people do not care about or do not own.

For example, consider autonomous business strategizing AI systems that are profitable additions to many companies, but in the long run accrue resources and influence and really just want certain businesses to nominally succeed, resulting in a worthless future. Suppose Bob is considering whether to get a business strategizing AI for his business. It will make the difference between his business thriving and struggling, which will change his life. He suspects that within several hundred years, if this sort of thing continues, the AI systems will control everything. Bob probably doesn’t hesitate, in the way that businesses don’t hesitate to use gas vehicles even if the people involved genuinely think that climate change will be a massive catastrophe in hundreds of years.

When the business strategizing AI systems finally plough all of the resources in the universe into a host of thriving 21st Century businesses, was this misuse or misalignment or accident? The strange new values that were satisfied were those of the AI systems, but the entire outcome only happened because people like Bob chose it knowingly (let’s say). Bob liked it more than the long glorious human future where his business was less good. That sounds like misuse. Yet also in a system of many people, letting this decision fall to Bob may well have been an accident on the part of others, such as the technology’s makers or legislators.

Outcomes are the result of the interplay of choices, driven by different values. Thus it isn’t necessarily sensical to think of them as flowing from one entity’s values or another’s. Here, AI technology created a better option for both Bob and some newly-minted misaligned AI values that it also created—‘Bob has a great business, AI gets the future’—and that option was worse for the rest of the world. They chose it together, and the choice needed both Bob to be a misuser and the AI to be misaligned. But this isn’t a weird corner case, this is a natural way for the future to be destroyed in an economy.

_Thanks to Joe Carlsmith for conversation leading to this post._
# Reading/listening list for the US failing or other significant shifts?

I have a tentative hypothesis that the US is quite precarious. Although I believe we have likely avoided the worst outcomes this year, I think the next 10-30 yrs have some reasonable chance of going very badly. I don't know what that means, nor what the implications are for top cause areas.

I want to create a reading/listening list for exploring all this. Things I'm interested in:

*   history, particularly broad arcs/analysis of history (e.g., maybe I should read some Peter Turchin stuff)
*   history, particularly significant changes in global power dynamics (where power also involves technological advancement or similar)
*   ....other things I'm not thinking of that feel relevant

I have no background here at all. So even very basic recommendations would be appreciated.
# Interest survey: Forming an MIT Mystery Hunt team (Jan. 15-18, 2021)

The upcoming [MIT Mystery Hunt](http://web.mit.edu/puzzle/www/) will be all-remote, which presents an opportunity to form a widely distributed team for the LW/SSC community. The Mystery Hunt can be a lot of fun and is a great exercise in solving challenging problems. With enough people on the team we might even have a chance of winning!
 
If you think you might be interested please fill out the [survey](https://forms.gle/vicczrtfLFGBLHnH8).
# What are some good examples of fake beliefs?

I'm reading through Rationality from AI to Zombies again and just finished the section on [Fake Beliefs](https://www.lesswrong.com/s/7gRSERQZbqTuLX5re). I understand the ideas and now I'm trying to ask myself whether the ideas are useful.

To me, the biggest reason for them being useful is if I myself have some fake beliefs, it'd be good to identify and get rid of them. I don't think I do though.

Another reason why they might be useful is to identify them in others and better understand what is going on. Actually, this doesn't seem particularly useful, but it at least is somewhat interesting/satisfying. Anyway, I'm having trouble thinking of good examples of fake beliefs in others. The examples in the blog posts seem pretty contrived and perhaps uncharitable to what is actually going on in their heads.

So, to wrap this question up, I am most interested in hearing about examples of fake beliefs that rationalists might be prone to having, but I am also interested in hearing any other examples.
# On Arguments for God

This post is about God, but of course, it isn't really about God, but about a particular pattern in general.

We're pretty much all in agreement that God doesn't exist.

This is correct or at least let's say it is for the purposes of this post. However, this also poses a trap. 

Suppose there are forty arguments for God. Even if we know definitely for a fact that God doesn't exist, it doesn't mean that all of these forty arguments are wrong.

It would if all of these arguments claimed to definitely prove that he exists, but not if some of these arguments only claim he is more likely to exist than not or that he isn't as unlikely as we might think.

In fact, it'd actually be suspicious if all forty of these arguments came out *against* God. Surely we should expect the advantage to belong to the deists in at least one or two?

But since we have very good reasons to believe God doesn't exist and someone presents us with those arguments, surely we'll assume that they *have* to be wrong. And so we'll search very hard, until we find something that is plausibly an error or at least more plausible than God and talk ourselves into believing it. 

And once we've introduced that first error, we've opened up the door for more.
# European Master's Programs in Machine Learning, Artificial Intelligence, and related fields

While there is no shortage of detailed information on master’s degrees, we think that there is a lack of perspectives from students that have actually completed the program and experienced the university.

Therefore we decided to write articles on multiple European master's programs on Machine Learning, Artificial Intelligence, and related fields. The texts are supposed to give prospective students an honest evaluation of the teaching, research, industry opportunities, and city life of a specific program. Since many of the authors are [Effective Altruists](https://www.effectivealtruism.org) and interested in AI safety a respective section is often included as well.

It may not always be obvious, but there are many English-language degrees across Europe. Compared to America, these can be more affordable, offer more favorable visa terms, and a higher quality of life. We hope that you will consider bringing your talents to Europe. These are the articles that have already been written:

*   [University of Amsterdam (Master's Artificial Intelligence)](https://www.lesswrong.com/posts/Qx3xNCpkmJeCeaCSE/university-of-amsterdam-uva-msc-artificial-intelligence)
*   [University of Edinburgh (Master's Artificial Intelligence)](https://www.lesswrong.com/posts/pRqvHp6NBKwwpvEde/university-of-edinburgh-master-s-artificial-intelligence)
*   [ETH Zürich (ML related M.Sc. programs)](https://www.lesswrong.com/posts/gnsuXqdhZdGbsH2BF/eth-zurich-ml-related-msc-programs)
*   [EPF Lausanne (ML related M.Sc. programs)](https://www.lesswrong.com/posts/s48TWaCc6Z9xktdi9/epf-lausanne-ml-related-msc-programs)
*   [University of Oxford (Master's Statistical Science)](https://www.lesswrong.com/posts/Btecf6Qykpr7igez4/university-of-oxford-master-s-statistical-science)
*   [University of Tübingen (Master's Machine Learning)](https://www.lesswrong.com/posts/9uCr5gKF2fABi3dCj/university-of-tuebingen-msc-machine-learning)
*   [University of Darmstadt (Master's Computer Science with ML focus)](https://www.lesswrong.com/posts/oKB9awts5cjkKruxJ/tu-darmstadt-computer-science-master-s-with-a-focus-on)

This selection of Master’s programs is not an ultimate list of “good” master's programs – we just happened to know the authors of the articles. If you want to add an article about any ML-related program anywhere in the world don’t hesitate to contact us and we will add it to the list. We also don't claim that this is a complete overview of the respective programs and want to emphasize that this does not necessarily reflect the perception of all students within that program. 

Authors: Marius (lead organizer), Leon (lead organizer), Marco, Xander, David, Lauro, Jérémy, Ago, Richard, James, Javier, Charlie,  Magdalena, Kyle.
# Stuart Russell at the SlateStarCodex Online Meetup

Professor Stuart Russell will speak briefly on his book "Human Compatible", and then will take questions. The event begins Dec. 6, 2020 at 20:30 Israel Standard Time, 10:30 Pacific Standard time, 18:30 UTC.

Please [register here](https://forms.gle/G4Kk5wSN4VHperyf7) and we will send you an invitation.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/d17c8182e0213252512e60da0814f08e16c7172dc24e9b6d.jpg)

Stuart Russell is a Professor of Computer Science at the University of California at Berkeley, holder of the Smith-Zadeh Chair in Engineering, and Director of the Center for Human-Compatible AI. His book "Artificial Intelligence: A Modern Approach" (with Peter Norvig) is the standard text in AI, used in 1500 universities in 135 countries. His research covers a wide range of topics in artificial intelligence, with an emphasis on the long-term future of artificial intelligence and its relation to humanity.
# Beware Experiments Without Evaluation

Sometimes, people propose "experiments" with new norms, policies, etc. that don't have any real means of evaluating whether or not the policy actually succeeded or not.

This should be viewed with deep skepticism -- it often seems to me that such an "experiment" isn't really an experiment at all, but rather a means of sneaking a policy in by implying that it will be rolled back if it doesn't work, while also making no real provision for evaluating whether it will be successful or not.

In the worst cases, the process of running the experiment can involve taking measures that prevent the experiment from actually being implemented!

Here are some examples of the sorts of thing I mean:

*   *Management at a company decides that it's going to "experiment with" an open floor plan at a new office. The office layout and space chosen makes it so that even if the open floor plan proves detrimental, it will be very difficult to switch back to a standard office configuration.*
*   *The administration of an online forum decides that it is going to "experiment with" a new set of rules in the hopes of improving the quality of discourse, but doesn't set any clear criteria or timeline for evaluating the "experiment" or what measures might actually indicate "improved discourse quality".*
*   *A small group that gathers for weekly chats decides to "experiment with" adding a few new people to the group, but doesn't have any probationary period, method for evaluating whether someone's a good fit or removing them if they aren't, etc.*

Now, I'm not saying that one should have to register a formal plan for evaluation with timelines, metrics, etc. for any new change being made or program you want to try out -- but you should have at least some idea of what it would look like for the experiment to succeed and what it would look like for it to fail, and for things that are enough of a shakeup more formal or established metrics might well be justified.
# Brigaded Rounds

A group of [round singers](https://constellationpress.com/rounds/) were interested in trying out the [bucket brigade singing program](https://echo.jefftk.com), and I realized that with a small tweak it could support rounds directly. We just needed to write audio both at the place it belonged to, and an appropriate distance into the future. This would let everyone hear everyone else, though the leader still has to wait a bit before they start to hear others.

The system already needs to know BPM (beats per minute) so it can run a metronome, but the support around it also needs to know BPR (beats per repeat). For example, "Row, Row, Row, Your Boat" is a 16 beat round with participants entering every four beats, so you would probably set BPR to be 16. Four (or more) people could sing, with one person leading and each other person joining after the right number of beats as if they were singing together in person. Once everyone has been singing for 16 beats they all hear everyone else, and also themself.

You can get interesting effects by setting the BPR to some other multiple of the part interval, such as 12. This lets you sing along with yourself an offset, so that you appear to be one of the other singing voices.

It's also possible to configure the system with multiple repeats, so you would hear each iteration of your voice, say, three times. This can be fun if you have a small group of people and want to sing many parts, or even if you're playing around by yourself.

One thing this made me realize is that the latency calibration is really very important, and the previous way we were handling it was not sufficient. Originally, it started off trying to get a very accurate estimation of your latency, but if it wasn't doing very well it would slowly lower its standards. After about 30 seconds, it would let in people even with pretty inaccurate estimates. I've updated it now to be very careful, and if it can't estimate your latency to within 2 ms, it lets you know and gives you the choice between trying again, and joining without sending any of your audio to the server.

[![](https://www.jefftk.com/latency-calibration-warning.png)](https://www.jefftk.com/latency-calibration-warning-big.png)

Feel free to play with it: [echo.jefftk.com](https://echo.jefftk.com).

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100193464836752)*