Harley-ml/lesswrong huggingface dataset:
# Betting Thread

At Ought, we’ve started making bets on our continuous predictions. We’ve found it a fun way to hold ourselves accountable and to combat overconfidence.

Here’s a thread for people to share continuous beliefs they’re willing to bet on, and make bets on other people’s beliefs. You can use [Elicit](https://elicit.org/) to automatically generate a fair bet with positive expected value based on each person’s beliefs ([this post](https://www.lesswrong.com/posts/KafZRToDv9a5Ehi9F/notebook-for-generating-forecasting-bets) explains more about how we generate the bet).

Make a bet
----------

**To share a belief you’re willing to bet on:**

1.  Go to [elicit.org](https://elicit.org/), type in your question, and make a distribution of your beliefs
2.  Take a snapshot
3.  Post the snapshot URL and a screenshot image in a comment on this thread

**To bet on someone else’s belief:**

1.  Click on their snapshot URL. Click on the title to get a blank distribution
2.  Make your own distribution and take a snapshot
3.  Import your snapshot into theirs by pasting your snapshot URL into the ‘Show more’ dropdown on their snapshot
4.  Go to the Interpret tab to see the bet

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/animations/0b9bfe5419ba65e38c18934acd73331955e168a574987a79.gif)

Example Bet
-----------

Here is a bet we proposed between 538 and The Economist on [Biden's electoral college votes](https://elicit.ought.org/builder/WqryobaX8): 

**If the result is between 290.5 and 398.1, 538 pays The Economist $35.98. Otherwise, The Economist pays 538 $64.02.**

![](https://lh6.googleusercontent.com/dOptw_Me86b_wWthChoXkUoac0OTYXy7wYtQDH7h3WcOcRwYEzZmq-UheYdx4W12Rkr-2-dcabu7EFxua0k2xpUtoGltypW6vvZhRVoXJxlUYRHN7zt52XzsGG0M1GBmDYL_mo83)

![](https://lh3.googleusercontent.com/RblYAOPUWq8YDeLbcUqlrw-5jLA0qm0lazpfuO0stvBoMp6CYQkBKJfFun7XcFuj-I4ck-jd4N2s4nMl93ubScOMzVDmYAzDfjyQXAfXFcvfBzE052NXfzrQZzP3MagPtfDGa9rn)
# Problems Involving Abstraction?

I'm working on a post of examples for how to formulate problems involving abstraction (using the [abstraction formulation here](https://www.lesswrong.com/s/ehnG4mseKF6xALmQy/p/vDGvHBDuMtcPd8Lks)). This isn't going to solve problems, just show how to set them up mathematically.

To that end, I'd like to hear particular problems people are interested in which intuitively seem to involve abstraction. Examples of the sort of thing I have in mind: 

*   Humans generally seem to care about abstract objects, not individual atoms, so it seems like abstraction should be relevant to [impact measures](https://www.lesswrong.com/tag/impact-measures). How would we formalize that?
*   Humans can figure out what a new word means with ridiculously few examples, suggesting that we already have some "latent space" with a simple representation of the-class-of-things-corresponding-to-the-new-word. That sounds like it has something to do with abstraction. What's going on there?
*   The sort of "maps" we use in the real world (like street maps, for instance) are lossy, abstract representations of the territory (i.e. streets). How can we usefully formulate map-territory correspondence for such abstract maps? Is possible for a system to use its abstract map to [recognize flaws in its own abstract-map-making process](https://www.lesswrong.com/posts/46qnWRSR7L2eyNbMA/the-lens-that-sees-its-flaws)?

There is a high chance that your request (or at least something very similar to it) will be incorporated in the post. So, what examples would people like to see?
# Stupid Questions October 2020

The stupid questions thread was one of the regular threads on LessWrong 1.0. It's a place where no question is to stupid to be asked and anybody who answers is encouraged to be kind.

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
# Reward large contributions?

Sometimes I want people to get $$$ for their contributions.

When someone posts a long, exceptional sequence? I don't get that feeling.

When folks working on the site itself roll out a sweet new feature? I don't get that feeling.

But when [https://www.lesswrong.com/users/lsusr](https://www.lesswrong.com/users/lsusr) creates a fun and relevant competition requiring them to code up quite a bit of framework/bots and coordinate people? ([https://www.lesswrong.com/posts/AHTRyQJtiRin22kth/the-darwin-game-1](https://www.lesswrong.com/posts/AHTRyQJtiRin22kth/the-darwin-game-1)) I wonder if there's a pool to reward over-and-beyond contributors, and if there is, how much I'd be willing to contribute to it.

Does something analogous exist? "Should" it? If so, what should be "eligible"?
# What's the difference between GAI and a government?

I have zero technical AI alignment knowledge. But this question has kept recurring to me for like a year now so I thought I'd ask.

A lot of the arguments for the danger of GAI revolve around the notion that an agent that is smarter than a human is un-boxable, self-creating and self-enhancing, and not necessarily aligned with human interests.

That pattern-matches very well onto "governments," "corporations," and other forms of collective agencies. They have access to collective intelligence far beyond what's accessible to an individual. That intelligence brings them power beyond the ability of even the most clever individual to avoid long-term. Their goals aren't necessarily aligned with human values. They use their intelligence and power to enhance their own intelligence and power. They're not always successful, but they are often able to learn from their mistakes. If one agency destroys itself, another takes its place.

How much bearing does this have on technical AI alignment work? Can AI alignment work translate into solutions for the problems we presently have in aligning these agencies to human values? Do the restraints that have so far prevented governments/corporations from paperclipping the world map onto any proposed strategies for AI alignment?
# The Trouble With Babbles

I haven't yet participated in any of our babble challenges, even though I think they're a good idea. My reason is that they are ambiguous and prone to Goodhart's law. If the challenge is to list 50 ways of getting a golf ball to the moon, no matter how silly, I could do this:

1.  Carry it while jumping to the moon after getting really buff
2.  Carry it  while jumping to the moon with rocket boots
3.  Carry it while jumping to the moon from a high platform
4.  Carry it while jumping to the moon from a high ladder
5.  Carry it while jumping to the moon from the top of a human pyramid

You can see that I'm optimizing here for getting to 50 by proposing minor variations on the same core idea. The intention of babble is to get you out of your mental ruts and past your blockages. It's not clear that this exercise on its own helps you do that very much. Yet if I said to myself "no more 'jumping' entries," I'd be pruning.

Can we solve this problem, while keeping the spirit of the exercise (which itself is a commendable example of babble)?

Let me babble a few ideas:

1.  Every 20 babbles, go back and categorize your ideas. Make the cutoff "At least 50 babbles, and 10 categories."
2.  Try problems in a conjecture + counter-example format. Topics that might work well here are social, philosophical, or aesthetic.
3.  Choose problems that are not technical problems (getting an object to the moon), but personal/social problems ("babble 50 ways to incentivize better teaching").
4.  Allow respondents to choose their own problem.
5.  Babble baseball. One team's "at bat" and they have to babble ideas. The other team's "in the field" and they have to categorize (or sub-categorize) the ideas. Then they have to propose their own ideas within each category. The "at bat" team has a 1-minute head start to list some ideas, which they publish all at once. They then continue publishing new ideas while the "in the field" team goes to work. The trick is that the "at bat" team has to publish ideas *sequentially* \- i.e. "batter 1" has to publish an idea before "batter 2", "batter 2" before "batter 3" and so on. Only when the last "batter" has published an idea does "batter 1" get to go again. By contrast, the "in the field" team gets to work all at once.

Please add your own babbles in the comments!
# What is the current bottleneck on genetic engineering of human embryos for improved IQ

That is, what is the technological constraint that is most limiting our ability to significantly enhance the cognitive performance of people born this year? 

Links to long-form sources that answer this, and related questions, would be appreciated.
# Should we use qualifiers in speech?

Consider

> From what I understand, technological progress happens exponentially.

vs.

> Technological progress happens exponentially.

The difference is the "from what I understand" part. Other examples of such qualifiers include:

*   It seems to me
*   My impression is that
*   I could be wrong, but
*   Perhaps
*   Probably

Using such qualifiers may be annoying. It is probably easier and quicker to just skip them. It also might add "fluff" and distract from the main point.

On the other hand, I worry that omitting them would lead to overconfidence. If you say  "X is true" enough times instead of "It seems to me that X is true; I'm pretty confident but not super confident", it seems likely that you'd develop a nontrivial overconfidence in X.

Of course, the answer to "Should we use qualifiers in speech?" is almost certainly, "It depends". But despite that, it still seems like it'd be pretty useful to figure out what the "default" or "your go-to" should be.

Personally I lean pretty strongly towards using them when doing intellectual things, like here on LessWrong. But in everyday life I lean towards avoiding them, because it goes against norms, is a little awkward, and doesn't have nearly as big a benefit as when you're doing intellectual things.
# What is our true life expectancy?

I [recently learned](https://www.lesswrong.com/posts/3t3CWWTvvtLMLQzZc/stupid-questions-october-2020?commentId=rabfDpHZhddYWhDZt) that when they talk about a life expectancy of eg. 80 years, it's assuming that the future looks similar to the past. But with exponential progress in technology, that doesn't seem like a good assumption. For example, according to [Bostrom's survey](http://www.nickbostrom.com/papers/survey.pdf) (90% likelihood), the median pessimistic year for AGI is 2075.

So then, taking into account technological progress, what do we expect life expectancy to be?
# “Prediction” and “explanation” are not causation

Everyone knows that correlation is not causation. Many people don't know that in scientific jargon, “predict” and “explain” are *also* not causation. They are forms of correlation.

(Technically, “association” might be a better term than “correlation”, which can have a narrower technical meaning in statistics. But since I'm writing this for non-experts, I'm going to use the term “correlation” in the colloquial, wider sense.)

These terms can cause extreme miscommunication:

In lay usage, “X predicts Y” implies that X comes *before* Y. Predictions are about the future. In statistics, there is no time implication at all. It is just a type of correlation. If I said that I could use 2020 data to “predict” things that happened in 2019 (or 1920), most people would laugh at me. But this is a perfectly legitimate usage of statistical “prediction”.

Similarly, the general sense of “explanation” means a conceptual understanding of a phenomenon. In statistics, “explanation” implies no such understanding, only a certain type of correlation.

Because of the time-implication of “prediction” and the conceptual-understanding implication of “explanation”, most people are likely to interpret these as evidence for or even proof of causation. But in many cases, they merely mean correlation.

If you are reading scientific papers, be careful with how you interpret these terms. If you are reading reports about science, know that the reporter might not be clear on this point. If you are reporting science yourself, be responsible with what you write.

(By the way, without [@NeuroStats](http://twitter.com/neurostats) I wouldn't know this stuff either. Any value in this post is thanks to her; errors are mine alone.)
# Why Huntington-Hill?

The US House of Representatives uses the [Huntington-Hill method](https://en.wikipedia.org/wiki/Huntington%E2%80%93Hill_method) to figure out how many representatives each state should have. First you give each state one representative, because even the smallest state is guaranteed one, and then you assign the remaining spots, one at a time, to whichever state has the highest "priority". Priority being ratio of its population to the geometric mean of the number of seats it currently holds and the number it would hold if it received this extra seat:

Ps = state population
Rs = state reps

      Ps
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
sqrt(Rs*(Rs+1))

Where does this come from? I had a shot at deriving it, and it actually makes a lot of sense. First, we restate the problem has one of error minimization. At every stage, we want to assign the next seat wherever it would most minimize representational inaccuracy. Current error is, summed over all states:

Pt = total population
Rt = total (target) reps

| Pt   Ps |
| \-\- \- \-\- | \* Ps
| Rt   Rs |

For each state we might give a seat to, the effect that would have on total error is:

| Pt    Ps  |        | Pt   Ps |
| \-\- \- \-\-\-\- | \* Ps - | -- - -- | * Ps
| Rt   Rs+1 |        | Rt   Rs | 

We would like to identify the state that minimizes this quantity. Since we are adding representatives one by one, `Pt/Rt` will always be greater than `Ps/Rs` \[1\] and we can remove the absolute value and distribute the `Ps`.

PtPs   PsPs   PtPs   PsPs
\-\-\-\- \- \-\-\-\- \- \-\-\-\- \+ ----
 Rt    Rs+1    Rt     Rs

Cancel the `PtPs/Rt` and we have:

PsPs   PsPs
\-\-\-\- \- \-\-\-\-
 Rs    Rs+1

Combine the two fractions and cancel again:

  PsPs
\-\-\-\-\-\-\-\-\-
Rs*(Rs+1)

Since we're trying to identify the state that minimizes the quantity, we can instead identify the state that minimizes its square root:

      Ps
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
sqrt(Rs*(Rs+1))

Which is in the prioritization of Huntington-Hill.

I initially tried to derive this from squared error, which did not work and ended up with an enormous amount of scribbles on paper.

  
\[1\] This is not quite true, as we get to assigning the very last representatives, but I think it still works?

*Comment via: [facebook](https://www.facebook.com/jefftk/posts/10100189924172272)*
# Yet another world spirit sock puppet

*Crossposted from* [*world spirit sock puppet*](https://worldspiritsockpuppet.com/)*.*

I have almost successfully made and made decent [this here](https://worldspiritsockpuppet.com/) my new blog, in spite of little pre-existing familiarity with relevant tools beyond things like persistence in the face of adversity and Googling things. I don’t fully understand how it works, but it is a different and freer non-understanding than with WordPress or Tumblr. This blog is more mine to have mis-built and to go back and fix. It is like not understanding why your cake is still a liquid rather than like not understanding why your printer isn’t recognized by your computer.

My plan is to blog at [worldspiritsockpuppet.com](https://worldspiritsockpuppet.com/) now, and cross-post to my older blogs the subset of posts that fit there.

The main remaining thing is to add comments. If anyone has views about how those should be, er, tweet at me?
# Increasing Isolation Again

The coronavirus is getting much worse in Massachusetts, as it is in many places. Wastewater levels do not look good:

[![](https://www.jefftk.com/boston-wastewater-covid-2020-10-25.png)](https://www.jefftk.com/boston-wastewater-covid-2020-10-25-big.png)

Neither does the percentage of tested individuals who are positive:

[![](https://www.jefftk.com/ma-positive-tests-by-date-2020-10-24.png)](https://www.jefftk.com/ma-positive-tests-by-date-2020-10-24-big.png)

This is p8 of the [daily coronavirus dashboard for today](https://www.mass.gov/doc/covid-19-dashboard-october-24-2020/download), and each day's is available at [mass.gov](https://www.mass.gov/info-details/covid-19-response-reporting#covid-19-daily-dashboard-) when ready. The "percentage of all molecular tests that are positive" number is much less useful: we also don't want to think things are good just because MIT is retesting their students at an incredible rate.

When our house [relaxed our strict isolation](https://www.jefftk.com/p/partially-stepping-down-isolation) in early July things were looking good: rates were down, testing was up. The per-individual positive test percentage was ~1.5-2%:

[![](https://www.jefftk.com/ma-covid-2020-08-19.png)](https://www.jefftk.com/ma-covid-2020-08-19-big.png)

Hospitalizations are starting to trend upwards as well, though are still far below their peak:

[![](https://www.jefftk.com/ma-covid-hospitalizations-2020-10-25.png)](https://www.jefftk.com/ma-covid-hospitalizations-2020-10-25-big.png) *[COVID Tracking Project](https://covidtracking.com/data/state/massachusetts/hospitalization)*

We talked yesterday as a [house](https://www.jefftk.com/p/shared-house-setup), and decided on new house rules:

*   Our default is the same as it has been [since March](https://www.jefftk.com/p/stepping-up-isolation): you can only interact with other people outside the household if you're outside, masked, and at least six feet apart. I recommend buying [snowpants](https://www.jefftk.com/p/snowpants) for comfortable outdoor cold-weather socializing.
    
*   We're still treating a small number of people who have had coronavirus and tested positive for antibodies on a reliable test as fully safe. There is still some chance that they could get it again, with immunity feeding overtime, but we think the additional risk from them is very low.
    
*   [Previously](https://www.jefftk.com/p/partially-stepping-down-isolation) Each of us could pick one person each week to spend time with us unrestricted: no masks, kissing etc ok. We're not doing this anymore.
    
*   Now, each of us can pick one person each week to spend time with inside, as long as we wear a high-quality mask keep faces at least 2ft apart. It's now possible to get KN-95 (Chinese-spec N95) masks at about $2/mask ([ex](https://smile.amazon.com/dp/B0882TV6JM)).
    
*   We're still not going into stores to shop etc.
    

These rules don't bring our risk down to zero, but we think they're a reasonable trade off between danger of getting or spreading the coronavirus and the benefits of spending time with people.

We will of course revisit as things continue to change.
# But what kinds of puppets are we?

_Crossposted from [world spirit sock puppet](worldspiritsockpuppet.com/meteuphoric.html)._

I watched [The Social Dilemma](https://www.thesocialdilemma.com/) last night. I took the problem that it warned of to be the following:

1.  Social media and similar online services make their money by [selling your attention to advertisers](https://worldspiritsockpuppet.com/2020/10/14/the-bads-of-ads.html)
2.  These companies put vast optimization effort into manipulating you, to extract more attention
3.  This means your behavior and attention is probably very shaped by these forces (which you can perhaps confirm by noting your own readiness to scroll through stuff on your phone)

This seems broadly plausible and bad, but I wonder if it isn’t quite that bad.

I heard the film as suggesting that your behavior and thoughts in general are being twisted by these forces. But lets distinguish between a system where huge resources are going into keeping you scrolling say—at which point an advertiser will pay for their shot at persuading you—and a system where those resources are going into manipulating you directly to do the things that the advertiser would like. In the first case, maybe you look at your phone too much, but there isn’t a clear pressure on your opinions or behavior besides pro phone. In the second case, maybe you end up with whatever opinions and actions someone paid the most for (this all supposing the system works). Let’s call these distorted-looking and distorted-acting.

While watching I interpreted the film suggesting the sort of broad manipulation that would come with distorted-acting, but thinking about it afterwards, isn’t the kind of optimization going on with social media actually distorted-looking? (Followed by whatever optimization the advertisers do to get you to do what they want, which I guess is of a kind with what they have always done, so at least not a new experimental horror.) I actually don’t really know. And maybe it isn’t a bright distinction.

Maybe optimization for you _clicking_ on ads should be a different category (i.e. ‘distorted-clicking’). This seems close to distorted-looking, in that it isn’t directly seeking to manipulate your behavior outside of your phone session, but a big step closer to distorted-acting, since you have been set off toward whatever you have ultimately been targeted to buy.

I was at first thinking that distorted-looking was safer than distorted-acting. But distorted-looking forces probably do also distort your opinions and actions. For instance, as the film suggested, you are likely to look more if you get interested in something that there is a lot of content on, or something that upsets you and traps your attention.

I could imagine distorted-looking actually being worse than distorted-acting: when your opinion can be bought, the change in it is presumably what someone would want. Whereas when your opinion is manipulated as a weird side effect of someone trying to get you to look more, then it could be any random thing, which might be terrible.(Or would there be such weird side effects in both cases anyway?)
# Sam Altman at SlateStarCodex Online Meetup

Our online meetup on  November 8, at  10:30 AM Pacific Standard Time, 18:30 UTC, 20:30 Israel Standard Time, will feature Sam Altman, CEO of OpenAI, who will be answering questions. We will then have open video-chat rooms as usual.  
  
We hope to focus the Q&A on AI, AGI, GPT-N and OpenAI, but feel free to ask questions on other topics if you like.  
  
Please [register here](https://forms.gle/i7tBJYEvuuHjQLq68) , and we'll send you an invitation.
# Top Time Travel Interventions?

I encourage people to treat this as an exercise and avoid looking at the other answers before posting your own. (Maybe use spoiler blocks?) This is more for fun than for serious. However, it may potentially serve as [an analogy](https://www.lesswrong.com/posts/7khK4DShZBR8gfyHv/chronophone-motivations) to help think about what the top modern-day interventions might be.

The challenge: you have one opportunity to go back in time and change something, for the good of all humankind. ***Use your upvotes wisely; it's how we'll judge success. My suggestion: upvote based on what you think would actually do good; strong upvote what you think works best. If you want to give kudos for other reasons, such as insightfulness, I suggest using a comment instead.***

You don't have magical powers to change anything you want (so for example, you can't just "prevent the agricultural revolution"; you have to have a plan to do that). However, you do have the ability to take modern technology and resources back with you. For simplicity, though, let's keep it to one time-travel event -- your machine is single-use-only and you can't afford to build two. (You could bring the science of time travel back with you, but I don't know if that's a good idea...)

For the sake of the thought experiment, pretend that you can actually change things via time travel, giving rise to a new timeline (rather than there only being one consistent timeline, like we'd expect of real time travel).

What do you do, to do the most good?

Stretch goals: solve the problem under harsher constraints, such as time travel machines with restricted ranges. Block key things most important to your strategies so far, and see what you can still do.

I think this problem is challenging for some specific reasons, but I'll put them behind a spoiler block for those who want to just dive into it.  
  
ETA: I also [posted a similar challenge on the EA forums](https://forum.effectivealtruism.org/posts/Et6pTXpJvu5eSLkMo/best-time-travel-intervention).

I'm specifically interested in this challenge because of X-risk. I used to think that, in order to the most good, you basically take as much scientific knowledge as you can, as far back as you can, and try to teach it to people or something. But it's quite plausible that that just brings X-risk nearer, for a wide variety of sciences and technologies one might try to smuggle back.  
  
Similarly, you can just try to do basic good for people -- averting large historical disasters, etc. But again, this might just accelerate progress, giving humans the power to create global catastrophic risks sooner, and thus giving humanity less time overall.  
  
This makes things feel a little hopeless -- is it possible to do very much good at all?
# On the Dangers of Time Travel

It has been [obvious for decades](https://www.lesswrong.com/s/qMtriMPLdriNkAfSJ/p/rPhiu4RhMEgrp3CMH) to anyone who understands quantum mechanics that that the scientific consensus on theoretical physics [is wrong](https://www.lesswrong.com/s/Kqs6GR7F5xziuSyGZ). General relativity is background independent. Quantum field theory is background dependent. They cannot both be correct.

According to the Copenhagen interpretation of thermodynamics, entropy is emergent from time. Anyone with with half a Noble Prize in Physics can tell that this is backwards. Time is emergent from entropy.

Everything that can be said to physically exist, from an unobserverable wave function to living cat, is a macrostate. A random walk along microstates evolves in the direction of higher entropy macrostates even in the absence of time. In this way, entropy is more fundamental than time.

Putting entropy before time explains why all quantum fields [locally maximize proper time](https://www.lsusr.com/blog/intuitive-noether-current.html). It solves the problem of Baryon asymmetry and dissolves the [semantic stopsign](https://www.lesswrong.com/posts/FWMfQKG3RpZx6irjm/semantic-stopsigns) at the Big Bang.

If you have worked out the details of quantum gravity for yourself then you have seen in your equations exactly how to build a time machine.

The best theoretical physicists have managed to suppress this technology. In a historically unprecedented conspiracy, they [have not published anything useful](https://en.wikipedia.org/wiki/String_theory) for decades.

While the physicists buy time, the rest of us must invent systems to [maximize the benefits](https://www.lesswrong.com/posts/mASRjBQ5T95KX8vyn/top-time-travel-interventions)   of time travel while minimizing risk. As a consequence of this progress, there is a nonzero Bayesian probability we could turn on a time machine without instantly destroying the multiverse. But no weight of electronic paper discussing time travel safety can prevent a malevolent actor uninterested in safety.

As a last line of defense, the cosmic speed limit is below 88 everywhere in the solar system, it is illegal to buy plutonium in a drugstore and phone booths have been quietly disappeared. But we cannot expect such flimsy mechanisms to hold forever as technology advances.

[We must build a temporal displacer first, before the enemy does](https://youtu.be/dQw4w9WgXcQ).
# Announcing Data Secrets Lox: An Alternative to Slate Star Codex Open Threads

In light of the  [shutdown](https://slatestarcodex.com/2020/06/22/nyt-is-threatening-my-safety-by-revealing-my-real-name-so-i-am-deleting-the-blog/) of Slate Star Codex, and in particular, the lack of new SSC Open Threads, much of the SSC community has moved to a new forum: [Data Secrets Lox](https://www.datasecretslox.com/index.php). Data Secrets Lox is a sort of "permanent open thread" for the SSC readership, with many of the same commenters and cultural norms as the Slate Star Codex comments. 

If you enjoyed the Slate Star Codex commentariat, please join us at Data Secrets Lox!
# "Cartesian Frames" Talk #2 this Sunday at 2pm (PT)

Scott Garrabrant of MIRI is giving a second talk on Cartesian Frames this Sunday, Nov. 1 at 2pm (PT), covering the contents of his [Cartesian Frames posts](https://www.lesswrong.com/s/2A7rrZ4ySx6R8mfoT) this week. Afterwards, there will be a Q&A and freeform discussion from 3pm to 4pm.

This talk will assume that you've either:

1.  watched the first talk ([https://www.youtube.com/watch?v=H1tJdaCvcck](https://www.youtube.com/watch?v=H1tJdaCvcck.));
2.  or read the first two posts ([Introduction to Cartesian Frames](https://www.lesswrong.com/posts/BSpdshJWGAW6TuNzZ/introduction-to-cartesian-frames) and [Additive Operations on Cartesian Frames](https://www.lesswrong.com/posts/ewkYgtZapQRtDPT2F/additive-operations-on-cartesian-frames)).

The talk and discussion will be on Gather.Town (i.e., not on Zoom), and take place after the [Rationalist Town Hall: Pandemic Edition](https://www.lesswrong.com/posts/BfzPGuT6RxywkakA7/rationalist-town-hall-pandemic-edition) event.

**Directions**

To attend Scott's talk or the discussion:

1.  Go to [https://gather.town/app/aPVfK3G76UukgiHx/lesswrong-campus](https://gather.town/app/aPVfK3G76UukgiHx/lesswrong-campus), the LW Walled Garden.
2.  Head north and move your avatar onto the TARDIS / phone booth.
3.  Then move your avatar onto the words "Scott G's Office" and you'll be teleported to his office.
4.  For the talk, leave Scott's office, go up, and enter the room on the right.

The path to the TARDIS looks like this:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/5e5cc9cc5e926eaa3f72b6837e7e63e214a95ea0526f9cd1.png)

The path from Scott's office to the lecture hall looks like this:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/948d2e9494dd6fa494933c9c53370fd02185713dba0529b0.png)

**Other events: Nov. 8 and Nov. 15**

Scott is likely to host another talk or two on Cartesian Frames the following two Sundays at 2pm, also in Gather.Town.

Regardless, Scott will be hosting Cartesian Frames discussions on Gather.Town those Sundays (Nov. 8 and Nov. 15), 2-4pm.
# Draft papers for REALab and Decoupled Approval on tampering

Hi everyone, we (Ramana Kumar, Jonathan Uesato, Victoria Krakovna, Tom Everitt, and Richard Ngo) have been working on a strand of work researching tampering problems, and we've written up our progress in two papers. We're sharing drafts in advance here because we'd like to get feedback from everyone here.

The [first paper](https://drive.google.com/file/d/1z-uUa2OH0W3ldp98afZaODVyPwEP-qV7/view?usp=sharing) covers:

*   How and when tampering problems might arise in the real world
*   Key assumptions in standard RL frameworks we relax to allow modeling tampering
*   How we model and measure tampering empirically, through our internal platform REALab, and
*   How we formalize tampering problems, through our Corrupt Feedback MDP formalism

We particularly hope it clears up the concept of tampering (and why "but the agent maximized its given reward function" typically assumes the wrong framing), and internally, we've found REALab to be a useful mental model.

The [second paper](https://drive.google.com/file/d/1KPDob0fdMWHFRY_VkGFLUxojvWwa7c6c/view?usp=sharing) describes:

*   Decoupled approval, an algorithm closely related to approval direction and Counterfactual Oracles, and designed to be straightforwardly compatible with standard deep RL
*   An analysis of this algorithm (within the CFMDP formalism), and
*   Empirical validation (in REALab)

We'd love to get feedback on these; the current drafts are viewable in this Google Drive [folder](https://drive.google.com/drive/folders/1_J4ncrhjWvmhdMKXtsCAN18asUhUy0FB?usp=sharing). We're happy to discuss these on whichever of LessWrong/Alignment Forum/Google Drive comments, and would prefer to keep discussion on these forums for now, as we'll share the papers more widely after they're posted on arXiv in a few weeks. Looking forward to hearing your thoughts!
# What risks concern you which don't seem to have been seriously considered by the community?

There are a few things I'm worried about which I have not seen discussed much, and it makes me wonder what we're collectively missing.

This seems like a question which has likely been asked before, but my Google-fu did not find it.

You don't need to make a watertight case for something being important in order to share your concern, brief explanations are fine if the high bar of writing something detailed would put you off posting.
# What's holding back outsourcing to cloud labs?

I would have expected Emerald Cloud Lab or similar competitors to go a lot and be successful over the last five years. As far as I know, like Emerald Cloud Lab only had modest growth and there aren't competitors who grew strongly. Outsourcing to cloud labs seems like it allows the laberatory to have benefits of scale and virtualization that drives down costs and is easier to use then working in a wet lab. Is there something holding back this trend that I'm not seeing? Alternatively, what's going on?
# Secular Solstice Online (Americas)

Back in those days, there was darkness, and so we created light.   
There was hunger, and so we invented agriculture.   
There was sickness, and so we created medicine.  
  
Back in \*these\* days, well, there was still sickness sometimes, and many of us continued to create medicine... and some of us created brand new tools for online rituals that anyone could join the world-over.  
  
This December 19th, we'll be hosting Secular Solstice Online, for the American continental timezones. (Other timezones also welcome if you feel like staying up late/early). We'll be singing together using custom software that we built to enable Solstice, telling stories, and joining together for an online festival and afterparty.

...

[https://secularsolstice2020.org/](https://secularsolstice2020.org/)

  
Note that while the site is up, we may be doing random testing and bug fixes over the next 24 hours, so expect the site to behave weirdly.

Reminder:  
– Requires Chrome  
– Works much better with a WIRED headset (not bluetooth)  
– Works much better with WIRED internet (not wifi)  
– You may need to close other tabs or applications that are takng up lots of bandwidth.  
– Having candles is nice

The "virtual doors" will open at 4:30pm PT, 7:30 pm ET. There will be some initial period as we wait for people to file in. We will aim to start the ceremony at 4:45pm, but it might be a bit delayed depending on whether there are unexpected technical hiccups for a lot of people.
# The Darwin Game - Round 1

# Technical Difficulties

I messed up the game engine. I gave bots their own move in the `previous` parameter instead of their opponent's move.

This is the primary factor behind the CloneBots [diverging](https://www.lesswrong.com/posts/Tq8qeRxQ4pB3b5RKg/the-darwin-game-rounds-0-to-10?commentId=dRRtWxdwdr6N2XSnt) from each other. The bug caused CloneBot pairings to score 200-300 instead of 250-250.

I'm going to restart the tournament.

Sorry.

The original tournament, created by a [blind idiot](https://www.lesswrong.com/posts/pLRogvJLPPg6Mrvg4/an-alien-god) me, was exciting. It resembled real world population dynamics. I wrote an 8-part series with 11,277 words and 24 graphs.

Alas, the actual contest is between intelligent agents.

This will be fast.

# Edit: Everything below this line is in error too. Click [here](https://www.lesswrong.com/posts/HE7FZxmLA3QDWdvgq/the-darwin-game-conclusion) for the real, real game.

# Round 1

![](https://s3-us-west-2.amazonaws.com/www.lsusr.com/lesswrong/darwin-game/write-up-2-1/0-1.png)

![](https://s3-us-west-2.amazonaws.com/www.lsusr.com/lesswrong/darwin-game/write-up-2-1/bar-pie-0-1.png)

# Today's Obituary

| Bot         | Team | Summary            | Round |
| ---         | ---  | ---                | ---   |
| Silly 0 Bot | NPCs | Always returns `0` | 1     |

---

The next installment of this series will be posted on October 30, 2020 at 5 pm Pacific Time.

Edit 2020-10-29: I'm sick. Next installment may be delayed.

Edit 2020-11-05: Since this is taking so long I feel I owe you guys an explanation. I have not forgotten about the tournament. Over the last couple of weeks, several correlated problems in meatspace triggered a cascading failure in my real life affairs. All the problems are temporary but I am temporarily unable to write bug-free software. (I actually did write a post for October 30 but it had multiple errors in it.) I would be surprised if it took more than a month to resolve everything but can make no guarantees. I plan to continue the tournament as soon as I can write bug-free software. I apologize for the delays.

Edit: 2020-11-10: I found some more bugs in the code and will have to restart things again.
# Responses to Christiano on takeoff speeds?

It's been over two and a half years since Paul put [this blog post on takeoff speeds](https://sideways-view.com/2018/02/24/takeoff-speeds/) online. In particular, it argues that the "fast takeoff" undergone by humans is not very strong evidence that AIs will also undergo a fast takeoff, because evolution wasn't "optimising for" humans taking over the world.

I think this argument has been fairly influential - possibly disproportionately influential, given its brevity. I find it moderately persuasive, but not entirely so, and I'm currently working on a post explaining why. What I'm wondering is: have there been other critiques or responses to this argument? Because it currently seems to me like there's been very little public engagement with it.
# Trick-or-treating in Covid Times

Somerville has asked its residents not to trick-or-treat:

> Mayor Joseph A. Curtatone and the Somerville Board of Health announced Halloween guidance and related updates today. They strongly urge all community members to forgo trick-or-treating in favor of lower-risk activities as defined by the Centers for Disease Control such as at-home activities and holiday crafts.  
>    —[somervillema.gov](https://www.somervillema.gov/news/no-city-sponsored-or-permitted-halloween-events-allowed-somerville-due-pandemic-residents-urged)

They link to [MA](https://www.mass.gov/news/halloween-during-covid-19) and [CDC](https://www.cdc.gov/coronavirus/2019-ncov/daily-life-coping/holidays.html) guidelines.

I have generally supported Somerville's cautious approach, closing more quickly and opening more slowly than surrounding towns, keeping schools remote until we can make sure we have sufficient ventilation, organizing outdoor enrichment activities for students, etc. But I don't understand how any town that [allows indoor dining](https://www.somervillema.gov/departments/ospcd-economic-development/reopening-information-restaurants) can categorize trick-or-treating as impermissibly high risk?

At this stage in the pandemic, we know that transmission primarily happens through shared air, and outdoor activities are far safer than indoor ones. Especially indoor ones where people are, as in restaurants, generally unmasked and talking. Traditional trick-or-treating, where everyone rings the same doorbells, frequents centralized trick-or-treating corridors, gets quite close to each other, and reaches into the same bowls of candy, I agree is risky. The core experience of trick-or-treating, however, is walking around outside in costumes, going to neighbors' houses, and collecting candy. This can be done with very low risk:

*   Houses that want to participate should plan socially distanced distribution. Participation is even more optional than usual.
*   Trick-or-treat within walking distance of your house.
*   Don't go anywhere where you won't be able to social distance.
*   Set expectations with your kids about this being more of a search than usual, and clarify there will be less candy.
*   Only acquire candy if you're able to do it while keeping distance.
*   Wear a face mask, not just a Halloween mask.
*   Wait to eat until you're home.

Our fall and winter holidays and traditions are full of gathering indoors with food and song. Trick-or-treating is a rare exception, and we should make the most of it.

I'm planning to take the kids out tomorrow night, and I think if we're thoughtful about what we need to do differently it won't be any more risky than our usual evening walks around the neighborhood.
# Taking Ideas Seriously is Hard

Most people don’t practice taking ideas seriously. I think it’s because most people don’t know how to. I didn’t either, until I stumbled upon an implication.

For example, what would it mean to take compounding seriously?

Ugh. I can feel your aversion. You’ve already heard so much about compounding, how it works, how it’s the eight wonder of the world, etc. etc.

But, familiarity is not the same as taking it seriously.

Say you start with $100, and every year, make 10% more. This compounds, since the extra money is a function of how much you already have. The more you have, the more you get. It’s a positive loop that keeps on increasing.

That’s the familiar interpretation. The earlier you start, the more money you’ll make.

To take this interpretation seriously would mean investing your earnings for a similar return. [Reality has a surprising amount of detail](http://johnsalvatier.org/blog/2017/reality-has-a-surprising-amount-of-detail), and sometimes assumptions break. You don’t make 10% - which means you need to balance your investments somehow. That makes things complicated. However, this complication is not related to compounding.

Taking compounding seriously means taking it a step further. Your net worth is just one implication. What else compounds?

Your life experiences and knowledge. What would it mean to leverage compounding here?

If you’re taking compounding seriously, you’d learn the skills with the greatest return first. That means learning the broadest applicable skills you’d apply throughout your life first. That means learning how to think well - before learning the new fancy tech you want to learn.

Of course, sometimes you need a medium to learn the skill better. That makes sense: learn to think well via this new tech you wanted to learn. [Purposes are fragile](https://www.lesswrong.com/s/3HyeNiEpvbQQaqeoH) though, and it’s easy to get lost in the tool, instead of the overall goal.

![](https://neilkakkar.com/assets/images/divider.jpg)

What makes this example so good is that you’re probably very familiar with compounding. What else are you familiar with, but haven’t realised you’re not taking seriously?

A good way to practice this is to ask this question: What are the implications of (idea) in (field of interest)?

For example, asking myself What are the implications of compounding for my self-directed knowledge base led to the above insight of learning broader skills first.
# Aim to explain, not persuade

I love Less Wrong's Frontpage comment guidelines. I especially love the first guideline "Aim to explain, not persuade". The guideline averts arguments, improves my persuasiveness and enhances my intellectual triage.

# Intellectual Triage

It is easy to tell if a field is garbage when you are an expert in the field. But it is a waste of time to become an expert in a garbage field. You should figure out if a field is garbage before you decide whether to become an expert in it.

> Till one knows better, it's hard to distinguish something that's hard to understand because the writer was unclear in his own mind from something like a mathematical proof that's hard to understand because the ideas it represents are hard to understand.
>
> ―[*How to Do Philosophy*](http://paulgraham.com/philosophy.html) by Paul Graham

It is often impossible for a non-expert to determine whether an expert is telling the truth. But it is easy for a non-expert to determine whether an expert is attempting to tell a coherent truth. Coherent truth is specific, falsifiable, unambiguous and precise.

The opposite of coherence is bullshit. Bullshit is vague, unfalsifiable, ambiguous and [generic](https://srconstantin.wordpress.com/2016/10/20/ra/). You can cut through bullshit by communicating via [positive statements](https://www.lsusr.com/blog/speak-in-the-positive.html). Don't waste your time cutting holes in an argument that was never intended to hold water. Instead, ask "How do you know?" and "What do you mean by $x$?". If you cannot quickly climb down to [self-evident reality](https://www.lesswrong.com/posts/pFvZXFWbtvKvGiACJ/how-specificity-works#Ground_Your_Terms) then your interlocutor is not worth your attention.

In other words, "aim to explain, not persuade".

Truth and falsehood are parity inversions of the same meme. If someone convinces you of a falsehood then you can repair the damage with a checksum and a single bit flip. Bullshit is worse than falsehoods because if you eat bullshit then it will gunk up your epistemic failsafes.
# Generalize Kelly to Account for # Iterations?

On my post [weird things about money](https://www.lesswrong.com/posts/H2mPGJYWjrLM7rYXW/weird-things-about-money), Bunthut [made the following comment](https://www.lesswrong.com/posts/H2mPGJYWjrLM7rYXW/weird-things-about-money?commentId=gf2iQTZkW3Sja5P2t):

> I think the interesting question is what to do when you expect many more, but only finitely many rounds. It seems like Kelly should somehow gradually transition, until it recommends normal utility maximization in the case of only a single round happening ever. Log utility doesn't do this. I'm not sure I have anything that does though, so maybe it's unfair to ask it from you, but still it seems like a core part of the idea, that the Kelly strategy comes from the *compounding*, is lost. 

Kelly betting seems somehow *less justified* when we're not doing it a bunch of times. If I were making bets left and right, I would feel more inclined to use Kelly; I could visualize the growth-maximizing behavior, and know that if I trusted my own probability assessments, I'd see that growth curve with high probability.

Several prediction markets have recently offered [a bet](https://www.predictit.org/) at around 62¢ which [superforecasters assign](https://goodjudgment.io/superforecasts/#1338) around 85% probability. This resulted in a rare temptation for me to Kelly bet. Calculating the Kelly formula, I found that I was supposed to put 60% of my bankroll on this.

Now, 60% of my savings seems like a lot. But am I really more risk-averse than Kelly? It struck me that if I were to do this sort of thing all the time, I would feel more strongly justified using Kelly.

Bunthut is suggesting a generalization of Kelly which accounts for the number of times we expect to iterate investment. In Bunthut's suggestion, we would get *less* risk-averse as number of iterations dropped, approaching expectation maximization. This would reflect the idea that the Kelly criterion arises *because of* long-term performance over many iterations, and normal expectation maximization is the right thing to do in single-shot scenarios.

But I sort of suspect this "origin of Kelly" story is wrong. So I'm also interested in number-iteration formulas which reach different conclusions.

The obvious route is to modulate Kelly by the probability that the result will be close to the median case. With arbitrarily many iterations, we are virtually certain that the fraction of bets which pay out approaches their probabilities of paying out, which is the classic argument in favor of Kelly. But with less iterations, we are less sure. So, how might one use that to modulate betting behavior?

I suggest explicitly stepping outside of an expected-utility framework here. The classic justification for Kelly is very divorced from expected utility, so I doubt you're going to find a really appealing generalization via an expected-utility route.