trentmkelly/LessWrong-43k huggingface dataset:
Scott Alexander 2021 Predictions: Market Prices
by SimonM — 2021-04-27T14:03:10.995Z

Scott Alexander has posted some predictions for 2021. Taking Zvi's approach from here and rather than making my own adjustments, making my best estimate of what various prediction (and financial) markets are saying the odds are.

If anyone has seen a market for any of the questions listed here which I haven't found one, let me know and I'll add it.

EDIT: Final results here

US/WORLD

1. Biden approval rating (as per 538) is greater than 50%: 80%

Metaculus gives this 61%. (Much lower than both Zvi and Scott)

2. Court packing is clearly going to happen (new justices don’t have to be appointed by end of year): 5%

PredictIt gives a lower bound on this at 5%.
Metaculus gives a 27% chance to court packing by 2030. Assuming the chances of this happening is mostly weighted sooner rather than later (4/3/2/1 over Biden's term + 30% '24 - '30). would give this a 4% chance in this year

3. Yang is New York mayor: 80%

Metaculus is at 60%
Smarkets is at 73%
Betfair is at 77%
PredictIt is at 69%

4. Newsom recalled as CA governor: 5%

BetOnline.ag has this at ~7%

5. At least $250 million in damage from BLM protests this year: 30%
6. Significant capital gains tax hike (above 30% for highest bracket): 20%
7. Trump is allowed back on Twitter: 20%

Not aware of any markets

8. Tokyo Olympics happen on schedule: 70%

Metaculus has this at 80%
FTX has this at 75%
Smarkets has this at 75%

Note that Metaculus is talking about an Olympics at any point in 2021, whereas the others are about the Olympics being on schedule.

9. Major flare-up (significantly worse than anything in past 5 years) in Russia/Ukraine war: 20%

Metaculus has a related question at 32%. If we assume that question is about reaching a level approximately the same as was achieved in the past, this is presumably putting the chances of something worse at ~16%

10. Major flare-up (significantly worse than anything in past 10 years) in Israel/Palestine conflict: 5%
11. Major flare-up (significantly worse than anything in past 50 years) in China/Taiwan conflict: 5%

Not aware of any markets (specific to 2021).

12. Netanyahu is still Israeli PM: 40%

PredictIt has this at 22%

13. Prospera has at least 1000 residents: 30%

Not aware of one. I think Metaculus are making one though

14. GME >$100 (Currently $170): 50%

Possibly the most concrete one. The options market is giving this 60% chance right now. (Actually, it's giving that until 21-Jan-22, so the chances ...
Modern Monetary Theory for Dummies
by theflowerpot — 2021-04-27T13:11:53.108Z

Modern Monetary Theory, called MMT, is a modern theory of how fiat money works and some implications of this. I don't know the intricate details of this theory but there are some broad conclusions you can draw about someone who talks positively about it.

I'll try to very briefly summarize the understanding I have got from how MMT people view the world. Fiat money is different from real-world commodities in a significant way because it is possible to print any amount of it. The American government can never default on their debts if their debts are in the form of dollars because the US government is able to print literally any amount of dollars. A significant and fundamental reason why fiat money is then valuable (even if the supply could become infinite) is that you need it to pay taxes in the US. The government does not accept tax payments by ANY other means. This means that if you are a US citizen when the tax day comes you will be forced to "buy" dollars from other people in order to avoid going to jail, this creates a demand for the dollar irrespective of the supply. Moreover, the tax amount is also decided by the government, which means that they can create whatever level of demand they want or the dollar by US taxpayers.

So how does fiat work? Firstly, every single dollar in the world has been printed by the American government (FED) and the act of printing and the act of government spending are exactly the same. The only way to spend is to print and the only way to print is to spend. However, since all spending prints money, the government tries to also destroy money. It does this via taxation, you can look at it like every dollar taxed is burned. Generally, however, the government spends more than it taxes, which is called running a deficit, which means that they print more money than they destroy. This creates something called inflation.

You can look at inflation as a tax on liquidity which is just a fancy word for cash. If the government prints the same amount of money that is in circulation at the time then everyone's dollars effectively lose half of their value. So imagine that you had a hundred dollars when the government printed this money you would effectively have lost 50 dollars.

This might sound a bit crazy, if they have total control over the supply and over the demand and they can make the demand sky-high while keeping the supply sky-high don...
Can you improve IQ by practicing IQ tests?
by ForensicOceanography — 2021-04-27T11:28:56.688Z

As an European, I did never have any IQ test, nor I know anybody who (to my knowledge) was ever administered an IQ test. I looked at some fac-simile IQ tests on the internet, expecially Raven's matrices.

When I began to read online blogs from the United States, I started to see references to the concept of IQ. I am very confused by the fact that the IQ score seems to be treated as a stable, intrinsic charachteristic of an individual (like the height or the visual acuity).

When you costantly practice some task, you usually become better at that task. I imagine that there exists a finite number of ideas required to solve Raven matrices: even when someone invents new Raven matrices for making new IQ tests, he will do so by remixing the ideas used for previous Raven matrices, because -as Cardano said- "there is practically no new idea which one may bring forward".

The IQ score is the result of an exam, much like school grades. But it is generally understood that school grades are influenced by how much effort you put in the preparation for the exam, by how much your family cares for your grades, and so on. I expect school grades to be fairly correlated to income, or to other mesures of "success".

In a hypothetical society in which all children had to learn chess, and being bad at chess was regarded as a shame, I guess that the ELO chess ratings of 17 year olds would be highly correlated with later achievements. Are IQ tests the only exception to the rule that your grade in an exam is influenced by how much you prepare for that exam? Is there a sense in which IQ is a more "intrinsic" quantity than, for example, the AP exam score, or the ELO chess rating?
Agents Over Cartesian World Models
by Mark Xu — 2021-04-27T02:06:57.386Z

Thanks to Adam Shimi, Alex Turner, Noa Nabeshima, Neel Nanda, Sydney Von Arx, Jack Ryan, and Sidney Hough for helpful discussion and comments.

Abstract

We analyze agents by supposing a Cartesian boundary between agent and environment. We extend partially-observable Markov decision processes (POMDPs) into Cartesian world models (CWMs) to describe how these agents might reason. Given a CWM, we distinguish between consequential components, which depend on the consequences of the agent's action, and structural components, which depend on the agent's structure. We describe agents that reason consequentially, structurally, and conditionally, comparing safety properties between them. We conclude by presenting several problems with our framework.

Introduction

Suppose a Cartesian boundary between agent and environment:[1]

There are four types: actions, observations, environmental states, and internal states. Actions and observations go from agent to environment and vice-versa. Environmental states are on the environment side, and internal states are on the agent side. Let A,O,E,I refer to actions, observations, environmental states, and internal states.

We describe how the agent interfaces with the environment with four maps: observe, orient, decide, and execute.[2]

observe:E→ΔO describes how the agent observes the environment, e.g., if the agent sees with a video camera, observe describes what the video camera would see given various environmental states. If the agent can see the entire environment, the image of observe is distinct point distributions. In contrast, humans can see the same observation for different environmental states.

orient:O×I→ΔI describes how the agent interprets the observation, e.g., the agent's internal state might be memories of high-level concepts derived from raw data. If there is no historical dependence, orient depends only on the observation. In contrast, humans map multiple observations onto the same internal state.

decide:I→ΔA describes how the agent acts in a given state, e.g., the agent might maximize a utility function over a world model. In simple devices like thermostats, decide maps each internal state to one of a small number of actions. In contrast, humans have larger action sets.

execute:E×A→ΔE describes how actions affect the environment, e.g., code that turns button presses into game actions. If the agent has absolute contro

...
What topics are on Dath Ilan's civics exam?
by jimrandomh — 2021-04-27T00:59:04.749Z

Dath Ilan is a parallel Earth on which human civilization has its act together, in ways that actual-Earth does not. Like actual-Earth, citizens of Dath Ilan sometimes take standardized tests, both to figure out what sort of jobs they'd be suited for, to make sure that its educational institutions are functioning, and to give people guidance about what they might want to study. Unlike Earth's, Dath Ilan's tests have had a lot of thought put into the choice of topics: rather a lot more economics, rather a lot less trigonometry and literature. Topics are selected based on cost/benefit; something that takes a long time to learn would need to be a lot more useful, or have major positive externalities to more people knowing it.

I want to create a test, that will tell people what topics they ought to learn, and enable people to make their knowledgeability legible.

What topics belong on it?
Scott Alexander 2021 Predictions: Buy/Sell/Hold
by Zvi — 2021-04-27T00:40:01.309Z

Scott Alexander is out with his (late) 2021 predictions. You know what that means. It’s time to find things to disagree with!

Scott has the tough job here. He’s putting out a hundred plus predictions with probabilities attached. All I’m doing is saying where I definitely disagree with him.

Epistemic Status: Writing this quickly and off the cuff seems more appropriate and fair. I’m going to explain my reasoning here while also not trying to do a bunch of research on these questions. In general, if something seems reasonable and say or imply that I’m holding, that’s not a strong ‘this is also my probability strongly held’ answer, it’s in the ballpark but likely weakly held.

US/WORLD

1. Biden approval rating (as per 538) is greater than 50%: 80%

Biden’s approval rating is clearly steady. There’s always some honeymoon effect, but it would take a surprising event to send it that far down. 80% seems like it’s in the ballpark. Hold.

2. Court packing is clearly going to happen (new justices don’t have to be appointed by end of year): 5%

Indeed do many things come to pass, and ‘clearly going to happen’ isn’t a clear definition. If this is ‘legislation expanding the size of the court has passed’ then this seems high to me because not only does it seem unlikely Biden gets 50 votes on this, it seems unlikely he’d get them this quickly with so much else on the agenda, but also they’re talking about it, Biden’s already gone gangbusters on giant bills and 5% isn’t that high. So I can’t disagree strongly. Hold.

3. Yang is New York mayor: 80%

Yang is only at 69% on PredictIt, although PredictIt tends to be too low on favorites in this range (although not enough to justify trading on its own). He’s ahead, but he’s prone to rather silly things and there’s plenty of time to mess up, so I think I’m with PredictIt on this and I’ll stick with 70%. Sell.

4. Newsom recalled as CA governor: 5%

Depending on what counts as ‘recalled’ this is either at least 10%, or it’s damn near 0%. I don’t see how you get 5%. Once you get an election going, anything can happen. Weird one, I’d need more research.

5. At least $250 million in damage from BLM protests this year: 30%

With the verdict in, I don’t see what causes this kind of damage in the next 7 months. That doesn’t mean it can’t happen, but $250 million is a lot. I’m selling this down at least to 20%. Sell.

6. Significant capital gains tax ...
Announcing the Alignment Research Center
by paulfchristiano — 2021-04-26T23:30:02.685Z

(Cross-post from ai-alignment.com)

I’m now working full-time on the Alignment Research Center (ARC), a new non-profit focused on intent alignment research.

I left OpenAI at the end of January and I’ve spent the last few months planning, doing some theoretical research, doing some logistical set-up, and taking time off.

For now it’s just me, focusing on theoretical research. I’m currently feeling pretty optimistic about this work: I think there’s a good chance that it will yield big alignment improvements within the next few years, and a good chance that those improvements will be integrated into practice at leading ML labs.

My current goal is to build a small team working productively on theory. I’m not yet sure how we’ll approach hiring, but if you’re potentially interested in joining you can fill out this tiny form to get notified when we’re ready.

Over the medium term (and maybe starting quite soon) I also expect to implement and study techniques that emerge from theoretical work, to help ML labs adopt alignment techniques, and to work on alignment forecasting and strategy.
[Linkpost] Treacherous turns in the wild
by Mark Xu — 2021-04-26T22:51:44.362Z
Link: http://lukemuehlhauser.com/treacherous-turns-in-the-wild/

Mesa-optimizers potentially escape behavioral adversarial training by realizing they're being tested, gaming the test, then defecting when they're deployed. Luke Muelhauser recounts an example of such a treacherous turn in the wild.

Most relevant quote:

As part of a project studying the evolution of (simulated) organisms, computer scientist Charles Ofria wanted to limit the replication rate of a digital organism. So, he programmed the system to pause after each mutation, measure the mutant’s replication rate in an isolated test environment, and delete the mutant if it replicated faster than its parent. However, the organisms evolved to recognize when they were in the test environment and “play dead” (pause replication) so they would not be eliminated and instead be kept in the population where they could continue to replicate outside the test environment. Once he discovered this, Ofria randomized the inputs of the test environment so that it couldn’t be so easily detected, but the organisms evolved a new strategy, to probabilistically perform tasks that would accelerate their replication, thus slipping through the test environment some percentage of the time and continuing to accelerate their replication thereafter.

The full story is given in the original post.
FAQ: Advice for AI Alignment Researchers
by Rohin Shah — 2021-04-26T18:59:52.589Z
Link: https://rohinshah.com/faq-career-advice-for-ai-alignment-researchers/

To quote Andrew Critch:

I get a lot of emails from folks with strong math backgrounds (mostly, PhD students in math at top schools) who are looking to transition to working on AI alignment / AI x-risk. There are now too many people “considering” transitioning into this field, and not enough people actually working in it, for me, or most of my colleagues at Stuart Russell’s Center for Human Compatible AI (CHAI), to offer personalized mentorship to everyone who contacts us with these qualifications.

From math grad school to AI alignment, Andrew Critch

I’m pretty sure he wrote that at least 4 years ago (2016 or earlier). The field has grown enormously since then, but so has the number of people considering it as a research area. So far, I’ve tried to give at least 10 minutes of my time to anyone who emails me with questions; that probably won’t be sustainable for much longer. So now I’m answering the questions I get most frequently. I hope to keep this up to date, but no promises.

Usually, I write a blog post when I think I have something important and novel to say, that I am relatively confident in. That’s not the case for this post. This time, I’m taking all the questions that I frequently get and writing down what I’d say in response. Often, this is (a) not that different from what other people would say, and (b) not something I’m very confident in. Take this with more grains of salt than usual.

Thanks to Neel Nanda, Nandi Schoots, and others who wish to remain anonymous for contributing summaries of conversations.

See the linked post for the FAQ; which will hopefully be kept up to date over time.
Spoiler-Free Reviews: Monster Slayers, Dream Quest and Loop Hero
by Zvi — 2021-04-26T15:00:01.515Z

Recently my love of card-based roguelikes caused me to check out these two games.

First, Loop Hero.

Loop Hero is a unique Tier 2 (Worth It) game that came out recently.

If you’re up for a unique retro-graphical semi-auto-battling rogue-like, stop reading here and go play it, as the game benefits from the highest possible degree of blindness.

At its core, you go around a loop auto-battling, as your battles give you loot in the form of better equipment and drawing cards from your deck. The cards in your deck give you more enemies to face to get more better loot, they give you bonuses, and they give you resources to take back after your foray is over. Each loop, your enemies get harder. Once you’ve played enough cards, the boss appears. When you’re done with the level, you take your resources back and improve your camp, which makes you stronger, and you go out for another meta-loop on the loop, until you win.

Loop Hero has a lot of new ideas in it, it’s easy to learn and get into, and its atmosphere and vibe are unique and cool. You make a bunch of interesting decisions, there’s a bunch to try and learn, and you can tune the challenge in various ways to keep it balanced if you’d like. The mission of ‘figure things out and win in the minimum number of expeditions’ was how I approached it, and that seemed good. A hardcore player might start from scratch to try and win as fast as possible, and that’s its own thing.

There are some issues. The graphics are very old school, and while some are done quite well, some (especially your hero icon on the map) are ugly. Half the cards you’ll draw will be terrain cards, and you’ll have to go through the process of playing them every time, which stops being interesting quickly. There’s a lot of ‘forced move’ style stuff that ends up taking up a bunch of time. And finally, once you realize what the ‘right answers’ are, things get less interesting in many sense.

If they ever make Loop Hero 2, there’s a lot of room to expand upon these ideas and make them better.

At some point I’d like to discuss this stuff in spoiler-included detail, since I find the choices interesting, but for now that’s about all you need to know.

Second, Monster Slayers and Dream Quest.

Back in 2014 there was this game called Dream Quest. Dream Quest was an awesome game, much more subtle and well balanced than it appeared. You progressed through three levels...
Bayesian and Frequentists Approaches to Statistical Analysis
by Christopher Clark — 2021-04-26T00:18:00.668Z

In this post, I want to give a from-first-principles account of Bayesian and frequentists approaches to statistical analysis.
Bayesian and frequentism have been discussed here often, although LessWrong posts on these topics are often focused on Bayesians and frequentists as it comes to probability and rationality, or its use in AI.
Here, I am going to focus on statistical analyses that are intended for human readers.
This is a somewhat experimental attempt to cobble together a sensible framework from the often confusing and mismatched pieces I have encountered when trying to learn about this topic myself, and to keep things grounded on what statistical analysis is actually trying to accomplish rather than on abstract philosophical positions.
I will assume the reader is familiar with the mathematics of probability and Bayes rule.

Introduction

To start at the beginning: As staticians, we are interested in helping a reader understand and make inferences using data. We will do so by performing some computations on the data, and then presenting the results to the reader who will, hopefully, be able to use those results to draw conclusions faster and more accurately than they would if trying to parse the data on their own. The main thing we need to decide is what should be computed.

To put this in mathematical terms, let d be the data we have observed. Let h be a hypothesis about how the data was generated, and let D be a random variable over possible values we could have observed. I will assume we also have a model of P(D|h), meaning the probability distribution over those possible values given that a particular hypothesis, h, is true. As a concrete example, imagine a friend asks us to help them decide whether a coin is fair (meaning it has a 50/50 chance of landing on heads if flipped), and they tell us they observed 13 heads after flipping the coin 20 times. Then D would be a random variable with a probability distribution over the integers 0 to 20, h will be a real value between 0 and 1 representing the odds of the coin landing on heads when flipped, and d=13. P(D|h) will be a binomial distribution with p=h. Our task is to use d and our model of P(D|h) to give the audience some key bits of information that will help them decide if the coin is fair.

If you are willing to view the reader as a Bayesian reasoner we can formalize this a bit more. In that case, we kno...