trentmkelly/LessWrong-43k huggingface dataset:
Bet on Rare Undesirable Outcomes when Seeding a Prediction Market
by lsusr — 2022-02-10T00:20:03.695Z

Suppose you run an insurance company that insures buildings against burglars. You recently sold insurance to a client with a building in a new city you have never operated in before. You need to know how likely it is for a burglar to break into your client's building.

The simplest way to know for sure if a burglar will break into your client's building is to hire a burglar yourself for $2,000. Hiring a burglar yourself is worse than doing nothing at all because it's more important to protect your client from burglary than to get your predictions right.

A less direct way to establish the probability of whether a burglar will break into your client's building is to fund a prediction market. The prediction market's implied odds will converge to the probability a burglar will break into your client's building. Suppose the prior probability of a burglar breaking into your building is 5%. The price of "the building gets burgled" ought to rest at 0.05 and the price of "the building does not get burgled" ought to rest at 0.95.

Prediction markets require liquidity. Suppose you seeded your prediction market with liquidity such that an investor can invest $10,000 into the market without noticeably moving the prices. A savvy investor would put $10,000 into "the building gets burgled" which pays $200,000 if the building gets burgled and then hire a burglar for $2,000 to guarantee the building gets burgled. The investor pockets $200,000 - $10,000 - $2,000 = $188,000. By funding a prediction market, you have just paid $190,000 to get your building burgled. The net result is $188,000 worse than hiring a burglar yourself to break into your client's building.

Prediction markets have two sides. In the previous example, we incentivized burglary because we seeded both sides (including "the building does not get burgled") with liquidity. Buying shares of "the building does not get burgled" incentivizes others to burgle the building. What happens if you seed the market by buying shares only of "the building gets burgled"?

The prediction market starts out empty. We offer to buy shares in "the building gets burgled" for 0.05. Traders can therefore sell shares in "the building gets burgled" for 0.05 which equals buying shares of "the building does not get burgled" for 0.95. Traders are incentivized to influence the outcome by protecting our building, which is what we want.

We also generate...
Why rationalists are not much concerned about mortality?
by SurvivalBias — 2022-02-10T00:11:46.068Z

As of 2022, humans have a life expectancy of ~80 years and a hard limit of ~120. Most rationalists I know agree that dying is a bad thing and at minimum we should have an option to live considerably longer and free of the "diseases of the age", if not indefinitely. It seems to me that this is exactly the kind of problem where rationality skills like "taking things seriously", "seeing with fresh eyes" and awareness of time discounting and status quo bias should help one to notice something is very very wrong and take action. Yet - with the exception of cryonics[1] and a few occasional posts on LW - this topic is largely ignored in the rationality community, with relatively few people doing the available interventions on the personal level, and almost nobody actively working on solving the problem for everyone.

I am genuinely confused, why is this happening? How is it possible that so many people who are equipped with epistemological tools to understand they and everyone they love are going to die, understand it's totally horrible, understand this problem is solvable in principle, can keep on doing nothing about it?

There is a number of potential answers to this question I can think of, but none of them is satisfying and I'm not posting them to avoid priming.

[ETA: to be clear, I have spent a reasonable amount of time and effort making sure that the premise of the question is indeed the case - whether rationalists are insufficiently concerned about mortality - and my answer is unequivocal "yes". In case you have evidence to the contrary, please feel free to post them as an answer]

^

It's an interesting question exactly how likely cryonics is to work and I'm planning to publish my analysis of this at some point. But unless you assign a ridiculously optimistic probability to it working, the problem largely remains. Even 80% probability of success would mean your chances are worse than in Russian roulette! Besides, my impression is that only a minority of rationalists is signed up anyway.
Bernal Heights: acquisition of a bicycle
by KatjaGrace — 2022-02-09T21:00:13.559Z

The measure of a good bicycle, according to me, is that you can’t ride it without opening your mouth in joy and occasionally exclaiming things like ‘fuck yeah bicycle’. This is an idiosyncratic spec, and I had no reason to think that it might be fulfilled by any electric bicycle—a genre I was new to—so while I intended to maybe search through every electric bicycle in The New Wheel for one that produced irrepressible, physically manifest joy, I also expected this to likely fail, and to be meanwhile embarrassingly inexplicable and irritating to bike shop employees—people who often expect one’s regard for bikes to map fairly well to facts about their price, frame shape, and whether the gears are Shimano. But several bikes in, when I uncomfortably explained to a guy there that, while the ones I had tried so far were nice, bicycles had been known to make me, like, very happy, he said of course we should find the bicycle that I loved. So I at least felt somewhat supported in my ongoing disruption of his colleague’s afternoon, with requests that bicycle after bicycle be brought out for me to pedal around the streets of Bernal Heights.

The guy would maneuver each bike out of the crowded shop and to the sidewalk, and adjust it to fit me, and we would chat, often about his suggestion that I maybe ride up to the hill on the other side of the main road. Which I would agree might be a good idea, before riding off, deciding that turning left was too hard, and heading in the other direction, through back streets and around a swooping circle park with a big ring road, where I would loop a few times if the mood took me.

Some bicycles were heavy, and rode like refrigerators. Most bicycles were unsteady, and urged even my cycling-seasoned bottom to the seat while pedaling. Most bicycles added considerable assistance to going up hills. Many bicycles seemed fine.

Bernal Heights, on the other hand, seemed awesome. As I paused before my habitual turn-in-the-wrong-direction one time, the house kitty-corner to me was playing music louder than I recall ever hearing anything play music that wasn’t a large-concert speaker. It was truly not considerate. And a middle-aged guy on my corner was having a great time, laughing, and was like, ‘Welcome to Cortland Avenue’. I pulled up and said that I hadn’t been here before actually, and didn’t know what he was talking about. He explained that ...
My investment protects you. Your investment protects me.
by Nicole Dieker — 2022-02-09T20:12:23.712Z

Crossposted. Previous reading includes Money-generating environments vs. wealth-building environments (or "my thoughts on the stock market").

For “market volatility,” read people are redeeming cash from the market.

For “market growth,” read people are putting more cash into the market.

For continued market growth, you need more people putting cash in and fewer people taking cash out.

My investment protects you.

Your investment protects me.

After all, “panicked investors” can also be read as people redeeming cash from the market — and if too many people take cash out of the market before we get a chance to redeem our own investment value, then you and I won’t be able to take out as much cash as we were hoping to.

At least, not until more people put cash in.

Right?

(You’re going to say “but market growth is also based on businesses being better at businessing, Facebook’s stock cratered last week because Facebook did a bad job being Facebook,” and I’m going to say “ummmmmmm Facebook’s stock cratered last week because people [or bots] decided they’d rather have the cash value and/or were worried that other people [or bots] would redeem the cash value before they got a chance, and remember that even GameStop’s stock can rise if people put cash into it.”)

❤️
Epistemic Legibility
by Elizabeth — 2022-02-09T18:10:06.591Z

Tl;dr: being easy to argue with is a virtue, separate from being correct.

Introduction

Regular readers of my blog know of my epistemic spot check series, where I take claims (evidential or logical) from a work of nonfiction and check to see if they’re well supported. It’s not a total check of correctness: the goal is to rule out things that are obviously wrong/badly formed before investing much time in a work, and to build up my familiarity with its subject.

Before I did epistemic spot checks, I defined an easy-to-read book as, roughly, imparting an understanding of its claims with as little work from me as possible. After epistemic spot checks, I started defining easy to read as “easy to epistemic spot check”. It should be as easy as possible (but no easier) to identify what claims are load-bearing to a work’s conclusions, and figure out how to check them. This is separate from correctness: things can be extremely legibly wrong. The difference is that when something is legibly wrong someone can tell you why, often quite simply. Illegible things just sit there at an unknown level of correctness, giving the audience no way to engage.

There will be more detailed examples later, but real quick: “The English GDP in 1700 was $890324890. I base this on $TECHNIQUE interpretation of tax records, as recorded in $REFERENCE” is very legible (although probably wrong, since I generated the number by banging on my keyboard). “Historically, England was rich” is not. “Historically, England was richer than France” is somewhere in-between.

“It was easy to apply this blog post format I made up to this book” is not a good name, so I’ve taken to calling the collection of traits that make things easy to check “epistemic legibility”, in the James C. Scott sense of the word legible. Legible works are (comparatively) easy to understand, they require less external context, their explanations scale instead of needing to be tailored for each person. They’re easier to productively disagree with, easier to partially agree with instead of forcing a yes or no, and overall easier to integrate into your own models.

[Like everything in life, epistemic legibility is a spectrum, but I’ll talk about it mostly as a binary for readability’s sake]

When people talk about “legible” in the Scott sense they often mean it as a criticism, because pushing processes to be more legible cuts out illegible...
Create a prediction market in two minutes on Manifold Markets
by Austin Chen — 2022-02-09T17:36:56.320Z

Crossposted from https://manifoldmarkets.substack.com/p/above-the-fold-manifold-is-ready

TL;DR: Manifold Markets is:

A new prediction market platform
Where anyone can ask questions and resolve them
With grants from Scott Alexander and Paul Christiano

Our story

We started with a crazy new twist on prediction markets. You come up with a question for traders to predict, and then decide the outcome yourself.

For example, you could create a market on “Will my date with [X] go well?” Anyone can bet on it, and the bets create a forecast on the chance your date goes well. After the date is over, you get to judge the result and reward the traders who picked the correct side.

There are so many ways for this mechanism to go wrong. The creator of the market can be dishonest in deciding the outcome, or you may just disagree with their resolution, with no recourse.

Nevertheless, we pitched user-created prediction markets in a grant proposal to the blog Astral Codex, without having any previous background in prediction markets, and

somehow we won

!

It’s been an exciting two months of us hacking away to realize this idea, and today, we’re happy to announce the official launch of Manifold Markets! We’ve gotten so much support from early users who are just as passionate about prediction markets as we are.

You might ask: Is there a reason to create a prediction market, and not a Twitter poll? Are outcomes really that different from a simple person-by-person vote?

Yes, I believe so. The magic driving prediction markets is accountability. When wagering a scarce currency, those proven right live on to make future bets, whereas those with less-savvy bets have their influence diminished. Prediction markets succeed because they reward accuracy, and that makes all the difference!

Our goal is to make prediction markets an order of magnitude easier for you to create and share. It should be as frictionless as a Twitter poll. As part of that philosophy, we’re launching with a play money currency, which we believe is just as fun and predictive.

We’ve already built up a passionate community of predictors and market creators, including writers like

Richard Hanania

and

James Medlock

, who have predicted everything from

CDC recommendations

to

newsletter subscriptions

to

fatal shark attacks

.

There’s so much unexplored space to ask questions and get valuable forecasts. For example, with conditional markets, you...
Mailing Lists for Event Announcements
by jefftk — 2022-02-09T17:30:02.675Z

Let's say I'm organizing a repeated event like a meetup or dance
series: how should I let people know about it so they can decide
whether they want to attend? You can break the world down into three
groups:

People who wouldn't be interested, even after fully learning
about the event. Perhaps they don't enjoy the activity, have full
schedules, or live too far away.

People who would be interested, but don't know about your
events. Perhaps they didn't realize there were in-person gatherings
for this kind of thing, or didn't know there was one in their city.

People who already know about your events in general, but need
reminders or notifications about specific instances. Perhaps your
event doesn't have a consistent schedule, or someone only wants to
attend when they like topic.

You don't want to reach the people in (1): getting your event in front
of them just wastes their time and yours. If I went through the
phone book and called everyone in my city to tell them about my event,
since they're almost all in (1) I'd annoy a lot of people.

You do want to reach the people in (2), but it's hard. Many methods
of reaching this group will also reach many people in (1), and so be
spammy.

On the other hand, the people in (3) should be really easy to reach:
they want your notifications and know they want them. In my social
circles, the main tool people used to use was Facebook events: you
would join a group, and then you would get notified for every event
created in that group.

Unfortunately, in practice this resulted in a lot of event invitations
being sent to people in (1). Someone might be only a little bit
interested in the group's events. Or they used to be interested, but
now they've moved on. Or they never even joined the group, but someone
else added them because Facebook experimented with groups working that
way for a while. Or it's a huge group mainly used for discussion and
then someone creates an event which few people actually want to see.
Through a combination of limits, both algorithmic (events may or may
not be shown to people based on Facebook's predictions) and strict
(you can only invite the subset of a group which you are individually
Facebook friends with), this has become much less useful. Then add in
the general migration away from Facebook and this is not a good way to
reach people in (3).

So what do you do...
Paradigm-building: The hierarchical question framework
by Cameron Berg — 2022-02-09T16:47:57.119Z

The framework

Here, I will briefly introduce what I hope is a fundamental and potentially comprehensive set of questions that an AGI safety research agenda would need to answer correctly in order to be successful. In other words, I am claiming that a research agenda that neglects these questions would probably not actually be viable for the goal of AGI safety work arrived at in the previous post: to minimize the risk of AGI-induced existential threat.

I have tried to make this set of questions hierarchical, by which I simply mean that particular questions make sense to ask—and attempt to answer—before other questions; that there is something like a natural progression to building an AGI safety research agenda. As such, each question in this framework basically accepts the (hypothesized) answer from the previous question as input. Here are the questions:

What is the predicted architecture of the learning algorithm(s) used by AGI?
What are the most likely bad outcomes of this learning architecture?
What are the control proposals for minimizing these bad outcomes?
What are the implementation proposals for these control proposals?
What is the predicted timeline for the development of AGI?

Some immediate notes and qualifications:

As stated above, note that each question Q directly builds from whatever one’s hypothesized answer is to Q-1. This is why I am calling this question-framework hierarchical.
Question 5 is not strictly hierarchical in this sense like questions 1-4. I consider one’s hypothesized AGI development timeline to serve as an important ‘hyperparameter’ that calibrates the search strategies that researchers adopt to answer questions 1-4.
I do not intend to rigidly claim that it is impossible to say anything useful about bad outcomes, for example, without first knowing an AGI’s learning algorithm architecture. In fact, most of the outcomes I will actually discuss in this sequence will be architecture-independent (I discuss them for that very reason). I do claim, however, that it is probably impossible to exhaustively mitigate bad outcomes without knowing the AGI’s learning algorithm architecture. Surely, the devil will be at least partly in the details.
I also do not intend to claim that AGI must consist entirely of learning algorithms (as opposed to learning algorithms being just one component of AGI). Rather, I claim that what makes the AGI safety control prob

...
[Intro to brain-like-AGI safety] 3. Two subsystems: Learning & Steering
by Steven Byrnes — 2022-02-09T13:09:11.945Z

(Last revised: October 2024. See changelog at the bottom.)

3.1 Post summary / Table of contents

Part of the “Intro to brain-like-AGI safety” post series.

In the previous post I defined the notion of “learning from scratch” algorithms—a broad category that includes, among other things, any randomly-initialized machine learning algorithm (no matter how complicated), and any memory system that starts out empty. I then proposed a division of the brain into two parts based on whether or not they learn from scratch. Now I’m giving them names:

The Learning Subsystem is the >90% of the brain that “learns from scratch”, including the cortex, hippocampus, striatum, amygdala, cerebellum, and more.

The Steering Subsystem is the <10% of the brain that doesn’t “learn from scratch”—basically the hypothalamus and brainstem.

(See previous post for a more detailed anatomical breakdown.)

This post will be a discussion of this two-subsystems picture in general, and of the Steering Subsystem in particular.

In Section 3.2, I’ll talk about the big picture of what these subsystems do and how they interact. As an example, I’ll explain why each subsystem needs its own sensory-processing circuitry—for example, why visual inputs get processed by both the visual cortex in the Learning Subsystem, and the superior colliculus in the Steering Subsystem.
In Section 3.3, I’ll acknowledge that this two-subsystem picture has some echoes of the discredited “triune brain theory”. But I’ll argue that the various problems with triune brain theory do not apply to my two-subsystem picture.
In Section 3.4, I’ll discuss three categories of ingredients that could go into a Steering Subsystem:

Category A: Things that are plausibly essential for general intelligence (e.g. an innate drive for curiosity),
Category B: Everything else in the human steering subsystem (e.g. an innate drive to be kind to your friends),
Category C: Any other possibility that an AGI programmer might dream up, even if it’s radically different from anything in humans or animals (e.g. an innate drive to correctly predict stock prices).

In Section 3.5, I’ll relate those categories to how I expect people to build brain-like AGIs, arguing that “brain-like AGIs with radically non-human (and dangerous) motivations” is not an oxymoron; rather, it’s the default expected outcome, unless we work to prevent it.
In Section 3.6, I’ll discuss the fact tha

...
Prediction Markets are for Outcomes Beyond Our Control
by lsusr — 2022-02-09T09:54:40.891Z

Betting markets are the gold standard of expert predictions because bets are the ultimate test of what people truly believe.

The best betting markets are highly liquid. A liquid market is one where you can place a large bet without moving the price very much. Liquid prediction markets work when no individual person can influence the outcome. Bettering markets are a great way to find out if "it will rain tomorrow" or whether "candidate x will be elected president next year".

But what if a single person can influence the outcome? For example, what would happen if I created a betting market for "Lsusr will publish a blog post tomorrow"?

Suppose I am ambivalent about whether I will publish a blog post tomorrow. If the price of "Lsusr will publish a blog post tomorrow" drops below 1.00 then I will buy shares of "Lsusr will publish a blog post tomorrow" and then pocket a risk-free profit by posting a blog post tomorrow. If the price of "Lsusr will publish a blog post tomorrow rises above 0.00 then I will buy shares of "Lsusr will not publish a blog post tomorrow" and then pocket a risk-free profit by not posting a blog post tomorrow.

The market equilibrium occurs even if I am unaware that the prediction market exists. Suppose the price drops to 0.99. A trader could buy shares and then pay me a small fee to influence the outcome.

I am not truly ambivalent. Suppose I'm willing to influence the outcome in exchange for $500. What happens? If the market liquidity is less than $500 then we have a functional prediction market. If the market liquidity is more than $500 then we have a regular market.

Prediction markets function best when liquidity is high, but they break completely if the liquidity exceeds the price of influencing the outcome. Prediction markets function only in situations where outcomes are expensive to influence.
How to Legally Conduct "Rationalist Bets" in California?
by Davis_Kingsley — 2022-02-09T08:16:16.741Z

So, in theory betting on future events can be a useful practice for establishing an epistemic record, "putting your money where your mouth is", etc. However, I recently noticed an issue -- most forms of betting appear to be illegal in the state of California, including not just sports betting but also "casual bets." In fact, if such bets are conducted online or have more than $2500 at state, they are actually felonies!

(After this issue came up in a recent conversation elsewhere, someone asked a lawyer about the matter to confirm, and the lawyer's opinion was that this was indeed illegal but unlikely to be enforced.)
To me, this poses a major obstacle to conducting these sorts of bets and makes me much more reluctant to do so. Are there any good methods for avoiding such issues that people are aware of? (For instance, I've heard the idea posed before that you bet in the form of conditional donations to charity -- I don't think this actually gets around the relevant issues but would be interested in hearing more proposals like that if people have them.)