trentmkelly/LessWrong-43k huggingface dataset:
2024 ACX Predictions: Blind/Buy/Sell/Hold
by Zvi — 2024-01-09T19:30:06.388Z

My 2023 ACX predictions showed a clear lack of confidence in taking on the market. I won 30 markets for an average of +185 each, and lost 12 for an average loss of -185 each1. When one goes 30-12, hitting a 71% mark versus the about 58% average price initially paid, that is worth noticing. It is possible that I generally benefited from 2023 being a year where not much happened outside of AI, but I think it’s time to know what we really think.

That means this year I’m going to add a phase, where I predict blind. Blind means I’m not allowed to look at any prediction markets. I can still look up facts, and financial markets are fair game, but nothing beyond that. Only after that will I look at Manifold. Metaculus makes this viable, as they have the ACX questions without listing probabilities.

Note that it would still be a mistake to make major moves on Manifold right away. While I did well in the end, I had plenty of opportunity through the year to double down.

The questions are listed on Manifold and Metaculus in a random order without being numbered, so I’m going to attempt to organize them reasonably.

If you are scoring me in the future for calibration or Brier score, this means you have three sets of numbers:

Blind. No market influence at all.

Bought/Sold to. Mostly market prices that I adjust slightly.

Metaculus predictions. Me after adjusting for the market.

If you are comparing me to other predictors, please use the Metaculus predictions, although of course all the numbers have some interest.

International Politics

Will there be 100 or more military conflict deaths between Ethiopia and Eritrea in 2024?

I hadn’t heard about this, so I read a few articles. The hypocrisy of comparing our reaction of ignoring the much larger tragedies and ‘war crimes’ that have played out around Ethiopia, while the same people cry (well, basically everything) and spew hate around other conflicts, is palpable throughout.

I got burned in Bayes points thinking (up until only days prior) that Russia was unlikely to actually invade Ukraine, one of my worst recent prediction mistakes. There have already been multiple live conflicts here, and there is a realpolitik reason for Ethiopia to fight for a Red Sea port – I know little about the area, but I was very surprised back in the day that they allowed themselves to be cut off when Eretria became independ...
Announcing the Double Crux Bot
by sanyer — 2024-01-09T18:54:15.361Z

Edit (April 2, 2024): You can now join the Double Crux Bot's own Discord server here.

TL;DR:

We are releasing a beta version of our chatbot powered by GPT, designed for facilitating Double Crux dialogues between two users on Slack or Discord. We're hoping to receive user feedback to evaluate the bot's usefulness and further work required. You can get the bot for your Slack workspace here and for your Discord server here.

Introduction

Double Crux

is a conflict resolution technique developed for the Center for Applied Rationality (CFAR) workshops, but it can be difficult to use because it might not apply to all problems, conversations can become emotional, or the participants might not have the requisite knowledge to embark on a double crux conversation themselves. Having a facilitator makes it easier, but facilitators aren’t readily available and are pretty much inaccessible outside of defined rationality workshops. There are also widely varying opinions on the best scenarios and strategies for approaching double crux. Our bot provides a low-effort way to engage in double crux for disagreements that people have in real-time, and it systematizes the method to an algorithm we have developed and continue to refine.

Example Scenario

Say you and your co-worker disagree about whether you should create a code of conduct policy for your organization. One of you is strongly against it, and the other one is strongly for it. Despite rehashing your reasoning multiple times, you cannot reach an agreement. The bot might help you to understand that your co-worker thinks a code of conduct policy might be counterproductive if it isn't enforced well, whereas you think that, even without enforcement, it has a positive net effect on overall culture. The double crux here might be whether conduct policies require enforcement.

The double crux bot is intended to help resolve these types of

action-oriented disagreements

. The bot helps people make their reasoning explicit and reflect on the crux of the issue. The bot helps people build better inferences about each other’s motivations and frameworks so that they can come to a resolution by addressing those key points upon which most of the decision-making hinges. The chatbot does not let people negotiate or outline the solution itself but instead helps to lay the foundation for doing so.

The Double Crux Algorithm

Double crux

has been...
Does AI risk “other” the AIs?
by Joe Carlsmith — 2024-01-09T17:51:47.020Z

(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app.

This essay is part of a series I'm calling "Otherness and control in
the age of AGI." I'm hoping that the individual essays can be read
fairly well on their own, but see here for a brief summary of the essays
that have been released thus far.)

In my last essay, I discussed the way in which what I've called "deep
atheism" (that is, a fundamental mistrust towards both "Nature" and
"bare intelligence") can prompt an aspiration to exert extreme levels of
control over the universe; I highlighted the sense in which both humans
and AIs, on Yudkowsky's AI risk narrative, are animated by this sort
of aspiration; and I discussed some ways in which our civilization has
built up wariness around control-seeking of this kind. I think we should
be taking this sort of wariness quite seriously.

In this spirit, I want to look, in this essay, at Robin Hanson's
critique of the AI risk discourse – a critique especially attuned the
way in which this discourse risks control-gone-wrong. In particular,
I'm interested in Hanson's accusation that AI risk "others" the AIs (see
e.g.
here,
here,
and
here).

Hearing the claim that AIs may eventually differ greatly from us, and
become very capable, and that this could possibly happen fast, tends
to invoke our general fear-of-difference heuristic. Making us afraid
of these "others" and wanting to control them somehow ... "Hate" and
"intolerance" aren't overly strong terms for this attitude.[1]

Hanson sees this vice as core to the disagreement ("my best one-factor
model to explain opinion variance here is this: some of us 'other' the
AIs
more").
And he invokes a deep lineage of liberal ideals in opposition.

I think he's right to notice a tension in this vicinity. AI risk is,
indeed, about fearing some sort of uncontrolled other. But is that
always the bad sort of "othering?"

Some basic points up front

Well, let's at least avoid basic mistakes/misunderstandings. For one:
hardcore AI risk folks like Yudkowsky are generally happy to care about
AI welfare – at least if welfare means something like "happy
sentience." And pace some of Hanson's accusations of bio-chauvinism,
these folks are extremely not fussed about the fact that AI minds
are made of silicon (indeed: come now). Of course, this isn't to say
that AI welfare (and AI rights) issu...
AI demands unprecedented reliability
by Jono — 2024-01-09T16:30:12.095Z

My AI pessimism is not addressed by optimistic arguments that do not promise the level of reliability required for a technology that our societies vital functions are expected to rely on.

Historically, useful improvements are hard to reverse and solutions can suddenly fail after we had become dependent on them. The potato seemed perfect for Ireland until they all blighted. Dutch dams seemed good, until they broke. Banks seem to keep suddenly failing on the regular.

AI is useful and we rely on it. Our culture is under heavy influence of search engines and algorithmic recommendations, people entrust their lives to self-driving cars and companies entrust sensitive or copy-righted data to generative models.

AI relies on AI. Some hypothetical ChatGPT call queried Bing which returned a page written by some human who, inspired by a Facebook recommendation, made Bart summarize the report of Covid prediction model that was co-written with Copilot. This low-stakes example fails all the time, but the state of affairs is still useful enough for people to build on top of and thereby increasingly rely upon.

The tool-AI future looks like one where man and machine rely on various interdependent models with medical diagnoses, mental help, bureaucracy, military protection, news, matchmaking, electricity supply, medicine production, weather forecasting, forecasting and AI development. The agentic-AI future has us rely on AI to build all of these itself.

We should demand a level of reliability that is stricter than any level of reliability our past technologies have achieved because our reliance on AI will be unprecedentedly large.

Addendum: unpersuasive arguments

I can only guess at what requirements for AI are good enough. Having humanity at AI's mercy and surviving is good enough, but not a situation we'd want to get into without knowing in advance that that is what will happen.
In the meantime, here is a list of argument types that should be insufficient for lowering anyone's P(Doom):

Counterarguments to failure modes
Unless your technique counteracts all possible (not only all named) failure modes, there is likely a failure mode we both missed.

Optimistic arguments that work in most situations
We are going to see all the situations. If your technique mostly works, it will surely fail.

Arguments against the reliability of humans
Humans are reliable because we live at humanity's mercy and...
Uncertainty in all its flavours
by Cleo Nardo — 2024-01-09T16:21:07.915Z

Acknowledgements:
This research began during the SERI MATS program, under the joint mentorship of John Wentworth, Nicholas Kees, and Janus. Thanks also to Davidad, Jack Sagar, and David Jaz Myers for discussion.

Abstract:
I think that there is a uniform correspondence between flavours of uncertainty and monads taking state-spaces to belief-state-spaces, for different characterisation of belief. In this essay, I describe this correspondence explicitly and list 15 diverse and well-motivated examples. I explore some applications to model-building and agent foundations. Along the way, I characterise infrabayesianism uncertainty as the minimal way to encompass possibilistic uncertainty, probabilistic uncertainty, and reward.
No prerequisites are required beyond a high-school familiarity with sets, functions, real numbers, etc. Feedback welcome.

Introduction

Suppose I'm facing the following problem. There's an upcoming election between n candidates, and you're uncertain who will win. How can I model both your belief about the election and the election itself in a coherent way? By "belief" here, I mean your epistemic attitude, your internal model, your opinion, judgement, prediction, etc, etc. Think map-territory distinction: the election is the territory, your belief is the map, and I need to model both the map and the territory coherently despite the fact that the map and the territory are (typically speaking) two completely different types of thing.

Well, to model the election itself, I'll use a set S={s1,s2,s3,…sn} with an element for each electoral candidate. To represent your belief about the election, I must find another set B(S) with an element for each belief that you might have about the election. I'll call S the state space and B(S) the belief-state space. A solution to our problem is given by a mathematical operator B sending each state-space S to the matching belief-state space B(S).

One may feel prompted to ask: does any operator B suffice here? Can the belief-state space be anything whatsoever, or must it carry some extra structure, possibly satisfying some additional constraints? Or, stated more philosophically, can any territory serve as a map for any other? I say no. Roughly speaking, the operator B must be a so-called monad, which will be the central object of ...
Compensating for Life Biases
by Jonathan Moregård — 2024-01-09T14:39:14.229Z
Link: https://honestliving.substack.com/p/compensating-for-life-biases

The most impactful practice of my life is the way I’m capturing insights. Whenever I realize something that alleviates a problem in my life, I write down the insight. Every week, I read through my list of insights and use them to improve an ever-evolving list of principles I use to guide my life.

I think of this list as a set of personal heuristics, used to compensate for recurring mistakes I make. My guess is that people all have their own set of “life biases” that lead to them acting in unskillful and suboptimal ways. A list of principles is a great way to compensate.

I’m not sure this system will work for anyone, but I suggest giving it a try. My principles have saved me a huge amount of time and suffering.

Example Principle

My first and favourite principle is “Don’t wait for things to get better”. I wrote this down to alleviate a recurring issue in my life. I occasionally find myself in dysfunctional situations that cause me to suffer. It can be a job I’ve grown dissatisfied with, a relationship that doesn’t work out, or similar.

A bad habit of mine is to “wait it out”, hoping that the situation will magically improve. My favourite principle helps me remember to opt for action instead of remaining dissatisfied. I either shift my mindset and find ways to enjoy the situation; or make an external change happen.

I recently quit my job. I waited for way too long, telling myself that I was in a special situation where the principle didn’t apply. I shouldn’t have waited for things to get better.

The weekly review

I recommend reviewing the principles once a week, reading down the list while considering whether any of the principles apply to a situation in your life. If it does, consider the implications and either act immediately, or add a task to handle it later.

Besides reviewing the list of principles, also read the list of accumulated insights. Clear the ones that feel unnecessary, integrate others into the list of principles, and leave some be, to be reread next week.

There’s no hard cap on the number of principles, but I find 8-10 to be a sweet spot. It’s usually possible to find common themes, allowing you to merge principles. Having too many principles makes the practice unwieldy, it’s better to keep to opt for fewer of them.

Having room for new ones is vital - if there is no room, innovation is blocked.

Make your own principles!

There is a vital di...
Can Morality Be Quantified?
by Julius — 2024-01-09T06:35:05.426Z

[Cross-posted from my blog]

A common criticism of effective altruism (EA) is its tendency to oversimplify complex issues into mere numbers. Critics argue that EA approaches amount to calculating moral decisions as if on a spreadsheet. They protest that you can’t put a number on human life or the impact of a charity because not everything can be quantified.

I disagree with these objections. I think they miss the fundamental purpose of quantification. Quantifications are not intended to be literally true; they are supposed to be a useful framework to guide decision-making. In this context, their utility is unmatched.

Calculating morality on a spreadsheet makes people uneasy, even without the googly-eyed glasses.

Mathematical theory is precise and logical, leading to exact answers. For example, if you start with three apples, add three more, eat four, then add one, you’re back to three apples again, exactly where you started. But, in reality, no two apples are identical, so you’re not exactly where you started. This doesn't imply that the mathematical model “doesn’t work”; it simply means its application has limits. It counts the apples accurately but doesn't account for their uniqueness.

This limited nature is true of all models. As the statistician George Box said, “All models are wrong, some are useful.” Economists, for instance, rely on price not because it perfectly reflects value, but because it is a highly useful tool for quantification. Money, as a universal quantifier, is incredibly helpful, yet it also has its limitations. It facilitates transactional clarity, even though it may not capture the full spectrum of value in every context.

Imagine two companies, Alfred's Apples and Bonnie's Bananas, both seeking loans from a bank. Alfred's Apples dominates the lucrative apple market, while Bonnie's Bananas is a major player in the less profitable banana market. Comparing these companies for loan purposes is challenging due to their different products. This is where money becomes a crucial tool.

By quantifying the worth of Alfred’s Apples at $100M and Bonnie’s Bananas at $50M, the bank can compare them on a common scale and decide on loan terms. Moreover, once these companies are on the stock market, their valuations reflect the collective judgment of all investors, condensed into a single number per company.

This process, where everything is converted to mon...
Brief Thoughts on Justifications for Paternalism
by Srdjan Miletic — 2024-01-09T00:36:25.329Z
Link: https://dissent.blog/2024/01/06/brief-thoughts-on-justifications-for-paternalism/

Lisa wrote an article on coercion, paternalism and irreversible decisions. I have some thoughts.

The basic groundwork goes something like this:

coercing people is bad

coercing people to prevent them infringing on the rights of others may be justified in some cases

coercing people to stop them doing something they freely want to do, and which has no impact on others, is on face value unjustified.

When can paternalistic coercion be justified? A few obvious justifications I can think of are

When a person is mistaken about the consequential outcomes of an act. (e.g: John thinks that taking act X will give him $1'000. Actually it will give him with -$1'000)

When a persons actions would lower their own utility

When a persons choice is not a reflection of their "true" preferences. (e.g: an addict who wants drugs but doesn't want to want drugs. A person in a temporary depression who wants to kill themselves)

When a persons choice has overly large effects on their future self, and those effects are locked in. e.g: selling yourself into slavery

The "You're Wrong" justification for paternalism

This justification is basically that someone is mistaken about the practical consequences of a choice they're going to make. Hence coercing them to do what would best suit their preferences is okay.

On one hand I'm not too convinced by this justification. If someone is mistaken about the consequences of a given action, you can always tell them why you think they're mistaken. If they still disagree, it's not clear what justification you have for coercion. Imagine you live in a society where the following is true

There are shops which sell "ill-advised consumer goods"

the goods in these shops are unregulated and hence often dangerous or harmful to the user

the government specifically advises you to not buy things from these shops and entering them requires you to sign a waiver stating you understand that it's a bad idea for most people who do so

If you don't trust yourself to make these kind of decisions in the moment, you can also choose at the beginning of every year to be banned from ill-advised goods for the entire year.

In this kind of society, people are very clearly

given a choice of whether they want to defer to the state/experts or not (the annual opt-in)

made aware of why society thinks a certain act is bad

I guess the general argument here is that if you think some...
Hiring decisions are not suitable for prediction markets
by SimonM — 2024-01-08T21:11:14.304Z

In a recent edition of

The Diff

($), Byrne Hobart pointed out

a talk

given by Robin Hanson at Manifest. The talk is well worth listening to. For example, Robin says:

I've always thought [Prediction Markets] was the best idea I've had in my life [...] My biggest contribution

The general thrust of the talk is...

Our world is full of big organizations that just make a lot of bad decisions because they find it hard to aggregate information [...], so prediction markets are a proven method for doing exactly that

... but that it's work to apply prediction markets.

Paraphrasing an entire section: You need to prove a technology works to get it accepted. It's insufficient to supply the technology. Robin gives the analogy of the motor - a motor on its own has no value. A motor hooked up to a pump in a coal mine… that's another story. In his analogy, the prediction market is the motor, and someone needs to find the "pump in the coal mine" to hook it up to.

What are the [...] high initial value things? The thing that most often comes to my mind is new hires.

Here is where I start to disagree with Robin.

The first question is:

Are decisions made during hiring poor?

My intuition here is "actually fairly good." Firms typically spend a decent amount on hiring processes - they run screening tests, conduct interviews, look at CVs, and ask for references. It's fair to say that companies have a reasonable amount of data collected when they make hiring decisions, and generally, the people involved are incentivized to hire well.

The second question is:

Would a prediction market work well?

If I look at the tests I mentioned, this prediction market is unappealing. We'd expect no cross-subsidies, mis-weighted demand, and noise traders (other than most participants won't be very good). There's little reason for the information to be dispersed - the company currently asks and gets the data.

There are further issues - the individual traders are unlikely to make lots of trades, so the mechanism by which better traders have more capital and make the market more efficient is absent.
Better Anomia
by jefftk — 2024-01-08T18:40:05.236Z

In Anomia you take turns revealing cards and putting them on your
pile. Each card has one of eight symbols (ex: a circle) and a unique
category (ex: 'frozen food'). If your top card has the same symbol as
someone else's, whoever first gives an example word from the other's
category takes their card. In the

official
rules any cards you take this way go into a separate face-down
"winning pile", but the game is much more exciting if instead you put
them face-up on your own pile.

What makes this fun is you often end up with a stack of same-symbol
cards on your pile, and so when someone takes your top card you're
still displaying matching symbols. Then either they name something
from your now-revealed card's category, or you name something for the
card they just took and steal it back. You continue until you no
longer match. I've played games where we ended up with a stack
of 10+ matching cards, which several people tried to steal, some
successfully and some not.

To prevent cycles where we both keep saying "ice cream" as quickly as
possible to steal and re-steal 'frozen food', this requires the
usually-optional 'no-repeats' rule. [1] But that rule is a good enough
one that I'd always prefer to play with it anyway.

Here's a walked example. Initially no one matches:

Drawing "Musical Instrument" causes a match:

"Bridge" is stolen with "Danyang-Kunshan", revealing "Bagel Flavor":

"Bagel Flavor" is stolen with "poppyseed", revealing "Cowboy Accessory":

In a shocking reversal, "Bagel Flavor" is stolen back with "egg":

And "Bridge" is stolen back with "Bay":

Finally, "Musical Instrument" is stolen with "drum":

They no longer match, and the face-off is over.

[1] The specific formulation we use is that any example that moves a
card has been used up and can't be used again. And if we play
multiple times with the same deck in one sitting repeats from the
previous time are still not allowed, though new players joining can
still use them. Note that we do allow words that have come up in
conversation or failed to move cards; this seems to be more fun than
the official formulation which prohibits using anything previously
spoken.