trentmkelly/LessWrong-43k huggingface dataset:
Results: Circular Dependency of Counterfactuals Prize
by Chris_Leong — 2022-04-05T06:29:56.252Z

Now that the deadline has passed, I thought I should announce the winner of my competition on the Circular Dependency of Counterfactuals.

The winner is tailcalled with their post Some thoughts on "The Nature of Counterfactuals".

There were a few things I liked about the post. Firstly, it directly engaged with the topic. Some people's responses were primarily about something else, but they tried to shape it to be relevant to the competition. That's totally understandable. If someone else posted a bounty and I was already interested in something similar but which was not quite the main focus, I'd probably try to my luck as well. However, that wasn't quite what I was looking for.

Secondly, it avoided the trap that some of the other responses fell into where they tried to demonstrate that counterfactuals weren't circular by showing that counterfactuals could be explained in terms of Y, but without explaining why Y might need counterfactuals in order to make sense. In contrast, Tailcall's argument was that counterfactuals could be defined in terms of causality and that it bottoms out there as causality has an ontologically real existence. They were able to give some plausible reasons for believing this. I'm still not sure whether I buy their argument or not, but at the very least they managed to identify a non-obvious argument that I failed to adequately engage with. Beyond this, it will also provide a solid target for engagement when I write on this topic in the future. Their post covered other topics too, but these aspects are what won them the prize.

I'm hoping I'll be able to find time to write up a proper response to their argument, but for now congratulations!

The prize will actually be $1100 USD since I had originally planned to award a social media prize, but that didn't really work out. No one really posted any Facebook posts or Twitter threads on this.
A Solution to the Unexpected Hanging Problem
by Dawn Drain — 2022-04-05T06:19:25.025Z

I was rereading the fantastic encyclopedia Mathematics 1001: Absolutely Everything that Matters in Mathematics in 1001 Bite-Sized Explanations over Christmas break. It really lives up to its title!, and is much more accessible than reading random Wikipedia articles.

There’s an interesting puzzle, The Unexpected Hanging, posed in the final section Games and Recreation. And, uh, more specifically in the chapter on Puzzles and Perplexities. The problem is as follows:

A logician was condemned to be hanged. The judge informed him that the sentence was to be carried out at noon, one day next week between Monday and Friday. But it would be unexpected: he would not know which day it would be, before it happened. In his cell, contemplating his doom, the logician reasoned as follows:

‘Friday is the final day available for my hanging. If I am still alive on Thursday afternoon, then I can be certain that Friday must be the day. Since it is to be unexpected, this is impossible. So I can rule out Friday.

Therefore Thursday is the last possible day on which the sentence can be carried out. If I am still here on Wednesday afternoon, I must expect to die on Thursday. Again, this conflicts with the execution’s unexpectedness, and is therefore impossible. I can rule out Thursday!..’

Repeating the same argument the logician was able to rule out Wednesday, Tuesday, and Monday, and went to sleep slightly [sic] cheered. On Tuesday morning the hangman arrived at his cell, completely unexpectedly! As he stood at the gallows, the logician reflected that the dreadful sentence was being carried out exactly as the judged had promised.

Where oh where was the flaw in the logician’s reasoning? Please feel free to puzzle over the problem yourself before reading my solution :).

.

.

.

.

.

.

.

.

Ok, here’s how I would approach the problem. One correct, but unsatisfactory answer, is to say that the judge’s warning, "the sentence was to be carried out at noon, one day next week between Monday and Friday. But it would be unexpected: he would not know which day it would be, before it happened”, is somehow logically impossible or logically meaningless. It’s like saying “this sentence is false” or “the flibbertygibbets gobswallow” or “green is the best color” or “the water is very hot”, and isn’t the kind of unambiguous proposition that takes a definite truth value. I mean, this is true, and explains the logician’...
I was asked about how to resolve strong emotional fear concerns around the basilisk and this is my instructions as to how to do that.
by Elo — 2022-04-05T06:14:15.374Z

So you just read something about the basilisk and it freaked you out. There’s some emotional argument around being simulated, tortured and extorted by a future AI. You read it, feel quite scared, concerned or uncomfortable about it and now don’t know what to do about the phenomena.

This won’t be the last time you heard of a monster that scared you, although maybe previous monsters somehow didn’t seem real or like they applied to you. For example zombies. Seem rather slow and stupid. Unless they are fast zombies, in which case you can still see them and make plans with the other humans. Humans always outsmart the monsters right? And zombies are super improbable anyway.

Back to the Basilisk. So problem, you now know of an idea that seems scary. You have two problems really. The first is that there’s a class of information that now seems dangerous. You could in the future find out a new alpha basilisk that is also scary and more urgent to address than the current information hazard. The second problem is “there’s a scary thing”.

If we look at the greater context of this problem. As a ~5 year old or young child, if you found a scary thing, like the dark or a mean person, you could turn to an adult and get their assistance. As an adult, you just kinda “have the skills” to handle scary things, but as a person in-between child and adult, there are now no adults to turn to (or maybe no adults who will take you seriously) and you don’t yet have adult tools for dealing with emotions. (there’s a long debate about whether people ever turn into adults or just do the same as they have been doing and eventually just have learned competence by practice, while still not being an adult but that’s a different conversation)

To repeat the problem again, there’s an intellectual idea (basilisk) that is bothering the emotional mind (fear) and I don’t have the tools to deal with this problem.

So what are the tools that help with emotions?

A non exhaustive list:

Recognition of the emotion
Capacity to stay with the emotion
Strong stable foundation of emotional territory that feel familiar and native
Skill of knowing how to respond to the emotion

Recognition of the emotion

Lucky for us, emotions are a finite set. Or at least humans have already made a reasonably comprehensive list of emotions. Lan...
Why Iraq is so violent
by Tim Liptrot — 2022-04-05T05:13:10.674Z

Thought this might be an interesting topic for you all.

THE CAUSES OF VIOLENCE IN IRAQ

Iraq has struggled with internal violence for most of its history post-independence. 4 coups occurred between independence and Saddam Hussein’s dictatorship, about 1 per decade. This unstable period ended not with a power-sharing agreement but with the personalization of the regime under Saddam, who eliminate the collective action capacity of elites rather than share. Since the demise of Saddam Hussein, conflict returned with a series of civil wars mostly along ethnic lines.

This post reviews the primary reasons for Iraq’s continued civil conflict. I start by applying evidence from cross-national regressions, which suggest that armed groups are unusually easy to start and maintain in Iraq. Next I explain how the 2005 elections failed to resolve Iraq's powersharing problems.

1.1	INCENTIVES TO START ARMED GROUPS IN IRAQ

Starting a rebellion is hard, and continuing it is even harder. Rebel movements and militia present considerable organizational challenges Collier and Hoefler. Unlike the spontaneous protest movement, they require hierarchical organization and revenue to pay workers (especially when some workers may die in combat). Workers and investors must forego other activity to fund and participate. The rebel force must avoid total annihilation by the government, as the Chinese Communist Party did in the Long March. These severe practical constraints shape where rebellions and military disunity begin or do not. Imagine trying to convince your own friends/colleagues to join a rebellion!

Oil is the single most important factor for Iraq. In 2021 Iraq’s oil exports generated $ 75 billion, which paid 90% of the state budget. That is about $2000 per person and a stunning 45% of Iraq’s GDP. This is important because taxation is quite difficult in most fragile states; producers often live in dense informal urban communities, company registration is low, and tax compliance is spotty. Why would people pay taxes to a state that fails its most basic obligation of controlling violence? Primary commodity exports sidestep this problem because even a weak state can simply control ports or border crossings (this is why border crossings were hotly contested in Afghanistan). No wonder that regression analysis consistently find that primary commodity exports are the single most important influe...
Why learn to code?
by Tim Liptrot — 2022-04-05T04:40:46.071Z

I hear the phrase "learn to code" around the rat and postrat communities occasionally. From context I get the main argument, the income is good, the work isn't too difficult. Has anyone written a fuller argument for learning to code? Or what is your favorite version of the argument? I'm considering a career change and want to hear it out.
You get one story detail
by Gordon Seidoh Worley — 2022-04-05T04:38:36.022Z

So, as you know, you have about five words. That is, if you want to convey an idea to a bunch of people, you can only reliably communicate ideas simply enough to fit in five words.

I have a related theory.

If you want a lot of people to remember a story, you have to keep it tiny. How tiny? I'll explain.

If you're talking to someone one-on-one, you can go into great detail telling a nuanced story. They can ask questions and you can help them understand the deep motivations for why things happened. You can successfully convey a story with a lot of nuance so that they can retell it.

If you're talking to a small group you can tell a complex story. There's several of people so you don't have time to check with each of them explicitly at each story beat, but you can read the faces of your listeners and get a sense for what they are understanding. You can go into more detail when they look confused. In this way you can convey a complex story so each person could walk away and tell it to others mostly intact.

If you're talking to a big group you can tell a simple story. There's too many people for you to engage with anyone directly. You might still be able to check a few faces for confusion or understanding, but not all of them. And you now have a large enough audience that you can't trust that your audience has a sufficiently similar background to grasp all the details from a shared version of it. So you can convey only a fairly simple story that relies on tropes the audience is familiar with if you hope to have them retell it.

If you're talking to a huge group you can tell a tiny story. At this point you're probably not even going to get to see if anyone really understands or not, and certainly not in realtime. Your audience is literally or metaphorically obscured in darkness from you. If you want to tell the a story they'll be able to retell, it's going to have to be tiny. Five words tiny. You can get away with some connecting words, but your story has to have a very simple structure and not a lot of details.

Specifically, I think a tiny story can have at most one detail. More than that and folks are going to get lost. What's a detail? "Dog bites man" is one detail. "Man bites dog" is another detail. "Dog bites man, then man bites dog back" is a story with two details. I think a one detail story is the best you can do for a huge audience without details getting lost.

Why...
Down By 30
by Adam Zerner — 2022-04-05T03:29:39.264Z

Imagine that you are a play caller for a football team that is down by 30 points with, say, 8 minutes to go in the 4th quarter. What's your strategy?

Aggression. That's your strategy. Lot's, and lot's, of aggression.

Let's take a step back. What does a normal strategy in football look like, as far as play calling goes? On offense, it's a mix of passive and aggressive calls. You'll keep the defense honest with a few running plays. A few short passes. A few screens. These are all safe, passive play calls. They're likely to yield you a few yards, but unlikely to yield you a lot of yards.

This is a problem however when you're down by 30 with 8 minutes to go: if you pursue this strategy, even if you execute it successfully, you will lose. You will make slow but steady progress chugging your way down the field. It's probably enough time to score one touchdown. Two if you're very lucky. But when you're down by 30, you need four touchdowns. So no, this strategy will not cut it.

Ok, so what's the alternative? Simple: deep passes. You need lots of yards, and you need them fast. Throwing the ball far down the field to a receiver is the way to obtain that.

Note that this is scary, and perhaps counterintuitive. Such a deep pass probably won't be completed. It feels futile. Maybe there's a 95% chance that it is incomplete. Maybe it is futile.

On the other hand, the defense is probably willing to concede a nice, easy, 10 yard completion to you. Such completions are usually hard to come by. It feels very nice to make such completions. It feels like you are being productive. It feels like solid progress is being made.

Sadly, it is not. In the unfortunate scenario you find yourself in, there simply is not nearly enough time left in the game for such a strategy to work.

Oh, in addition to completing deep passes, you also probably want to conserve the clock. That would be helpful. Try to get it to your receivers in such a way where they could get out of bounds before being tackled. When that happens, the clock stops. When they are instead tackled in bounds, the clock keeps running, draining a very precious resource from you.

You might also want to try a few trick plays. Trick plays are an even more extreme version of the deep pass to some extent: they're even less likely to work, but when they do, they can yield a lot of yards.

One downside to trick plays is that you expose your hand, so...
Yudkowsky Contra Christiano on AI Takeoff Speeds [Linkpost]
by aog — 2022-04-05T02:09:22.096Z

Linkpost for “Yudkowsky Contra Christiano on AI Takeoff

Speeds

”, published by Scott Alexander on March 4, 2021. Here is the original conversation. Cross-posted on the EA Forum here.

Key Quotes:

Around the end of last year, Paul and Eliezer had a complicated, protracted, and indirect debate, culminating in a few hours on the same Discord channel. Although the real story is scattered over several blog posts and chat logs, I’m going to summarize it as if it all happened at once.

Core Debate

Paul Christiano's position:

Paul sums up his half of the debate as:

“There will be a complete 4 year interval in which world output doubles, before the first 1 year interval in which world output doubles. (Similarly, we’ll see an 8 year doubling before a 2 year doubling, etc.)”

That is - if any of this “transformative AI revolution” stuff is right at all, then at some point GDP is going to go crazy (even if it’s just GDP as measured by AIs, after humans have been wiped out). Paul thinks it will go crazy slowly. Right now world GDP doubles every ~25 years. Paul thinks it will go through an intermediate phase (doubles within 4 years) before it gets to a truly crazy phase (doubles within 1 year).

Why? Partly based on common sense. Whenever you can build a cool thing at time T, probably you could build a slightly less cool version at time T-1. And slightly less cool versions of cool things are still pretty cool, so there shouldn’t be many cases where a completely new and transformative thing starts existing without any meaningful precursors.

But also because this is how everything always works. [See explanations from Katja Grace and Nintil.]

Eliezer Yudkowsky's response:

Eliezer counters that although progress may retroactively look gradual and continuous when you know what metric to graph it on, it doesn’t necessarily look that way in real life by the measures that real people care about.

(one way to think of this: imagine that an AI’s effective IQ starts at 0.1 points, and triples every year, but that we can only measure this vaguely and indirectly. The year it goes from 5 to 15, you get a paper in a third-tier journal reporting that it seems to be improving on some benchmark. The year it goes from 66 to 200, you get a total transformation of everything in society. But later, once we identify the right metric, it was just the same rate of gradual progress the whole time.

...
Terminal Preferences
by jefftk — 2022-04-05T02:00:04.454Z

I do most of my coding on the command line, so: some views on
terminals.

I like having lots of terminals side-by-side, to see many different
things simultaneously. Most places I've worked have kept code to 80
columns, with two-space indentation, which is a good fit for my
approach. Here's how I usually have my terminals set up:

The monitor is 27", at 2560x1440 ("QHD"). This has room for five 83x101
terminals, though I usually divide my rightmost terminal vertically
for long-running commands or interactive prompts.

On traditional resolution (non-retina) displays, I have a strong
preference for bitmap fonts without antialiasing. They are much
clearer at a given size, which allows fitting much more text on a
screen. In iterm2 this means selecting Monaco Regular at 10pt,
and leaving the "antialiased" box unchecked, under Profiles >
Text.

At some point I could see moving to a higher resolution display, but
probably not until 5K monitors (my current resolution, but 4x the
pixels) are widely available and reasonably cheap. High resolution
displays are a bit nicer, but I don't mind the regular kind.

Aside from moving from Linux to Mac, and getting wider monitors, this
is a setup I've had for a long
time.

Comment via: facebook
[Cross-post] Half baked ideas: defining and measuring Artificial Intelligence system effectiveness
by David Johnston — 2022-04-05T00:29:16.992Z

This is a cross-post from my blog.

Artificial Intelligence safety researchers are concerned with the behaviour of “highly capable” AI systems. One of the challenges of this line of research is the fact that it’s hard to say, looking at today’s moderately capable AI systems, what highly capable systems will do or how they will work.

There are two intuitions that suggest understanding highly capable systems is particularly important for safety research:

Highly capable AI systems could be very useful, so we expect that if they can be built they probably will be
Highly capable AI systems raise much more serious safety concerns than less capable systems

This seems to make sense to me, but what does anyone actually mean by “highly capable”? I know Shane Legg and Marcus Hutter have had a go at understanding the related idea of “intelligence”: they collected many different definitions and proposed their own definition, which they informally summarise as

Intelligence measures an agent’s ability to achieve goals in a wide range of
environments

I think this is a good definition, especially the more precise version Legg and Hutter offer later in their paper. It also does a good job of explaining the first intuition: A system with high intelligence can achieve many different goals in many environments. This means that a system that is very intelligent can achieve malign goals in environments where people are trying to stop it from doing so. This means that we want to be quite sure that the goals a highly capable system is pursuing are not malign, which is one of the major concerns of AI safety researchers.

It can also account, to some extent, for the first intuition: a system that can achieve many different goals can, presumably, do a better job of achieving our goals than a less intelligent system. However, in this case there is a gap in the argument: we have to assume that, because the system can achieve some goal that it is likely to achive our goals. That is: a more intelligent AI system is more useful if that intelligence can be controlled.

This matters because AI safety research is trying to come up with comptetitive proposal for highly capable AI systems that are both competitive with unsafe systems and safe. My proposal is that competitiveness is not just a function of intelligence (per Legg and Hutter), but of effectiveness, which can be roughly understood as “controllabl...
Duels & D.Sci March 2022: Evaluation and Ruleset
by aphyer — 2022-04-05T00:21:28.170Z

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

There is a web interactive here where you can test your submission against your rival's deck (or against a couple other NPCs I added for your amusement, if you think yourself mighty enough to challenge those who wield the legendary power of the Sumerian Souvenirs).

NOTE: Win rates in the interactive are Monte-Carlo-d with small sample sizes and should be taken as less accurate than the ones in the leaderboard.

RULESET

Code is available here for those who are interested.

A game is played as follows:

There are up to 6 turns in the game.
Each turn, each player simultaneously draws 2 cards from their deck. (So over the full duration of the game, each player will draw their entire deck.)
Then, each player will simultaneously play either 0 or 1 of those cards:

Every card has a mana cost, ranging from 1 to 6.
On turn X, you can play a card that has a cost of X or less.
If both your cards are valid plays, you will play the higher-cost one.

For example, if you draw a 2-cost and a 4-cost card:

If it is Turn 1, you will play nothing.
If it is Turn 2 or 3, you will play the 2-cost card.
If it is Turn 4 -6, you will play the 4-cost card.

Any cards you didn't play are discarded - you will not be able to play them on later turns.
Note that:

You cannot play two cards in the same turn, even if you would in theory have enough mana to do so.
You cannot be clever about which card you play. In some cases a lower-cost card might be better than a higher-cost card: you will play the higher-cost one anyway if you can. If you draw two cards of the same cost, you will play one at random.

After both players have played a card if they can, each player calculates their Total Power by adding up the Power of all cards they have on their board.
If one player's Total Power exceeds the other's by at least 2000 Power, that player wins an immediate victory and the game ends.
If not, the game continues to next turn.
After 6 turns, if no player has won an immediate victory, the game ends anyway, and whoever has higher Total Power wins.
If players are tied on Total Power at this point, their cards clash in a mighty battle and the winner is decided randomly.

CARD LIST

Cards available to you were the following:

Card Name	Mana Cost	Power	A

...