trentmkelly/LessWrong-43k huggingface dataset:
The Meta-Puzzle
by DanielFilan — 2021-11-22T05:30:01.031Z

This post was inspired by the Raymond Smullyan book The Riddle of Scheherazade.

This post contains two puzzles: first, a puzzle to warm up, then a meta-puzzle about the first puzzle. I strongly encourage you to give your best shot at the first puzzle—it is achievable, and the rest of the post will spoil the solution. I also encourage you to try the meta-puzzle before reading my solution.

The puzzle

This puzzle is set in a far-off, mystical island called “Australia”. In Australia, everyone either worships God or Satan. The God-fearing only speak the truth, while the Satanists always lie and never speak the truth. Furthermore, in Australia, both Satanists and the God-fearing can either be married or single.

While in Australia, you meet a stranger. You don’t know who this stranger worships, or whether the stranger is married or not. The stranger says a sentence, and once you hear the sentence, you are convinced that the stranger is single, without needing to check for proof. What could the sentence possibly have been?

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Solution to the puzzle

Take the sentence “I worship Satan, or I am single”—where “or” is used in the sense that both could be true. No Satanist could say such a sentence, because if a Satanist said it, it would be true. Therefore, the person who spoke it must have been God-fearing. Since the person is God-fearing, they do not worship Satan, so they must be single, since otherwise the sentence would be false.

The meta-puzzle

All of this is very similar to how normal logic works, but in some ways very different. First, note that in Australia, a sentence is true if and only if the person saying it worships God. So, wherever you see “I worship Satan”, you can substitute “This sentence is false”, and wherever you see “I worship God”, you can substitute “This sentence is true”—all the reasoning will go thru the same.

Now, let’s think about how logic works off the island. I, the author of this post, am currently in California, one of the united states of America—far away from Australia. Suppose I say to you “This sentence is false, or I am single”. You could reason thusly:

If the sentence is false, then the sentence is false or he is single—but then the sentence is true, because that’s what the sentence says. Therefore it’s true. If the sentence is true, then “this sentence is false, or Daniel is single” is true—but “th

...
Some real examples of gradient hacking
by Oliver Sourbut — 2021-11-22T00:11:35.047Z

Speculation and an invitation to make further suggestions or clarifications

I've been wondering what might count as an analogous example of gradient hacking if any has ever occurred, and if it could be useful to collect such analogies (while mindful not to overfit our expectations). Presumably so far the only examples will be for genetic natural selection (perhaps particularly in humans), and perhaps cultural/memetic selection (leaving aside the technicality of whether natural selection has a gradient per se).

I'm looking for deliberate behaviours which are intended to affect the fitness landscape of the outer process.

In biology

Sexual selection

I think examples of sexual selection are adjacent but don't qualify as gradient hacking. While they constitute an execution of a policy whose main legible effect is indeed on natural selection, presumably in nearly all cases there is not a deliberate process of influence on the natural selection process as such. The policies themselves are encoded by genetic natural selection for the most part, so I'd say this qualifies more as one kind of divergent training trajectory of the outer training process. There remains a plausible analogy that at least some animals and humans engaging in sexual selection are doing so 'because' they are inner misaligned (they are agents with preferences which are mere proxies for the 'true goals' of genetic natural selection and sexual selection is just one such proxy preference).

Examples of sexual selection where the preferences are socially learned seem closer, and there are examples in animals and humans. Even in this case, even though the policy is adapted 'at runtime' (the genetic policy might be something like 'choose a mate which my society codes as attractive', which incidentally presumably encourages other genetic policies like 'try to conform to my society's attractiveness code' and maybe even 'try to nudge my society's attractiveness code towards me and my kin') it is still not carried out with the runtime deliberate intention of affecting the outer process of natural selection (even though it is a deliberate action which does in fact have this side effect, the intended goal is the proxy).

Eugenics

Perhaps the first and only qualifying attempts at actual gradient hacking are efforts towards eugenics practiced by humans, up to and including some instances of (attempted or successfu...
Vitalik: Cryptoeconomics and X-Risk Researchers Should Listen to Each Other More
by Emerson Spartz — 2021-11-21T18:53:47.638Z

5 years ago, Ethereum co-founder Vitalik Buterin wrote an important but little-known blog post that deserves more visibility, so I'm posting it here. This way, it'll also cross-post to The Nonlinear Library, where it'll be available in podcast form.
Inspired by Vitalik, I just set up an X-risk + Crypto Discord - let's cross-pollinate together!

Also inspired by Vitalik, I'm putting out a $100 bounty for any post on the EA Forum/Alignment Forum/LW that cross-applies ideas from crypto/web3 to x-risk that gets at least 25 upvotes, subject to my discretion. Alternatively, you can email me at emersonspartz@nonlinear.org and if your ideas are interesting enough I'll still pay out the bounty.

Semi-unrelated, but if you're interested in this kind of stuff, Anna Riedl and I set up a Telegram for EAs into Complexity and Systems Science. We think this is another valuable interdisciplinary combination.

---
Why Cryptoeconomics and X-Risk Researchers Should Listen to Each Other More

Vitalik Buterin

Jul 4, 2016·6 min read

Special thanks to Jaan Tallinn for early feedback and comments.

There has recently been a small but growing number of signs of interest in blockchains and cryptoeconomic systems from a community that has traditionally associated itself with artificial intelligence and various forms of futuristic existential risk research. Ralph Merkle, inventor of the

now famous cryptographic technology

which

underpins

Ethereum’s

light client

protocol, has

expressed interest

in DAO governance. Skype co-founder Jaan Tallinn

proposed

researching blockchain technology as a way to create mechanisms to solve

global coordination problems

. Prediction market advocates, who have long understood the potential of prediction markets as governance mechanisms, are now looking at Augur. Is there anything interesting here? Is this simply a situation of computer geeks who were previously attracted to computer-geek-friendly topic A now also being attracted to a completely unrelated but also computer-geek-friendly topic B, or is there an actual connection?

I would argue that there is, and the connection is as follows. Both the cryptoeconomics research community and the AI safety/new cyber-governance/existential risk community are trying to tackle what is fundamentally the same problem: how can we regulate a very complex and very smart system with unpredictable emergent properties using a very simple a...
Giving Up On T-Mobile
by jefftk — 2021-11-21T16:50:05.212Z

I'd been thinking of moving away from T-Mobile since their
depressingly cavalier account messaging a few weeks ago where

they sent
what were effectively "your account is in the process of being hacked"
notifications as part of migrating to a new billing system.

This billing migration continues to be poorly handled, and after a
series of conflicting messages I no longer have service:

2021-11-02: As part of your new My T-Mobile experience, your Autopay
feature needs to be re-setup prior to your next bill cycle date. To
avoid a service interruption, please login and re-setup your Autopay
at [url]

2021-11-18: Please refill your plan within the next 3 days to ensure
continued T-Mobile service at [url] or press 611. Please
disregard if you've already paid for next month's service.

I tried to log in and update my billing information. The service was
extremely slow and it timed out after I entered my credit card
information. I thought I probably hadn't successfully enabled
autopay, but I wasn't sure.

2021-11-18: Your T-Mobile plan will renew in two days via AutoPay
with an estimated charge of $30.00 before taxes/fees/surcharges. Thank
you for being a T-Mobile customer.

I took this to mean that my credit card change had actually gone
through, and I was all set.

2021-11-20: Please refill your plan before 12:01am tomorrow to ensure
continued T-Mobile service by visiting [url] or dial
611. Please disregard if you've already paid for next month's service.

I received three of these, one after another. This was surprising, but
it did say to disregard if I had already paid. I thought there was a
good chance that my service was going to be cancelled, but at this
point I was grumpy enough that I figured that if they cancelled my
service that was fine with me.

2021-11-20: Auto-Pay is cancelled. Your next T-Mobile plan payment
will not be paid automatically. To re-enroll, call *233 or visit
[url].

2021-11-21: Your T-Mobile service has been suspended. To restore
service, contact your Primary Account Holder or make a payment at
[url].

None of this is that bad, but it was frustrating enough that I decided
to move away. I care about cost, hassle, and the chance that my
account is stolen, and Google Fi (disclosure: I work for Google,
speaking only for myself) seemed like a solid option. The marginal
cost...
From language to ethics by automated reasoning
by Michele Campolo — 2021-11-21T15:16:20.304Z

Posted also on the EA Forum.

Introduction

In

Naturalism and AI alignment

I claimed that we might get an aligned AI by building an agent which derives its own preferences (a partial order over world-histories) from its knowledge of the natural world.

Now I am working on a formal (i.e. mathematical or algorithmic) model of that agent. When AI capabilities will reach human level in natural language understanding, the formal model will help turn a natural language AI system that understands the world at human level of intelligence into an AI that is able to reason about ethics and possibly do EA-style research.

Here, I elaborate on the idea that an AI which reasons like a human might be aligned. The basic argument is that:

Some humans act and think ethically
If we understand the causes of such behaviour and replicate them in AI, we get AI that is roughly aligned. If we also manage to eliminate some human cognitive biases, we should get AI that is more robustly aligned.

Factors underlying altruistic behaviour in humans

At least some humans act and think in ethical or altruistic terms. To explain why this happens, we can consider the different factors that may lead to such behaviour.

Positive and negative first-person experiences: if humans couldn’t perceive any kind of pleasure or pain, survival in general would be significantly harder, let alone making considerations about others’ subjective experiences.

Theory of mind

: without it, humans probably could not reason about what is good or bad for others.
Empathy: in addition to theory of mind, humans can also experience a positive or negative response depending on what somebody else is feeling. Empathy incentives harm reduction and makes positive emotions spread.
Social and cultural drives: culture influences what actions are perceived as good or bad; and humans, as social animals, tend to adjust their behaviour accordingly, in order to be seen as good persons by the people around them.
Game-theoretical reasoning: one can end up acting altruistically and cooperatively simply because doing so provides the greatest benefit to themself.
Moral reasoning: some humans classify actions as good or bad according to a system of beliefs held to be true. Two examples: religious people often rely on a set of dogmas that guide their decisions during life; some philosophers wonder “Given what I know about the world, what matters most?”,

...
Split and Commit
by Duncan Sabien (Inactive) — 2021-11-21T06:27:07.557Z

This is an essay describing a basic sanity-inducing mental movement that I use approximately ten times per week, and suspect other people would benefit from adopting (and regularly reminding each other to do). I've talked about it elsewhere, but until now it didn't have its own linkable reference post.

P1: Things are often not what they seem.

This can be because the seeming itself is broken, e.g. the progression that goes:

"Well, here I am in 1700's America and it sure seems to me that these black folk are fundamentally intellectually and morally and spiritually inferior to white folk" or "Well, here I am in 1800's America and it sure seems to me that these women are constitutionally incapable of holding political office" or "Well, here I am in 1900's America and it sure seems to me that these gay men are an active threat to everyone's safety, including our women (somehow) and children" or "Well, here I am in 2000's America and it sure seems to me that ████ ██ ██████ ██ ████ ████████████ █ ███ ██ ██████ ██ ████ █████ ████████ ██ ████ ███ ████ ████ ██████ ███."

(Or "Well, here I am inventing physics and it sure seems to me that force consistently and exactly equals mass multiplied by acceleration.")

It can also be because the seeming is generally correct, but there exists variance. If 95% of the marbles in a bag are red, and 5% of the marbles in the bag are green, it's correct to guess that the next randomly selected marble will be red, because betting on red makes you the least likely to be wrong. But that doesn't mean that it can't be green, or that you should be shocked if it's green. If you bet on red the whole way through the bag, you know you'll be wrong five percent of the time.

P2: Every observation is evidence in favor of more than one hypothesis.

If my friend tells me my hair looks good, this is an update in favor of my hair looking good and an update in favor of my hair not looking good but my friend lying to me for various reasons (kindness, prank). In order to know which update is larger, you need some other information about what kind of person my friend is, and what our relationship is like.

If I see some breaking news about some person having maybe done something horrible, this is an update in favor of that person having done something horrible, and an update in favor of that person being the target of a smear campaign/co...
What’s the weirdest way to win this game?
by Adam Scherlis — 2021-11-21T05:18:57.831Z
Link: https://adam.scherlis.com/2021/11/14/whats-the-weirdest-way-to-win-this-game/

You and three other people draw cards from a deck and hold them to your foreheads (so that everyone else can see your card, but you can't). You all guess the suits of your own cards simultaneously. You all win if *at least one* person guesses correctly.

You can all agree on a strategy before drawing cards. What do you do?

(Warning: Spoilers! The post is mostly about finding weirder / more general solutions.)

Plus:

A very incomplete introduction to group theory
Strong opinions about algebraic structures
The worst theorem in math
A stained glass window
Eat the cute animals instead
by Andrew Vlahos — 2021-11-21T01:06:54.408Z

Note: I was trying to criticize the hypocrisy of people who treat farm animals in ways they would never treat pets. It seems people missed this.

Here's a 150% serious explanation of why eating cats and dogs is a great idea! A lot of people reading this have no problem eating cows and chickens, but for some odd reason become squeamish when the meat is cute. It's time to put aside your inhibitions and start going to animal shelters instead of grocery stores. Here's why:

Better for the environment

Growing crops, then feeding them to livestock, uses a lot of land and water. Cats and dogs, however, are so common that animal shelters are overwhelmed. In fact, they are invasive species which are destroying the ecosystem. Capturing and eating cats and dogs would help the environment, while raising cows and chickens only hurt it.

Resists factory farming

Factory raised animals are in horrible conditions.

Meanwhile, there is no comparable factory farming operation for cats and dogs, so you don't need to worry about supporting this (if you don't pay extra for "purebred" status).

Fewer animals get eaten

Cats and dogs are obligate carnivores, meaning that they need to eat meat to live. Leaving a cat or dog alive means that many other animals get eaten. If you think predation is wrong, why aren't you reducing it?

They're delicious

They even consider themselves delicious!

Why haven't you started already? If you would eat cows and chickens but not cats and dogs, what would it take to change your mind?
Chris Voss negotiation MasterClass: review
by VipulNaik — 2021-11-20T22:39:19.756Z

This post is about the negotiation MasterClass taught by Chris
Voss
as well as a related MasterClass Session by Chris
Voss
and more broadly about the techniques and worldviews shared in
those. I watched the original MasterClass in April 2020 and watched
the Session in December
2021.

My post discusses some of the relationship -- similarities and
differences -- with rationalist advice. These portions should be of
particular relevance to LessWrong and are part of the reason for
posting to LessWrong.

This is a fairly lengthy post, so please use the sections to navigate
to the portions you are most interested in.

I cover a lot in this review:

What is this a review of?

Why am I doing this review?

Worldview

Negotiation principles

Negotiation techniques

Other ideas

General concerns

ETA 2021-11-24: I added a few more subsections and made some edits to
existing content. You can see the change set
here.

ETA 2021-11-25: I added a subsection in response to a point raised in
the comments. You can see the change set
here.

ETA 2022-01-02: I expanded and restructured the review to cover the
MasterClass Session that I consumed in December 2021. The original
post, written in November 2021, did not cover the Session. You can see
the change set
here.

What is this a review of?

Who is Chris Voss?

Christopher Voss
worked at the FBI as a lead hostage negotiator. In 2008 he left to
form his own company, The Black Swan
Group, that offers coaching to
people on negotiation techniques. Their customers include people in
real estate and many other kinds of business.

What is MasterClass?

MasterClass is a player
in the growing edutainmment space, combining education and
entertainment. There are 100+ "classes" in MasterClass. Each class is
taught by a subject matter expert (the teacher), and includes several
videos (ranging from 2 to 30 minutes, usually 5 to 20 minutes). The
typical video format just includes the teacher speaking to the camera,
but there are occasionally interactive sessions with other
participants, or other co-instructors.

Course notes can be downloaded. There are also community features on
MasterClass.

You may have seen video ads for MasterClass on YouTube!

What is MasterClass Sessions?

In late 2021, MasterClass introduced
Sessions, a more interactive
form of MasterClass that is designed a bit more like a course. A few
differences between traditional MasterClasses and sessions:

...
The Maker of MIND
by Tomás B. — 2021-11-20T16:28:56.327Z

After my first rebirth - my mind labile, full of that indescribable sense of renewal, everything new again, everything fresh, everything tinged with a subtle hilarity - I assisted Bartosz Sumner, now rather well-known in some parts as the author of The Making of MIND.

Bart had lived many more lives than I, and he was spending his latest life studying the events that occurred during his first.

MIND, as you know, was created on June 8th, 2034. Bart was only 31. I asked him once about his first-hand memories of the event.

He told me, “It happened so fast there is not much of a story. One moment, I was writing Javascript at the office, and next thing I knew I was informed by MIND that it was now in control, that I no longer needed to work to justify my existence, and that if I would like I could keep programming but for the sake of my mental health it highly recommended I forget Javascript ever existed.”

I then asked him what Javascript was, and he told me he could not answer that, as he claimed he had taken MIND's advice.

I had just spent a rather hectic decade playing the villain in a fantasy server (standard evil wizard trying to monopolize magic storyline). I like to think I did a good job of it, perhaps a bit hammy but, well, that's half the fun, isn't it? When I was assassinated by a subordinate who had somehow disabled all my resurrection spells, several heroes PMed me afterward and told me I was the best villain their server had had so far.

I suppose I was upset about my death being a bit anticlimactic (I was hoping to, at least, be taken out by a hero), but I was also kind of relieved. Living in a story is exciting, but it can get exhausting after a while.

Several friends offered me juicy roles in some sims they were starting up, but I guess I was getting tired of stories and magic.

I was having the cliche "first-life crisis" the old always rib the young about. And if I was going to be cliche, I felt there was little point in mincing around it, so I asked MIND for a rebirth, and a quiet place where I could learn to surf and meet some people who weren't into the fantasy sim scene. For its own inscrutable reasons, it suggested San Adrastea and a beach house two doors down from Bartosz Sumner.

Rebirth is a bit of a misnomer in that you do not become a fundamentally new person. I am told by people who care about such things th...