trentmkelly/LessWrong-43k huggingface dataset:
Algorithmic Intent: A Hansonian Generalized Anti-Zombie Principle
by Zack_M_Davis — 2020-07-14T06:03:17.761Z

"Why didn't you tell him the truth? Were you afraid?"

"I'm not afraid. I chose not to tell him, because I anticipated negative consequences if I did so."

"What do you think 'fear' is, exactly?"

The Generalized Anti-Zombie Principle calls for us to posit "consciousness" as casually upstream of reports of phenomenological experience (even if the causal link might be complicated and we might be wrong about the details of what consciousness is). If you're already familiar with conscious humans, then maybe you can specifically engineer a non-conscious chatbot that imitates the surface behaviors of humans talking about their experiences, but you can't have a zombie that just happens to talk about being conscious for no reason.

A similar philosophical methodology may help us understand other mental phenomena that we cannot perceive directly, but infer from behavior. The Hansonian Generalized Anti-Zombie Principle calls for us to posit "intent" as causally upstream of optimized behavior (even if the causal link might be complicated and we might be wrong about the details of what intent is). You can't have a zombie that just happens to systematically select actions that result in outcomes that rank high with respect to a recognizable preference ordering for no reason.

It's tempting to think that consciousness isn't part of the physical universe. Seemingly, we can imagine a world physically identically to our own—the same atom-configurations evolving under the same laws of physics—but with no consciousness, a world inhabited by philosophical "zombies" who move and talk, but only as mere automatons, without the spark of mind within.

It can't actually work that way. When we talk about consciousness, we do so with our merely physical lips or merely physical keyboards. The causal explanation for talk about consciousness has to either exist entirely within physics (in which case anything we say about consciousness is causally unrelated to consciousness, which is absurd), or there needs to be some place where the laws of physics are violated as the immaterial soul is observed to be "tugging" on the brain (which is in-principle experimentally detectable). Zombies can't exist.

But if consciousness exists within physics, it should respect a certain "locality": if the configuration-of-matter that is you, is conscious, then almost-identical configurations should also be conscious

...
What are the mostly likely ways AGI will emerge?
by Craig Quiter — 2020-07-14T00:58:50.126Z

I'd rather not bias answers with my specific guesses, but what I'm trying to get at are bottom-up economic, engineering, and organizational realities-based answers. For example, if this question were asked in the year 2000 about the most likely way the next massive communities of people would form, answers might take into account the broadening adoption of the internet, the relative ease with which new web communities could be created with the LAMP stack, and that advertising would be a likely form of revenue. This may have predicted something akin to the social networks and provided some impetus for preventing societal manipulation through the targeted advertising that we saw in the 2016 U.S. elections. I realize predicting such a thing in 2000 would have been extremely difficult, but it feels like this is the type of question that if asked now, could help provide some guidance on how to create AGI safely.

I'll provide my own guesses in a few days after I've given some time for folks to weigh in.
How to persuade people/groups out of sunk cost fallacy?
by vernamcipher — 2020-07-13T22:51:50.860Z

First, a scope limiting definition: "The sunk‐cost fallacy occurs when prior investments instead of future returns influence decisions about future investments."

Does anyone on this forum know of psychology research around sunk cost fallacies and how to persuade people out of them? It seems straightforward enough to persuade people into them ("We've lost thousands of lives and spent billions of dollars already! If we end the [unwinnable] war now it will all have been for nothing!") but very hard to persuade people out of them.

If you are not aware of any psychological research, please feel free to provide ideas or anecdotes about your own success/lack thereof in persuading others out of their sunk cost thinking.
Life Through Quantum Annealing: How a quantum computing technique could shape existence
by ChrisM — 2020-07-13T22:48:23.018Z

Crossposted from Vessel Project. A couple of short video elements are embedded in the original article; in this post they are simply linked.

Overview

We explain how quantum annealing works and discuss how it could form our universe, drawing evidence from physics, cognition, and biology.

We then use a thought experiment to consider how a quantum annealing universe could benefit from being cyclic, or starting over, and we entertain the idea that such a model may even explain common human behavior.

Quantum Annealing: a search for the best state

Quantum annealing is a type of quantum computing used mainly for optimization problems; these problems can have many variables with many possible solutions, but we want to find the best solution. A classic example is the “travelling salesman” problem: Given a list of cities, what is the shortest route that visits every city and returns home? There are many possible routes, but not all are ideal.

A network of interacting quantum bits, or qubits, are used to solve problems in quantum annealing. A qubit represents a variable, having a “spin” of either up or down to indicate how that variable fits into the whole solution, like ones and zeros in binary code. But unlike other forms of computing, quantum annealing does not actively control the state of every variable. Instead, it allows qubits to naturally evolve based on simply programmed criteria.

Each qubit begins in superposition, but ultimately ends up with a defined spin — and entangled with all other qubits. Superposition means there is some degree of uncertainty to the spin; it can only be described by probability. Using an external magnetic field, a “bias” is applied to control how likely it is for a qubit to end up with a certain spin. “Couplers” are also used to determine how qubits affect each other and become entangled. If two qubits should always have the same spin — or opposite spins — a coupler is used to define that relationship. Over time, qubits become more entangled with each other, meaning they can no longer be described independently; only as a single state. Biases and couplers are the only ways to program a quantum annealer, which moves from an overall state of uncertainty to certainty, and non-entangled to entangled.

It’s important to understand quantum annealing in terms of ene...
3-P Group optimal for discussion?
by AiresJL — 2020-07-13T22:23:44.644Z

Assertion 1: Optimal size for a weekly discussion group is 3 and the optimal setting is in person

After 8 years hiatus on Lesswrong, i'd like to revisit lesswrong. In my time away, I have learned that categorically the best medium to learn something is through real human to human interaction bar none. Therefore, in order to catch-up with the latest body of knowledge in less-wrong, i'd like to ask for help.

Ask 1: Specifically, are there 2 volunteers that would like to meet 90 minutes once a week in-person in SF starting July/August to discuss 2 things:

1. What is an emotion in the context of information and decision theory?

2. Can we replicate emotion in AI?

These questions may seem stupid at first given the nature of the term "emotions," however in practical terms, GPT-3 used for NLP will fail any version of the turing test that analyzes whether the AI violates predictive emotional congruity. In fact, I would bet that all chatbots trained on just linguistic sequences will fail any and all emotional congruity tests.

-----------------------------------------------------------------------------------------------

***Emotions are a trick of language. Fundamentally there is an ordinal ranking of all actions a you make take at time t. The selection of an action constitutes either random selection or some ranking based on an objective function (more on this later) where you select the highest ranking action. These values associated with each action that necessitates an apple to apple comparison is effectively a vector v, that represents the relative value of an action. This vector is in fact the emotional vector that corresponds to the free-energy principle biological process proposed by Karl Friston in 2018. It manifests as what you feel although typically our attention sensor queue is only sensitive to high intensity emotions, emotional tensors t about an action a which as an effect of emergent complexity represents the entire spectrum of human thought. Through drop-out causation analysis we can radically reduce the solution space for the so-called mind-body philosophy problem.

TLDR; Let me know if you'd like to join a small (3 person) discussion group interested in emotions, decision theory and AI.

I'll also have friends drop-in such as Peter Dayan, Joscha Bach, Kenneth Stanley, Matei Zaharia and Jeff Clune.
Book Review: Fooled by Randomness
by Sherrinford — 2020-07-13T21:02:36.549Z

The book "Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets" by Nassim Nicholas Taleb was published in 2001. On Amazon, the book has a rating of 4.3, rated by 1,124 people, with 62% of reviewers giving 5 stars. On Goodreads, it has a rating of 4.07 from 1,968 reviews. "The book was selected by Fortune as one of the 75 'Smartest Books of All Time.'", notes Wikipedia. So this should be a great book. I forgot who wrote this, and I forgot the exact words, but I remember reading on twitter that Taleb is something like a modern genius because in his books he develops a worldview based on fundamental insights on randomness, and derives important conclusions including a system of ethics from that. Now that sounds promising! This worldview is developed in Taleb's Incerto series of five books. In the description of the German publisher you can read that the order in which you read the books does not matter, but then it seemed like a good idea to start with the first one nonetheless ("The Black Swan" surely is the better-known title and has 2,083 reviews).
The book seems to have been a revelation to many, judging by the enthusiastic reviews. In a five-star Amazon review, reviewer Alex Bush writes (November 3, 2015) that the book "revolutionized how I view the world. In multiple ways. It's hard to overstate how rarely a book changes your ideas about how the world works once, let alone multiple times". He thinks that Taleb "has managed to weave a fantastically engaging and entertaining book out of what could very easily be a dry and technical topic", that FbR is "the most general and therefore most widely applicable" book by Taleb, and suggests that it teaches people "the survivorship and hindsight biases, as well as the difference between conditional and unconditional probability". Indeed, these are things you may learn from the book in a non-technical way. You may also learn that people find causality where there is mere randomness; that the past of a time series can lead you to a false feeling of certainty if rare events are relevant; that it is often disregarded that the expected value of a random distribution may differ from the median, when in fact it would be critical to keep it in mind if possible rare events would have strongly negative effects. Thes...
Roll for Sanity
by  — 2020-07-13T16:39:38.902Z

[This is very much a personal-diary type post, but it ends up touching on predictive processing and other aspects of how our brains work. Feels related to Choosing the Zero Point. Cross-posted from Grand, Unified, Crazy.]

I. Looking for Trouble

In the card game Munchkin, there is a mechanic called “Looking for Trouble”, whereby if you haven’t yet fought a monster on your turn, you can play a monster from your hand and fight that. You don’t have to do this – it’s optional, and can carry stiff penalties if the monster ends up defeating you – but since killing monsters is one of the key ways to win at Munchkin, it’s an important mechanic.

Obviously you don’t want to fight a monster if you think that you’re going to lose. A brand new munchkin “Looking for Trouble” with a level 20 Plutonium Dragon is literally… looking for trouble [1]. And even if you think you might win, it’s often a good idea to wait a turn or two in order to try and collect more spells, stronger weapons, etc. It would be a pretty terrible Munchkin strategy to go looking for trouble on every possible turn, regardless of your equipment or which monsters you actually have in your hand.

And yet… this terrible strategy feels like a metaphor for my life recently.

Between work, personal relationships, and the chaos caused by the pandemic, I’ve been dealing with a pretty big set of stressors (monsters) already in my life. But like an idiot Munchkin, every time I’m not dealing with an immediate personal problem, I find myself Looking for Trouble. And the internet makes this soooooo easy.

Instead of taking a break, relaxing, and recharging my mental and emotional batteries, I find myself checking the latest coronavirus stats, seeing which of my favourite pieces of media have been cancelled, reading hot takes on the death of democracy, or just plain “doomscrolling” on social media. Unsurprisingly, I have not been at my best the last little while.

As best I can tell, this unfortunate behavioural pattern is a classic instance of predictive-processing gone awry. In other words, so much has gone wrong recently that my brain has decided the world must always be on fire, and that’s just the way things are. My subconscious is predicting disaster so strongly that when there’...
Null-boxing Newcomb’s Problem
by Yitz — 2020-07-13T16:32:53.869Z

The leering statue of the unnamed trickster god stared down at him from its sacred alcove, motionless as always.
Maxwell sighed. He had been praying at the base of the statue for the past hour, and like every time previous, he hadn’t felt anything. The priests had promised salvation from his fiscal woes if he would only pray for an hour a day at the local temple for two weeks, and now two weeks had passed, but nothing had happened. Maxwell rose wearily to his feet, recited the closing prayer quickly, and turned to go.
That was when he heard the voice.

“STOP WHERE YOU ARE.“

Maxwell whirled around, and found himself standing face-to-face with the leering stone statue, whose marble eyes were now glowing a dull red. It had stepped out of its alcove. Maxwell screamed, in accordance with the ancient and honored custom of screaming at creepy glowing statues that had moved right behind you.

“BE NOT AFRAID, MORTAL, FOR I HAVE COME IN ANSWER TO YOUR PRAYERS...ACTUALLY, I TAKE THAT BACK. BE MILDLY AFRAID, BECAUSE I AM ADMITTEDLY KIND OF CREEPY AND DON’T HAVE THE SAME UTILITY FUNCTIONS AS YOU.”

“What the heck...”

“I AM THE UNNAMED TRICKSTER GOD, COME TO DELIVER YOU FROM FINANCIAL SUFFERING.“

“Well thanks, unnamed trickster god, that’s just swell. So, uh... what’s the catch?”

“THE CATCH?”

“You’re a trickster god aren’t you? Don’t trickster gods always come with some sort of catch to their bounties?”

“OH YEAH, THANKS FOR REMINDING ME, I TOTALLY FORGOT ABOUT THAT.”

“Well it’s okay, you don’t have to—“

“NOPE, JUST MESSING WITH YOU, OF COURSE THERE’S A CATCH.”

“Ah well, I guess that is to be expected. So what is it then? I’m really desperate, so I’m willing to sacrifice pretty much whatever you need...”

“YOU ARE IN LUCK, YOUNG MORTAL, FOR YOUR REQUIRED SACRIFICE IS A CONDITIONAL ONE, WHICH INVOLVES MONEY YOU ARE NOT YET AWARE OF OWNING. I NOW PRESENT TO YOU THESE TWO BOXES.”

Upon saying this, the chest of the statue of the unnamed trickster god split open, revealing two black boxes inside a hollow cavity. The two boxes levitated soundlessly into the air, and hovered in front of Maxwell, slowly rotating. The leftmost box began to smoke, and the blackness dripped away, revealing clear glass. Inside the leftmost box sat ten crisp one hundred dollar bills. The rightmost box remained impenetrably black. Maxwell tentatively reached for the leftmost box.

“WAIT.”

Maxwell jerked his hand back, and w

...
2020 LessWrong Demographics Survey Results
by B Jacobs — 2020-07-13T13:53:47.700Z

Thank you to everyone who participated! You can see the original post here, but the survey is now closed. Here are the results:

Google forms results: https://docs.google.com/forms/d/e/1FAIpQLSfHJtg9ju6cCDl7T1PZvuBC9aIYIEN4Qb_MoQFPa9iMkUIFLQ/viewanalytics

Copy of the original survey (to view the questions in context): https://docs.google.com/forms/d/e/1FAIpQLSfBt1l3XBCZHx8DTJhWWTSK_bH7Tw6W4PdYYLKmMyplt1k0Ag/viewform

[Anyone wanting to revive the old (larger) annual LW-surveys can contact me if they need help/funding.]
What You Are
by Jarred Filmer — 2020-07-13T11:35:23.234Z

I wrote this after a bad trip some years ago, in which one of the challenges was a almost total loss of context for my experience. I'd hoped this would give me some kind of experiential ground to stand on if it happened again. If you have your own sequence of fundamentals to bootstrap an understanding of "what's going on" please share.
You Exist
Which means that you are rather than are not
You are Conscious
This means there is something that it is like for you to Exist
One imagines there is nothing that it is like to Exist as a rock
And perhaps something that it is like to Exist as a cat
Somewhere from rock to cat a subjective point of view develops, and there is Consciousness
You are an Agent
Which means you prefer, and act
To prefer is to have a valenced to your experience, positive and negative
To act is make change happen in ways that interact with your preferences
You are a Human
Humans are the best general purpose Agents we know of
Which means you can prefer and do a great many things
Update more slowly!
by Davidmanheim — 2020-07-13T07:10:50.164Z

My experience forecasting has led me to realize that a key mistake that is often made is updating on new data too quickly. This seems backwards, but I think that often the biggest reason that people both under- and over-react to evidence is that they don't consider the evidence clearly enough, and immediately start revising their model to account for the evidence, instead of actually thinking before updating.

Let's deconstruct how rapid updating is misleading with a simple notional example. Someone shows up with a coin and claims that she is psychic and can predict coinflips. You are skeptical, and challenge her to do so. She immediately takes out a coin and correctly predict heads 3 times in a row. You can update a few ways:

Conclude that the claim is now much more likely than before, and you give some credence to the idea that she is psychic
Conclude that she was lucky, and 7:1 odds is not very strong evidence, so you continue to hold on to your prior strongly
Conclude that she is cheating using a coin which has heads on both sides

Notice that these possibilities which spring to mind are completely different cognitive strategies. Either you update to believe her more, you decide the evidence is insufficient, or you immediately seize on a new hypothesis that explains the evidence.

But these are all mentally lazy strategies. If you thought about the situation for longer, you could easily generate a half dozen additional hypotheses. Perhaps the she has trained herself to flip coins a certain way. Perhaps she simply lied and said the coin landed heads each time, and didn't really give you time to see it well when it didn't. Perhaps she is substituting the coin as it lands. Perhaps, perhaps, perhaps.

My advice, per the title, is to slow down. You might decide to be a good Bayesian, and preserve multiple hypotheses, updating marginally - but doing this means that you assume the correct hypothesis is in your prior set. There are a million hypotheses that can explain a given set of events, and the most cognitively available ones are those that allow you to be lazy.

Don't worry, take your time. If the issue matters enough to bother trying to update your model, taking five minutes to reflect is better than jumping the gun. And if you don't need to make any decisions, at least file everything away and decide that it's unclear instead of qu...
In praise of contributing examples, analogies and lingo
by Adam Zerner — 2020-07-13T06:43:48.975Z

Examples, analogies and lingo all make posts easier to understand. So then, I want to encourage people to write more comments like:

X is an example of what you're describing.
Y might be a good analogy for this.
Z could be a good term for that.

I see comments like this sometimes, but much less frequently than I would like. And I suspect that readers often have thoughts like "Z could be a good term for that", but don't think it's worth saying. I think these comments are almost always worth saying.

Crowdsourcing

Examples, analogies and lingo seem like such great things to crowdsource. Even if the post author is intimately familiar with the post's content, I would expect 100 readers who are vaguely familiar with the content to do a better job of coming up with examples, analogies and lingo than the single post author. They're just the sorts of things that benefit a lot from a diversity of viewpoints, it seems to me.

Non-expert explanation

Let's say I'm wrong in the above section and the reality is that the post author is better than the crowd at coming up with good examples, analogies and phrases. Even so, sometimes the "good" example from the post author doesn't make much sense to a particular reader whereas a "mediocre" example from a commenter is what makes it click.

Scott Alexander does a great job of talking about this concept in his post Non-Expert Explanation.

Post authors don't always do a good job

Coming up with good examples, analogies and lingo is hard! There's a lot of times when I have an idea in my mind that I want to write about but I get stuck because I have trouble coming up with good examples, analogies or lingo. There are also times when I post it anyway, knowing that I haven't done a good job with those things.

I doubt I'm alone. I doubt that every time someone clicks "Submit" on a post that they think they've done as good a job as possible in the examples, analogies and lingo they've used.