trentmkelly/LessWrong-43k huggingface dataset:
Deconfusing Deception
by J Bostock — 2022-01-29T16:43:53.750Z

What does deception look like from the outside? I notice I am confused.

Imagine you have some algorithm which learns to predict sensory inputs, like a human. The internal structure of this will come to correspond to the external world in some way, as a part of generating a predictive model. Imagine the algorithm is walking around a building, looking at the objects in it. Most of the objects are ordinary, made-of-atoms objects. The algorithm learns to predict where the objects are, how they interact, and basically gets very low input-prediction error.

Then it comes across a teapot. The teapot isn't a real object, but in fact a hallucination projected onto the sensory arrays of the algorithm by a daemon. At first, the daemon only has to project some visual data. But then the algorithm picks up the teapot, so the daemon must project some tactile data.

For a "successful" deception, the algorithm must not "notice" that their purely-physical model of the world is being violated.

When the algorithm decides to make a cup of tea, the daemon must not only fake the sensations of a full, brewing pot of tea, but hide the sight of the water and teabag falling onto the floor. Later, the algorithm falls over by slipping on the spilled water, so the daemon must from then on fabricate all the sensory data that the algorithm gets.

Or, the daemon could fabricate some other reason for the sensory data. The water on the floor could have leaked in from a crack in the ceiling. For this to work, the algorithm must have already been uncertain about whether there was a crack in the ceiling or not, and the daemon must have known this.

This highlights two ways you can deceive a learning algorithm. Both involve control of all the information flowing between a system (in this case the whole world) and an algorithm. One requires modelling of the system, the other requires modelling the algorithm.

How else can we look at this? It seems like if there's a daemon making a fake teapot, the process of moving information world→model(world) has a discontinuity around the part of the algorithm defining a model of the teapot. The reason that a real teapot leads to a model of the teapot in the algorithm follows a different-looking causal chain to the reason a daemon leads to a model of the teapot in the algorithm.

This discontinuity defines a region which grows as the fake object interacts with the...
The Map of the Map
by LightStar — 2022-01-29T10:03:13.527Z

The map is not the territory.
Our understanding perception of reality is not fully accurate. This is important to understand. Not being aware of it, assuming that you are percieving reality as it is when in fact you aren't, is a source of error.
But our understanding of our model of reality is not fully accurate either. Not being aware of that is also a source of error.

Reality is the territory.
Our mental model of reality is the map.
What we think our mental model of reality is, is the map of the map.

We have mental models of other people's minds. We also have a mental model of our own mind. It is not fully accurate.

Specifically, I want to talk about concepts.
A word/concept has two different meanings, what we think it means to us (map of map), and what it actually means to us when processed by our brain (map). (while the similarity cluster of reality that the concept is meant to capture would be the territory).

Or, a different way to put it, a concept has two different interpretations depending on which mental machinery we use to process it, and yet we aren't automatically aware of that.

As you can imagine, this can cause all kinds of errors.

Motte and bailey
Non-central fallacy
Hidden inferences
Positive/negative emotional connotations of words

Tabooing helps. What do you do when a word doesn't mean to your brain what you think it means, or when you are aware its meaning is somehow ambigous/contradictory/misleading? Remove the word from the equation entirely, and replace it with actual meaning you want to use.
Simply saying, "I define 'human' to mean 'featherless biped'" is not good enough - actually remove the word entirely, in order not to contaminate what you are saying with whatever meaning our brains already have attached to the tabooed word.
Mildly Photochromic Lenses?
by jefftk — 2022-01-29T02:30:06.232Z

After wearing glasses for about half my life, I decided to try

photochromic
lenses. These darken when exposed to sunlight, as if I had put on
sunglasses. It's pretty neat!

I got them in early summer, and I have mixed feelings. It was
nice to be able to hang out in really bright sunlight without needing
to squint or shade my eyes. This has never been something that
bothered me very much, and I never used to wear sunglasses, but it was
nice to have a bit less light.

On the other hand, as I wore them more, it was starting to feel a bit
depressing. Colors were not as bright, and the world seemed less
alive. Especially as we got farther into winter I was really
preferring the view around the edges of my glasses. I decided to get
an otherwise identical pair, but with regular lenses—it's pretty
great how buying glasses online has made them so much cheaper that I
can get an extra pair without feeling extravagant!

So far I like the regular glasses better, but I think I would prefer
photochromics in really strong sunlight? Is there such a thing as
mildly photochromic lenses, which stay clear on a typical day but darken
in intense sunlight?

Comment via: facebook
Book notes: "The Origins of You: How Childhood Shapes Later Life"
by juliawise — 2022-01-28T20:00:56.056Z

Cross-posted from Otherwise, where I'm writing these days.
Other book note posts on books about children: The Anthropology of Childhood, Don't Shoot the Dog
The book: The Origins of You: How Childhood Shapes Later Life by Jay Belsky, Avshalom Caspi, Terrie E Moffitt, Richie Poulton
[edit: see comments for useful caveats; I probably wasn't skeptical enough of the book while reading.]

Main idea

The “dual-risk hypothesis” appears in a lot of their research: “an underlying or latent 'vulnerability' that puts one at risk of developing a problem, but according to this hypothesis, the problem only arises when a particular stress is experienced or encountered.”

A lot of the findings in the book have some element of this – some risk factor (situational or genetic) is associated with worse outcomes only when it’s combined with some other risk factor (usually related to parenting: insecure attachment, or harsh or inattentive parenting.)

Personally my takeaway from this book is that warm, attentive parenting, especially in the first three years, is protective against a number of bad outcomes. And it’s under my control in a way that some other things are not.

About the book

I’m not covering most of the findings, just the ones I happened to take notes on.

The authors talk a lot about methodology. At first I loved this – instead of just saying “here’s a finding” they break down a bunch of research decisions they had to make. You definitely come away with a picture of why it’s hard to do good social science research.

But I was less interested in this as the book went on – each chapter was largely about the research process, and didn’t get to findings until 2/3 through each chapter. After a while I just started skipping to the findings in each section.

People have strong opinions about Belsky, the main author, and he might need to be taken with a grain of salt. He says people don’t want to listen to him because he’s taken a politically incorrect stance that daycare has disadvantages. Others say he’s overstating the conclusiveness of his research because of his own ideology.

They follow three major observational studies:

Dunedin Multidisciplinary Health and Development Study, observing all the 1037 children born in a New Zealand town in 1972, following them from age 3 and still doing followups.
the Environmental Risk Study, following 2232 twins born in England and Wales
the Na

...
Advertising as a Parasite on the Semantic Commons
by ThomasMore — 2022-01-28T17:38:36.340Z

‘Tis the season to be bombarded with adverts for useless crap you don’t need or want. But it's not Christmas you say? Correct: it is always the season to be inundated with adverts. Continuing the tradition of ad-bashing on LessWrong, I want to pick up on a particular 'bad of ad': cultural poisoning.

I find ads infuriating and depressing. But not just any ads, the worst offenders are those which start off with some wholesome scene, tugging at the heartstrings, only to transition to selling shoes or a car. Why? I think it’s because they poison our cultural environment. The notion of ‘cultural poisoning’ is, however, a little unclear, and I want to give it some more shape using the concept of the Semantic Commons.

Semantic Commons

Shared concepts can only suffer so much misuse in a community before their meaning changes. Consider the sad fates of ‘literally’, ‘actually’, ‘in reality’, 'liar’, and ‘girthy.’ These terms have either lost their force; had their meanings altered to something different; or had unpleasant connotations grafted on. Now they are useless for the wider community.[1]

It seems the frequency; visibility; and vividness of the misuse all contribute to this process. And many times it is deliberate. Take, for example, the strategic expansion of situations considered ‘rape’ for political purposes. Scott Alexander conceptualises this as the semantic commons: the more users ‘take’ from the commons and use a concept to make a point, the more they risk denaturing and diluting its meaning.

Thermodynamics of Cool

Adverts are possibly the heaviest users of the semantic commons. Many (though not all) adverts rely on linking existing terms, concepts, and shared experiences in a community to the products they sell.[2]They thereby abuse what my father calls the ‘Thermodynamic Laws of Cool.’ Take an advert which shows Ceilidh dancers drinking Coca Cola. Ceilidhs are cool because they are fun, social, sincere cultural events. Coke less obviously so – it’s a sugary drink produced by a multinational conglomerate. Advertising Gurus know Coke seems cooler if it is associated with the spontaneous dancing of the Ceilidh so they make adverts showing Ceilidh dancers drinking Coke.

But the Thermodynamic Laws of Cool suggest the total amount of coolness in a system is constant: for every bit Coke seems cooler, Ceilidhs seem...
Newcomb's Grandfather
by Yair Halberstadt — 2022-01-28T08:56:53.417Z

In front of you are two boxes.
Box A contains $1000.
If Omega predicted you would just take Box B, Box B contains $1,000,000.
If Omega predicted you would take both boxes, Box B contains nothing. Omega will also go back in time and kill your grandfather.
Nobody has any idea how accurate Omega's predictions are.
Does the fact that you exist prove that Omega thought you would just take one box, so Box B definitely contains the million dollars, and if you took both boxes it would imply that Omega was a terrible predictor so you might as well just take both boxes and run?
2020 Review: Final Voting
by Raemon — 2022-01-28T03:39:59.606Z

Click here to begin voting. Click here for a general overview of the 2020 Review.

We're now in the final week of the LessWrong 2020 Review. We've had over a month of preliminary voting and discussion. Now it's time to finalize our votes. In this post I cover how to vote, why to vote, and what will happen with the outcome (in no strict order).

The winning posts will be assembled into the longterm annals of the Best of LessWrong[1]. This year, each top-rated post will be displayed with a donation button for its author, so readers can directly show their support for posts that were important to them.

The Best of LessWrong: where we remember the greatest contributions

Quick Recap on The Review

The LessWrong Review has two major goals.

Improve the LessWrong community's longterm feedback and reward cycle.
Build common knowledge about the best ideas we've discovered on LessWrong.

Voting sends a signal about which posts were most important. This can feed into various compilation and distillation efforts.

Who Can Vote?

All users registered before 2020 can vote. The LessWrong curation team will weight the votes of users with 1000+ karma more highly when assembling sequences, books, or other projects.

How do I vote?

Go to the Review Voting page. There, you'll see posts that you haven't yet voted on sorted first, with posts you've previously given a karma-vote to sorted to the very top (they have a green stripe along the left side. Strong upvotes have a darker green).

The posts will include the reviews that got written about that post. This is intended to help you make an informed vote. You can read reviews that look interesting or highly upvoted to get a better sense of how the post held up.

You vote by clicking buttons that assign a post a score. A score of 1 means roughly "this post was good." A score of 4 means "this post was quite important". A score of 9 means "this post was extremely important." A vote of 0 means "I don't have a strong opinion."

(A vote of "4" is 4x as strong as a vote of "1", a vote of "9" is 9x)

If you intuitively mark posts as "good/important/extremely-important", you'll probably do fine expressing your votes.

Okay but I'm a nerd and like knowing the details of things. How does this work under the hood?

The system is built on top of a quadratic voting system. In quadratic voting, you have a limited number of points to spend. You can vote thing...
How A Highly Sensitive Person Sees Words
by LightStar — 2022-01-28T03:16:10.876Z

Preface: I wrote this post partly to try and capture/demonstrate/introduce/explain a concept that I haven't found an existing explanation on that would match my mental model. And partly to clear up my own confusion about the topic and put my thoughts in more precise words.

So what this is post going to be about?
I guess I would put it like, "how a highly sensitive person sees words, and communication in general". An exploration of some hidden-complexity behind words that already exists, but that maybe we aren't that good at paying attention to.

I'll start this as a dialogue, with some imaginary person introducing thoughts that are partially-flawed, then exploring how they are flawed.

"Martin Luther King is a criminal!"

This is a misleading thing to say. Martin Luther King was a prisoner of conscience, arrested for breaking what he saw as unjust laws in the course of his civil rights work. If by criminal we mean 'anyone who has ever has broken a law', then yes, Martin Luther King is a criminal, but we'll have to concede he is a 'good kind of criminal'

"And what if I genuinely believe that breaking the law is always bad? That there is no such thing as 'a good kind of criminal'?"

Just to be clarify, I assume you don't actually think that, for example, jaywalking is as bad as murder?

"They are not equally bad, but they are both bad. Both of them are crimes, and both a jaywalker and a murderer are criminals"

This might be Fallacy of Gray? Stubbing my toe is bad. Extinction of the human species is bad. There is a reason we typically use different words for concepts with vastly different magnitudes. I wouldn't say that stubbing my toe is a 'catastrophe', or that the extinction of the human species is 'a bit of a downer'.

"You are being disingenuous. There are plenty of general, vast-encompassing concepts. A 'celestial object' can mean an asteriod, a planet, a galaxy, a nebula, or anything in between."

Perhaps it's more important to use specific rather then generic concepts when talking about political, "sensitive" or emotionally-charged topics? For example, the concept of "a person who has ever done anything bad, whether it's stealing a plastic bag or genocide" might be too muddled to be useful. Even if you don't intend it to, the lack of precision in the terminology you choose to use contributes to a confusing and misleading message.

"I don't think there is anything misleadin...
The Skill of Writing Facetiously
by lhc — 2022-01-27T22:44:10.356Z

TL;DR Tone can function like (im)precision. So work on the ability to choose its variation instead of some arbitrary thing choosing it for you.

The ability to write in a facetious tone is a wonderful addition to one's writing toolset, equivalent to the ability to use fewer significant digits. This is a separate feature from the features of "fun to read" and "irreverent". People routinely mistake formalese for completeness/rigor, and the ability to counter that incorrect inference in them is very useful.

At some point in Eliezer's post against bioanchors, he roleplays Open Philanthropy as saying:

OpenPhil: We have commissioned a Very Serious report on a biologically inspired estimate of how much computation will be required to achieve Artificial General Intelligence, for purposes of forecasting an AGI timeline. (Summary of report.) (Full draft of report.) Our leadership takes this report Very Seriously.

Here's Holden's reply to that specific bit:

It's true that Open Philanthropy's public communication tends toward a cautious, serious tone (and I think there are good reasons for this); but beyond that, I don't think we do much to convey the sort of attitude implied above. The report's publication announcement was on LessWrong as a draft report for comment, and the report is still in the form of several Google docs. We never did any sort of push to have it treated as a fancy report.

The tendency towards mistaken impressions from formalese is common knowledge (at least, after this post). So "We never did any sort of push to have it treated as fancy" becomes about as defensible as "We never pushed for the last digits in the estimate 'There are 45,321 hairs on my head' to be treated as exact", which... is admittedly kinda unfair to Holden. Mainly because incorporating a writing style is a much harder execution than replacing some digits with zeros.

Eliezer is facetiously pointing out the non-facetiousness in Open Phil's report (and therefore doing some of the facetiousness-work for them), and Holden solemnly points out the hedged-by-default status of it, which... is admittedly kinda fair for Holden to do. But I'm glad this whole exchange (enabled by facetiousness) happened, because it causes the hedged-by-default-ness to be emphasized.

(He did say they have good reasons for a serious tone. And I do believe there are good reasons for it,...
A little game theory on discursive conflict
by David Hugh-Jones — 2022-01-27T21:19:42.286Z
Link: https://wyclif.substack.com/p/a-little-game-theory-on-discursive

[Slightly updated version here.]

There’s a really widespread belief in the social sciences that discourse matters, that language and narrative isn’t just a transparent medium through which we view the world, but also something that creates winners and losers, and is therefore fought over. Michel Foucault is the very obvious name to check here, but I don’t think the belief is widespread just because it was espoused by Foucault. I think it’s the other way round: Foucault is influential because he was really on to something. Even if you don’t drink the postmodernism kool-aid, “discourse” is just too popular a concept to be completely mistaken.

Economists have not traditionally paid much attention to language and narrative, but that’s been changing. Some big names have written papers and books, either saying “hey, maybe narrative is important, we should think about it”, or actually trying to think about narrative in the framework of an economic model. This is a bit ironic, because the essence of formal modelling is that you throw out all the thick description and reduce everything to narrative-free mathematics. But OK, fine, economists have to bring their unique approach to bear.

So, I’ll sketch how one might think about discourse from a game-theoretic point of view. This is a “paper I’m too lazy to write”, or probably one that’s too trivial to write: I’m no economic theorist, it’s a simple idea, make of it what you will. I’ll focus on discursive conflict — the idea that people might have conflicting interests, and there might be resulting struggles, over how things are described.

Here’s a classic 2 x 2 game: the Battle of the Sexes. Like all such games it has a cute story attached.

The story is: Jim wants to go to the boxing, and Jane wants to go to the ballet. (Game theory developed in an age of gender stereotypes.) But both of them would like to go with the other person. Jim chooses an action in the rows; Jane chooses an action in the columns. The numbers are Jim’s payoff, then Jane’s payoff. So, if they both go to the boxing, Jim gets 2 and Jane gets 1. If they both go to the ballet, Jim gets 1 and Jane gets 2. But they both prefer either of these alternatives to the sad situation where they go to different events alone.

There are two equilibria here: both go to the ballet, or both go to the boxing. Those are equilibria because neither person unilatera...
What happened when I applied metrics to my piano practice (a five-part essay about the process of learning)
by Nicole Dieker — 2022-01-27T20:45:06.367Z

Crossposted from Nicole Dieker Dot Com. The third and final part in my piano practice series, as promised.

If you follow me on Twitter —

and I’m not at all sure if you should, since Twitter seems to be less valuable to me every successive day —

which in turn makes me less interested in

adding value to it

—

but anyway, if you read the tweets, you might have seen this one:

Nicole Dieker @HelloTheFuture

I'm going to write more about this tomorrow, but last night I told L that I had discovered "the secret to learning" and he told me that I was correct, except for the part where I was the first person to discover it

January 26th 2022

2 Likes

Over the past month I have, in fact, discovered the secret to learning — and even if I am not the first person to make this particular discovery, it still counts.

Now I have the somewhat difficult task of telling you what it is.

Here’s how I explained it to L:

Define win condition.
Define action you are going to take to achieve win condition.
Take defined action.
Evaluate action both against its original definition (that is, did you do what you said you were going to do or did you do something else) and against the win condition.

If you’re me, write down the results. If you’re L, keep them in your head. (He keeps all of this in his head. I have no idea how his head can handle it. He told me that he might have more storage space in his working memory because he thinks of things in bits and symbols instead of words.)

Ask yourself what is keeping you from achieving your win condition. Describe it as specifically as possible.

If you’re me, write it down.

Define action you are going to take to solve/address/eliminate obstacle preventing win condition.
Repeat 2-7 until win condition is achieved.
STOP.

Step #8 — STOP AFTER WIN — is more important to the process than I originally realized. At first I assumed that once you hit WIN you could REPEAT WIN, maybe REPEAT WIN 5 CONSECUTIVE PASSES, but it doesn’t work that way.

Once your brain hits WIN, it’s done with that particular problem for that particular practice session. Successive passes during the same practice session are more likely to be unfocused fails, which introduce inconsistencies that have to be resolved by running additional (time-consuming, frustrating) learning loops.

Plus, STOPPING AFTER WIN sets up a work environment in which you are CONTINUALLY RETURNING TO WON SEQUENCES.

I feel like I just c...