trentmkelly/LessWrong-43k huggingface dataset:
The different kinds of confidence
by masasin — 2018-09-30T14:16:06.806Z

Hello! This is my first ever LessWrong post!

The following is extracted from a conversation I had with DaystarEld on Facebook, and I have his permission to share it on LessWrong.

tl;dr: Personal skills you can be confident about can be broken down into three types:

Absolute skill: Are you good at what you do, relative to the general population?
Relative skill: Are you good at what you do, relative to other people who do the same thing?
Instrumental skill: Are your skills applicable to this situation? Is it part of the reference class, or do you need to learn/try something else? Should you ask someone else, or try and handle it yourself?

DaystarEld asserted that personal confidence is not just about your own belief in your abilities, but also your perceptionn of common knowledge of your capabilities.

DaystarEld: I've had experiences where I was confident in my own competence, but was not sure those I would be working with were aware of it, or whether they would value the things I was competent in, and this effectively served to make me hesitant and reserved until I received enough positive signals from those around me, and then felt more comfortable acting with agency and status.

The more I think about it, the more I wonder about how generalized this is. Because clearly there are people who act confident but do not have their competencies acknowledged by those around them, or just have no reason to believe that it is yet.

He wondered if someone who is well-calibrated as to their ability would be considered overconfident or arrogant if they acted confidently without social signals of respect.

DaystarEld: I'm curious to know if there another word for being overconfident in how your abilities are perceived by others, regardless of how accurate your own perceptions of your abilities are. And is it a GOOD thing to not act confident when you lack social signals of common knowledge? Or should people care more about their own calibration of their abilities than what others think? I can see cases for both, but on a purely what-gets-best-results level, how valuable have others found it to ramp up your confidence with each individual or group of people you interact with, opposed to starting at the point that you feel you most deserve?

I reasoned that, assuming I'm well-calibrated and confident in my ability, others not acknowledging your skill shouldn't really make a difference in...
What is being done?
by marcus_gabler — 2018-09-30T13:20:49.518Z

Hello,

my name is Marcus, I live in Germany.

I figured out a lot of the knowledge shared here by myself over the years and now reached a point where I just had to start working on my own website before I bust from all those ideas wanting to go out.

I tried to find out if such content already existed but first failed. I finally discovered this and a few similar websites yesterday, after Spencer Greenberg of www.clearerthinking.org answered my desperate email and pointed me to "the rationalist movement".

Bang! Finally I wasn't alone anymore, and there is even an LW meetup in my home city Frankfurt!

Now I am hoping to find more answers here.

Apparently, there are no websites / organisations similar to LW, EA or CFAR (Jeeez! I can't believe I'm already using these shorts that were 100% mysterious to me yesterday) in Germany. Does that really mean I am the spearhead of world saving here once I go live???

Anyway, I wonder what is being done to give impact to "the cause"?

The rational (sic!) things to be done seem to be:

Building a website and make "How To" videos that will spread the (boiled down and simplified) word to the widest possible audience
Start online petitions to implement rationalism into regular school education
Ask Bill Gates for a few billions in donations
Found a political party (OK, probably no use in the U.S., but certainly doable in Germany)

I am just thinking out loud here.

Or has this already been done and failed?

What is actually MOVING in the rationalist movement?

Who has an overview of whats going on?

Happy to be here and anxious about feedback! :-)

Take care, Marcus.
Am I understanding the problem of fully updated deference correctly?
by Davide_Zagami — 2018-09-30T10:55:58.965Z

I understand that one solution to AI alignment would be to build an agent with uncertainty about its utility function, so that by observing the environment and in particular us, it can learn our true utility function and optimize for that. And according to the problem of fully updated deference, trying to accomplish this would not significantly simplify our work because it involves two steps:

1) Learning our true utility function V (easy step)

this "merely" consists of knowing more about the world (if a perfect description W of the universe, which contains our true utility V somewhere in it, could be fed into the agent, then this step would be complete)
even a permanently misaligned agent would try to learn V for instrumental reasons (e.g. fooling us into thinking it is on our side)

2) Actually optimizing for V (hard step)

this requires that its meta utility function looks at its model of the world W, and uses a hardcoded procedure P that reliably points to the object that is our utility function V (if P is misspecified, the agent ends up optimizing V′≠V)
P is a probability distribution over utility functions that depends on W (that is, it's a way for the agent to "update" on its own utility function)
there is no "universal" way to specify P (an alien race would specify a different P, which means they would specify a different update rule, whereas e.g. the rules of Bayesian updating would be the same from star to star)
specifying P may be easier than specifying V, but is still hard (e.g. it may require that the programmers know in advance how to define a human so that the agent will be able to find the human objects in its world model and extract their V)
the agent would oppose attempts to modify P, much like agents who are certain about their utility function oppose attempts to modify it (the problem reproduces at the meta level)
EDT solves 5 and 10 with conditional oracles
by jessicata — 2018-09-30T07:57:35.136Z

Introduction and motivation

The starting point for this post is a comment I wrote on Paul's post on EDT vs CDT:

One argument for CDT over EDT that you didn’t mention in this post: Suppose you live in a deterministic universe and know your own source code. Suppose you are deciding between taking a 5 dollar bill and a 10 dollar bill. Suppose your world model says you take the 5 dollar bill with 100% probability. Now conditioning on taking the 10 dollar bill gives you complete garbage, since you are conditioning on a probability 0 event. If you use EDT, then depending on other details of your decision procedure, this could lead to you always taking the 5 dollar bill. So then your world model would be accurate. (This is the “5 and 10” problem often discussed at MIRI; I don’t know if it has been written up anywhere)

CDT never generates undefined expected utility estimates like EDT does. It takes the 10 dollar bill in this problem. However, if it always takes the 10 dollar bill, then its counterfactual for taking the 5 dollar bill is strange because it is one in which a physical law is violated. The violation of a physical law could have important consequences other than which action the agent takes.

Both decision theories have trouble with this problem, but at least CDT always produces a defined answer.

Here’s another way of thinking about this problem. A fully Bayesian version of EDT must construct all possible worlds and then condition on taking a certain action. But each of these possible worlds contains a running copy of the EDT algorithm. So, absent some defined method for taking a fixed point, this leads to an infinite loop, and you can’t actually have a fully Bayesian version of EDT.

(What if you use reflective oracles to allow EDT to select some fixed point? We could specify that the reflective oracle returns arbitrary results when asked to condition on a probability 0 event (I think this is what the most natural way to emulate conditional queries on a reflective oracle results in, but I haven’t checked). Now there are multiple possible reflective oracles (i.e. fixed points); it’s possible to always take the 10 dollar bill and think bad things will happen conditional on taking the 5 dollar bill, and it’s also possible to always take the 5 dollar bill and think bad things will happen conditional on taking the 10 dollar bill.)

A fully Bayesian version of CDT mu

...
bumping into financial unknown unknowns 
by brown — 2018-09-30T02:39:50.079Z

subtitle: "I think there are a few black boxes in my professional life as well."

Current problem: my finances are a little tight these days.

How I imagine personal finances is kind of like a stream. I need as much water coming in as going out. I'd like to create more value for others than I consume [0]. Thing is, when you're starting out; "how the hell do you create value for random strangers?" I get the feel I'm lacking in models.

The standard template seems to be convince some large company you're worth their salt then figure out how to tolerate it. Eg “get a job” [1]

A slight aside,
I've run a meetup for two years. It was mostly hanging out with rationally-inclined uni friends & eating college funded food.

I've found myself in the position sometimes when talking to someone else about how they might also run a meetup. They say words to the effect of "I'm not pro enough to do that" or "I don't think I can do it". This can sometimes lead into self-doubt, but from my perspective it's feels like: "you're great, you just don't know how"

It's been said before [2] that "running a meet-up" is too big a task. And instead:
*I’m going to hang out at this place with some food & people can come chill if they want

But maybe also, observe someone else running a thing & ask questions.

I’ve watched my father start multiple small businesses, an ea friend get uni funding and other people writing lesswrong comments & posts. Being nearby someone allows me to gradually acquire their models until: "hey maybe I could do that”. [3]

So given this is how some humans react to the unknown, what are the things that I think I cannot do?

I'm pretty sure I've got similar thinking around "I couldn't run a business". The boxes are more, "how do I find some users & notice a problem they need fixing", "how do I find out if this is a valuable thing for people without wasting their time / attention", "how do I get clients" which I guess becomes "how do I build trust" [4] and so on.

And how might I gently try on the hypothesis of “maybe I'm missing some models here" instead of either "I'm not allowed / not capable enough to run a business" [5][6]

Now to figure out how I can bump into / and acquire these models.

[0] And capture some of it! :)

[1] On not getting a job as an option

[2] how to run a lesswrong meetup handbook

[3] And here we are.

[4] Critch: deserving of trust

[5] Nate’s: confide

...
Case Study II
by abstractapplic — 2018-09-30T00:37:32.974Z

Introduction and motivation

My last excruciatingly-thorough analysis was well-received. Since I posted it, I’ve thought of – and had pointed out to me, h/t johnswentworth – some more tricks I could have tried. And as luck would have it, the person who provided the last dataset has released a similar one. So I’ll be taking this as my cue to run another analysis, with emphasis on everything I failed to consider the first time around.

The dataset

Nicky Case ran a poll asking fans which of four educational projects fans thought they should work on next: topics were `web` (web design), `write` (science writing), `stats` (unconventional/simplified statistics), and `learn` (learning about the process of learning). They also asked – putting aside the level of interest in the projects themselves – how interested people are in the skills these projects would aim to teach. The projects and the skills were ranked from 1 to 5 stars by each respondent.

The poll was run simultaneously on Twitter and Patreon, creating two datasets (Case suspected that these populations might behave in non-negligibly different ways). Results were clear and unambiguous: people wanted to learn about learning. So this time, there’s no tie-breaker to hunt for, and I have no pretext or starting point other than “root around for things Case and/or LW might find interesting”.

Exploration I

I decided to start with the Twitter data. I checked for missing records, set a seed, split the dataset into equally-sized exploration and test datasets, and started poking around in the exploration set.

[First mistake to correct: nonstandard nomenclature. These concepts are hard enough to communicate under the best conditions, and when people start getting sloppy with naming conventions, it gets much worse. (I have personally wasted over a week of my life because people who should know better used a nonstandard definition of GINI)

So, contra my first analysis, the data subset you don’t use until the end of the project is the test data, not the verification data. Similarly, the thing I kept referring to as purity last time is more commonly called not overfitting. Mea culpa on both counts.]

My first step was to eyeball the distributions of all eight variables on their own. They were all highly right-skewed, and had roughly the same shape. More importantly, they were all visibly non-G...
Criticism Scheduling and Privacy
by Lulie — 2018-09-29T13:50:17.501Z

Scheduling criticism

"Two times two is four ‘tis true
But too empty and too trite
I would rather seek a clue
To some matters not so light."

— W. Busch, Schein und Sein (Karl Popper's translation)

Being in control of when, how, and what you get criticism on is a vital part of the growth of knowledge.

Scheduling criticism isn't being closed-minded. It's the only way to deal with the fact that there is an infinity of possible criticism, and no mechanical way to make that infinity smaller. You need some way of choosing what is useful.

The self-development idea of 'growth mindset' says: learn from criticism, don't ignore it.

There is an exaggerated version of this idea, namely: "Because criticism is not actually bad and in fact helps you learn, criticism is always good (or at least neutral)."

The grain of truth in this is: Identifying errors gives you access to the problems in your ideas. Any criticism could help. Finding conflicts between your ideas is the first step to solving them. A criticism on its own, as an idea in the ether, is only information — and this is information that can be used to make progress.

So why isn't criticism purely a good thing? How could noticing errors be bad?

Well, it's true that someone might have an irrational aversion to making mistakes (perhaps after having been penalised in school for getting things wrong), which causes them to be averse to even noticing mistakes.

But if we got over this, and didn't take criticism personally, wouldn't criticism just be a gift?

No. Criticism, when unwanted, can effectively destroy the means of error correction and the growth of knowledge. Put differently: it can structurally cripple thinking.

When I say "unwanted", I'm not referring to emotional reactions to criticism. One can feel bad about criticism by taking it personally — and it is possible to avoid this kind of bad feeling, by learning to take criticism better — but that isn't what I'm talking about here. That's a personal hangup, whereas I'm pointing at something more fundamental.

Criticism is how we learn

Criticism is part of how we learn. We have a problem we're trying to solve or a question we're trying answer, we come up with potential answers, and we criticise those answers to narrow them down or polish them. Once we have a promising solution, we criticise it further to understand it better and to test that it actually works as a solutio...
other people are occasionally rather weird
by brown — 2018-09-29T02:25:05.019Z

So here's a dynamic I find myself in sometimes:

I've just explained a problem to a friend / family member and they offer advice that feels plain wrong.

Normally this leaves me either:

trying to somehow make the advice fit (it doesn't)

or deciding maybe this person is not as great as I thought they were (They’re pretty great so I’m not crazy keen to do this either)

Here's hopefully, pointing at a third alternative.

What might be going on:
When I hear another persons problem. What this brain tends to do is run a simulation of my friend in that situation. Yet, the complexity of my human friend is much greater than my simulation of them. There's bound to be a bit of compression.

I've heard, how humans store faces is that we have a default face template. And then to minimise storage cost a new friend's face is stored as lots of "deviations" from that default [1].

I'd be pretty surprised if something similar isn't going on with my models of other people. When I imagine a friend, they're basically me but with a few tweaks. Oh yeah, Sally doesn't like noise or John's real tidy.

Here instead are some intuitions behind other people being super weird:

Many folks showering habits are super strange. From The Typical Sex Life Fallacy, Thing of Things:

People may take anywhere between five minutes and forty minutes to shower. They may wash their hair daily, once a week, or not at all. They may wash their bodies thoroughly, only clean the parts that look dirty, only clean certain parts (such as the armpits or genitals), or just stand under the water. They may use a loofah, a sponge, or nothing. They may bring in a comb to comb out the conditioner. They may sing. They may zone out. They may jerk off. They may bathe instead, and bathing may involve reading a book or bath bombs or lighting candles and drinking a nice bottle of wine or bubble bath or none of those things at all. The one thing that is consistent is that everyone thinks the way they shower is the way normal people shower.

what common universals are you missing?
Synesthetics often don't realise everyone else isn’t seeing the colours too. People don't realise they're colour blind until they see (or don’t see) one of those number patters.

When you move in with friends, you find all of these funny rules. Oh you put the salad dressing in the fridge, "you recycle that?" etc.

Turns out that most people's typing strate

...
Code Switch
by Elo — 2018-09-29T00:32:18.302Z

Original post: http://bearlamp.com.au/code-switching/

Related to: Hedging

Code switching is this useful linguistics concept that is not overtly known enough.

The type that I care about is a cross between "group identity" and "lexical need":

Group identity: People may alter their language to express group identification. This can happen, for example, when introducing members of a particular group to others.[33]

Lexical need: People often use some technical terms or words written in another language. In that case, if people try to translate those words, that might distort the exact meaning and value of the word or term. In this case, code switching occurs to maintain the exact meaning of the word.

I care about cultural code switching. But also a very particular problem where words in one "code" start to not be capable of describing the nature of the informational transfer. Where words start to break down.

"have you tried turning up the volume on your attention?"

My recent investigations have shifted from the scientific structural rational playing field to the spiritual mystical enlightenment playing field. If I had lexical problems before, they hit an all new level where each person's lexical map is just comfortable being independent to each other person's map.

How do I describe a wordless experience? How do I transmit, record, encode or remind myself - what is the difference between "attention", "awareness" and "focus"?

Awareness is broad and soft. Attention is sharp, focus is the pointy part of attention. If I think of focus as a lens pointing light to a single dot, that's focus. Awareness is a broad focus, but objects are not blurry per se, just taken to be as a holon, not as an individual.

And on it goes, with energy, awareness, emotions, colours, spirits, powers, understanding, confusion and so many many more codes.

The benefit of code switching, over say - rejecting someone for using certain words, is that it's way more inclusive. If I reflect internally that maybe I have misunderstood their code, then ask for clarification - I'm much more likely to treat them as a steelman than a strawman. Take charitably what is said and from there... grow.
Three Levels of Motivation
by DragonGod — 2018-09-28T10:48:27.572Z
Link: https://docs.google.com/document/d/1JLx9Hef3iVZb1YDEa7bz_eQbkblYwpK-jocb6KlEwT0/edit?usp=drivesdk

Epistemic Status: This is basic information, and while it is original to me, I make no claim that the the content thereof is new. Nevertheless, I think it is important enough to be worth posting.
Law without law: from observer states to physics via algorithmic information theory
by avturchin — 2018-09-28T10:07:30.042Z
Link: https://arxiv.org/pdf/1712.01826.pdf

TL;DR: Nothing exists except random data strings and Solomonoff induction between them.

There are papers that you dream you will write yourself one day, when you have plenty of time. As the idea is so crazy, you don’t expect that anyone else will dare to even consider the topic. But then, you find that someone already wrote it—and did it better than you ever could hope to do—using exactly the words and examples that were percolating against the background of your consciousness. It could be called the “bias of illusionary capability”, or, as my friend Lisa used to say: “I don’t believe in lost opportunities”.

Muller created a compelling—likely not true, but still exciting—theory of a world where nothing exists. Only two things are postulated: that there are strings of binary numbers, called observer states, and that Solomonoff induction could be used to calculate probability that one string will be next to another. The magic of this paper is that all of the main properties of the observable universe are derived from only these two assumptions, via around 80 pages of dense math. The author works in three academic institutions, the paper was supported by grants, and cites Bostrom, Sandberg and S. Armstrong. Lee Smolin at least participated in discussions leading to its publication, according to the acknowledgements. The ideas discussed within it possess some similarities to the idea of Boltzmann simulations recently discussed by Armstrong.

He wrote:

"1.To every sequence of binary strings x = (x1, . . . , xn), there is an associated “observer state” that can be interpreted as the first-person perspective of an observer that has previously experienced x1, . . . , xn−1 and experiences xn now.

2. From this first-person perspective, another experience y is going to come next, with probability (chance) of P(y|x; A), where A = (V, E,Λ) is a complete observer graph with Λ = x1, V = {0, 1} ∗ and E = V × V .”

And conclusion:

"• Instead of assuming the existence of a “physical world”, we have postulated that the fundamental laws act directly on the level of observers (or rather of their observations).

• In the simplest case, there is only one such law, determining the probability of future observations, given all of the observer’s current (and possibly past) observations. We have argued that this should correspond to some kind of “universal apriori probabil...