trentmkelly/LessWrong-43k huggingface dataset:
10 posts I like in the 2018 Review
by Ben Pace — 2020-01-11T02:23:09.184Z

I see basically every post that gets submitted to LessWrong, whereas many users come in and read things more occasionally, so I thought I'd list 10 posts I like in the 2018 Review that people might have missed.

I wasn't able to write quick reviews of each of them, so this is more like a list of nominations. I've left out a few posts that I expect will naturally be very popular.

Naming the Nameless by Sarah Constantin

This points to a variety of specific mechanisms and gears by which culture and aesthetics affect our judgments and our choices, and combines them into an essay that walked me much further than I was before-hand in noticing these effects. I think it might be in my top 5 posts of 2018.

Explicit and Implicit Communication by lionhearted

This has some powerful arguments about when not to say things out loud - when not to make background assumptions explicit - which I think is a really powerful datapoint for a rationalist to take on board. And the anti-Nazi 'Simple Sabotage Field Manual' is amazing.

Challenges to Christiano's capability amplification proposal + Paul's research agenda FAQ by Eliezer Yudkowsky and zhukeepa (respectively)

These together are maybe the first time I really understood what Paul's ideas were. Really helpful. I'm mostly talking about the Eliezer-Paul dialogue, both on Eliezer's post, and in the comments of Alex's post. I appreciate zhukeepa putting in the explanatory work post as a necessary step for that dialogue to continue.

Varieties of Argumentative Experience by Scott Alexander

So. Many. Examples. This is a strong step forward in conceptualising arguments and disagreements, written by someone who's read and been involved in an incredible amount of good (and bad) ones on the internet.

Unrolling social metacognition: Three levels of meta are not enough. by Academian

This very clearly lays out the iterative process by which social emotions and attitudes are built up. Foundational for a lot of social modelling.

Act of Charity by Jessicata

A thoughtful dialogue throwing into contrast a lot of key intuitions around deception, self-deception, and ethics.

Public Positions and Private Guts by Vaniver

This post says a lot about how formal communication ignores many messy parts of what it's actually like being a human, that helps show how what communication needs to be changes at scale.

Optimization Amplifies by Scott Garrabrant

One of the most imp

...
Is there a moral obligation to respect disagreed analysis?
by km42 — 2020-01-11T01:22:25.550Z

I wish to perform some action which invites material risk upon myself and another person P, as well as inviting benefits. My personal evaluation is that the benefits far outweigh the risk, but the risk is both speculative and partially subjective so I decided to consult P before taking the action.

P was vigorously opposed to the action and gave his reasoning, a list of factors that he claims suggests the risk is much higher and the benefits much lower. Some of the factors given were of the inherently subjective variety, such as feeling proud of the current status quo.
Afterwards, I sat and thought about these factors and still reached my initial conclusion that the benefits far outweigh the risk. Furthermore, not only can I perform this action unilaterally but P will not even be aware I have performed this action unless one of the Bad Outcomes (which in my evaluation are exceeding unlikely) occur.
If the risk were greater than the benefit I would not take this action but P failed to convince me this is the case.

My question:
If my analysis is correct, with overwhelming probability not only will P be unharmed by my actions but also be entirely unaware of them. With this in mind, do I have a moral obligation to return to P and argue my case and obtain consent before performing my action?
A method for fair bargaining over odds in 2 player bets!
by Agrippa Kellum — 2020-01-11T01:18:23.185Z

Alice and Bob are talking about the odds of some event E. Alice's odd's of E are 55% and Bob's are 90%. It becomes clear to them that they have different odds, and being good (and competitive) rationalists they decide to make a bet.

Essentially, bet construction can be seen as a bargaining problem, with the gap in odds as surplus value. Alice has positive EV on the "No" position for bets at >55% odds. Bob has neutral or better EV on the "Yes" position for bets at <90% odds.

Naive bet construction strategy: bet with 50/50 odds. Negative EV for Alice, so this bet doesn't work.

Less naive bet construction strategy: Alice and Bob negotiate over odds. The problem here, in my eyes, is that Alice and Bob have an incentive to strategically misrepresent their private odds of E in order to negotiate a better bet. If Alice is honest that her odds are 50%, and Bob lies that his odds are 70%, so they split the difference at 60%, Bob takes most of the surplus value.

If both were honest and bargaining equitably, they'd have split the difference at 72.5% instead. So I'll call 72.5% the "fair" odds for this bet.

A nicer and more rationalist aligned bet construction strategy wouldn't reward dishonesty! So, here it is.

1. Alice and Bob submit their maximum bets and their odds.

2. Take the minimum of the two maximum bets. Let's say its $198.

3. Construct 99 mini bets*; one at 1% odds of E, 2% odds of E... 99% odds of E. Each player automatically places 2$ on each mini bet that is favorable according to their odds ($198/99 = $2).

*99 chosen for simplicity. You could choose a much higher number for the sake of granularity.

So, in this case, Alice accepts the No position on all bets at =>55% odds, and Bob accepts the Yes position on all bets at =<90% odds, so they make 35 $2 bets, the average odds of which are 72.5%, which is the fair odds.

Observe that there is no incentive for either player to have misrepresented their odds. If Alice overrepresented her odds as 60%, she would just deny herself the ability to bet on bets at 56% through 59%, which have positive EV for her.

Note that Alice and Bob only bet $70 -- less than half of the maximum bet. If Bob wanted Alice to bet more money than she was really willing to risk, he might try to convince her that his odds were close to hers, such that a high maximum...
Of arguments and wagers
by paulfchristiano — 2020-01-10T22:20:02.213Z

Automatically crossposted from ai-alignment.com

(In which I explore an unusual way of combining the two.)

Suppose that Alice and Bob disagree, and both care about Judy’s opinion. Perhaps Alice wants to convince Judy that raising the minimum wage is a cost-effective way to fight poverty, and Bob wants to convince Judy that it isn’t.

If Judy has the same background knowledge as Alice and Bob, and is willing to spend as much time thinking about the issue as they have, then she can hear all of their arguments and decide for herself whom she believes.

But in many cases Judy will have much less time than Alice or Bob, and is missing a lot of relevant background knowledge. Often Judy can’t even understand the key considerations in the argument; how can she hope to arbitrate it?

Wagers

For a warm-up, imagine that Judy could evaluate the arguments if she spent a long enough thinking about them.

To save time, she could make Alice and Bob wager on the result. If both of them believe they’ll win the argument, then they should be happy to agree to the deal: “If I win the argument I get $100; if I lose I pay $100.” (Note: by the end of the post, no dollars will need to be involved.)

If either side isn’t willing to take the bet, then Judy could declare the case settled without wasting her time. If they are both willing to bet, then Judy can hear them out and decide who she agrees with. That person “wins” the argument, and the bet: Alice and Bob are betting about what Judy will believe, not about the facts on the ground.

Of course we don’t have to stick with 1:1 bets. Judy wants to know the probability that she will be convinced, and so wants to know at what odds the two parties are both willing to bet. Based on that probability, she can decide if she wants to hear the arguments.

It may be that both parties are happy to take 2:1 bets, i.e. each believes they have a 2/3 chance of being right. What should Judy believe? (In fact this should always happen at small stakes: both participants are willing to pay some premium to try to convince Judy. For example, no matter what Alice believes, she would probably be willing to take a bet of $0.10 against $0.01, if doing so would help her convince Judy.)

If this happens, there is an arbitrage opportunity: Judy can make 2:1 bets with both of them, and end up with a guaranteed profit. So we can continuously raise the

...
Book review: Rethinking Consciousness
by Steven Byrnes — 2020-01-10T20:41:27.352Z

(Update September 2024: My thinking has developed a bit since 2020. See [Intuitive self-models] 2. Conscious Awareness for my own Rethinking-Consciousness-adjacent theorizing.)

Princeton neuroscientist Michael Graziano wrote the book Rethinking Consciousness (2019) to explain his "Attention Schema" theory of consciousness (endorsed by Dan Dennett![1]). If you don't want to read the whole book, you can get the short version in this 2015 article.

I'm particularly interested in this topic because, if we build AGIs, we ought to figure out whether they are conscious, and/or whether that question matters morally. (As if we didn't already have our hands full thinking about the human impacts of AGI!) This book is nice and concrete and computational, and I think it at least offers a start to answering the first part of that question.

What is attention schema theory?

There are two ingredients.

For the first ingredient, you should read Kaj Sotala's excellent review of Consciousness and the Brain by Stan Dehaene (or read the actual book!) To summarize, there is a process in the brain whereby certain information gets promoted up to a "Global Neuronal Workspace" (GNW), a special richly-connected high-level subnetwork of the brain. Only information in the GNW can be remembered and described—i.e., this is the information of which we are "aware". For example, if something flashes in our field of view too quickly for us to "notice", it doesn't enter the GNW. So it does get processed to some extent, and can cause local brain activity that persists for a couple seconds, but will not cascade to a large, widespread signal with long-lasting effects.

Every second of every day, information is getting promoted to the GNW, and the GNW is processing it and pushing information into other parts of the brain. This process does not constitute all of cognition, but it's an important part. Graziano calls this process attention.[2]

The second ingredient is that the brain likes to build predictive models of things—Graziano calls them "schemas" or "internal models". If you know what an apple is, your brain has an "apple model", that describes apples' properties, behavior, affordances, etc. Likewise, we all have a "body schema", a deeply-rooted model that tracks where our body is, what it's doing, and how it works. If you have a phantom limb, that means your body schema has a limb where your actual b...
Example: Markov Chain
by johnswentworth — 2020-01-10T20:19:31.309Z

The previous post in this sequence discussed how to throw away information in causal DAGs. This post provides a detailed example using a Markov chain.

Suppose we have an n-state Markov chain (CS people: picture a finite state machine with n states and random state transitions at each timestep). A matrix of state transition probabilities Tji gives the probability of transitioning to state j when the system starts the timestep in state i. Writing the state at time t as Xt, we have P[Xt+1=j|Xt=i]=Tji. More generally, P[Xt+m=j|Xt=i]=(Tm)ji, where Tm denotes a matrix power (i.e. T matrix-multiplied by itself m times). To complete the specification, we’ll assume that the system starts in a random state X0 at time 0, with the initial distribution P[X0] given.

As a causal DAG, this system is just a chain: the state at time t depends only on the state at time t−1:

X0→X1→X2→X3→...

People typically draw basic Markov chains the same way we draw finite state machines: a graph with one node for each state, and arcs indicating transitions. Unlike an FSM, where the next arc is chosen by a symbol from some input stream, here the next arc is chosen randomly - so each arc has a probability associated with it. An example:

This is NOT a causal diagram, it is a state transition diagram. It says that, if the system is in state 1, then at the next timestep it will randomly transition to state 1, 2, or 5. (I haven’t included the probabilities on each arc; all that matters for our purposes is that each arc shown has nonzero probability.) Since we have two graph representations of the system (the state transition diagram and the causal DAG), I will generally refer to vertices in the state transition diagram “states” and vertices Xt in the causal diagram as “nodes”.

What happens if we

throw away long-term-irrelevant information

from a node in this Markov chain?

Here’s the idea:

Pick the node Xt
Pick the set of nodes from X0 to Xt+m−1 for some large-ish m (we’ll denote this set X<t+m)
Throw away all info from Xt which is not relevant to nodes outside X<t+m: replace Xt with X′t, a representation of the function x→P[Xt+m=x|Xt].

Let’s think about what that last piece looks like. Xt c...
Offer of co-authorship
by Vanessa Kosoy — 2020-01-10T17:44:00.977Z

I am offering interested persons to become a co-author on (a revised version of) my paper "forecasting using incomplete models". The paper was rejected from the Electronic Journal of Statistics with possibility to resubmit. The main criticism of the reviewers was: not enough citations and discussion of relation to other work. The job of the collaborator would be to revise the paper in order to (i) address this concern (ii) improve the overall presentation and make it more accessible. After that, the paper will be resubmitted to EJS (and/or other publications venues, according to what we will decide together).

Naturally, I will work closely with the collaborator and review the changes ey make, but ey are expected to do the lion share of the work in the revision.

The requirements for a candidate collaborator are:

Must: The knowledge required to understand the paper in full, including all the technical details. It is fine if ey need to refresh eir memory or look up particular theorems / definitions, but ey need to have enough background for that to be efficient. To see what sort of knowledge is necessary, one is advised to skim the paper. Mostly it involves measure theory, probability theory and functional analysis.

Advantage: Strong background in statistics, familiarity with the relevant literature

Advantage: Experience with doing literature surveys on technical subjects

Advantage: Experience with writing good explanations of technical subjects

Advantage: Experience with peer-reviewed academic publications

In order to apply, email me at rot13 of inarffn.xbfbl@vagryyvtrapr.bet. In your email, please share with me (i) your name (ii) your background (iii) the extent to which you are compatible with the above requirements and (iv) your motivation for applying.
What is Success in an Immoral Maze?
by Zvi — 2020-01-10T13:20:01.090Z

Previously in Sequence: Moloch Hasn’t Won, Perfect Competition, Imperfect Competition, Does Big Business Hate Your Family?, What is Life in an Immoral Maze?, Stripping Away the Protections

Immoral Mazes are terrible places to be. Much worse than they naively appear. They promise the rewards and trappings of success. Do not be fooled.

If there is one takeaway I want everyone to get from the whole discussion of Moral Mazes, it is this:

Being in an immoral maze is not worth it. They couldn’t pay you enough. Even if they could, they definitely don’t. If you end up CEO, you still lose. These lives are not worth it. Do not be a middle manager at a major corporation or other organization that works like this. Do not sell your soul.

When one works for an immoral maze, what is one hoping for? What is success?

Suppose you persevere. You make the sacrifices. Become the person you need to become. Put in the work day after day. Fortune smiles on you and you win out against all the others doing the same thing. You succeed.

What is success? What do you get in exchange?

For some managers, the drive for success is a quest for the generous financial rewards that high corporate position brings. For others, success means the freedom to define one’s work role with some latitude, to “get out from under the thumb of others.” For still others, it means the chance to gain power and to exert one’s will, to “call the shots,” to “do it my way,” or to know the curiously exhilarating pleasure of controlling other people’s fates. For still others, the quest for success expresses a deep hunger for the recognition and accolades of one’s peers. (Location 955, Quote 118)

This a warning. “Success,” in context, does not mean happiness. It does not make you healthy. It does not improve your reproductive fitness. It does not reflect or spread the values that you (one would hope) had when you stared down that road.

It gives you money. But in terms of actual meaningful personal consumption, you can’t really do much with it beyond status competitions. If you had plans to do something good with the money, by the time the day arrives, it is highly unlikely you’ll do it. You have changed yourself to succeed on your journey.

Even after you ‘succeed’ you probably keep putting tons of hours into the job in ways no amount of money can compensate for, once you already had basically enough.

What was the point? Wha...
On Being Robust
by TurnTrout — 2020-01-10T03:51:28.185Z

Inspired in part by Being a Robust Agent. Flipside of Half-assing it with everything you've got.

Do you ever feel... fake? Like, at any minute, Scooby Doo and the gang might roll up and unmask you as a freeloading fraud impostor in front of everyone?

There are a lot of things to say about the impostor syndrome on a psychological basis (the fears are often unrealistic / unmerited, etc). But I'd like to take another angle. For a few years, I've tried to just make a habit of being un-unmaskable. Although this is a useful frame for me, your mileage may vary.

My point isn't going to just be "do the things you know you should". I think we're often bad at judging when corners are okay to cut, so you probably do better just by having the policy of not cutting corners, unless it's extremely obviously alright to do so. That is, generally err against using scissors when confronted with corners, even if it makes sense in the moment.

Concrete examples

Making insights truly a part of you. This doesn't mean one should freak out about the Math Gestapo checking whether you've memorized what Jordan normal form is. Rather... when I was just beginning to learn formal proof-based math, I worried "I'm about to go work with some of the smartest people in the world, and they'll instantly see I'm a fake who just picked up shallow knowledge". The internal response was "just get good enough that in no conceivable world could you be a fake who secretly can't do formal math".

Working out regularly, taking care of the small things, building the key good habits. Having your shit together.

Learning a lot of related areas, just in case they have key insights.

Regularly and automatically backing up your files, in multiple locations.

Using a password manager to generate and store strong passwords, automatically syncing your database over Dropbox, etc.

Rather than having embarrassing things on Facebook which you hope people won't find, just use a tool to search-and-delete incriminating cringey material from your past.

Keep your exhaustive resume up-to-date, using a slick LATEX template like you know you should.

Following best practices (e.g. when writing code, so there isn't a secret layer of gross code underneath the most prominent functions; when dealing with git repos, so future collaboration / merging works out okay).

Responding to emails after reading them. Not leaving people on read by mis

...
Farewell to the Entertainer
by jefftk — 2020-01-10T02:30:01.702Z

For decades

EV

TAPCO 100M
"Entertainer" systems supported contra dances everywhere:

It's a 10-channel mixer, with 8 XLR inputs and two high-impedance 1/4"
inputs. It has two powered output channels, one for mains and one for
monitors. Three-band parametric EQ, eight-band graphic EQ on the
mains and monitors, and phantom power. It came out in the late 1970s
or early 1980s, and it's impressive how far ahead of its time it is.

Forty years later it seems to me like the last few dances that were
using Entertainers are moving on. I wanted I wanted to know more about
their history, so I called Walter
Lenk. Walter has been working in audio professionally since the
early 1970s, building custom
audio equipment, designing systems, and running sound. He had a
lot to tell me!

Before about 1980 there weren't good options for this sort of sound
reinforcement gig. Most groups were running 6-channel mono
mixer-amplifiers like the Shure Vocal Master:

With only a single output channel, you can't run a separate monitor
mix, and what is helpful to for the band to hear is often very
different than what's helpful to the people in the hall.

The Entertainer came out in 1981, and was a huge improvement. It
could run two output channels, mains and monitors, each piece was
relatively light, and while it wasn't very powerful it had automatic
limiters on the amplifier that would slightly turn down the gain as
needed.

Ad in Billboard, 1981-07-11

Walter remembers seeing one at a trade show in New York in Fall 1981
and buying one as soon as he got back to Boston. He was very
impressed with it, bought several more, and convinced a lot of
Boston-area folks to get them. He also remembers writing a piece for
the CDSS
News where he talked about systems and recommended the
Entertainer.

One remarkable thing about the Entertainer system is that it remained
a reasonable choice for most groups for decades, with many groups
buying used ones after EV stopped selling them. It was ahead of its
time, but the combination of a separate mix for monitors and a
relatively low maximum volume left it a bit niche. It was a good fit
for churches and folk dances, but rock bands generally wanted
something louder and didn't care as much about having separate
monitors. Still, they were a good fit for us, and we got an
impressive amount of service out of them!

...