trentmkelly/LessWrong-43k huggingface dataset:
Response to Glen Weyl on Technocracy and the Rationalist Community
by John_Maxwell — 2019-08-22T23:14:58.690Z

Economist Glen Weyl has written a long essay, "Why I Am Not A Technocrat", a major focus of which is his differences with the rationalist community.

I feel like I've read a decent number of outsider critiques of the rationalist community at this point, and Glen's critique is pretty good. It has the typical outsider critique weakness of not being fully familiar with the subject of its criticism, balanced by the strength of seeing the rationalist community from a perspective we're less familiar with.

As I was reading Glen's essay, I took some quick notes. Afterwards I turned them into this post.

Glen's Strongest Points

The fundamental problem with technocracy on which I will focus (as it is most easily understood within the technocratic worldview) is that formal systems of knowledge creation always have their limits and biases. They always leave out important consideration that are only discovered later and that often turn out to have a systematic relationship to the limited cultural and social experience of the groups developing them. They are thus subject to a wide range of failure modes that can be interpreted as reflecting on a mixture of corruption and incompetence of the technocratic elite. Only systems that leave a wide range of latitude for broader social input can avoid these failure modes.

So far, this sounds a lot like discussions I've seen previously of the book Seeing Like a State. But here's where Glen goes further:

Yet allowing such social input requires simplification, distillation, collaboration and a relative reduction in the social status and monetary rewards allocated to technocrats compared to the rest of the population, thereby running directly against the technocratic ideology. While technical knowledge, appropriately communicated and distilled, has potentially great benefits in opening social imagination, it can only achieve this potential if it understands itself as part of a broader democratic conversation.

...

Technical insights and designs are best able to avoid this problem when, whatever their analytic provenance, they can be conveyed in a simple and clear way to the public, allowing them to be critiqued, recombined, and deployed by a variety of members of the public outside the technical class.

Technical experts therefore have a critical role precisely if they can make their technical insights part of a social and democra

...
Why so much variance in human intelligence?
by Ben Pace — 2019-08-22T22:36:55.499Z

Epistemic status: Practising thinking aloud. There might be an important question here, but I might be making a simple error.

There is a lot of variance in general competence between species. Here is the standard Bostrom/Yudkowsky graph to display this notion.

There's a sense that while some mice are more genetically fit than others, they're broadly all just mice, bound within a relatively narrow range of competence. Chimps should not be worried about most mice, in the short or long term, but they also shouldn't worry especially so about peak mice - there's no incredibly strong or cunning mouse they ought to look out for.

However, my intuition is very different for humans. While I understand that humans are all broadly similar, that a single human cannot have a complex adaptation that is not universal [1], I also have many beliefs that humans differ massively in cognitive capacities in ways that can lead to major disparities in general competence. The difference between someone who does understand calculus and someone who does not, is the difference between someone who can build a rocket and someone who cannot. And I think I've tried to teach people that kind of math, and sometimes succeeded, and sometimes failed to even teach basic fractions.

I can try to operationalise my hypothesis: if the average human intelligence was lowered to be equal to an IQ of 75 in present day society, that society could not have built rockets or do a lot of other engineering and science.

(Sidenote: I think the hope of iterated amplification is that this is false. That if I have enough humans with hard limits to how much thinking they can do, stacking lots of them can still produce all the intellectual progress we're going to need. My initial thought is that this doesn't make sense, because there are many intellectual feats like writing a book or coming up with special relativity that I generally expect individuals (situated within a conducive culture and institutions) to be much better at than groups of individuals (e.g. companies).

This is also my understanding of Eliezer's critique, that while it's possible to get humans with hard limits on cognition to make mathematical progress, it's by running an algorithm on them that they don't understand, not running an algorithm that they do understand, and only if they understand it do you get nice properties about them being aligned in the...
Logical Counterfactuals and Proposition graphs, Part 1
by Donald Hobson — 2019-08-22T22:06:01.764Z

Within this sequence of posts I will outline a procedure for logical counterfactuals based on something similar to proof length. In this post I present a reimagining of propositional logic in which proving a theorem is taking a walk around a graph of equivalent proposititions.

Within this post, I will use Latin symbols, for specific symbols within proof strings. I will sometimes use symbols like ∀ to represent a function with particular properties.

I will use Greek letters to represent an arbitrary symbol, upper case for single symbols, lower case for strings.

Respecifying Propositional logic

The goal of this first section is to reformulate first order logic in a way that makes logical counterfactuals easier. Lets start with propositional logic.

We have a set of primitive propositions p,q,r,... as well as the symbols ⊤,⊥. We also have the symbols ∨,∧ which are technically functions from Bool2→Bool but will be written p∨q not ∨(p,q) . There is also ¬:Bool→Bool

Consider the equivalence rules.

1. α≡¬¬α

2. α∧β≡β∧α

3. (α∧β)∧γ≡α∧(β∧γ)

4. ¬α∧¬β≡¬(α∨β)

5. α∧⊤≡α

6. ¬α∨α≡⊤

7. ⊥≡¬⊤

8. α∧(β∨γ)≡(α∧β)∨(α∧γ)

9. ⊥∧α≡⊥

10.α≡α∧α

Theorem

Any tautology provable in propositional logic can be created by starting at ⊤ and repeatedly applying equivalence rules.

Proof

First consider α⟹β to be shorthand for ¬α∨β.

Lemma

We can convert ⊤ into any of the 3 axioms.

α⟹(β⟹α) is a shorthand for

¬α∨(¬β∨α)≡1

¬¬(¬α∨(¬β∨α))≡4

¬(¬¬α∧¬(¬β∨α))≡4

¬(¬¬α∧(¬¬β∧¬α))≡1

¬(¬¬α∧(β∧¬α))≡2

¬(¬¬α∧(¬α∧β))≡3

¬((¬¬α∧¬α)∧β)≡4

¬(¬(¬α∨α)∧β)≡6

¬(¬⊤∧β)≡7

¬(⊥∧β)≡9

¬⊥≡7

⊤

Similarly

(α⟹(β⟹γ))⟹((α⟹β)⟹(α⟹γ))

(¬α⟹¬β)⟹(β⟹α)

(if these can't be proved, add that they ≡⊤ as axioms)

End Lemma

Whenever you have α∧(α⟹β), that is equiv to

α∧(¬α∨β)≡8

(α∧¬α)∨(α∧β)≡1

(¬¬α∧¬α)∨(α∧β)≡4

¬(¬α∨α)∨(α∧β)≡6

¬⊤∨(α∧β)≡1

¬¬(¬⊤∨(α∧β))≡4

¬(¬¬⊤∧¬(α∧β))≡1

¬(⊤∧¬(α∧β))≡2

¬(¬(α∧β)∧⊤)≡5

¬¬(α∧β)≡1

α∧β

This means that you can create and apply axioms. For any tautology, look at the proof of it in standard propositional logic. Call the statements in this proof p1,p2,p3...

suppose we have already found a sequence of substitutions from ⊤ to p1∧p2...∧pi−1

Whenever pi is a new axiom, use (5.) to get p1∧p2...∧pi−1∧⊤, then convert ⊤ into the instance of the axiom you want. (substitute alpha and beta with arbitrary props in above proof schema)

Using substitution rules (2.) and (3.) you can rearrange the terms representing lines in the proof and igno...
Time Travel, AI and Transparent Newcomb
by johnswentworth — 2019-08-22T22:04:55.908Z

Epistemic status: has "time travel" in the title.

Let's suppose, for the duration of this post, that the local physics of our universe allows for time travel. The obvious question is: how are paradoxes prevented?

We may not have any idea how paradoxes are prevented, but presumably there must be some prevention mechanism. So, in a purely Bayesian sense, we can condition on paradoxes somehow not happening, and then ask what becomes more or less likely. In general, anything which would make a time machine more likely to be built should become less likely, and anything which would prevent a time machine being built should become more likely.

In other words: if we're trying to do something which would make time machines more likely to be built, this argument says that we should expect things to mysteriously go wrong.

For instance, let's say we're trying to build some kind of powerful optimization process which might find time machines instrumentally useful for some reason. To the extent that such a process is likely to build time machines and induce paradoxes, we would expect things to mysteriously go wrong when trying to build the optimizer in the first place.

On the flip side: we could commit to designing our powerful optimization process so that it not only avoids building time machines, but also actively prevents time machines from being built. Then the mysterious force should work in our favor: we would expect things to mysteriously go well. We don't need time-travel-prevention to be the optimization process' sole objective here, it just needs to make time machines sufficiently less likely to get an overall drop in the probability of paradox.
Embedded Naive Bayes
by johnswentworth — 2019-08-22T21:40:05.972Z

Suppose we have a bunch of earthquake sensors spread over an area. They are not perfectly reliable (in terms of either false positives or false negatives), but some are more reliable than others. How can we aggregate the sensor data to detect earthquakes?

A “naive” seismologist without any statistics background might try assigning different numerical scores to each sensor, roughly indicating how reliable their positive and negative results are, just based on the seismologist’s intuition. Sensor i gets a score s+i when it’s going off, and s−i when it’s not. Then, the seismologist can add up the s+i scores for all sensors going off at a given time, plus the s−i scores for sensors not going off, to get an aggregate “earthquake score”. Assuming the seismologist has decent intuitions for the sensors, this will probably work just fine.

It turns out that this procedure is equivalent to a

Naive Bayes model

.

Naive Bayes is a

causal model

in which there is some parameter θ in the environment which we want to know about - i.e. whether or not there’s an earthquake happening. We can’t observe θ directly, but we can measure it indirectly via some data {xi} - i.e. outputs from the earthquake sensors. The measurements may not be perfectly accurate, but their failures are at least independent - one sensor isn’t any more or less likely to be wrong when another sensor is wrong.

We can represent this picture with a causal diagram:

From the diagram, we can read off the model’s equation: P[θ,{xi}]=P[θ]∏iP[xi|θ]. We’re interested mainly in the posterior probability P[θ|{xi}]=1ZP[θ]∏iP[xi|θ] or, in log odds form,

L[θ|{xi}]=lnP[θ]P[∼θ]+∑ilnP[xi|θ]P[xi|∼θ]

Stare at that equation, and it’s not hard to see how the seismologist’s procedure turns into a Naive Bayes model: the seismologist’s intuitive scores for each sensor correspond to the “evidence” from the sensor lnP[xi|θ]P[xi|∼θ]. The “earthquake score” then corresponds to the posterior log odds of an earthquake. The seismologist has unwittingly adopted a statistical model. Note that this is still true regardless of whether the scores used are well-calibrated or whether the assumptions of the model hold - the seismologist is implicitly using this model, and whether the model is correct is an entirely separate question.

The Embedded Naive Bayes Equation

...
Intentional Bucket Errors
by Scott Garrabrant — 2019-08-22T20:02:11.357Z

I want to illustrate a research technique that I use sometimes. (My actual motivation for writing this is to make it so that I don't feel as much like I need to defend myself when I use this technique.) I am calling it intentional bucket errors after a CFAR concept called bucket errors. Bucket errors is about noticing when multiple different concepts/questions are stored in your head as a single concept/question. Then, by noticing this, you can think about the different concepts/question separately.

What are Intentional Bucket Errors

Bucket errors are normally thought of as a bad thing. It has "errors" right in the name. However, I want to argue that bucket errors can sometimes be useful, and you might want to consider having some bucket errors on purpose. You can do this by taking multiple different concepts and just pretending that they are all the same. This usually only works if the concepts started out sufficiently close together.

Like many techniques that work by acting as though you believe something false, you should use this technique responsibly. The goal is to pretend that the concepts are the same to help you gain traction on thinking about them, but then to also be able to go back to inhabiting the world where they are actually different.

Why Use Intentional Bucket Errors

Why might you want to use intentional bucket errors? For one, maybe the concepts actually are the same, but they look different enough that you won't let yourself consider the possibility. I think this is especially likely to happen if the concepts are coming from very different fields or areas of your life. Sometimes it feels silly to draw strong connections between e.g. human rationality, AI alignment, evolution, economics, etc. but such connections can be useful.

Also I find this useful for gaining traction. There is something useful about constrained optimization for being able to start thinking about a problem. Sometimes it is harder to say something true and useful about X than it is to say something true and useful that simultaneously applies to X, Y, and Z. This is especially true when the concepts you are conflating are imagined solutions to problems.

For example, maybe I have an imagined solution to counterfactuals that has a hole in it that looks like understanding multi-level world models. Then, maybe I also have have an imagined solution to tiling th...
Computational Model: Causal Diagrams with Symmetry
by johnswentworth — 2019-08-22T17:54:11.274Z

Consider the following program:

f(n):
if n == 0:
return 1
return n * f(n-1)

Let’s think about the process by which this function is evaluated. We want to sketch out a causal DAG showing all of the intermediate calculations and the connections between them (feel free to pause reading and try this yourself).

Here’s what the causal DAG looks like:

Each dotted box corresponds to one call to the function f. The recursive call in f becomes a symmetry in the causal diagram: the DAG consists of an infinite sequence of copies of the same subcircuit.

More generally, we can represent any Turing-computable function this way. Just take some pseudocode for the function, and expand out the full causal DAG of the calculation. In general, the diagram will either be finite or have symmetric components - the symmetry is what allows us to use a finite representation even though the graph itself is infinite.

Why would we want to do this?

For our purposes, the central idea of embedded agency is to take these black-box systems which we call “agents”, and break open the black boxes to see what’s going on inside.

Causal DAGs with symmetry are how we do this for Turing-computable functions in general. They show the actual cause-and-effect process which computes the result; conceptually they represent the computation rather than a black-box function.

In particular, a causal DAG + symmetry representation gives us all the natural machinery of causality - most notably counterfactuals. We can ask questions like “what would happen if I reached in and flipped a bit at this point in the computation?” or “what value would f(5) return if f(3) were 11?”. We can pose these questions in a well-defined, unambiguous way without worrying about logical counterfactuals, and without adding any additional machinery. This becomes particularly important for embedded optimization: if an “agent” (e.g. an organism) wants to plan ahead to achieve an objective (e.g. find food), it needs to ask counterfactual questions like “how much food would I find if I kept going straight?”.

The other main reason we would want to represent functions as causal DAGs with symmetry is because our universe appears to be one giant causal DAG with symmetry.

Because our universe is causal, any computation performed in our universe must eventually bottom out in a causal DAG. We can write our programs in any language we ple...
[AN #62] Are adversarial examples caused by real but imperceptible features?
by Rohin Shah — 2019-08-22T17:10:01.959Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by commenting on this post.

Audio version

here

(may not be up yet).

Highlights

Call for contributors to the Alignment Newsletter

(Rohin Shah): I'm looking for content creators and a publisher for this newsletter! Apply by September 6.

Adversarial Examples Are Not Bugs, They Are Features

(Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom et al) (summarized by Rohin and Cody): Distill published a discussion of this paper. This highlights section will cover the full discussion; all of these summaries and opinions are meant to be read together.

Consider two possible explanations of adversarial examples. First, they could be caused because the model "hallucinates" a signal that is not useful for classification, and it becomes very sensitive to this feature. We could call these "bugs", since they don't generalize well. Second, they could be caused by features that do generalize to the test set, but can be modified by an adversarial perturbation. We could call these "non-robust features" (as opposed to "robust features", which can't be changed by an adversarial perturbation). The authors argue that at least some adversarial perturbations fall into the second category of being informative but sensitive features, based on two experiments.

If the "hallucination" explanation were true, the hallucinations would presumably be caused by the training process, the choice of architecture, the size of the dataset, but not by the type of data. So one thing to do would be to see if we can construct a dataset such that a model trained on that dataset is already robust, without adversarial training. The authors do this in the first experiment. They take an adversarially trained robust classifier, and create images whose features (final-layer activations of the robust classifier) match the features of some unmodified input. The generated images only have robust features because the original classifier was robust, and in fact models trained on this dataset are automatically robust.

If the "non-robust features" explanation were true, then it should be possible for a model to ...
Implications of Quantum Computing for Artificial Intelligence Alignment Research
by Jsevillamol — 2019-08-22T10:33:27.502Z

[Crossposted from arXiv]

ABSTRACT: We introduce a heuristic model of Quantum Computing and apply it to argue that a deep understanding of quantum computing is unlikely to be helpful to address current bottlenecks in Artificial Intelligence Alignment. Our argument relies on the claims that Quantum Computing leads to compute overhang instead of algorithmic overhang, and that the difficulties associated with the measurement of quantum states do not invalidate any major assumptions of current Artificial Intelligence Alignment research agendas. We also discuss tripwiring, adversarial blinding, informed oversight and side effects as possible exceptions.

KEYWORDS: Quantum Computing, Artificial Intelligence Alignment, Quantum Speedup, Quantum Obfuscation, Quantum Resource Asymmetry.

EPISTEMIC STATUS: Exploratory, we could have overlooked key considerations.

Introduction

Quantum Computing (QC) is a disruptive technology that may not be too far ahead in the horizon. Small proof-of-concept quantum computers have already been built

[1]

and major obstacles to large-scale quantum computing are being heavily researched

[2]

.

Among its potential uses, QC will allow breaking classical cryptographic codes, simulate large quantum systems and faster search and optimization

[3]

. This last use case is of particular interest to Artificial Intelligence (AI) Strategy. In particular, variants of the Grover algorithm can be exploited to gain a quadratic speedup in search problems, and some recent Quantum Machine Learning (QML) developments have led to exponential gains in certain Machine Learning tasks

[4]

(though with important caveats which may invalidate their practical use

[5]

).

These ideas have the potential to exert a transformative effect on research in AI (as noted in

[6]

, for example). Furthermore the technical aspects of QC, which put some physical limits on the observation of the inner workings of a quantum machine and hinder the verification of quantum computations

[7]

, may pose an additional challenge for AI Alignment concerns.

In this short article we introduce a heuristic model of quantum computing that captures the most relevant characteristics of QC for technical AI Alignment research.

We then apply our model to abstractly answer in which areas we expect knowledge of QC might be relevant, and discuss four specific avenues of current research where it might come into play: tripw...
Body Alignment & Balance. Our Midline Anatomy & the Median Plane. 

by leggi — 2019-08-22T10:24:59.156Z

Alignment & Balance of the Human Body.

Epistemic status: An introduction post for my Base-Line Theory of Human Health and Movement. Facts and definitions first ...

Introduction.

When referring to the human body,
What do balance and alignment mean?

Two (of the many) definitions for balanced:

1. Different parts of something that exist in equal or correct amounts.

2. A state of equilibrium, being in harmonious arrangement.

Alignment has many definitions, the two most relevant to 'body alignment' are:

1. Arrangement in a straight line.

2. Arranged in the correct relative positions.

Alignment, Balance & The Median Plane.

The median plane (a.k.a. midsagittal plane) is the plane that splits the body into left and right halves.

A straight line when viewed from the front or back. A 2-D shape from the side.

The body must be "correctly arranged" to create the median plane, where:

The body is aligned.

All midline anatomy is arranged on the straight line of the median plane.

The body is balanced.

Left and right sides in equilibrium either side of the median plane.

The position of the rest of the body should be considered relative to the midline anatomy and the median plane.

Midline Anatomy.

Palpable Anatomical Structures.

Some easy to find midline anatomical structures include the:

Anus. (asshole)
Pubic symphysis of the pelvis. (bone between the legs, front midline)
Navel. (belly button)
Xiphoid process of the sternum. (inverted "V" bottom of rib cage, front midline)
Jugular notch of the sternum. ( "V" top of rib cage, front midline)
External occipital protuberance. (back of head - midline bump at the base of the skull).

Feel for the relative positioning of these midline markers to increase your awareness of your body's midline.

Midline Linear Structures.

The linear anatomical structures on our midline need to be at full extension to be truly aligned - straight, smooth, no kinks or tensions so they can line up on the median plane.

The main linear structures we should focus on for body alignment are:

The linea alba. Midline at the anterior (front) of the body.

Our primary anatomical guide for body alignment.

The nuchal ligament & supraspinous ligament. Forming a continuous structure midline at the posterior (back) of the body.

Our secondary anatomical guides for body alignment.

Linea Alba.

The linea alba (Latin for 'white line') is a strip of strong connective tissue midline at the fro...