trentmkelly/LessWrong-43k huggingface dataset:
The National Security Commission on Artificial Intelligence Wants You (to submit essays and articles on the future of government AI policy)
by quanticle — 2019-07-18T17:21:56.522Z
Link: https://warontherocks.com/2019/07/in-search-of-ideas-the-national-security-commission-on-artificial-intelligence-wants-you/

The National Security Commission on Artificial Intelligence, a body created by Congress to study the impact of advances in AI on US national security has put out a call for essays which analyze how advances in artificial intelligence could affect US government and security policy. There are 5 specific prompts, which I summarize as follows:

How will AI affect the nature of war, and more generally interstate competition at or below the level of armed conflict?

What kind of military and non-military AI capabilities should the US government invest in? What AI-related skills will be necessary for the national-security workforce in the future?

What kind of infrastructure, institutions and organizational structures will be best suited for ensuring AI development?

What kinds of AI research should the US national security community engage in?

Will this research require the creation of new institutions?

What other kinds of infrastructure are necessary to create a sustainable advantage in artificial intelligence, and what are the ethical concerns with attempting to create a sustainable advantage?

How will government acquisition processes need to change?

What kinds of data are necessary for developing AI applications and tools? What are the ethical and security concerns related to collecting, analyzing and storing this data?

What should the US do to influence global norms around artificial intelligence? Given that there are many nations pursuing AI, what should the US do to influence adversaries' AI development?

How should the government interact with the private sector? How can the private sector educate the government about the capabilities and risks of AI? What can the government and the private sector do that AI systems used for national security are trusted, by the public, strategic decision-makers, and allies?

I'm heartened to see the government taking the risks of AI more seriously, and I think submitting an essay here is a relatively low-effort way that someone, as a member of the public, can get their opinions regarding the risks and opportunities around AI development in front of a body specifically tasked with creating recommendations for future AI development and regulation.
Laplace Approximation
by johnswentworth — 2019-07-18T15:23:28.140Z

The last couple posts compared some specific models for 20000 rolls of a die. This post will step back, and talk about more general theory for Bayesian model comparison.

The main problem is to calculate P[data|model] for some model. The model will typically give the probability of observed data x (e.g. die rolls) based on some unobserved parameter values θ (e.g. the p's in the last two posts), along with a prior distribution over θ. We then need to compute

P[data|model]=∫θP[data|θ]dP[θ]

which will be a hairy high-dimensional integral.

Some special model structures allow us to simplify the problem, typically by factoring the integral into a product of one-dimensional integrals. But in general, we need some method for approximating these integrals.

The two most common approximation methods used in practice are Laplace approximation around the maximum-likelihood point, and MCMC (see e.g. here for application of MCMC to Bayes factors). We'll mainly talk about Laplace approximation here - in practice MCMC mostly works well in the same cases, assuming the unobserved parameters are continuous.

Laplace Approximation

Here's the idea of Laplace approximation. First, posterior distributions tend to be very pointy. This is mainly because independent probabilities multiply, so probabilities tend to scale exponentially with the number of data points. Think of the probabilities we calculated in the last two posts, with values like 10−70 or 10−20 - that's the typical case. If we're integrating over a function with values like that, we can basically just pay attention to the region around the highest value - other regions will have exponentially small weight.

Laplace' trick is to use a second-order approximation within that high-valued region. Specifically, since probabilities naturally live on a log scale, we'll take a second order-approximation of the log likelihood around its maximum point. Thus:

∫θelnP[data|θ]dP[θ]≈∫θelnP[data|θmax]+12(θ−θmax)T(d2lnPdθ2|θmax)(θ−θmax)dP[θ]

If we assume that the prior dP[θ] is uniform (i.e. dP[θ]=dθ), then this looks like a normal distribution on θ with mean θmax and variance given by the inverse Hessian matrix of the log-likelihood. (It turns out that, even for non-uniform dP[θ], we can just transform θ so that the prior looks uniform near θmax, and transform it back when we're done.) The result:

∫θelnP[...
Normalising utility as willingness to pay
by Stuart_Armstrong — 2019-07-18T11:44:52.272Z

I've thought of a framework that puts most of the methods of interteoretic utility normalisation and bargaining on the same footing. See this first post for a reminder of the different types of utility function normalisation.

Most of the normalisation techniques can be conceived of as a game with two outcomes, and each player can pay a certain amount of their utility to flip from one one outcome to another. Then we can use the maximal amount of utility they are willing to pay, as the common measuring stick for normalisation.

Consider for example the min-max normalisation: this assigns utility 0 to the expected utility if the agent makes the worst possible decisions, and 1 if they make the best possible ones.

So, if your utility function is u, the question is: how much utility would you be willing to pay to prevent your nemesis (a −u maximiser) from controlling the decision process, and let you take it over instead? Dividing u by that amount[1] will give you the min-max normalisation (up to the addition of a constant).

Now consider the mean-max normalisation. For this, the game is as follows: how much would you be willing to pay to prevent a policy from choosing randomly amongst the outcomes ("mean"), and let you take over the decision process instead?

Conversely, the mean min-mean normalisation asks how much you would be willing to pay to prevent your nemesis from controlling the decision process, and shifting to a random process instead.

The mean difference method is a bit different: here, two outcomes are chosen at random, and you are asked now much you are willing to pay to shift from the worst outcome to the best. The expectation of that amount is used for normalisation.

The mutual Worth bargaining solution has a similar interpretation: how much would you be willing to pay to move from the default option, to one where you controlled all decisions?

A few normalisations don't seem to fit into the this framework, most especially those that depend on the square of the utility, such as variance normalisation or the Nash Bargaining solution. The Kalai–Smorodinsky bargaining solution uses a similar normalisation as the mutual worth bargaining solution, but chooses the outcome differently: if the default point is at the origin, it will pick the point (x,x) with largest x.

This, of course, would incentivise you to lie - but that problem is unavoidable in bargaining an

...
Dialogue on Appeals to Consequences
by jessicata — 2019-07-18T02:34:52.497Z
Link: https://unstableontology.com/2019/07/18/dialogue-on-appeals-to-consequences/

[note: the following is essentially an expanded version of this LessWrong comment on whether appeals to consequences are normative in discourse. I am exasperated that this is even up for debate, but I figure that making the argumentation here explicit is helpful]

Carter and Quinn are discussing charitable matters in the town square, with a few onlookers.

Carter: "So, this local charity, People Against Drowning Puppies (PADP), is nominally opposed to drowning puppies."

Quinn: "Of course."

Carter: "And they said they'd saved 2170 puppies last year, whereas their total spending was $1.2 million, so they estimate they save one puppy per $553."

Quinn: "Sounds about right."

Carter: "So, I actually checked with some of their former employees, and if what they say and my corresponding calculations are right, they actually only saved 138 puppies."

Quinn: "Hold it right there. Regardless of whether that's true, it's bad to say that."

Carter: "That's an appeal to consequences, well-known to be a logical fallacy."

Quinn: "Is that really a fallacy, though? If saying something has bad consequences, isn't it normative not to say it?"

Carter: "Well, for my own personal decisionmaking, I'm broadly a consequentialist, so, yes."

Quinn: "Well, it follows that appeals to consequences are valid."

Carter: "It isn't logically valid. If saying something has bad consequences, that doesn't make it false."

Quinn: "But it is decision-theoretically compelling, right?"

Carter: "In theory, if it could be proven, yes. But, you haven't offered any proof, just a statement that it's bad."

Quinn: "Okay, let's discuss that. My argument is: PADP is a good charity. Therefore, they should be getting more donations. Saying that they didn't save as many puppies as they claimed they did, in public (as you just did), is going to result in them getting fewer donations. Therefore, your saying that they didn't save as many puppies as they claimed to is bad, and is causing more puppies to drown."

Carter: "While I could spend more effort to refute that argument, I'll initially note that you only took into account a single effect (people donating less to PADP) and neglected other effects (such as people having more accurate beliefs about how charities work)."

Quinn: "Still, you have to admit that my case is plausible, and that some onlookers are convinced."

Carter: "Yes, it's plausible, in that I don't have a full refutation,

...
Why it feels like everything is a trade-off
by Sunny from QAD — 2019-07-18T01:33:04.764Z

Epistemic status: A cute insight that explains why it might feel like you always have to make sacrifices along one metric to get improvements along another. Seems tautological once you understand it. Might be obvious to everyone.

Meta-epistemic status or something: My first post. Testing the waters.

Tl;dr: Skip to the last paragraph.

Example of a trade-off

I'm a programmer. I'm also a design prude. I'm also lazy. This all means that I spend a lot of my time chasing some different metrics in my code:

1) How easy it is to read.

2) How long it takes to run.

3) How long it takes to write.

4) ...

These metrics are often at odds with one another. Just the other day I had to make a trade-off involving a function I'd written to evaluate a polynomial at a given point. Originally, it was written in a way that I felt was self-explanatory: it looped over the derivatives-at-zero of the polynomial, which were passed in as a list, and summed up the appropriate multiples of powers of x — a Taylor sum. Pseudo-code:

def apply_polynomial( deriv, x ):
sum = 0
for i from 0 to length( deriv ):
sum += deriv[i] * pow(x, i) / factorial(i)
return sum

It turned out that this function was a significant bottleneck in the execution time of my program: about 20% of it was spent inside this function. I was reasonably sure that the

pow

and

factorial

functions were the issue. I also knew that this function would only ever be called with cubics and lower-degree polynomials. So I re-wrote the code as follows:

def apply_cubic( deriv, x ):
sum = 0
len = length( deriv )
if len > 0:
sum += deriv[0]
if len > 1:
sum += deriv[1] * x
if len > 2:
square = x * x
sum += deriv[2] * square / 2
if len > 3:
cube = square * x
sum += deriv[3] * cube / 6
return sum

Sure enough, this improved the runtime significantly — by nearly the whole 20% that had been being spent inside this function. But notice that the code no longer contains the elements that define a Taylor sum: the loop is gone, and the factorials (0!, 1!, 2!, 3!) have been replaced with their values (1, 1, 2, 6). It also isn't obvious why the length comparisons stop at 3. The code no longer explains itself, and must be commented to be understood. Readability has been sacrificed on t...
Prereq: Cognitive Fusion
by Hazard — 2019-07-17T19:04:35.180Z

In a post by Kaj Sotala, he introduces the very useful idea of cognitive fusion.

Cognitive fusion is a term from Acceptance and Commitment Therapy (ACT), which refers to a person “fusing together” with the content of a thought or emotion, so that the content is experienced as an objective fact about the world rather than as a mental construct. The most obvious example of this might be if you get really upset with someone else and become convinced that something was all their fault (even if you had actually done something blameworthy too).
In this example, your anger isn’t letting you see clearly, and you can’t step back from your anger to question it, because you have become “fused together” with it and experience everything in terms of the anger’s internal logic.

You can become fused to an emotion, a voice in your head, a political view, and experience it to "just be true". I see this as a similar sort of fusion I hear musician talk about, where after years of practice their instrument begin to feel like a part of their body. They aren't "using their index finger to press the black note on a piano" they are "just playing G". This is analogous to being so caught up in your own anger that your partner is "just wrong and terrible" as opposed to "it sorta looks like you intentionally did something to annoy me and I'm worried about if you'll do this again in the future." (or whatever the actual case is)

Sometimes I think of there being a general fusion process where the brain collapses levels of inference. All of the steps that go into a given physical motion or thought process get compressed into a single dot. The thought process will be experienced as "just true" and the physical motion will be experience as an atomic action available to you. Sometimes you can "uncompress" the chain, and sometimes you can't.

Problems can arise when you fuse to a thought or emotion that doesn't have an accurate view of the world, and you unknowingly take it's broken map as the territory.

Isn't this just "Don't make assumptions"?

Not quite, though it is similar. Assumptions don't really capture the more general fusing process that you can also see with physical movement. "I can't believe that you just ass...
Why did we wait so long for the bicycle?
by Ruby — 2019-07-17T18:45:09.706Z
Link: https://rootsofprogress.org/why-did-we-wait-so-long-for-the-bicycle

h/t alyssavance

The bicycle, as we know it today, was not invented until the late 1800s. Yet it was a simple mechanical invention. It would seem to require no brilliant inventive insight, and certainly no scientific background.

Why, then, wasn’t it invented much earlier?

I asked this question on Twitter, and read some discussion on Quora. People proposed many hypotheses, including:

+ Technology factors. Metalworking improved a lot in the 1800s: we got improved iron refining and eventually cheap steel, better processes for shaping metal, and ability to make parts like hollow tubes. Wheel technology improved: wire-spoke (aka tension-spoked) wheels replaced heavier designs; vulcanized rubber (1839) was needed for tires; inflatable tires weren’t invented until 1887. Chains, gears, and ball bearings are all crucial parts that require advanced manufacturing techniques for precision and cost.

+ Design iteration. Early bicycles were inconvenient and dangerous. The first version didn’t even have pedals. Some versions didn’t have steering, and could only be turned by leaning. (!) The famous “penny-farthing” design, with its huge front wheel, made it impossible to balance with your feet, was prone to tipping forward on a hard stop, and generally left the rider high in the air, all of which increased risk of injury. It took decades of iteration to get to a successful bicycle model.

+ Quality of roads. Roads in the 1800s and earlier were terrible by modern standards. Roads were often dirt, rutted from the passage of many carts, turning muddy in the rain. Macadam paving, which gave smooth surfaces to roads, wasn’t invented until about 1820. City roads at the time were paved with cobblestones, which were good for horses but too bumpy for bicycles. (The unevenness was apparently a feature, assisting in the runoff of sewage—leading one Quora answer to claim that the construction of city sewers was what opened the door to bicycles.)

+ Competition from horses. Horses were a common and accepted mode of transportation at the time. They could deal with all kinds of roads. They could carry heavy loads. Who then needs a bicycle? In this connection, it has been claimed that the bicycle was invented in response to food shortages due to the “Year without a Summer”, an 1816 weather event caused by the volcanic explosion of Mt. Tambora the year earlier, which darkened skies and lowered tempera

...
1st Athena Rationality Workshop - Retrospective
by Linda Linsefors — 2019-07-17T16:51:36.754Z

During June 7-10 we ran the first Athena Rationality Workshop at the EA Hotel. The workshop taught the Ease process, which is a mental self debugging framework developed by Matt Goldenberg. The Ease framework incorporates various things he learned and developed during his 5 years of working as a personal coach.

The Ease process is a step by step method going through

Awareness - noticing you thoughts, feelings and other mental objects
Introspection - finding the root cause of what is going on in your mind
Acceptance - accepting that you are who you are in the present and the choices you have done in the past
Alignment - creating peace between your internal parts that are involved in the issue you are trying to fix
Ecology - checking that whatever shift you made in you mind during the alignment step, does not create new problems elsewhere
Creation - building the attitude, habits and external support that you need to keep on track with your new solution
Integration - deeply integrate and reinforce new habits and attitude into your mind

Most of the material was taught by Matt. Ryan Thomas, Deni Pop, Toon Alfrink and Linda Linsefors (me) gave one or two lectures each and, to different extents, helped participants during exercises.

Overall the workshop went really well. We got several pieces of positive feedback during the workshop, and from the feedback form as well, although there is room for improvement (see further down). On average the participants estimated the time spent at the workshop to 9.5 times more productive compared to what they would have done otherwise, with answers ranging from 0.5 to 50.

"I was very impressed by the quality of the workshop; to be able to string together so many disparate theories, tools, and techniques into a single coherent framework, is nothing short of genius. Brilliant work!" - Olivier Maas (participant)

Who we are

Matt Goldenberg has been working intensely on self-improvement for the past 15 years, was a professional coach for 5 years and has been running applied rationality group workshops for the past 18 months.

Linda Linsefors organizes workshops. She gets the job done.

Toon Alfrink firmly decided to turn around his mental health issues once and for all. Somehow, this time, it actually worked. He visited a Zen Monastery for 3 months and hasn’t been depressed ever since. His productivity benefited just as much, going from ...
Wolf's Dice II: What Asymmetry?
by johnswentworth — 2019-07-17T15:22:55.674Z

In the previous post, we looked at Rudolph Wolf's data on 20000 rolls of a pair of dice. Specifically, we looked at the data on the white die, and found that it was definitely biased. This raises an interesting question: what biases, specifically, were present? In particular, can we say anything about the physical asymmetry of the die? Jaynes addressed this exact question; we will test some of his models here.

Elongated Cube Models

Jaynes suggests that, if the die were machined, then it would be pretty easy to first cut an even square along two dimensions. But the cut in the third dimension would be more difficult; getting the length to match the other two dimensions would be tricky. Based on this, we'd expect to see an asymmetry which gives two opposite faces (1 & 6, 2 & 5, or 3 & 4) different probabilities from all the other faces.

Here's what the model looks like for the 1 & 6 pair:

1 & 6 each have the same probability p2
2, 3, 4 & 5 each have the same probability 1−p4
Uniform prior on p (i.e. dirichlet with α=1)

Let's call this model1,6.

I will omit the details of calculations in this post; readers are welcome to use them as exercises. (All the integrals can be evaluated using the dirichlet-multinomial α=1 formula from the previous post.) In this case, we find

P[data|model1,6]=n!1!(n+1)!((n1+n6)!n1!n6!(12)n1+n6)((n2+...+n5)!n2!…n5!(14)n2+n3+n4+n5)≈2.2∗10−59

For the other two opposite face pairs, we get:

2,5: 1.4∗10−63
3,4: 8.5∗10−29

... sure enough, an asymmetry on the 3,4 axis goes a very long way toward explaining this data.

Recall from the previous post that the unbiased model gave a marginal likelihood P[data|model] around 10−70, and the biased model with separate probabilities for each face gave around 10−20. So based on the data, our 3,4 model is still about a billion times less probable than the full biased model (assuming comparable prior probabilities for the two models), but it's getting relatively close - probabilities naturally live on a log scale. It looks like the 3-4 asymmetry is the main asymmetry in the data, but some other smaller asymmetry must also be significant.

Just for kicks, I tried a model with a different probability for each pair of faces, again with uniform prior on the p's. That one came out to 1.7∗10−30 - somewhat worse than the 3,4 model. If you're used to traditional statis... ,
Nutrition heuristic: Cycle healthy options
by aaq — 2019-07-17T12:44:25.657Z

It also doesn't help that different people have contradictory theories, e.g. meat, eggs, and diary are either very important to eat, or very important to avoid. More precisely, the best form of meat is fish. Except you shouldn't eat fish, because they are full of deadly mercury.

Villiam, Nutrition is Satisficing

I really liked this comment, so I wanted to hone in on it.

Suppose we have 3 options for a protein source for dinner: Chicken, steak, or fish. They're all considered to be pretty good sources of protein, by most doctors, but they all have downsides:

Steak is linked to heart disease,
fish contains higher levels of heavy metals, and
chicken is just kind of bland.

What should you do?

A nutrition plan is a little bit like a stock portfolio, in that you can diversify away risk by investing a small amount in several different companies at once. However, many of the risks we talk about with nutrition are linked to the overconsumption of specific foods. That means that diversification is super effective!

So the smart satisficer's move is to have all three, but cycle them. If we're planning for the week, we might do chicken Monday-Wednesday-Friday, and then on the other 4 days alternate between steak and fish. This does require a bit of planning on your part, but if you stick to variations between just a handful of pretty-good sources, I think it's pretty feasible.

(Note that I'm considering "Grilled chicken breast is really healthy, but I had it every night for three weeks and I'm never touching it again" as a strong long term and short term risk of overconsumption. In fact, since this seems to happen often with the healthiest foods we eat, I would strongly advise you to update on this prior as well: Extremely healthy foods need to be cycled in and out with their only very healthy, but tasty, options.)