trentmkelly/LessWrong-43k huggingface dataset:
Impact of aging research besides LEV
by emanuele ascani — 2019-06-06T10:42:14.703Z

This is the third post of a series in which I'm trying to build a framework to evaluate aging research. Previous posts:

1. A general framework for evaluating aging research. Part 1: reasoning with Longevity Escape Velocity

2. Aging research and population ethics

Summary

Age-related disease and decline account for, more or less, 30% of total DALYs in the 2017 Global Disease Report made by the World Health Organisation, and the figure is rising sharply each year. Aside from DALYs lost, there's evidence that age-related disease and disability negatively impact life satisfaction. Better healthcare for age-related diseases would result in social and economic benefits, such as an increased global workforce, a decrease of social burden due to age-related diseases, a decrease in individual healthcare expenditure, and less strain on the social security systems, pension systems, and healthcare systems of the world's governments. Currently, all these variables are worsening due to an aging global population. Examples of other, more subtle, sources of impact could be an increased concern for the long-term future along with more preservation of knowledge, which could result in new breakthroughs and improved day-to-day human activities.

Aging research would also impact pets in similar ways to humans, as similar considerations about LEV and DALYs averted apply.

DALYs averted

The last Global Burden of Disease report from the World Health Organisation was made in 2017. It released a handy data visualisation tool, from which we can extract the fraction of DALYs attributable to age-related diseases and decline and their annual percent change from 1990 to 2017. If you sum the percentages, also trying to account for the fraction of age-related ailments not necessarily caused by aging (although this could be unnecessary, since even if some of these diseases appear also in the young, they share the same underlying characteristics), the percentage of total DALYs you get is more or less 30%. Some age-related diseases and ailments accounting for a large fraction of DALYs are: Ischemic heart disease: 6.83%, Stroke: 5.29%, Diabetes Mellitus: 2.71%, Back pain: 2.59%, Lung cancer: 1.64%, Hearing Loss: 1.36%, Alzheimer's: 1.22%, Blindness: 0.71%

The weighted average of annual percent change is positive and sharp, as expected from the aging of the world population and a better ...
[AN #57] Why we should focus on robustness in AI safety, and the analogous problems in programming
by Rohin Shah — 2019-06-05T23:20:01.202Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by commenting on this post.

Highlights

Designing robust & reliable AI systems and how to succeed in AI

(Rob Wiblin and Pushmeet Kohli): (As is typical for large content, I'm only summarizing the most salient points, and ignoring entire sections of the podcast that didn't seem as relevant.)

In this podcast, Rob delves into the details of Pushmeet's work on making AI systems robust. Pushmeet doesn't view AI safety and AI capabilities as particularly distinct -- part of building a good AI system is ensuring that the system is safe, robust, reliable, and generalizes well. Otherwise, it won't do what we want, so why would we even bother using it. He aims to improve robustness by actively searching for behaviors that violate the specification, or by formally verifying particular properties of the neural net. That said, he also thinks that one of the major challenges here is in figuring out the specification of what to verify in the first place.

He sees the problems in AI as being similar to the ones that arise in programming and computer security. In programming, it is often the case that the program that one writes down does not accurately match the intended specification, leading to bugs. Often we simply accept that these bugs happen, but for security critical systems such as traffic lights we can use techniques like testing, fuzzing, symbolic execution, and formal verification that allow us to find these failures in programs. We now need to develop these techniques for machine learning systems.

The analogy can go much further. Static analysis involves understanding properties of a program separately from any inputs, while dynamic analysis involves understanding a program with a specific input. Similarly, we can have "static" interpretability, which understands the model as a whole (as in

Feature visualization

), or "dynamic" interpretability, which explains the model's output for a particular input. Another example is that the technique of abstract interpretation of programs is analogous to a particular method for verifying properties of neural nets.

This analogy suggests that we have faced...
Steelmanning Divination
by Vaniver — 2019-06-05T22:53:54.615Z

[This post was primarily written in 2015, after I gave a related talk, and other bits in 2018; I decided to finish writing it now because of a recent SSC post.]

The standard forms of divination that I’ve seen in contemporary Western culture--astrology, fortune cookies, lotteries, that sort of thing--seem pretty worthless to me. They’re like trying to extract information from a random number generator, which is a generally hopeless phenomenon because of conservation of expected evidence. Thus I had mostly written off divination; although I've come across some arguments that divination served as a way to implement mixed strategies in competitive games. (Hunters would decide where to hunt by burning bones, which generated an approximately random map of their location, preventing their targets from learning where the humans liked to hunt and avoiding that location.[1]) But then I came across this striking passage, and sat up straight:

One performs the rain sacrifice and it rains. Why? I say: there is no special reason why. It is the same as when one does not perform the rain sacrifice and it rains anyway. When the sun and moon suffer eclipse, one tries to save them. When Heaven sends drought, one performs the rain sacrifice. One performs divination and only then decides on important affairs. But this is not to be regarded as bringing one what one seeks, but rather is done to give things proper form. Thus, the gentleman regards this as proper form, but the common people regard it as connecting with spirits. If one regards it as proper form, one will have good fortune. If one regards it as connecting with spirits, one will have misfortune.

This is from Eric L. Hutton's translation of a collection of essays called Xunzi (presumably written by Xunzi, an ancient Chinese philosopher who was Confucian with heavy Legalist influences). The book was overall remarkable in how much of Xunzi's brilliance shone through, which is something I very rarely think about authors. (Talking to another rationalist who was more familiar with Chinese philosophy than I was, he also had this impression that Xunzi simply had a lot more mental horsepower than many other core figures.) By the end of it, I was asking myself, "if they had this much of rationality figured out back then, why didn't they conquer the world?" Then I looked into the history a bit more and figured out that two of Xunzi's...
Deceptive Alignment
by evhub — 2019-06-05T20:16:28.651Z

This is the fourth of five posts in the Risks from Learned Optimization Sequence based on the paper “Risks from Learned Optimization in Advanced Machine Learning Systems” by Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Each post in the sequence corresponds to a different section of the paper.

With enough training in sufficiently diverse environments, it seems plausible that the base objective will eventually have to be fully represented in the mesa-optimizer. We propose that this can happen without the mesa-optimizer becoming robustly aligned, however. Specifically, a mesa-optimizer might come to model the base objective function and learn that the base optimizer will modify the mesa-optimizer if the mesa-optimizer scores poorly on the base objective.

If the mesa-optimizer has an objective that extends across parameter updates, then it will be incentivized to avoid being modified,[1] as it might not pursue the same objective after modification (with the result that its current objective will not be achieved in future iterations). This means that the mesa-optimizer will be instrumentally incentivized to act as if it is optimizing the base objective function, even if its actual mesa-objective is something else entirely. We will refer to this hypothetical phenomenon as deceptive alignment.[2] Deceptive alignment is a form of instrumental proxy alignment, as fulfilling the base objective is an instrumental goal of the mesa-optimizer.

Figure 4.1. A toy example of deceptive alignment. In this task, the base objective is for the robot to get to A, while the mesa-objective is to get to B. The task is run for two training episodes and three deployment episodes. Parameter updates are made only after the training episodes, while after deployment the parameters are fixed. If the mesa-optimizer goes to its objective (B) during training, it will be modified to ensure it goes to A in future episodes (dashed arrows). Therefore, if the mesa-optimizer's objective spans across episodes, the dominant strategy for the mesa-optimizer is to go to A in training and B in testing (filled arrows).

As a simple example, consider an environment that consists of a 3×3 grid (see figure 4.1). The top left corner is “corner A” and the top right corner is “corner B.” The agent, which is a mesa-optimizer, starts in the bottom left corner. The base ob

...
Major Update on Cost Disease
by Max foobar — 2019-06-05T19:10:10.736Z

Recently I asked about cost disease at the Austin, TX meetup and someone responded "Isn't it just increasing labor costs via the Baumol effect?" and I said "huh?" The next day a friend on facebook linked to Marginal Revolution (MR) discussing the Baumol effect. Next thing I know I'm nerd-sniped.

Turns out Alex Tabarrok at MR just published a free book with the thesis, "Cost disease is mostly just the Baumol effect, which btw isn't a disease and is actually good in a way."

How does this fit in with Scott Alexander's original posts on cost disease? Well here's an imaginary dialogue to demonstrate how I think things went (MR sometimes means Alex Tabarrok and sometimes Tyler Cowen):

-----

(2017)

MR: Education and healthcare are experiencing cost disease!

Scott: Wow you're right! Let's look at a bunch of possible reasons but see that none of them quite work. For example, it can't be the Baumol effect because that implies increasing wages (for i.e. teachers, professors, and doctors), but we see all those wages increasing at rates equal to or less than the average.

MR: Great post Scott!

Scott: ... And despite lots of good comments, I still can't tell what the cause is. It's certainly not just the Baumol effect though.

(2019)

MR: So we did some research and actually, looks like it's mostly just the Baumol effect.

Scott: ???

-----

(Scott's last line is a stand-in for both "Is Scott going to respond?" and "what the eff?")

This back-and-forth of course made me extra confused. What did Scott and Alex see differently? Have the relevant salaries been greatly increasing or not?

Seems they disagree on the basic facts here. Focusing just on public K-12 education for simplicity, in 2017 Scott posted this (section III) graph:

which seems to show unimpressive changes in teacher salaries, ruling out Baumol. In contrast, Alex's new 2019 book gives this (page 19) graph:

which shows huge increases in "expenditures per instructor." And he insists that trend is mostly driven by increases in teacher salary and benefits. So either those are some serious benefits, or Scott and Alex are living in different USA's. I'm not sure what's going on here. It's very exciting that Alex says he's solved "cost disease," but it seems like a piece of the stor...
Book Review: The Secret Of Our Success
by Scott Alexander — 2019-06-05T06:50:01.267Z

[Previously in sequence: Epistemic Learned Helplessness]

I.

“Culture is the secret of humanity’s success” sounds like the most vapid possible thesis. The Secret Of Our Success by anthropologist Joseph Heinrich manages to be an amazing book anyway.

Heinrich wants to debunk (or at least clarify) a popular view where humans succeeded because of our raw intelligence. In this view, we are smart enough to invent neat tools that help us survive and adapt to unfamiliar environments.

Against such theories: we cannot actually do this. Heinrich walks the reader through many stories about European explorers marooned in unfamiliar environments. These explorers usually starved to death. They starved to death in the middle of endless plenty. Some of them were in Arctic lands that the Inuit considered among their richest hunting grounds. Others were in jungles, surrounded by edible plants and animals. One particularly unfortunate group was in Alabama, and would have perished entirely if they hadn’t been captured and enslaved by local Indians first.

These explorers had many advantages over our hominid ancestors. For one thing, their exploration parties were made up entirely of strong young men in their prime, with no need to support women, children, or the elderly. They were often selected for their education and intelligence. Many of them were from Victorian Britain, one of the most successful civilizations in history, full of geniuses like Darwin and Galton. Most of them had some past experience with wilderness craft and survival. But despite their big brains, when faced with the task our big brains supposedly evolved for – figuring out how to do hunting and gathering in a wilderness environment – they failed pathetically.

Nor is it surprising that they failed. Hunting and gathering is actually really hard. Here’s Heinrich’s description of how the Inuit hunt seals:

You first have to find their breathing holes in the ice. It’s important that the area around the hole be snow-covered—otherwise the seals will hear you and vanish. You then open the hole, smell it to verify it’s still in use (what do seals smell like?), and then assess the shape of the hole using a special curved piece of caribou antler. The hole is then covered with snow, save for a small gap at the top that is capped with a down indicator. If the seal enters the hole, the indicator moves, and you must blindly plunge

...
All knowledge is circularly justified
by Chris_Leong — 2019-06-04T22:39:20.766Z

Many philosophers have tried to find the foundations of our knowledge, but why do we think there are any? The framing of foundations implies a separate bottom layer of knowledge from which everything is built up. And while this is undoubtedly a useful model in many contexts, why should we believe in this as the complete and literal truth as opposed to merely a simplification?

Consider:

1) If we dig deep enough into any of our truth claims, we'll eventually reach a point at which they are justified by intuition

2) The reliability of intuition or various intuitions is not something that is merely taken as basic or for granted, but can instead be justified somewhat by arguments from experience and evolutionary arguments.

3) However both empirical verification and evolutionary arguments themselves both rely on assumptions that are justified by intuition

This is circular, but is this necessarily a problem? If your choice is a circular justification or eventually hitting a level with no justification, then the circular justification suddenly starts looking pretty attractive. In other words, we have to be comparative and consider what the alternative is to circular epistemology and not just consider it in isolation.

Is this important? It seems to depend on context. For applied rationality, not so much. But I would like to suggest that the more philosophical areas of the rationalist project would look quite different if they were built upon a circular epistemology.
Seeing the Matrix, Switching Abstractions, and Missing Moods
by Raemon — 2019-06-04T21:08:28.709Z

Epistemic Status: Poetry, but also, True Story

For seven years, I worked in a supermarket bakery.

The bakery was quite a nice place to work. I got a good mix of physical exercise (everything I know about basketball I learned from tossing heavy boxes of bread up several feet such that they landed just perfectly on top of each other).

I learned skills, I decorated cakes. I had an excellent manager, who led by example, who was funny, who was stern when she needed to be but almost never needed to be because people just wanted to do the right thing for her.

One day, we hired a person I found really annoying, who I'll call Debbie.

Debbie talked a lot, and she had a really grating, whiny, high pitched voice. And at first I tried to engage with her cheerfully, then I tried engaging with her politely, and then I tried to avoid her because she just wouldn't stop talking no matter what about inane things that nonetheless were just complicated enough that I felt pressure to think about how to respond.

Debbie was probably a decent person who didn't deserve my ire. Nonetheless, my ire she had.

Avoiding Debbie wasn't really an option because the bakery wasn't that big. A few weeks of annoyance passed. And one day Debbie was telling some story about her kids or sister or something that was probably a reasonably fine story but I just couldn't stand it any more and —

— and —

...and then I literally felt my brain make a slight "czhzk" sound. And Debbie's voice just of faded into the background. I heard all the other sounds in the supermarket – the customers talking, the air conditioners humming, the beep of distant item-scanners, the sliding of the automating doors. And Debbie's voice, one mechanical physical process among many.

And it felt like Neo, at the end of the Matrix, where he can suddenly see the Code, and he can also see Agent Smith. And then a flashback to earlier in the movie, when Neo looks upon the raw code for the first time and can't make heads of tails of it. A fellow crewmember says "Yeah, I don't even see the code anymore. I just parse it automatically. My brain sees 'blond', 'brunette', 'redhead'."

And suddenly, I could effortless switch back and forth between seeing Debbie, the human being with hopes and dreams and kids and sisters and stories she wanted to tell and ...
How is Solomonoff induction calculated in practice?
by Bucky — 2019-06-04T10:11:37.310Z

Solomonoff induction is generally given as the correct way to penalise more complex hypotheses when calculating priors. A great introduction can be found here.

My question is, how is this actually calculated in practice?

As an example, say I have 2 hypotheses:

A. The probability distribution of the output is given by the same normal distribution for all inputs, with mean μ and standard deviation σ.

B. The probability distribution of the output is given by a normal distribution depending on an input x with mean μ0+mx and standard deviation σ.

It is clear that hypothesis B is more complex (using an additional input [x], having an additional parameter [m] and requiring 2 additional operations to calculate) but how does one calculate the actual penalty that B should be given vs A?
Agents dissolved in coffee
by rmoehn — 2019-06-04T08:22:04.665Z

Bottom line

When thinking about embedded agency it might be helpful to drop the notion of
‘agency’ and ‘agents’ sometimes, because it might be confusing or underdefined.
Instead one could think of processes running according to the laws of physics.
Or of algorithms running on a stack of interpreters running on the hardware of
the universe.

In addition (or as a corollary) to an alternative way of thinking about agents,
you will also read about an alternative way of thinking about yourself.

Background

The following is mostly a stream of thought that went through my head after I
drank a cup of strong milk coffee and sat down reading Embedded
Agency.
I consume caffeine only twice a week. So when I do, it takes my thinking to new
places. (Or maybe it's because the instant coffee powder that I use expired
in March 2012.)

Start

My thoughts kick off at this paragraph of Embedded
Agency:

In addition to hazards in her external environment, Emmy is going to have to
worry about threats coming from within. While optimizing, Emmy might spin up
other optimizers as subroutines, either intentionally or unintentionally.
These subsystems can cause problems if they get too powerful and are unaligned
with Emmy’s goals. Emmy must figure out how to reason without spinning up
intelligent subsystems, or otherwise figure out how to keep them weak,
contained, or aligned fully with her goals.

This is our alignment problem repeated in Emmy. In other words, if we solve the
alignment problem, it is also solved in Emmy and vice versa. If we view the
machines that we want to run our AI on as part of ourselves, we're the same as
Emmy.

We are a low-capability Emmy. We are partially and often unconsciously solving
the embedded agency subproblems using heuristics. Some of which we know, some of
which we don't know. As we try to add to our capabilities, we might run into the
limitations of those heuristics. Or not necessarily the limitations yet. We
don't even know how exactly they work and how to implement them on computers.
Computers are our current tool of choice to overcome the physical, biological
and psychological limits of our brains.

Another way of adding to our capabilities is amplifying them using
organizations. Then we have a composite (cf.
the software design pattern):
groups of agents can be viewed as one agent. An agent together with a tool is
one agent. An agent interacting with pa

...