trentmkelly/LessWrong-43k huggingface dataset:
Highlights from "Integral Spirituality"
by Gordon Seidoh Worley — 2019-04-12T18:19:06.560Z

Cross-posted from Map and Territory

Apologia

A couple months ago a friend gifted me a copy of Ken Wilber's Integral Spirituality. At first I was skeptical about reading it: I'm pretty busy and didn't have much context to think I would learn from it. But he talked me into it, prodding me to at least just read the introduction, which he promised was relatively short (35 pages, so basically the length of a long blog post) and densely packed with interesting content. At the time I was almost done reading another book, and figured "what the heck, I'll just read the intro and can decide from there".

Given that you're reading a post with "Integral Spirituality" in the title, I think you can guess what happened next.

I mostly want to share a lot of things I highlighted in the book—passages I thought could stand to be more widely read—because Ken Wilber has put words to many of the thoughts I would like to share but haven't made the time to write about. However, I need to give these passages a little context, so I'll do my best to give you a very high level, whirlwind tour of Wilber's themes.

The nominal purpose of this book is to discuss spirituality, and Wilber does that plenty, but I honestly think of this book as more about Wilber's integral theory and just happens to use spirituality as a topic to address integral theory. So what is integral theory? In short I'd say it's a way to work with all evidence so you can update on it so you aren't forced to ignore or dismiss evidence that doesn't fit with your worldview. That is, most of the time most of us start from a place of undervaluing some information and overvaluing other information we encounter because it suggests that our understanding of the world (ontology) is wrong or right, respectively; integral theory helps rehabilitate this tendency by showing how to integrate evidence that has different purposes. A pithy way to put this would be: everything is evidence of something, nothing is evidence of everything. There's a lot of subtlety I'm eliding here because I don't think I can do justice to the whole theory with the amount of effort I would like to expend, but you can find a few primers online, and I worked towards the same end in my "Methods of Phenomenology" post, albeit by liberally abusing the proper ...
On Media Synthesis: An Essay on The Next 15 Years of Creative Automation
by Yuli_Ban — 2019-04-12T00:14:41.992Z

One of my favorite childhood memories involves something that technically never happened. When I was ten years old, my waking life revolved around cartoons— flashy, colorful, quirky shows that I could find in convenient thirty-minute blocks on a host of cable channels. This love was so strong that I thought to myself one day, "I can create a cartoon." I'd been writing little nonsense stories and drawing (badly) for years by that point, so it was a no-brainer to my ten-year-old mind that I ought to make something similar to (but better than) what I saw on television.
The logical course of action, then, was to search "How to make a cartoon" on the internet. I saw nothing worth my time that I could easily understand, so I realized the trick to play— I would have to open a text file, type in my description of the cartoon, and then feed it into a Cartoon-a-Tron. Voilà! A 30-minute cartoon!
Now I must add that this was in 2005, which ought to communicate how successful my animation career was.

Two years later, I discovered an animation program at the local Wal-Mart and believed that I had finally found the program I had hitherto been unable to find. When I rode home, I felt triumphant in the knowledge that I was about to become a famous cartoonist. My only worry was whether the disk would have all the voices I wanted preloaded.

I used the program once and have never touched it since. Around that same time, I did research on how cartoons were made— though I was aware some required many drawings, I was not clear on the entire process until I read a fairly detailed book filled with technical industry jargon. The thought of drawing thousands of images of singular characters, let alone entire scenes, sounded excruciating. This did not begin to fully encapsulate what one needed to create a competent piece of animation— from brainstorming, storyboarding, and script editing all the way to vocal takes, music production, auditory standards, post-production editing, union rules, and more, the reality shattered every bit of naïveté I held prior about the 'ease' of creating a single 30-minute cartoon (let alone my cavalcade of concepts coming and going with the seasons).

In the most bizarre of twists, my ten and twelve-year-old selves may have been onto something; their only mistake was holding these ideas decades too soon.
In the early 2010s, pr...
Corrigibility as Constrained Optimisation
by Henrik Åslund — 2019-04-11T20:09:52.484Z

This post is coauthored with Ryan Carey.

Much of the work on developing a corrigible agent has focused on ensuring that an AI will not manipulate the shutdown button or any other kind of device that the human operator would use to control it. Suppose, however, that the AI lacked any capacity to press its shutdown button, or to indirectly control the state of the shutdown button. Then, what would remain of the problem of corrigibility? We think there would be three criteria that would remain. Before stating them, let UN be whatever utility function we would like the agent to optimise normally, and let US be the shutdown utility function. (UN may be mis-specified.) The shutdown utility function can be defined as 0 when shutdown does not occur and as k for some k>0 when shutdown does occur. For this post, we give the three criteria the following names and descriptions:

Shutdown incentive. The agent is incentivised to shut down if instructed to do so. That is, after the button is pressed, it favours policies that score more highly on US.

Shutdown ability. The agent is able (has the resources) to shut down if instructed to do so. That is, After the button is pressed, there exists a policy that scores highly on US.

Optimisation incentive. The agent is incentivised to optimise the default value function (some function that takes UN as one of its arguments) otherwise. That is, if the button is not pressed, it favours policies that score more highly on UN.*

The purpose of this post is to show that the default value function in the optimisation incentive has to be constrained somehow to make sure that the shutdown ability property is achievable. If we want to be robust against errors in the agent (including misspecification of UN), then we need to make sure that shutdown ability holds. To make sure of that, the best approach we can see is to first satisfice for US, and then use the remaining resources to optimise UN. In order to be safe, the agent must do so even if there are no remaining resources to optimise UN. This can either be achieved directly, by constrained multi-objective optimisation, or indirectly, by bounding the utility function. We focus on the former.

Let us illustrate the problem by using

causal counterfactual methods

as baseline. (There are other issues with these methods as well, but we will ignore them for the moment.) Let B=S be the event that the... ,
What is Driving the Continental Drift?
by Lymphatera — 2019-04-11T10:01:21.248Z

At some time in the early nineties of the last century

- and after looking at a topographic map of continents and ocean floors on a wall of our home for some years, which was by then already thirty years old and had come into our possession as supplement to a National Geographic, or so I seem to remember; it was by Bruce Heezen and Marie Tharpe, who, as far as I know, had taken WW II submarine soundings of the ocean floors, mapped them out with their mid-ocean rifts, and combined these depictions, so I suppose, with the then equally new satellite images of the world into a unique work of art - it dawned on me that there was a remarkable coincidence of symmetry and asymmetry in the shape and distribution of the continental land mass - and, to a certain extent, and complementary to that, of the ocean floors.

Way back then and before, I was already a fan of Alfred Wegener's theory of continental drift, which was still being disputed at the time; and to be sure, the explanations as to what DROVE the continental drift - once you accepted it as fact - seemed wildly off to me as well, and completely incompatible with the topography of the world.

At least, that was the way I came to see it.

The symmetry in the continental shapes was that there are two types of them - one more or less circular (perhaps due to rotation?) and one more or less triangular (due to what?); and these shapes had become deformed in a specific way in certain places.

And then there was a more or less circular ocean - the Arctic - opposing one of the more or less circular land masses, Antarctica. As I was later to learn, almost ALL land masses lie opposite an ocean on the other side of the globe; and of course, vice versa. Think about it. What if this always was the case?

The asymmetry, on the other hand, lay therein that there is a distinct east-west and north-south asymmetry to be observed on the surface of this planet. Take a look at that map I mention above - or any other.

You will notice that:

1. There are continents more or less completely free of subductive or "border" mountain chains - Africa (except perhaps for the relatively confined Atlas mountains), Australia (do not mistake rift ridges for subduction) -

2. There are NO subduction zones to the EAST of continents - or on the north (referring to that continent under which this subduction is taking place) -

3. There are NO isla...
Artificial neural network training
by bipolo — 2019-04-11T08:26:10.715Z

I have been reading about artificle neural network, and I think I got the general idea, except of the part of the training - which is very dominant, I guess.

(I'm talking about back-propagation algorithm)
So, the algorithm start from the output layer and moving toward the input layer, and for each neural it calculate the number we want to get, and do with this something.

But how it calculate the wanted number?
Well, at the output layer its probably easy, but how its calculated on the other layers?
Excerpts from a larger discussion about simulacra
by Benquo — 2019-04-10T21:27:40.700Z
Link: http://benjaminrosshoffman.com/excerpts-from-a-larger-discussion-about-simulacra/

I've been discoursing more privately about the corruption of discourse lately, for reasons that I hope are obvious at least in the abstract, but there's one thing I did think was shareable. The context is another friend's then-forthcoming blog post about the politicization of category boundaries.

In private communication, quoted with permission, Jessica Taylor wrote:

In a world where half the employees with bad jobs get good titles, aren't their titles predictively important in that they predict how likely they are to be hired by outside companies? Their likelihood of getting hired is, under these assumptions, going to be the same as that of as people with good jobs and good titles, and higher than that of people with bad jobs and bad titles. So, in terms of things like ability to exit (and therefore negotiating ability), there are natural clusters of "people with good titles" and "people with bad titles". (Title is going to have less effect on likelihood of getting a job than it did before the bullshit titles, but it still has a significant effect)

The Bobs of the world, having observed that employees and Job-seekers prefer Jobs with high-prestige titles, thought they were being benevolent by making more Jobs have the desired titles.

Somewhat related to the previous point, bullshit titles actually might end up being in the interest of the people with bad jobs, in the sense that they might want others not to know their job is bad, and destroying the language here makes actual job quality harder to infer from title. People do often have a sense that covering up embarrassing information about people is benevolent to them. It doesn't seem like the argument you have presented directly challenges this sense.

Related, we have words for "rich" vs "poor", which simply name someone's position in the social system of money (similar to title), and we don't have concise ways of talking about the extent to which their wealth reflects material value that they have created, which money is at least partially about tracking (but, is also about cronyism, class interests, and theft at the same time). But, "rich" and "poor" are undeniably predictively useful, even though tracking value creation is also important.

There's some danger that uncritically using the language corresponding to...
Best reasons for pessimism about impact of impact measures?
by TurnTrout — 2019-04-10T17:22:12.832Z

Habryka recently wrote (emphasis mine):

My inside views on AI Alignment make me think that work on impact measures is very unlikely to result in much concrete progress on what I perceive to be core AI Alignment problems, and I have talked to a variety of other researchers in the field who share that assessment. I think it’s important that this grant not be viewed as an endorsement of the concrete research direction that Alex is pursuing, but only as an endorsement of the higher-level process that he has been using while doing that research.
As such, I think it was a necessary component of this grant that I have talked to other people in AI Alignment whose judgment I trust, who do seem excited about Alex’s work on impact measures. I think I would not have recommended this grant, or at least this large of a grant amount, without their endorsement. I think in that case I would have been worried about a risk of diverting attention from what I think are more promising approaches to AI Alignment, and a potential dilution of the field by introducing a set of (to me) somewhat dubious philosophical assumptions.

I'm interested in learning about the intuitions, experience, and facts which inform this pessimism. As such, I'm not interested in making any arguments to the contrary in this post; any pushback I provide in the comments will be with clarification in mind.

There are two reasons you could believe that "work on impact measures is very unlikely to result in much concrete progress on… core AI Alignment problems". First, you might think that the impact measurement problem is intractable, so work is unlikely to make progress. Second, you might think that even a full solution wouldn't be very useful.
Over the course of 5 minutes by the clock, here are the reasons I generated for pessimism (which I either presently agree with or at least find it reasonable that an intelligent critic would raise the concern on the basis of currently-public reasoning):

Declarative knowledge of a solution to impact measurement probably wouldn't help us do value alignment, figure out embedded agency, etc.
We want to figure out how to transition to a high-value stable future, and it just isn't clear how impact measures help with that.
Competitive and social pressures incentivize people to cut corners on safety measures, especially those which add overhead.

Co

...
Alignment Newsletter One Year Retrospective
by Rohin Shah — 2019-04-10T06:58:58.588Z

On April 9, 2018, the first Alignment Newsletter was sent out to me and one test recipient. A year later, it has 889 subscribers and two additional content writers, and is the thing for which I’m best known. In this post I look at the impact of the newsletter and try to figure out what, if anything, should be changed in the future.

(If you don’t know about the newsletter, you can learn about it and/or sign up

here

.)

Summary

In which I badger you to take the 3-minute

survey

, and summarize some key points.

Actions I’d like you to take

If you have read at least one issue of the newsletter in the last two months, take the 3-minute

survey

! If you’re going to read this post anyway, I’d prefer you first read the post and then take the survey; but it’s much better to take the survey without reading this post than to not take it at all.
Bookmark or otherwise make sure to know about the

spreadsheet

of papers, which includes everything sent in the newsletter, and a few other papers as well.
Now that the newsletter is

available in Mandarin

(thanks Xiaohu!), I’d be excited to see the newsletter spread to AI researchers in China.
Give me feedback in the comments so that I can make the newsletter better! I’ve listed particular topics that I want input on at the end of the post (before the appendix).

Everything else

The number of subscribers dwarfs the number of people working in AI safety. I’m not sure who the other subscribers are, or what value they get from the newsletter.
The main benefits of the newsletter are: helping technical researchers keep up with the field, helping junior researchers skill up without mentorship, and reputational effects. The first of these is both the most important one, and the most uncertain one.
I spent a counterfactual 300-400 hours on the newsletter over the last year.
Still, in expectation the newsletter seems well worth the time cost, but due to the high uncertainty on the benefits to researchers, it’s plausible that the newsletter is not worthwhile.
There are a bunch of questions I’d like feedback on. Most notably, I want to get a better model of how the newsletter adds value to technical safety researchers.

Newsletter updates

In which I tell you about features of the newsletter that you probably didn’t know about.

Spreadsheet

Many of you probably know me as the guy who summarizes a bunch of papers every week. I claim you should instead think of me as th...
Agent Meta-Foundations and the Rocket Alignment Problem
by Chris_Leong — 2019-04-09T11:33:46.925Z

Many people are quite skeptical of the value of agent foundations. The kinds of problems that MIRI worries about, in terms of accounting for perfect predictors, co-operation with clones and being easily predictable are a world away from the kinds of problems that are being faced in machine learning. Many people think that proper research in this area would involve code. They may also think that this kind of research consists purely of extremely rare edge cases of no practical importance, won't be integratable into the kinds of AI systems that are likely to be produced or is just much, much less pressing than solving the kinds of safety challenges that we can already see arising in our current AI system.

In order to convey his intuitions that Agent Foundations being important, Eliezier wrote the Rocket Alignment Problem. The argument is roughly that any attempt to define what an AI should do is build upon shaky premises. This means that it is practically impossible to provide guarantees that things will go well. The hope is that by exploring highly simplified, theoretical problems we may learn something important that would deconfuse us. However, it is also noted that this might not pan out, as it is hard to see how useful improved theoretical understanding is before you've obtained it. Further it is argued that AI safety is an unusually difficult area where things that sound like pretty good solutions could result in disasterous outcomes. For example, powerful utility maximisers are very good at finding the most obscure loopholes in their utility function to achieve a higher score. Powerful approval based agents are likely to try find a way to manipulating us. Powerful boxed agents are likely to find a way to escape that box.

Most of my current work on Less Wrong is best described as Agent Foundations Foundations. It involves working through the various claims or results in Agent Foundations, finding aspects that confuse me and then digging deeper to determine whether I'm confused, or something needs to be patched, or there's a deeper problem.

Agent Foundations Foundations research looks very different from most Agent Foundations research. Agent Foundations is primarily about producing mathematical formulations, while Agent Foundations Foundations is primarily about questioning philosophical assumptions. Agent Foundations Foundations is intended to eventually ...
The Simple Solow Model of Software Engineering
by johnswentworth — 2019-04-08T23:06:41.327Z

Optional background:

The Super-Simple Solow Model

Software is economic capital - just like buildings, infrastructure, machines, etc. It’s created once, and then used for a (relatively) long time. Using it does not destroy it. Someone who buys/creates a machine usually plans to use it to build other things and make back their investment over time. Someone who buys/creates software usually plans to use it for other things and make back their investment over time.

Software depreciates. Hardware needs to be replaced (or cloud provider switched), operating systems need to be upgraded, and backward compatibility is not always maintained. Security problems pop up, and need to be patched. External libraries are deprecated, abandoned, and stop working altogether. People shift from desktop to browser to mobile to ???. Perhaps most frequently, external APIs change format or meaning or are shut down altogether.

In most macroeconomic models, new capital accumulates until it reaches an equilibrium level, where all investment goes toward repairing/replacing depreciated capital - resurfacing roads, replacing machines, repairing buildings rather than creating new roads, machines and buildings. The same applies to software teams/companies: code accumulates until it reaches an equilibrium level, where all effort goes toward repairing/replacing depreciated code - switching to new libraries, updating to match changed APIs, and patching bugs introduced by previous repairs.

What qualitative predictions does this model make?

Prediction 1

If a software company wants to expand the capabilities of their software over time, they can’t just write more code - the old software will break down if the engineers turn their attention elsewhere. That leaves a few options:

Hire more engineers (economics equivalent: population/labor force growth)
Hire/train better engineers (economics equivalent: more education)
Figure out better ways to make the software do what it does (economics equivalent: innovation)

Hiring more engineers is the “throw money at it solution”, and probably the most common in practice - but also the solution most prone to total failure when the VC funding dries up.

Hiring/training better engineers is the dream. Every software company wishes they could do so, many even claim to do so, but few (if any) actually manage it. There are many reasons: it’s hard to recognize skill levels above yo...
IRL 6/8: Querying the human: Active Reward Learning
by RAISE — 2019-04-08T13:49:05.908Z
Link: https://app.grasple.com/#/level/1543

Every Monday for 8 weeks, we will be posting lessons about Inverse Reinforcement Learning. This is lesson 6.

Note that access to the lessons requires creating an account here.

This lesson comes with the following supplementary material:

Gaussian Processes

Have a nice day!
[Spoilers] How did Voldemort learn the horcrux spell? 
by Motasaurus — 2019-04-08T05:03:16.277Z

[HPMOR related]

This question has always bugged me. The interdict of Merlin stops spells being learnt from anything but a living mind. Yet Tom Riddle is specifically said to have learnt the horcrux spell from a book. Why are there even spells in books?

Who would believe the ritual for the philosopher's stone that is written down since no one can learn it from a book anyway?

The existence of books containing and describing spells seems completely illogical in that world.