trentmkelly/LessWrong-43k huggingface dataset:
The AI is the model
by Charlie Steiner — 2019-10-04T08:11:49.429Z

A Friendly AI is not a selfish AI constrained by a special extra conscience module that overrides the AI's natural impulses and tells it what to do. You just build the conscience, and that is the AI.

Eliezer Yudkowsky, Ghosts in the Machine

When I started thinking about value learning, I thought the goal was to extract simple objects that described the essence of morality. Not so simple as a verbal definition, but something like a utility function. Something separate from planning or reasoning, that was purely about preferences, which you could plug into an AI which would then do some totally separate work to turn preferences into choices.

Turns out that runs into some serious obstacles.

I

The difficulty of value learning is that there is no One True Utility Function to be assigned the globs of atoms we call humans. To think about them as having desires at all requires viewing them at a suitable level of abstraction - though of course, there's no One True Level Of Abstraction, either. (I promise this is my last post that's basically just consequences of needing the intentional stance for a while.)

Call the world-model the AI uses to best predict the world its "native ontology." If I want to go to the gym, we want the AI to look at the atoms and see "Charlie wants to go to the gym." The thing that I want is not some specific state of the AI's native ontology. Instead, I can only "want" something in an abstracted ontology that not only contains the AI's intentional-stance model of "Charlie," but also intentional-stance-compatible abstractions for "go" and "gym." In short, abstraction is contagious.

This is like the idea of an umwelt (oom-velt), introduced by early philosopher of biology Jakob Johann von Uexküll. In nature, different organisms can have different effective models of the world even though they live in the same environment. They only evolve to model what is necessary for them to survive and reproduce. The umwelt is a term for this modeled world. The umwelt of a bloodsucking tick consists largely of things to climb on and warm-blooded mammals, which are perceived not by sight but by a certain smell and body temperature.

I think of the AI's intentional stance as not just being a specially abstract model of me, but also being a model of my entire umwelt. It needs an abstraction of the gym because the gym is a part of my inner world, an abstr...
Conscious  Proprioception.
by leggi — 2019-10-04T04:33:14.317Z

Conscious Proprioception: Increased Awareness of your Sense of Position, Motion, Alignment & Balance.

After an introduction to the anatomy of alignment and balance and the 5 main muscles of movement, consider what you experience if you focus on the position (posture) and motion of your body:

How much of your sense of proprioception are you aware of ?

Origin page is located here: conscious proprioception @ baselinehealing.com

Introduction.

Standard definition of proprioception:

Proprioception: "The ability to sense stimuli arising within the body regarding position, motion, and equilibrium."

The basic physiological model of proprioception:

'Sensors' located throughout the body generate sensory feedback, 'information' that is sent, via the nervous system, to the brain. Sensory feedback concerning the body's position, motion and balance is our sense of proprioception.
The brain 'processes/interprets' this feedback about positioning and then sends signals around the body to maintain a 'functional posture', the positioning of the body that we use day-to-day.

I feel it fair to say that the different mechanisms and various proprioceptors ('sensors') involved in proprioception are far from being understood and our sense of "bodily awareness" appears in the philosophical section at Stanford.

This post is my attempt to explain my experience of an increased consciousness of the proprioceptive feedback my body is continually producing - and how everyone can increase their awareness of their physical selves - feeling the positioning of the parts of your body, your state of motion and equilibrium.

Conscious proprioception.

Awareness of:

The relative position of the parts of your body.
The motion of your body.
Your equilibrium - whether you are balanced or not.

According to:

Base-Line Theory of Health and Movement. (part 2)

(BLTH part 1)

Increased awareness of the body's positioning, balance and state of alignment - of the body's posture - comes with focusing on the right anatomy.

Working from the body's 'Base-Line' muscles (pelvic floor Base, rectus abdominis Line) starts to connect us with our midline anatomy and guides for body alignment.

Position, Motion, Equilibrium.

To describe the position of something you need a reference.
→ The position of the rest of your body is relative to your Base-Line.

To describe a motion (a change in position) a reference is also needed...
Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More
by Ben Pace — 2019-10-04T04:08:49.942Z

An actual debate about instrumental convergence, in a public space! Major respect to all involved, especially Yoshua Bengio for great facilitation.

For posterity (i.e. having a good historical archive) and further discussion, I've reproduced the conversation here. I'm happy to make edits at the request of anyone in the discussion who is quoted below. I've improved formatting for clarity and fixed some typos. For people who are not researchers in this area who wish to comment, see the public version of this post here. For people who do work on the relevant areas, please sign up in the top right. It will take a day or so to confirm membership.

Original Post

Yann LeCun: "don't fear the Terminator", a short opinion piece by Tony Zador and me that was just published in Scientific American.

"We dramatically overestimate the threat of an accidental AI takeover, because we tend to conflate intelligence with the drive to achieve dominance. [...] But intelligence per se does not generate the drive for domination, any more than horns do."

https://blogs.scientificamerican.com/observations/dont-fear-the-terminator/

Comment Thread #1

Elliot Olds: Yann, the smart people who are very worried about AI seeking power and ensuring its own survival believe it's a big risk because power and survival are instrumental goals for almost any ultimate goal.

If you give a generally intelligent AI the goal to make as much money in the stock market as possible, it will resist being shut down because that would interfere with tis goal. It would try to become more powerful because then it could make money more effectively. This is the natural consequence of giving a smart agent a goal, unless we do something special to counteract this.

You've often written about how we shouldn't be so worried about AI, but I've never seen you address this point directly.

Stuart Russell: It is trivial to construct a toy MDP in which the agent's only reward comes from fetching the coffee. If, in that MDP, there is another "human" who has some probability, however small, of switching the agent off, and if the agent has available a button that switches off that human, the agent will necessarily press that button as part of the optimal solution for fetching the coffee. No hatred, no desire for power, no built-in emotions, no built-in survival insti...
[Link] Tools for thought (Matuschak & Nielsen)
by ioannes — 2019-10-04T00:42:32.116Z

https://numinous.productions/ttft

An excerpt:

We're often asked: why don't you work on AGI or [brain-computer interfaces (BCI)] instead of tools for thought? Aren't those more important and more exciting? And for AGI, in particular, many of the skills required seem related.

They certainly are important and exciting subjects. What's more, at present AGI and BCI are far more fashionable (and better funded). As a reader, you may be rolling your eyes, supposing our thinking here is pre-determined: we wouldn't be writing this essay if we didn't favor work on tools for thought. But these are questions we've wrestled hard with in deciding how to spend our own lives. One of us wrote a book about artificial intelligence before deciding to focus primarily on tools for thought; it was not a decision made lightly, and it's one he revisits from time to time. Indeed, given the ongoing excitement about AGI and BCI, it would be surprising if people working on tools for thought didn't regularly have a little voice inside their head saying “hey, shouldn't you be over there instead?” Fashion is seductive.

One striking difference is that AGI and BCI are based on relatively specific, well-defined goals. By contrast, work on tools for thought is much less clearly defined. For the most part we can't point to well-defined, long-range goals; rather, we have long-range visions and aspirations, almost evocations. The work is really about exploration of an open-ended question: how can we develop tools that change and expand the range of thoughts human beings can think?

Culturally, tech is dominated by an engineering, goal-driven mindset. It's much easier to set KPIs, evaluate OKRs, and manage deliverables, when you have a very specific end-goal in mind. And so it's perhaps not surprising that tech culture is much more sympathetic to AGI and BCI as overall programs of work.

But historically it's not the case that humanity's biggest breakthroughs have come about in this goal-driven way. The creation of language – the ur tool for thought – is perhaps the most important occurrence of humanity's existence. And although the origin of language is hotly debated and uncertain, it seems extremely unlikely to have been the result of a goal-driven process. It's amusing to try imagining some prehistoric quarterly OKRs leading to the development of language. What sort of goals could one possibly set? Per

...
Long-Term Future Fund: August 2019 grant recommendations
by habryka — 2019-10-03T20:41:16.291Z
Link: https://forum.effectivealtruism.org/posts/an9GrNXrdMwBJpHeC/long-term-future-fund-august-2019-grant-recommendations-1

Note: The Q4 deadline for applications to the Long-Term Future Fund is Friday 11th October. Apply here.

We opened up an application for grant requests earlier this year, and it was open for about one month. This post contains the list of grant recipients for Q3 2019, as well as some of the reasoning behind the grants. Most of the funding for these grants has already been distributed to the recipients.

In the writeups below, we explain the purpose for each grant and summarize our reasoning behind their recommendation. Each summary is written by the fund manager who was most excited about recommending the relevant grant (with a few exceptions that we've noted below). These differ a lot in length, based on how much available time the different fund members had to explain their reasoning.

When we’ve shared excerpts from an application, those excerpts may have been lightly edited for context or clarity.

Grant Recipients

Grants Made By the Long-Term Future Fund

Each grant recipient is followed by the size of the grant and their one-sentence description of their project. All of these grants have been made.

Samuel Hilton, on behalf of the HIPE team ($60,000): Placing a staff member within the government, to support civil servants to do the most good they can.
Stag Lynn ($23,000): To spend the next year leveling up various technical skills with the goal of becoming more impactful in AI safety.
Roam Research ($10,000): Workflowy, but with much more power to organize your thoughts and collaborate with others.
Alexander Gietelink Oldenziel ($30,000): Independent AI Safety thinking, doing research in aspects of self-reference in using techniques from type theory, topos theory and category theory more generally.
Alexander Siegenfeld ($20,000): Characterizing the properties and constraints of complex systems and their external interactions.
Sören M. ($36,982): Additional funding for an AI strategy PhD at Oxford / FHI to improve my research productivity
AI Safety Camp ($41,000): A research experience program for prospective AI safety researchers.
Miranda Dixon-Luinenburg ($13,500): Writing EA-themed fiction that addresses X-risk topics.
David Manheim ($30,000): Multi-model approach to corporate and state actors relevant to existential risk mitigation.
Joar Skalse ($10,000): Upskilling in ML in order to be able to do productive AI safety research sooner than otherwise.
Chris Chambers ($36,63

...
New Petrov Game Brainstorm
by Alexei — 2019-10-03T19:48:14.009Z

Big thanks to the LW team for putting together the Petrov Day experience! (Setup. Follow up.)

I looked over the comments and it seems like there there were a number of suggestions for how to do this better. Instead of waiting for the next year, let's do it right now.

My proposed setup:

1. Take the original 125 LW users. Take a prize pool of $1,250 (or more if people are willing to donate). The prize pool is split evenly between each player, but you have to survive the game to get paid. Everyone is anonymized in the game.

2. The game will last a minimum of 4 days (to give everyone enough time to act, strategize, and think). After 4 days, there will be an increasing probability that the game will end at any minute. (This is to prevent anyone trying to attack right when the game ends to avoid retaliation. In expectation, the game should last about a week.)

3. Each player will have the number of missiles equal to the number of players. They can launch any number of them.

4. When a missile is launched: a) the attacked player is notified that they are being attacked by a specific player (and therefore has an option to retaliate), b) 48 hours after the launch, the attacked player is declared dead: they can no longer perform any actions and will not receive a payout, c) 48 hours after the launch, the attacking player gets the target player's entire prize pool.

5. During the game there will be at least 125 fake alerts. They will be generated randomly (so some players might receive zero or more than one fake alerts). It will look the same as if some specific player has launched a missile against you. 48 hours after the notification, you'll find out whether it was real or not by whether or not you're still alive.

Additional details:

You can see who has been killed.
You can only know about missiles launches that you have done or that have been done to target you.
If you take money from a player who already took money from someone else, you get those too. So in theory, we could end up with one winner with the entire original prize pool.
When people create their accounts, they'll have an optional to either receive their winnings directly (let's say via PayPal) or to donate to LW. (Personally, I hope most people will choose the second option, which will make the payout much less of a hustle.)
I'm not sure how to easily structure this so that the playe

...
To Be Decided #1
by Ian David Moss — 2019-10-03T19:30:15.976Z

(Preface: This is the first edition of a quarterly email newsletter I started earlier this year called To Be Decided. I'm posting this as an experiment; if response here is positive, I'll post the two issues that have gone out since then as well as future issues as they come out. Feedback welcome!)

Welcome to the inaugural edition of To Be Decided, a quarterly newsletter about smarter decisions for a better world! TBD is all about deploying knowledge for impact, learning at scale, and making more thoughtful choices for ourselves and our organizations. Each edition will feature short and sweet reviews of important publications you don't want to miss but don't have time to read, along with a brief roundup of major developments in the world of learning and decision-making since last time.

Why Your Hard Work Sits on the Shelf—and What to Do About It

We've all been there. The time when the client seemed to forget the project ever happened as soon as the final check was cut. The time when your report stuffed full of creative recommendations got buried by risk-averse leadership. The time when stakeholders really did seem engaged by the findings, had lots of conversations, and then...nothing changed.

If you suspect these stories are more the rule than the exception, the evidence suggests you're right. And if the trend continues, chances are it's eventually going to catch up to those of us who generate and spread knowledge in the social sector. If we really want our work to be useful, we have to continue supporting decision-makers after the final report is delivered, working hand-in-hand with them to ensure whatever choices they make take into account not only the best information available but also other factors that matter to them, including their values, goals, and perceived obligations. For this reason, knowledge providers who want to see their work have greater impact might find value in partnering with a decision consultant in the form of a

"wrap-around" service for knowledge initiatives

.

(

Keep reading

)

What I've Been Reading

Rethinking the Purpose of Measurement
Measurement is not a simple act of observation disconnected from any larger plan. Instead, it’s an optimization strategy for reducing uncertainty about decisions we need to make. That’...
Solving the forgetting. Spaced repetition beyond rationality community.
by MrAKaDeus — 2019-10-03T15:03:06.903Z
Link: https://medium.com/@arkadiykossakovsky/how-to-make-spaced-repetition-mainstream-44072320f3bd?source=friends_link&sk=c3204e60f57cf27930bc3bd2fceeae44

Many of you have heard about spaced repetition. It's a learning technique that allows you to remember almost anything as long as you want. It works by repeatedly answering test questions with increasing intervals. The problem is it's not widely used (just like the art of rationality). Existing solutions are either limited to memorization of terms/foreign language words or require you to create all flashcards (test questions) yourself. But those who go through the struggle of creating flashcards for complex topics themselves show that spaced repetition can help in learning any topic.

The main hurdle to sharing flashcards is that you can't understand the question written by someone else if you don't know the topic quite well already. Therefore you have to start repetition when you understand the underlying concept. It seems that the best timing is right after you read about the relevant concept in a textbook or watch a lecture. The test question has to be integrated with the educational content.

The linked article describes the approach we take to implement this idea. We've already made a basic implementation. Now we're looking for those who want to try use it in their personal learning. But even more importantly those who are willing to experiment with creating courses for others. We'd appreciate any feedback you have!

https://medium.com/@arkadiykossakovsky/how-to-make-spaced-repetition-mainstream-44072320f3bd?source=friends_link&sk=c3204e60f57cf27930bc3bd2fceeae44
Can we make peace with moral indeterminacy?
by Charlie Steiner — 2019-10-03T12:56:44.192Z

The problem:

Put humans in the ancestral environment, and they'll behave as if they like nutrition and reproducing. Put them in the modern environment, and they'll behave as if they like tasty food and good feelings. Pump heroin into their brains, and they'll behave as if they want high dopamine levels.

None of these are the One True Values of the humans, they're just what humans seem to value in context, at different levels of abstraction. And this is all there is - there is no One True Context in which we find One True Values, there are just regular contexts. Thus we're in a bit of a pickle when it comes to teaching an AI how we want the world to be rearranged, because there's no One True Best State Of The World.

This underdetermination gets even worse when we consider that there's no One True Generalization Procedure, either. At least for everyday sorts of questions (do I want nutrition, or do I want tasty food?), we're doing interpolation, not extrapolation. But when we ask about contexts or options totally outside the training set (how should we arrange the atoms of the Milky Way?), we're back to the problem illustrated with train tracks in The Tails Coming Apart As Metaphor For Life.

Sometimes it feels like for every value alignment proposal, the arbitrariness of certain decisions sticks out like a missing finger on a hand. And we just have to hope that it all works out fine and that this arbitrary decision turns out to be a good one, because there's no way to make a non-arbitrary decision for some choices.

Is it possible for us to make peace with this upsetting fact of moral indeterminacy? If two slightly different methods of value learning give two very different plans for the galaxy, should we regard both plans as equally good, and be fine with either? I don't think this acceptance of arbitrariness is crazy, and some amount is absolutely necessary. But this pill might be less bitter to swallow if we clarify our picture of what "value learning" is supposed to be doing in the first place.

AIs aren't driving towards their One Best State anyhow:

For example, what kind of "human values" object do we want a value learning scheme to learn? Because it ain't a utility function over microphysical states of the world.

After all, we don't want a FAI to be in the business of finding the best position for all the atoms, and then moving the atoms there and freezing them...
Comments are back!
by jefftk — 2019-10-03T10:50:01.913Z

Instead of hosting comments on jefftk.com directly, I copy
over publicly-accessible comments from discussion elsewhere:

By default it's first-name only when copying from social media
(Facebook, Google Plus) and full name when copying from forums
(LessWrong, EA Forum), though I have it always use my full name for
clarity. While everything I copy over is already world-readable
without an account, some people don't want their comments copied (let
me know if that includes you!), in which case I show something like:

It's a little fragile, though, and about a year after I last fixed Facebook
comment inclusion it broke again. And Google Plus was turned off, so
I couldn't pull comments from there either. And the EA Forum and
LessWrong migrated to new software that didn't support the old (and
not very good) rss-based system. And my adapters for Hacker News and
Reddit broke too.

I've now gotten the main four working again, though it's not quite the
same as before:

Google Plus, being shut down, just serves a frozen archive as
of my last backup.

The EA Forum and LessWrong use
the GraphQL API and pull comments as needed, so crossposting is very
fast.

I have new code
for Facebook that runs selenium while logged out to build an archive,
and I'll figure out some system for updating it at some point.
Crossposting is very slow, like ~weeks.

At some point I may get Reddit and HN fixed, but I don't crosspost to
them very often so it's not much of a priority.

Comment via: facebook
[Link] (EA Podcast) Global Optimum: How to Learn Better
by Daniel_Gambacorta — 2019-10-03T00:29:01.964Z

I host the podcast Global Optimum. The goal of the podcast is to make altruists more effective. The most recent episode is about how to learn more effectively. I review the psychological literature on learning techniques and discuss the interplay between scholarship and rationality.

This episode features:

-What are the best and worst studying techniques?

-Do “learning styles” exist?

-How to squeeze more learning into your day

-How to start learning a new field

-How to cultivate viewpoint diversity

-How to avoid getting parasitized by bad ideas

-Should you study in the morning or at night?

-Can napping enhance learning?

Full transcript: http://danielgambacorta.com/podcast/how-to-learn-better/

The podcast is available on all podcast apps.

Listen here: http://globaloptimum.libsyn.com/how-to-learn-better