trentmkelly/LessWrong-43k huggingface dataset:
Models of moderation
by habryka — 2018-02-02T23:29:51.335Z

[Author's note: I will move this into meta in a week, but this is a bit more important than the usual meta-announcements, so I will have it in community for a bit.]

Meta

This post is trying to achieve roughly four things:

1. Be a future reference for some hopefully useful models about moderation

2. Give people a model of me in particular and how I think about moderation (since I have a bunch of control about moderation on the site)

3. Ask the community for input on the moderation systems we should have on LessWrong

4. Provide some context for moderation changes we have planned that we will explain in a follow-up post

Thoughts on moderation

I think when making my decisions about moderation, there are at least five major models that drive my decisions:

People are competing for limited resources, and moderation helps to move the people participating into a better Nash equilibrium by enacting legislation that rewards cooperative behavior and punishes defective behavior. [microeconomic frame]
There is a distinction between adversarial and non-adversarial states of mind, and the goal of a moderation policy is to cause participants to generally feel safe and deactivate their adversarial instincts. [safety frame]
There is a limited amount of bandwidth available to communicate, and the goal of a moderation policy is to allocate that bandwidth to users that will use that bandwidth for the greatest common good. [bandwidth allocation frame]
There is a shared methodology that underlies the rationalist community that establishes what forms of reasoning are effective, and what perspectives on the world are fruitful. The goal of a moderation policy is to nudge people to use the approaches to reasoning that actually work (and that the community agrees on work). [methodology frame]
We don't really know what moderation policies and technologies work before we see them, but we can judge fairly easily what discussions have been productive. Different people might also need different moderation policies to be effective. So the most important dimension is to encourage experimentation of different moderation principles and technologies, and then set up incentives for the most effective ways of moderation to propagate. [libertarian frame]

I think all of these describe meaningful aspects of reality and are lenses I often apply when thinking about moderation and culture on LessWrong. I wa...
Questions for an AGI project
by  — 2018-02-02T19:45:34.211Z

I've been thinking a bit about what would cause me to support an AGI project and thought it might be interesting to others, and I'd be interested in other risks or questions.

The questions would be about discovering the projects stance on various risks. By stance I mean.

How they plan to find out information about the risk ?
What their threshold is for acting on a risk?
What they will do when they reach the threshold?
Who owns this risk and process?

The types of risks I am interested in are.

Typical unfriendly foom situation
Asymetric deployment of AI causing potential war/political problems as discussed here
Uneven deployment of AI causing massive inequality and depression as people can no longer be actors in the world or their lives.
Deployment of AI causing humanity to speed up and magnify it's conflict and competition. Burning through it's resources. We had massively more brain power/compute since the industrial revolution, but it can seem touch and go that we are going to get off the planet permanently even with that, will AI been any better?

So for foom, they might do things like agi estimation where you try and estimate the capability of your part of an AGI at a task. If it turns out to be vastly better than you expect or your estimation is that it will do science vastly better than humans straight out of the box, you halt and catch fire and try and do some ethics and philosophy to get a good goal straight away.
Hammertime Day 5: Comfort Zone Expansion
by alkjash — 2018-02-02T18:00:00.545Z

This is part 5 of 30 in the Hammertime Sequence. Click here for the intro.

It would be hypocritical of me to write a post of my usual form to teach Comfort Zone Expansion. Instead, I’ll explain why the Disney song How Far I’ll Go is a triumphant call to exploration, and leave a short CoZE exercise that you should modify with the principles of Moana in mind.

Background

Comfort Zone Expansion (ironically named CoZE) is CFAR’s version of exposure therapy, designed to get people to try new things cautiously. When I first heard of CoZE, what came to mind was something like run naked into a crowded Starbucks and ask strangers to finger-paint my buttcheeks. Although there might be some value to such an exercise, CoZE is decidedly not that. The first step of CoZE is simply trying things you’ve never bothered to try, even though you have no resistance to them.

Let me call attention to some metaphors for talking about Comfort Zones.

Order and Chaos

One way to visualize your comfort zone as the dividing line between Order and Chaos.

Order is the known. Order is your social circle, the interior of your home, the streets you drive regularly. Order is the programming languages you’re familiar with, the sports you play, the languages you speak. Order is the rules you follow. Order is your comfort zone.

Chaos is the unknown – or worse the unknown unknown. Chaos is staring momentarily into a stranger’s eyes. Chaos is the antsy feeling you get turning just one street away from your usual route. Chaos is the feeling that the world has shifted beneath your feet when you break your code, when you find out you’ve been lied to, when you notice you’re deep into a mistake. Chaos is the amorphous shadow that expands gas-like to fill every space you don’t pay attention to.

Yang and Yin are Order and Chaos, and the Yin-Yang is the Daoist reminder that the proper Way through life is to navigate the twisting line between Order and Chaos.

For a more CS-friendly metaphor, consider staying within Order as Exploiting well-understood strategies and going into Chaos as Exploring new strategies. Moloch is the civilizational disaster that occurs everyone decides to Exploit by sticking within their comfort zones. Except for very young children, people categorically Explore too little and stagnate in local optima.

The Structure of Pop Songs

Jordan Peterson had an illuminating dialogue with composer Samuel And...
The Monthly Newsletter as Thinking Tool
by moridinamael — 2018-02-02T16:42:49.325Z

At the start of 2014 I accidentally started writing a monthly newsletter for a handful of close friends, and I've mostly kept it up since then. It was originally supposed to be kind of commitment device for myself, a way of holding myself accountable to my 2014 goals by commiting to sharing my progress with those friends at the end of each month. It continued to serve that function for the next four years, but also became something much more useful than that.

The easiest, very high ROI step is to simply open up a new Evernote document on the first of each month and title it "February 2018 Update" or whatever. Then just try to keep that tab open. You'll find stuff to put in there. What I end up writing generally falls into a small number of categories:

Links to things that I think my friend would be interested in, with a bit of discussion of why I think it's interesting. These also serve as future reference material for me.
Updates on my progress toward some goal or another, usually written in a style meant to be at least readably entertaining. This lets me look back over the years and see exactly what I was doing and when.
Discovery of some new thing that obsesses me briefly that prompts me to write 10,000 words of evangelism about it (e.g. meditation, hypnosis, trigger point therapy, Alexander technique, Ghokale method posture, jiu-jitsu, longevity supplements, some new AI architecture, Mr. Rogers, EverQuest as an exemplar of Fun Theory, the ketogenic diet, at least four different exercise regimens) which serves as very useful reference material, which I tend to frequently refer back to when the topic ends up being something that I make part of my life.
At one point I started writing a story for my friends and sending it to them in installements, which gradually turned into a 50,000 word book over the course of a year.

You don't even have to share it with anybody, I suppose, but I suspect you'll find it much more motivating to actually write in it if you do intend to share it. I created a private Blogger blog in which all these monthly newsletters reside. You could probably do the same thing with a public blog.

That might prompt you to remark, "Congratulations, genius, you've rediscovered blogging." Perhaps that's fair. But I think it does help that I know I'm only writing for a small handful of close fri...
The Utility of Human Atoms for the Paperclip Maximizer
by avturchin — 2018-02-02T10:06:39.811Z

TL;DR: use of humans’ atoms will increase AI’s expected utility by 0.00000000000000002 of total U.

The iconic example of the existential risk of superintelligence is the so-called paperclip maximizer, that is, a system which maximizes some random goal not aligned with human values. While a real paperclip maximizer is unlikely, convergent dynamic subgoals suggested by Omohundro, imply that almost any non-aligned AI will try to acquire as much computational power as possible, as well as other material resource. Thus, it will need the atoms of humans.

In this article, I will try to estimate the marginal utility of human atoms U¢ compared with total utility of all atoms’ U for the AI.

I assume that:

A) Superintelligent AI which is interested in human atoms, is limited to the Solar System. The reasons for this is that it may need human atoms to build spaceships to travel to the nearest stars, and if it reaches the stars, the value of most human atoms will be lost.

B) Superintelligent AI has powerful nanotechnology, which could be used to do whatever it wants with atoms, like build large space ships, deconstruct planets down to their final atoms etc.

C) The only resource that AI is interested in is atoms—not energy, not information etc.

D) The utility of atoms’ U is increasing linearly with increases in the available mass.

The weight of Solar System planets is 0.135% of the Sun’s mass, which is 1.98855×10E30 kg, and the mass of all planets is 2.6E1027kg, where most of this mass is Jupiter. Earth’s mass is 5.97237×10E24 kg.

The total mass of human beings on Earth is 70kg х 7.5x10E9 people = 5.25x10E11kg.

In that case, human atoms constitute marginal utility U’= 2x10E-16U. That is, use of the humans’ atoms will increase the AI’s utility by 0.00000000000000002 of total U.

But this is obviously flawed, as humans can’t survive in a vacuum, they need the whole Earth!

Imagine that AI wants to deconstruct the Earth for its atoms, but also decided to preserve human lives. It has two options:

1) Upload everybody into smaller computers. Based on various estimations of the Launder limit, and the computational capacity of human brains, the size of these computers will be different, but they could easily be 1000 times less than human bodies.

2) AI decides not to upload humans, but to build a space station, where humans can live approximately the same life as they do currently. As the typ...
AI Safety Research Camp - Project Proposal
by David_Kristoffersson — 2018-02-02T04:25:46.005Z

AI Safety Research Camp - Project Proposal

→ Give your feedback on our plans below or in the google doc
→

Apply

to take part in the Gran Canaria camp on 12-22 April (deadline: 12 February)
→

Join

the Facebook group

Summary

Aim: Efficiently launch aspiring AI safety and strategy researchers into concrete productivity by creating an ‘on-ramp’ for future researchers.

Specifically:

Get people started on and immersed into concrete research work intended to lead to papers for publication.
Address the bottleneck in AI safety/strategy of few experts being available to train or organize aspiring researchers by efficiently using expert time.
Create a clear path from ‘interested/concerned’ to ‘active researcher’.
Test a new method for bootstrapping talent-constrained research fields.

Method: Run an online research group culminating in a two week intensive in-person research camp. Participants will work in groups on tightly-defined research projects on the following topics:

Agent foundations
Machine learning safety
Policy & strategy
Human values

Projects will be proposed by participants prior to the start of the program. Expert advisors from AI Safety/Strategy organisations will help refine them into proposals that are tractable, suitable for this research environment, and answer currently unsolved research questions. This allows for time-efficient use of advisors’ domain knowledge and research experience, and ensures that research is well-aligned with current priorities.

Participants will then split into groups to work on these research questions in online collaborative groups over a period of several months. This period will culminate in a two week in-person research camp aimed at turning this exploratory research into first drafts of publishable research papers. This will also allow for cross-disciplinary conversations and community building, although the goal is primarily research output. Following the two week camp, advisors will give feedback on manuscripts, guiding first drafts towards completion and advising on next steps for researchers.

Example: Multiple participants submit a research proposal or otherwise express an interest in interruptibility during the application process, and in working on machine learning-based approaches. During the initial idea generation phase, these researchers read one another’s research proposals and decide to collaborate based on their shared ...
Further Progress on a Bayesian Version of Logical Uncertainty
by Diffractor — 2018-02-01T21:36:39.000Z

I’d like to credit Daniel Demski for helpful discussion.

So, the Eisenstat prior mentioned earlier seemed like a rather fundamental advance on the state of the art in logical uncertainty. Given that it was essentially a modification of the Demski prior, the obvious next question was “what’s the analogue of the modified Demski prior in this setting?”

As a quick refresher, the Demski prior sampled sentences, while the modified Demski prior sampled turing machines that would spit out a (potentially infinite) list of sentences. The key advance of the modified Demski prior over the Demski prior is that the modified Demski prior enables assigning positive probability to certain computable infinite sequences of sentences. The prototypical example was assigning positive probability to first-order PA with its infinite axiom schema, but this property also permitted the result in the logical induction paper where a turing machine could be asymptotically trusted to only output true things, even if the proof length of the statements grows ever-further beyond our ability to check.

The Demski and modified Demski prior, in their true forms, involve calling a halting oracle to determine whether a contradiction is present when a sentence/infinite sequence of sentences is added to the “sentence pool”.

A rather simple and general (though not very efficient) process for approximating the probability of an event under some prior involves specifying some sort of “prior-generating process”, and then monte-carlo sampling it a whole lot of times and seeing how many runs generated the event. In the case of the Demski and modified Demski prior, the use of a halting oracle means the prior-generating process can’t be emulated. However, since a halting oracle is limit-computable, throwing a whole lot of computing resources at consistency checking yields a better and better ability to emulate what the true version of the Demski/modified Demski priors are doing.

The key property of the Eisenstat prior that makes it ε-approximable instead of just limit-computable is that there’s a particular time by which you’re nearly guaranteed to run into either ϕ or ¬ϕ. Any sort of “modified Eisenstat prior” should preserve this property.

So, without further ado, I’ll present the algorithm, go into the philosophy of why it is the way it is, handwave at a few nice properties it probably has, and point out futur

...
The Self: Momentary vs Lifetime
by squidious — 2018-02-01T21:30:27.761Z

When you think of the Self, do you imagine it as something growing and changing, or consistent over a person's life? The definition of the Self is arguably one of the biggest differences between classical and postmodern psychology. Classical theories have their basis in the idea of a True Self, a fundamental unchanging part of a person that stays with them since birth. They may talk of upbringing, life circumstances, and social influence as things that change behavior, but they never change who you are underneath all of that. An artist may always be an artist, even in a world where they had no choice but to sell bread and raise children.

Postmodern theories of psychology look at the ways our behavior changes overtime, and instead see the Self as that which changes. Just as a person can change what they like, they can change who they are. A rambunctious teenager can change and become a stoic adult, and this isn't because they became any more or less in touch with a True Self, but because that Self changed, whether due to social pressure or just maturity.

These different perspectives change the ways clinicians approach their clients. A therapist who believes in the True Self may push a client to become more in touch with lost parts of themselves, while a more postmodern therapist might push a client to grow and change, exploring new hobbies and not remain attached to certain parts of their identity. I want to propose a third idea, which is that this theoretical rift is caused by two different definitions in the word Self.

In a sense, this debate is no different from nature versus nurture. While popular science shifts focus between the two, the truth is somewhere in the middle. Some of who we are is in our genes (brown eyes, wavy hair, a bit of a sweet tooth). The rest (taste in music, friends, hobbies) is influenced by how we are raised. So it can't be radical to suggest that between the True Self and the Changing Self, both as well as neither are true.

The two types of self I want to talk about are those I call the Momentary Self and the Lifetime Self. While there are parts of a person that never change (LS), it is impossible to ignore the ways in which parenting, culture, trauma, and major life events can change a person (MS). Difficulty comes with distinguishing which parts of a person are part of which Self.

Perhaps the most salient example of th...
Beta-Beta Testing: Frontpage Rework [Update - further tweak]
by Raemon — 2018-02-01T20:12:46.987Z

We have a rework of the front page on Lesserwrong, that we'd like to sanity-check before either deploying live or putting more work into.

We've deployed it to our beta-beta site, Lessestwrong.com. (The main shift here is in how the list of posts work, and how users will interact with them by default). If possible, I recommend logging in there since a lot of the features have to do with what information is presented to logged in users.

(it's a cloned database from a few months ago. If you created an account more recently you may need to recreate it there)

I'm interested both in people's first impressions, and followup thoughts after reading my explanation of what's going on and why.

www.lessestwrong.com
Hammertime Day 4: Design
by alkjash — 2018-02-01T19:00:01.156Z

This is part 4 of 30 in the Hammertime Sequence. Click here for the intro.

A central theme of Hammertime is that rationalists interact with reality first. We try for at least five minutes. We build habits to solve the bugs. We stick our necks out and ask reality for rapid feedback. Only after doing our due diligence and getting beaten back by reality should we turn to introspection.

That’s why the first five Hammertime techniques are for getting out and solving problems immediately. Only after interacting with reality and actually trying do we get to turn inwards to meditate, to question our motives, to get in touch with our feelings, and to make long-term plans.

Design is the most subtle approach for directly solving problems. It’s about permanently distorting the physical reality around you to propel you towards – instead of away from – completing your goals.

Day 4: Design

Design (sometimes taught as Systemization at CFAR) is rationalist fengshui. Its main goals are:

To inject intentions into the physical space you live and work in.
To free attention from unnecessary and repetitive distractions.

Design principles work across domains: in the establishment of routines, in the shaping of social environments, and in the organization of screen space. For the first cycle of Hammertime we will focus on Designing physical space to make concrete and immediate improvements.

Here are the three core principles of Design, according to yours truly.

1. Intentionality

The very first tear I shed at CFAR was in response to Valentine’s speech in Design class about the subtle machinations of Moloch infecting the very space around us:

The counter-top next to the front door that attracts mountains of junk like a gravitational well.

The dressers that hide away our running clothes and with them our good intentions.

The disarray that sends us stumbling back and forth to find our glasses, wallet, keys, and phone in the morning.

Moloch’s servants appear wherever attention is lacking.

The first principle of Design is Intentionality: things are where you intend them to be. Look around your room or desk. Everything should have a purpose. The purpose can be functional, but it also can be sentimental or aesthetic. You can intentionally organize things in the order you use them. You can intentionally arrange things in a pretty way. You can also intentionally leave a mess because you aspire to discover th...
Teaching rationality
by Sønderjye — 2018-02-01T16:07:11.672Z

I am interested in promoting rationality in my local community. As I don't know anyone who are involved in the movement I'll probably have to do some initial teaching. Which, in your oppinion, are the most essential/fundemental parts of rationality, and of these which are most accessible to newcomers?

While I learned a lot from Yudkowski's 'Rationality: from AI to Zombies', it's my impression that large parts of it would be confusing and discouraging to anyone without technical knowledge and in particular strong math skills.

The Bayesian Conspiracy, Waking up with Sam Harris and Rationality Speaking all provide valueable skills but from what I've experienced they all teach rationality in an indirect way rather than focusing on teaching it directly.

Does any exercises that help teach rationality exist?