trentmkelly/LessWrong-43k huggingface dataset:
Natural Value Learning
by Chris van Merwijk — 2022-03-20T12:44:20.272Z

The main idea of this post is to make a distinction between natural and unnatural value learning. The secondary point of this post is that we should be suspicious of unnatural schemes for value learning, though the point is not that we should reject them. (I am not fully satisfied with the term, so please do suggest a different term if you have one.)

Epistemic status: I'm not confident that actual value learning schemes will have to be natural, given all the constraints. I'm mainly confident that people should have something like this concept in their mind, though I don’t give any arguments for this.

Natural value learning

By a "value learning process" I mean a process by which machines come to learn and value what humans consider good and bad. I call a value learning process natural to the extent that the role humans play in this process is basically similar to the role they play in the process of socializing other humans (mostly children, also asocial adults) to learn and value what is good and bad. To give a more detailed picture of the distinction I have in mind, here are some illustrations of what I'm pointingat, each giving a property I associate with natural and unnatural value learning respectively:

Natural alignment

Unnatural alignment

Humans play the same role, and do the same kinds of things, within the process of machine value learning as they do when teaching values to children.	Humans in some significant way play a different role, or have to behave differently within the process of machine value learning than they do when teaching values to children.
The machine value learning process is adapted to humans.	Humans have to adapt to the machine value learning process.
Humans who aren’t habituated to the technical problems of AI or machine value learning would still consider the process as it unfolds to be natural. They can intuitively think of the process in analogy to the process as it has played out in their experience with humans.	Humans who aren’t habituated to the technical problems of AI or machine value learning would perceive the machine value learning process to be unnatural, alien or “computer-like”. If they naively used their intuitions and habits from teaching values to children, they would be confused in important ways.

...
How to prevent authoritarian revolts?
by lc — 2022-03-20T10:01:52.791Z

I was browsing reddit mindlessly when I saw this maybe ~10 posts down from the top of r/all, and I kind of snapped:

I am not a communist. I think violent communist revolt would result in, at the very least, a period of significant loss of life and economic destruction. Successful communist revolts usually (but not always) produce brutal, authoritarian police states.

Yet it it seems to me like violent communist uprisings are fairly overtly supported by a sizable fraction of the American populace, perhaps 1-5%. When I talk to the larger fraction of the American populace that does not actively sympathize, they appear to be much, much more comfortable with this than I am. They speak as if the FBI was somehow omnipotent, or as if the internet has removed the possibility for popular revolt inside countries with strong civil liberties.

This optimism is insane. Authoritarian factions are adversaries. They are run by real people with real political ambitions who attempt to move around what you think are the tripwires. They will mask themselves even to internal supporters as "social democracy" or "alternative right" or what-ever, and take off the mask right as they are lining you up against a wall. When they do, our turtle-paced glorified three-letter bureaucracies stand a real and substantial chance of being too slow to react.

A large part of this procedure is the revolutionary alerting everyone of their pending revolution before anything actually illegal happens. Statements like the above accomplish a task in this vein: demonstrate widespread sympathy and tolerance for murder. They serve as an important coordination mechanism, and that is the job they are performing in the above post.

In the general case I think most people intuitively understand why this should make everyone nervous. If 1-5% of the United States were upvoting posts on Reddit explaining how they "unironically" support a fascist uprising and gassing all Jewish Foreign Agents, well... finish the sentence. I would at least not have to explain why this political propagandist threatening to kill a large number of people is a bad omen and why it's a cause for concern.

So I'm asking: what can I do now to help prevent this from happening? I don't want to wait ten years so the problem can get 10-% worse. I don't want to deflect by saying that the meteor has got at least 30 more years on its flight path befo...
How I fixed exercise
by GabrielExists — 2022-03-20T04:52:19.997Z

Hi, I'm Gabriel. I recognize that people don't usually introduce themselves in posts on here, but this is my first one, and it's gonna be a pretty personal one, so I might as well be a bit personable.

You might have seen me on the University of Bayes discord or the unofficial LessWrong telegram channels. On the latter a discussion around exercise appeared, and some people wanted to hear how I "fixed" exercise for myself. I was recommended to post this on here, so this post is an adapted version of what I said then.

I didn't exercise much growing up, and this resulted in an unhealthy relationship to exercise as an adult. I was starting this journey with a lot of shame and frustration around trying over and over to start something stable, and always failing once passion stopped. I had a huge huge Ugh Field around exercise, which made everything harder. So a lot of the journey was incrementally going from "I try to exercise, but I keep failing" to "Exercise generally works for me. Sometimes I stop, due to being sick, stressed or forgetful, but it's no big deal, and it's easy to resume it". I now have the identity of "someone who exercises", regardless if I do at that moment. This is the superpower, because this means the Ugh Fields are dissolved and there's no shame or frustration in the way. Without psychological barriers, deciding to exercise and actually exercising follow each other tightly.
This took me one a and half years of having exercise as a primary focus in my life. I had been trying to create an exercise habit before that, but that was mostly doing the same thing over and over, so I'm discounting that. One and a half years might be the time it takes, or it may be possible to do in less time.

So here's a bullet point list of the things that would have shortened my journey to exercise. All advice is bad, all journeys are different, but parts of your journey may be similar to mine, and may be made easier from hearing about mine.

If nothing works in your life, as it did for me, find one thing and make it work. Accept that everything else falls to pieces. For me, I had huge piles of dishes, hadn't folded laundry in months (picked it out of the dryer when I needed a shirt), my sleep was fucked, I ate crap things at weird times, I didn't clean the floors ever, everything was cluttered with stuff. Giving up on the other things doesn't change the state on the wor

...
Know of any randomised trials teaching debiasing in schools? Want to help review them, or would that review help your work?
by MNoetel — 2022-03-20T04:52:13.302Z

Last year Elon Musk tweeted a list of cognitive biases saying they should be be taught at a young age.

He got over 325,000 likes, but is he right? We want to find out.

We're planning a systematic review of randomised trials on debiasing interventions in schools and universities. We think a review like this will help us figure out how best to improve this component of rationality. We hope an academic paper will provide credibility so—if the teaching works—it can be embedded in curricula. If you think that's worthwhile, there are a few ways you could help. If you think we've got come critical blind spots, please let us know too.

Let us know about any randomised trials we might have missed

We've collated 10 'starter' references for the review. Basically, the review's goal will be to 'find everything like this.' We can be more sure we're likely to find everything if we have more of those references up-front. Do you know of any randomised trials testing debiasing? If it's not on the list, can you add them to this draft protocol or to the comments below?

Let us know if this review has already been done

We've found systematic reviews on a range of nearby topics (see Our Rationale), but haven't seen a systematic review on this topic specifically. If there is a recent systematic review that covers the same territory, we'd be basically wasting our time. We'd love to know that is the case before we dive too deep. So, if you know of any, please let us know in the comments.

If you work on teaching rationality, maybe join the study as an advisor

We want to make sure this review directly helps the work of places like CFAR, Clearer Thinking, SPARC, and so on. If you work at a place like these formally teaching rationality, we're looking for a few advisors who can help make the review more useful for real-world implementation. If that sounds like you, please email Mike with what you do and why this would be valuable.

If you want to help to make the paper happen, volunteer as a reviewer

If you're not working at an organisation like that but still want to help, our team at READI Research will be calling on volunteers who want to learn how to do a systematic review. We recently published a meta-review on how to reduce animal product consumption, and have one about promoting charitable donations under second review (preprint here). Both reviews are stacked with amazing authors who volunte...
On Context And People
by Jan — 2022-03-19T23:38:36.701Z
Link: https://universalprior.substack.com/p/on-context-and-people?s=w

TL;DR: Contrasting views of the research landscape, a prototype for automating scientific debate, and determining the importance of authorship. Also an analysis of the moral aspects of wearing socks.

Disclaimer: This is coming from a rather neuroscience-centric perspective. I'd be curious if the patterns I describe here apply to other fields.

Previously in this series:

On Scaling Academia

,

On Automatic Ideas

, and

On (Not) Reading Papers

.

Bottlenecks and Context

I'm still hung up on that question of how we might make research "scalable"[1]. Answering this question involves identifying bottlenecks and finding efficient ways to circumvent them. "Generating ideas"

appeared to be one such bottleneck

. But, as language models like GPT-3 shine when it comes to systematically generating ideas, the bottleneck effectively reduces to a search problem. And while the hacky prototype I threw together in

a previous post

probably does not quite qualify as a solution to "generating ideas", I feel pretty optimistic that we can crack this with a larger language model and some elbow grease.

A great idea, however, is not enough to scale research. Ideas need to be contextualized[2]. Did somebody work on the idea before? What do we know about the topic already? What are disagreements, confusions, and holes in the research landscape?

Finding those answers is a severe bottleneck to scaling; the problem gets harder as the number of established insights increases. I've argued before that

the situation is

already

pretty bad

and that researchers tend to be overwhelmed by the available literature. Current solutions (like "not reading papers") are only band-aids that will not hold when we try to scale research. Therefore, we will need to change how we manage our established knowledge to make research scalable. This post is an exploration of what that might look like.

Statement-centric view

What does the research landscape look like? When I started out in research, my mental

picture of the scientific landscape

looked like this:

This graph demonstrates how we can transform several formulations of the same statement (p) into each other by applying “simple” transformation rules.

i.e. a graph of (hopefully) true statements connected with each other. Think "The brain is made of neurons" or "

The mitochondria is the powerhouse of the cell

" and how you can connect these statements across layers of abstractio...
When will kids stop wearing masks at school?
by Daniel Kokotajlo — 2022-03-19T22:13:16.187Z

I'm visiting Berkeley now and I walk past the local elementary school and I see all these kids running laps around the basketball court. Wearing masks. Gosh, PE class was bad enough back in my day... I talked to a high schooler walking by and they said that (in their school at least) they aren't officially required to wear masks but they all do it anyway. Even outdoors.

How long will this go on? I'm interested because if the answer is "years" this influences my choice of where to send my daughter to school. Presumably not every school district is like this.

I created a prediction market for (a variant of) this question: https://manifold.markets/DanielKokotajlo/will-most-boston-public-school-kids

(For context: I am pretty against kids wearing masks these days, especially outdoors. Because I think the amount of micromorts thereby prevented is really, really, ridiculously small, too small to justify the inconvenience. I'd be interested to see if anyone has crunched the numbers and come to a different conclusion. I haven't crunched the numbers myself recently.)
Exploring Finite Factored Sets with some toy examples
by Thomas Kehrenberg — 2022-03-19T22:08:55.817Z
Link: https://tm.kehrenberg.net/a/finite-factored-set-example/

Seeing as there is little secondary literature for the Finite Factored Set formalism, I thought I’d write up my experience of exploring it through some toy examples that are classic examples in the Pearlian paradigm. My goal was to see how these models that I understood very well in the Pearlian paradigm would work in Finite Factored Sets.

As a warning, this doesn’t make any use of the more exciting properties of Finite Factored Sets. It’s just an exploration of how this formalism handles the mundane stuff. This also means that I’m using the factored set directly, without the abstractions of the orthogonality database. Which I think is fine here, because these are tiny toy examples whose structure is fully known. (However, it’s possible that I’ve missed the entire point of Finite Factored Sets.)

The first example is the 3-variable collider that is very central to Pearl’s formalism. It is given by the following Causal Diagram:

A, B, and C are all binary variables (0=false, 1=true).

The intended meaning of a Causal Diagram (or rather, function causal model) is that the value of a node xi is given by a deterministic function that takes as input the parents, pa(xi), (indicated by the arrows) and an “error”, or “noise”, variable ui that is governed by a probability distribution that is independent from the other error/noise variables: xi=fi(pa(xi),ui). Thus, the value of C is given by c=fc(a,b,uC) where uC is noise or uncertainty that is not explicitly modeled, which we can visualize like this:

We could also split up A and B into a deterministic and a random part, but as they are root nodes, there is little point. It would just be a=fa(uA).

The Pearlian formalism runs on graphs, but Finite Factored Sets run on the set S of all possible outcomes – the sample space. So, the goal is now to construct a sample space that is consistent to the above graph. After that, we’ll find a factorization of that sample space.

I think it should be clear that to cover the whole sample space S, it is sufficient to consider all possible combinations of the outcomes of A, B, and UC (but not C), because if we know the value of these three, then we also know the value of C, via fc.

So, we simply define S as the Cartesian product of the sets of possible values of A and B: A={0,1}, B={0,1}, and the possible values of UC: UC, which I’ll leave und...
Rationalists Should Learn Lock Picking
by brook — 2022-03-19T18:00:18.354Z

To be clear, I'm not suggesting a widespread movement within rationalism to begin skilling up and stealing things to fund X-risk research. I promise.

What I am suggesting is that I found the first ~3-5 hours of learning to pick locks (which I did from sheer boredom) to be valuable in ways I hadn't anticipated, mostly to do with forming models of things with very limited (and weird) information about them. It requires smushing your brain up against the territory in interesting ways; it's motorcycle maintenance writ small.

Picking a lock is a pretty unique experience. A lock is made up of a cylinder inside another cylinder, with some 'pins' which are spring-loaded to enter the smaller cylinder and prevent it from rotating relative to the larger cylinder[1].

The pins are of different lengths, so the intended use is to open the lock with a key that pushes each of the pins to a height where they're not blocking rotation, then apply tension to the lock so that it opens.

Because of a seemingly hard-to-fix fault in the process of making locks, the holes that the pins fall into are slightly different sizes. This means that if you apply rotational tension to an empty lock, only one of the pins is stopping the lock opening at any given time.

This also means that if, while still applying tension, you put some funny-shaped tool into a lock to push down gently on the pins one by one, the one which is currently blocking the lock feels subtly different because of the friction with the side of the hole. There'll also be a subtle change[2] in the tensions (of the pins and of the cylinder) when that pin is in the right position; the cylinder will advance slightly. Rinse and repeat and voila, your lock is open.

Obviously it's not quite this easy in (most) real locks. But it is surprisingly easy. Those already afraid of their house being broken into need not apply.

The real point, though, is that you're trying to manipulate a machine which you only get information about through your hands, which you also have to use to manipulate the machine into doing what you want. It reminds me of the game Understand[3]; success is completely dependent on modelling the problem, actually noticing when (and how) your model is wrong, and trying to figure out why. It's also very frustrating.

If you do feel inspired, you might want to try starting with a tran...
Wargaming AGI Development
by ryan_b — 2022-03-19T17:59:27.846Z

I think wargames would be a useful tool for exploring AGI development scenarios. More specifically, I mean wargames of the kind used to train military officers in strategy. I think this format would help explore things like timelines and takeoff scenarios, in a way that is both concrete and transparent.

The thought cropped up because I have been popping into the AMA/Discussion post for Late 2021 MIRI Conversations over the course of the last week. Over the course of the MIRI conversations, and again in the AMA, I saw a lot of the tactic of offering or requesting concrete situations to test or illustrate some point. Wargaming feels to me like a good way to generate concrete situations. I think we could answer this question from So8res:

What's a story, full of implausibly concrete details but nevertheless a member of some largish plausible-to-you cluster of possible outcomes, in which things go well? (Paying particular attention to how early AGI systems are deployed and to what purposes, or how catastrophic deployments are otherwise forestalled.)

Over the same period I was reading the MIRI Conversations I was reading/re-reading a series of posts on wargaming at War on the Rocks, which I was linked to from the comments in a book-notes post about military innovation.

The oldest article of the lot focuses on the usefulness of wargames for teaching and learning; the original motivation for the author was how poorly they thought teaching a course on Thucydides went. As a highlight, I would like to point out a case where they considered a decision that had historically poor outcomes, but gained awareness of good motivations:

Remarkably, four of the five Athenian teams actually attacked Syracuse on Sicily’s east coast! As they were all aware that such a course had led to an Athenian disaster 2,500 years before, I queried them about their decision. Their replies were the same: Each had noted that the Persians were stirring, which meant there was a growing threat to Athens’ supply of wheat from the Black Sea. As there was an abundance of wheat near Syracuse, each Athenian team decided to secure it as a second food source (and simultaneously deny it to Sparta and its allies) in the event the wheat from the Black Sea was lost to them. Along the way, two of the teams secured Pylos so as to raise helot revolts that would damage the Spartan breadbasket. Two of the te

...
Can you be Not Even Wrong in AI Alignment?
by throwaway8238 — 2022-03-19T17:41:26.197Z

In January, I submitted a response to the Eliciting Latent Knowledge problem. I received a response with some short comments on what I labeled "Experience engaging with the problem", but no response to my approach, setup, or analysis. I subsequently followed up to no further response.
I suspect that the reason for the limited engagement has to do with the quality of my analysis. I think it's hard to read these things and people have limited time. If there were obvious deficiencies with my analysis in the context of how the field prefers to reason, I trust that those would be stated. However, if my analysis is Not Even Wrong, and the odds of me being helpful in the field is too low to warrant comment, then silence is a logical choice.
So, I present three questions:
1) Suppose my above suspicion is correct; that indeed, I am simply Not Even Wrong. Regardless of the field, how does one earnestly engage to find out why? I considered appealing to credentialism, but I don't want to reveal my identity. In my followup, I elected to demonstrate that I understood another poster's response and suggested that it fits into my analysis. What would you do differently?
2) AI safety is a domain where we want many different approaches. We want people from as many domains as possible working the problem. If a military general said they wanted to spend 5 years working on it, we would be foolish not to fund them. Hardware manufacturers, arms dealers, security professionals, legislators, power brokers, etc. In addition, we don't necessarily want these efforts to overlap. Most people in these industries would be Not Even Wrong. How can we attract and fund those efforts?
3) Below, I have included my full response, less one paragraph about contact information, as well as my followup. Am I Not Even Wrong? I want to stress that this is not particularly important, though it may impact my interest in contributing to AI safety now or in the future. I feel like an immediate response to this entire post is "well, what did you write?", and so I offer it at the end of this post.
Finally, for good measure, I want to make it clear that I am not at all trying to throw shade on ARC or the individual who responded to me. I am in the set of people who cares deeply about X-Risk and am thankful that talented people are dedicating their lives to AI alignment. They owe me nothing. They read my response, and...