trentmkelly/LessWrong-43k huggingface dataset:
The Case for a Bigger Audience
by John_Maxwell — 2019-02-09T07:22:07.357Z

How do people feel like LW 2.0 is going? I'm impressed with the number and quality of posts that are being made, especially relative to the baseline of what LW 1.0 was like right before the relaunch. But I miss the lively discussions in the comments from the "Golden Age" of LW 1.0. Consider the Craft and Community sequence, written right when Eliezer transitioned from writing for Overcoming Bias to writing for Less Wrong. Here are six posts from that sequence which were especially memorable. On average, they have 179 comments. Looking at the "Curated Content" on the homepage right now, the 3 curated posts average only 19 comments, even though they've all been up for at least 2 months.

A big audience lets your posts to have a greater impact. (Have any posts from LW 2.0 generated new conceptual handles for the community like "the sanity waterline"? If not, maybe it's because they just aren't reaching a big enough audience.) Sometimes your audience generates interesting new ideas you hadn't thought of. And there's a virtuous cycle: People will write more comments if they have a justified expectation of comment readership and replies.

I think the biggest risk to LW 2.0 at this point might be that authors who invest in making posts find that they don't seem to be getting significant readership, making a significant impact, or generating useful feedback. There are a lot of people making posts right now, but there's a risk those people will drift away. If we can get the virtuous cycle going, that risk is lessened.

There's always the fear of Eternal September, but I think the rest of the internet has gotten more addictive since LW 1.0, so just being a website where longform essays are posted already gets you an audience that's selected for having a long attention span. And of course, countering Eternel September is a huge part of the motivation for the new voting system.

Some promotional ideas to consider:

Make use of the Less Wrong Facebook and Twitter feeds to highlight new content. (Is there still an RSS feed going? If anyone is still using RSS, it's probably the rationalist crowd.) The Twitter account hasn't posted since 2016; could probably benefit from a relaunch tweet.

Send a one-time email to old LW 1.0 users with high karma who haven't logged in since the relaunch, announcing LW 2.0's launch and reminding them that their high karma puts them at

...
Can someone design this Google Sheets bug list template for me?
by Bae's Theorem — 2019-02-09T06:55:51.652Z

Project:

My local rationality group is going to perform a structured test of the 'Hammertime' sequence. We are making a "bug list" of various problems in our lives for this sequence. (For those familiar, this variant of the bug list is loosely inspired by the FMEA) We assign, on a scale of 1-5, three different values to each bug: "Impact" (on our life), "Frequency" (of occurrence, or need of occurrence for things we need to do), and "How cheap is the cost" (in time/effort/money, where 1 is most expensive, and 5 is cheapest). The products of "Impact" and "Frequency" will all be totaled together, resulting in our "RPN" ("Risk Priority Number"), and this is the number we will be testing before and after performing the sequence, to measure its success.

Goal:

As I adjust the assigned values of each bug, eliminate bugs, and add new ones, I want the spreadsheet to calculate the product of the first two values, and auto-sort the list by this number, giving me a live priority list for addressing the bugs. I also want the ability to sort the list by the product of all three numbers, allowing me to optionally factor in the difficulty. This will result in 6 columns:

1-Name of Bug

2-Impact

3-Frequency

4-"Cheap Cost?"

5-Product of 2 and 3 (default auto-sorting)

6-Product of 2, 3, and 4 (optional sorting)

As well as a final row (or column, or whatever) with the total sum of Column 5's values (the "RPN").

Problem:

I suck at spreadsheets, having never learned them. I already made an attempt, but failed. I could put the time and effort into learning this, but I don't foresee using this skill again, so it seems more optimal to use someone else's skills for this.
Reinforcement Learning in the Iterated Amplification Framework
by William_S — 2019-02-09T00:56:08.256Z

When I think about Iterated Amplification (IA), I usually think of a version that uses imitation learning for distillation.

This is the version discussed in the Scalable agent alignment via reward modeling: a research direction, as "Imitating expert reasoning", in contrast to the proposed approach of "Recursive Reward Modelling". The approach works roughly as follows

1. Gather training data from experts on how to break problems into smaller pieces and combine the results

2. Train a model to imitate what the expert would do at every step

3. Amplification: Run a collaboration of a large number of copies of the learned model.

4. Distillation: Train a model to imitate what the collaboration did.

5. Repeat steps 3 and 4, increasing performance at every step

However, Paul has also talked about IA using reinforcement learning (RL) to maximize the approval of the amplified model. What does this approach (RL-IA) look like? How does it relate to Imitation-IA and Recursive Reward Modelling?

Puzzling about RL-IA

To get an agent that takes good actions in an Atari game, we use Imitation-IA to build a system that answers the question "how good is it to take actions from this state", then train a reinforcement learner to "output the best action to take from a given state".

But there it seems like the improvement stops there - it's not clear how "ability to output the best action to take from a given state" could improve "ability to evaluate how good actions are good from a state" in any way that's different from running a traditional reinforcement learning algorithm (which usually involves taking some policy/value estimate and gradually improving it).

Clarifying what RL-IA does

Claim: There is a fairly straightforward correspondence between how Imitation-IA and RL-IA perform a task (given no computational limits). RL-IA does not change the class of tasks that Imitation-IA can perform or perform them in a radically different way.

Suppose we have a current version of the model M1 that takes questions and produces a distribution over answers. Let M2 be an amplified version of that model (ie. produced by running a number of copies of M1). Let Y be some question, with domain of answers D. We want to find the answer X* that is the answer in D which maximizes the approval of amplified overseer, M2("How good is answ...
HCH is not just Mechanical Turk
by William_S — 2019-02-09T00:46:25.729Z

HCH, introduced in

Humans consulting HCH

, is a computational model in which a human answers questions using questions answered by another human, which can call other humans, which can call other humans, and so on. Each step in the process consists of a human taking in a question, optionally asking one or more subquestions to other humans, and returning an answer based on those subquestions. HCH can be used as a model for what Iterated Amplification would be able to do in the limit of infinite compute. HCH can also be used to decompose the question of "is Iterated Amplification safe" into “is HCH safe” and “If HCH is safe, will Iterated Amplification approximate the behaviour of HCH in a way that is also safe”.

I think there's a way to interpret HCH in a way that leads to incorrect intuitions about why we would expect it to be safe. Here, I describe three models of how one could think HCH would work, and why we might expect them to be safe.

Mechanical Turk: The human Bob, is hired on Mechanical Turk to act as a component of HCH. Bob takes in some reasonable length natural language question, formulates subquestions to ask other Turkers, and turns the responses from those Turkers into an answer to the original question. Bob only sees the question he is asked and thinks for a short period of time before asking subquestions or returning an answer. The question of "is HCH corrigible" is about "how does the corrigibility of Bob translate into corrigibility of the overall system"? To claim that HCH is safe in this scenario, we could point to Bob being well-intentioned, having human-like concepts and reasoning in a human-like way. Also, since Bob has to communicate in natural language to other humans, those communications could be monitored or reflected upon. We could claim that this leads the reasoning that produces the answer to stay within the space of reasoning that humans use, and so more likely to reflect our values and less likely to yield unexpected outcomes that misinterpret our values.

Lookup Table: An AI safety research team lead by Alice writes down a set of 100 million possible queries that they claim capture all human reasoning. For each of these queries, they then write out the subquestions that would need to be written, along with simple computer code that combines the answers to the subquestions into an answer to the ...
The Hamming Question
by Raemon — 2019-02-08T19:34:33.993Z

This is a stub post, mostly existing so people can easily link to a post explaining what the Hamming question is. If you would like to write a real version of this post, ping me and I'll arrange to give you edit rights to this stub.

For now, I am stealing the words from Jacobian's event post:

Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.

CFAR developed the technique of "Hamming Questions" as different prompts to get your brain to (actually) think about the biggest problems, bottlenecks, and unspoken desires in your life.

A transcript of Hamming's extensive 1986 talk "You and your research", touches upon a several elements of Hamming's philosophy, and includes this anecdote about the canonical "Hamming Question":

Over on the other side of the dining hall was a chemistry table. I had worked with one of the fellows, Dave McCall; furthermore he was courting our secretary at the time. I went over and said, "Do you mind if I join you?" They can't say no, so I started eating with them for a while. And I started asking, "What are the important problems of your field?" And after a week or so, "What important problems are you working on?" And after some more time I came in one day and said, "If what you are doing is not important, and if you don't think it is going to lead to something important, why are you at Bell Labs working on it?" I wasn't welcomed after that; I had to find somebody else to eat with! That was in the spring.

In the fall, Dave McCall stopped me in the hall and said, "Hamming, that remark of yours got underneath my skin. I thought about it all summer, i.e. what were the important problems in my field. I haven't changed my research," he says, "but I think it was well worthwhile." And I said, "Thank you Dave," and went on. I noticed a couple of months later he was made the head of the department. I noticed the other day he was a Member of the National Academy of Engineering. I noticed he has succeeded. I have never heard the nam

...
Make an appointment with your saner self
by MalcolmOcean — 2019-02-08T05:05:49.784Z

Most people have had the experience of being able to articulate advice that they themselves do not follow, even though it applies to their situation as well. Usually this implies that there's some sort of internal conflict present—a competing commitment that gets in the way of doing the thing that the person might consider reasonable. I have written much on transcending and untangling internal conflict (see these posts) and I will write much more.

But transcending internal conflict can be a lengthy, complex, and non-monotonic process, and in the meantime you're still sitting around with a bunch of great advice you're not taking. A bunch of untapped potential.

There's a really straightforward technique that can help with this:

make an appointment with your saner self.

Put an event on your calendar, and treat it with the respect you'd give any other appointment. Which is to say: show up. Or, if for some reason it turns out you can't, then reschedule for the nearest appropriate time.

Then, when the time comes, take your own advice. You can do this literally—consider what advice you'd give a friend in your situation, then do that—or you can just do the obvious thing. You can do this with specific object-level situations, eg "I need to get around to submitting that application" or with more abstract things like "I really should take more time to reflect on my life."

Or perhaps you've got a technique that you know really helps you, whenever you do it, but you never seem to do it. "If I actually used the CFAR techniques, my life would be way better," said almost every CFAR alumnus ever. Well, make an appointment with your saner self (the one who does the techniques) and then show up and do them.

Ways this can fail (and some suggestions)

Make sure you're clear on what the appointment is. It's okay to leave it open-ended when you make the appointment, but once the appointment starts, don't take more than 5 minutes to figure out how you're going to spend it. Or decide "I'm going to spend it prioritizing". The key is not to let the time slip by while you wonder what the best way to spend it would be. Which of course you probably know on some level. The point of this technique is to tap into what you already know about how you can have a better life.

If you don't have enough self-trust to show up for an appointment if there isn't someone else who'll be left stood-up, then

...
What is learning?
by Pee Doom — 2019-02-08T03:18:10.859Z

I want to know what learning actually is. Ideally you can provide a model that is compact and useful as possible, such that when I encounter the word 'learn' in daily life I could replace it with this model while seeing a complete set of unique gears within it such that I can look at a real-world learning system and see how its traits correspond to each of those parts.

Generative subquestions:

What results from Tabooing it? What are its components? Is there a precise General Theory of Learning that underpins humans, animals, ML agents, or any other things that learn? If there are multiple constructs being pointed to with 'learn': what are the differences between systems that 'learn'? How many different kinds of 'learning' are there and what do they look like?

What isn't learning?
Is this how I choose to show up?
by Elo — 2019-02-08T00:30:27.798Z

Original post: http://bearlamp.com.au/is-this-how-i-choose-to-show-up/

Is this how I choose to show up? No.

I’m exhausted. I’m just trying to survive here and today I did that. Not every today. But I did this today. Yes.

Is this how I choose to show up? No.

I’m doing better than surviving but am I a good person? Did I do the right thing? Will I be going to heaven or hell for this. Is this how I choose to show up? Yes. I did the right thing. If I survive or not, I know I did the right thing.

Is this how I choose to show up? No.

I’ve aligned myself to the right people. If I follow them, then I know I’m a good person. They can help me survive. But are they the right people? How would I know? Yes. This is right. The gods are with us. And even if they aren’t, they can’t hate me for being on the side of the right people. The gods might smite us for being wrong. The gods might be on our side. I might survive being on this side, I might not. Yes. This is right.

Is this how I choose to show up? No.

I’m working in a team. We are building something for all of us. We are ordered and structured, that’s part of why the world is safe, because of our order. I don’t know if it’s the right people but at least we are working together. And hey – it’s a job, it’s worth it to do good work. It’s pay. It’s enough to survive. But is it enough for me? Am I getting what I want? Maybe if I knew better. The science, the tests to run I could get this team working better. How do I do that? Yes. It’s okay, I’ve got this how I am. I might not survive but at least I’m part of this big idea, and through this big idea I survive. It’s not that the gods might smite us, we are the gods now. We make the ideas. We live or die by the ideas we make and if they survive the long haul. It’s us against the gods of time. And of course the other people’s big ideas. Maybe our idea beats their idea by sheer will of structure, and I have all the right people with me, and even if I didn’t, that’s okay too I guess. Maybe we aren’t right, and I’m okay with that too, as long as we try. In the true arena of ideas, the best ideas win. Yes.

Is this how I choose to show up? No.

I’m running the tests. I’m getting that recognition for being right in the ways I’m right. In all the ways I know, I know that I’m doing well. All that unknown, it’s not safe, but I’m coming to conquer it. I have my team, but I ...
Sad!
by nws — 2019-02-07T19:42:59.580Z

I grew up in a strictly Christian home. Both of my father's parents were very involved in the protestant church, my grandmother hosts weekly prayer nights and my grandfather is an administrator at seminary school. To be clear, they hold on to strong beliefs typically held by evangelical republicans, for example, no position for women in church, and a ban on same-sex marriage (two topics mentioned in the Bible that are controversial in the Christian community today).

I am a Christian myself, and I wouldn't call myself an extreme rationalist, however, their religious nagging has brought quite inefficient, unwanted and impractical tendencies and customs into my life. However, I did keep my mouth shut for most of my childhood out of respect, along with the rest of my family. My grandparents are incredibly loving and kind people, however, I could only imagine how tough it must have been for my rationalist father and his two siblings growing up.

You could imagine my family's surprise when my aunt (dad's sister) came out as a lesbian to our family.

That's about 30+ years in the closet. To others, it may seem bizarre, but knowing my grandparents, I probably wouldn't have the courage to come out at all. Shortly after she came out, by announcing the three year anniversary of her and her girlfriend, my grandparents let us all know how they felt: "struggling with the reality of their daughter's lifestyle."

Things in my family have been tense ever since. My family and I have met my aunt's girlfriend. She is quite terrified and timid around my grandparents (obviously) but they try to treat her with respect. If you saw her and my family in a living room, you couldn't tell that they are in a relationship. No PDA, not even hand-holding out of the fear that outcry may arise.

Around a week ago, my aunt announced that she proposed to her girlfriend (after her girlfriend set a personal record at the Boston Marathon!) and she said yes. It took my grandparents about an hour after to let us all know that they will not be attending the wedding because their beliefs don't match up with their daughter's life. My parents are outraged and shocked, and our family is falling apart. I don't know what to make of this situation, so I ask you all:

If you were in my grandparents' situation, would you attend your daughter&...
Open Thread February 2019
by ryan_b — 2019-02-07T18:00:45.772Z

If it’s worth saying, but not worth its own post, you can put it here.

Also, if you are new to LessWrong and want to introduce yourself, this is the place to do it. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome. If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, and seeing if there are any meetups in your area.

The Open Thread sequence is here.
EA grants available (to individuals)
by Jameson Quinn — 2019-02-07T15:17:38.921Z

I'm considering applying for some kind of a grant from the effective altruism community. A quick sketch of the specifics is here. Raemon replied there with a list of possibilities. In this post, I'll look into each of those possibilities, to make this process easier for whoever comes next. In the order Raemon gave them, those are:

OpenPhil

"Can I apply for a grant?

In general, we expect to identify most giving opportunities via proactive searching and networking. We expect to fund very few proposals that come to us via unsolicited contact. As such, we have no formal process for accepting such proposals and may not respond to inquiries. If you would like to suggest that we consider a grant — whether for your project or someone else’s — please contact us. "

This looks like a case where it's at least partially about "who you know". I do in fact have some contacts I could approach in this regard, and I may do so as this search proceeds.

But this does seem like a bias that it would be good to try to reduce. I understand that there are serious failure modes for "too open" as well as "too closed", but based on the above I think it currently tilts towards the latter. Perhaps a publicly-announced process for community vetting? I suspect there are people who are qualified and willing to help sort the slush-pile that would create.

CEA (Center for Effective Altruism)

Applications for the current round have now closed. If you’d like to be notified when applications next open, please submit your contact information through this form.

The goal of Effective Altruism Grants is to enable people to work on projects that will contribute to solving some of the world’s most important problems. We are excited to fund projects that directly contribute to helping others, as well as projects that will enable individuals to gain the skills needed to do so.

...

CEA only funds projects that further its charitable objects.[1] However, we welcome applications that may be of interest to our partners who are also looking to support promising projects. Where appropriate, we have sometimes passed applications along to those partners.

This would seem to be a dead end for my purposes in two regards. First, applications are not currently open, and it's not clear when they will be. And second, this appears to focus on projects with immediate benefi...
X-risks are a tragedies of the commons
by David Scott Krueger (formerly: capybaralet) — 2019-02-07T02:48:25.825Z

Safety from Xrisk is a common good: We all benefit by making it less likely that we will all die.
In general, people are somewhat selfish, and value their own personal safety over that of another (uniform) randomly chosen person.
Thus individuals are not automatically properly incentivized to safeguard the common good of safety from Xrisk.

I hope you all knew that already ;)