trentmkelly/LessWrong-43k huggingface dataset:
East Coast Rationalist Megameetup 2018
by metaperture — 2018-10-29T17:46:42.563Z

NYC Secular Solstice this year will be on December 15. That weekend, from December 14 to 17, we'll be hosting a rationalist community gathering with accommodations for out-of-towners in the NYC area. If you'd like to join us, please register at rationalistmegameetup.com!

The stars are aligning for the solstice season to be soon upon us! The eighth annual Secular Solstice in NYC will be held on December 15. Every year, rationalist sojourners travel to New York to gather together and experience the darkest time of the year in the light of our community. Last year, Taymon organized the first East Coast Rationalist Megameetup here in NYC, and it was a great success. This year we are again organizing a gathering place that weekend for solstice sojourners and New Yorkers alike, to solidify that tradition into our second annual East Coast Rationalist Megameetup.

For those traveling from out of town, we're renting a big house together! Like last year, we're pricing the sleeping arrangements at $100 per person, and the other costs (for the activities listed below) at $25 per person, for a total cost of $125.

For New Yorkers or those who already have accommodations, who would like to join in the events and festivities but not sleep over, please register as well! Again, the megameetup-only cost is $25 per person.

The exact schedule for the weekend is up in the air, but we're planning on hosting:

three meals: Saturday lunch, Saturday dinner, and Sunday breakfast
Secular Solstice afterparty on Saturday night
an unconference on Sunday

Depending on the attendees and interest, we're hoping to also plan some other activities to further the megameetup tradition.

If you'd like to attend, please register as soon as possible so we can get an accurate headcount and book an ideal space for the event. For reference, last year we rented a 9 bedroom house which we stretched (with air mattresses and creativity) to sleep around 36 at peak occupancy (Saturday night). This year we're shooting for two to a bed and one per air mattress at peak occupancy -- non-peak occupancy is expected to be significantly less crowded. Solstice tickets are not included in either of these registrations, so please make sure to register for Solstice separately.

Anyone who would like to attend but cannot afford $125 should contact me directly for arrangements.

Orig...
Alignment Newsletter #30
by Rohin Shah — 2018-10-29T16:10:02.051Z

Highlights

Learning Complex Goals with Iterated Amplification (Paul Christiano et al): This blog post and the accompanying paper introduces iterated amplification, focusing on how it can be used to define a training signal for tasks that humans cannot perform or evaluate, such as designing a transit system. The key insight is that humans are capable of decomposing even very difficult tasks into slightly simpler tasks. So, in theory, we could provide ground truth labels for an arbitrarily difficult task by a huge tree of humans, each decomposing their own subquestion and handing off new subquestions to other humans, until questions are easy enough that a human can directly answer them.

We can turn this into an efficient algorithm by having the human decompose the question only once, and using the current AI system to answer the generated subquestions. If the AI isn't able to answer the subquestions, then the human will get nonsense answers. However, as long as there are questions that the human + AI system can answer but the AI alone cannot answer, the AI can learn from the answers to those questions. To reduce the reliance on human data, another model is trained to predict the decomposition that the human performs. In addition, some tasks could refer to a large context (eg. evaluating safety for a specific rocket design), so they model the human as being able to access small pieces of the context at a time.

They evaluate on simple algorithmic tasks like distance between nodes in a graph, where they can program an automated human decomposition for faster experiments, and there is a ground truth solution. They compare against supervised learning, which trains a model on the ground truth answers to questions (which iterated amplification does not have access to), and find that they can match the performance of supervised learning with only slightly more training steps.

Rohin's opinion: This is my new favorite post/paper for explaining how iterated amplification works, since it very succinctly and clearly makes the case for iterated amplification as a strategy for generating a good training signal. I'd recommend reading the paper in full, as it makes other important points that I haven't included in the summary.

Note that it does not explain a lot of Paul's thinking. It explains one particular training method that allows you to train an AI sy...
Boston Solstice 2018
by jefftk — 2018-10-28T20:37:47.679Z

Six years ago Julia and I took the bus down to NYC with a group from the Boston LessWrong meetup. We were headed to a Solstice celebration and megameetup Ray was organizing. It was a good weekend: lots of talking to people I'd only met online, various proto-EA conversations, friendly and relaxed atmosphere. The Solstice itself was in a crowded living room, and I remember playing guitar for Uplift, Thanksgiving Eve, and God Wrote the Sky. This was the second NYC Solstice, the first one being in 2011, and it was still a cosy event.

NYC Solstice 2012; I'm in the middle

We went again the next year, for the first big Solstice. There was a Kickstarter, it was a beautiful hall, full band of professional musicians. Good in a different way, still a really nice time to see friends and get together.

A week later we held the first Boston Solstice, in a friend's house, with basically no rehearsal. It was fun and silly, and I really liked it.

Boston also moved on to a larger event, in the MIT Chapel, for the following years, and while I enjoyed helping out with the music I missed the feeling of a smaller event.

As the long-term drift to the Bay has continued, it's time for Boston to go back to having a smaller, more casual, more communal Solstice. This year I'm planning to host one at our house, centered mostly around group singing, with a few readings. Ray's original vision had a large emotional arc, from happy and silly, to serious and solemn, to joyful and triumphant, anchored by a dark and meaningful speech, but the theatric intensity necessary for this isn't the sort of thing I'm up for. The idea is we'll still have a general arc to the evening, with some central darkness bookended by lighter pieces, but it won't be as quasi-spiritual/deep as people might be used to from other events.

So: if this sounds like something you would enjoy, save the date: Friday December 21st.

(Reading past Solstice discussion the differing expectations around kids are tricky. If you want something to be a community event, since kids and parents are part of the community you need to figure out how that's going to work. My current plan is for the kids to be around for the initial joyful section, which is the part they're likely to be most interested in, and then do rotating volunteer childcare upstairs for when kids stop being interested ...
On Doing the Improbable
by Eliezer Yudkowsky — 2018-10-28T20:09:32.056Z

(Cross-posted from Facebook.)

I've noticed that, by my standards and on an Eliezeromorphic metric, most people seem to require catastrophically high levels of faith in what they're doing in order to stick to it. By this I mean that they would not have stuck to writing the Sequences or HPMOR or working on AGI alignment past the first few months of real difficulty, without assigning odds in the vicinity of 10x what I started out assigning that the project would work. And this is not a kind of estimate you can get via good epistemology.

I mean, you can legit estimate 100x higher odds of success than the Modest and the Outside Viewers think you can possibly assign to "writing the most popular HP fanfiction on the planet out of a million contenders on your first try at published long-form fiction or Harry Potter, using a theme of Harry being a rationalist despite there being no evidence of demand for this" blah blah et Modest cetera. Because in fact Modesty flat-out doesn't work as metacognition. You might as well be reading sheep entrails in whatever way supports your sense of social licensing to accomplish things.

But you can't get numbers in the range of what I estimate to be something like 70% as the required threshold before people will carry on through bad times. "It might not work" is enough to force them to make a great effort to continue past that 30% failure probability. It's not good decision theory but it seems to be how people actually work on group projects where they are not personally madly driven to accomplish the thing.

I don't want to have to artificially cheerlead people every time I want to cooperate in a serious, real, extended shot at accomplishing something. Has anyone ever solved this organizational problem by other means than (a) bad epistemology (b) amazing primate charisma?

EDIT: Guy Srinivasan reminds us that paying people a lot of money to work on an interesting problem is also standardly known to help in producing real perseverance.
Schools Proliferating Without Practicioners
by Davis_Kingsley — 2018-10-26T05:25:03.959Z
Link: http://www.thelastrationalist.com/schools-proliferating-without-practitioners.html

Very interesting post from "The Last Rationalist" discussing how the rationalist community seems to have been slow to update on comparative impracticality of formal Bayes and on the replication crisis in psychology.

I don't fully agree with this post - for instance, my impression is that there is in fact a replication crisis in medicine, which the author seems unaware of or understates - but I think the key points provide useful food for thought.

(Note: this is my opinion as a private individual, not an official opinion as a CFAR instructor or as a member of any other organization.)
[Beta] Post-Read-Status on Lessestwrong
by Raemon — 2018-10-25T23:13:00.775Z

Over on Lessestwrong.com (our sometimes-staging-site), we're trying out an update to how post items display on the frontpage. Posts that you've read now appear slightly grey. The goal is to make it so you can tell when you've read a post at a glance, without making it feel like unread posts are shouting at you to read them or feel like an obligation.

You can log in on lessestwrong using your usual login, and see how it feels.
In favor of tabooing the word “values” and using only “priorities” instead
by chaosmage — 2018-10-25T22:28:34.920Z

I don’t know if this is trivial/obvious or absurd or anything in between, so maybe you guys would like to help me judge?

The idea is simple. I’m confused by the term “values” in the moral and ethical sense. The way I often see it used, it makes personal and societal “values” seem hard to define/operationalize, incommensurable, uncountable, frequently in conflict with each other and usually in unclear relationships to each other and to those who hold them. But the word “values” is everywhere in normative discussions, and normative discussions are important and interesting, so I wish they weren’t being muddled by that word.

Is it just me, too stupid to get some perfectly clear meaning? Or is “values” really as vague and mysterious as it seems to me?

I think all of the useful work that the word “values” does can also be done by the word “priorities”. Priorities tell you what to do, they help you decide between alternatives. They are a language for describing agreement and disagreement on normative questions.

And all of us, including people who think in terms of values, already think in terms of priorities when we’re in projects and in everyday life. The confusion of “values” is in more abstract, longer-term regions of our thought. I think it is better to extend our thinking about priorities into those regions, rather than use a completely different set of terms and operations.

“Priorities” can also more obviously build on, or be derived from, each other. A priority can be strictly subordinate to another, as a means to an end. We’re used to having “higher” and “lower” and “overarching” priorities, so we can use those qualifiers rather than need to invent subcategories of “values” like

“instrumental values” and “terminal values”

.

Example: I have a priority to finish this post. That's a means to my higher priority, which is to find out if this idea is useful, and float it in the rationality cluster if it is. And that in turn is a means to my next higher priority, to contribute usefully to the memeplex that makes our species increasingly powerful and helps accelerate the colonization of the galaxy. You might call colonization of the galaxy my terminal value, but I prefer to call it my overarching priority.

And “priorities” implies commensurability. Which is great, because

everything is commensurable

. The priorities are still in competition over resources of course, b...
When EDT=CDT, ADT Does Well
by Diffractor — 2018-10-25T05:03:40.366Z

Back in my first post about ADT, I conjectured that ADT might achieve sublinear regret relative to argmaxing for the true embedder, if some extra condition was imposed.

The Problem:

In the form of ADT stated in that post, the game of chicken is a counterexample to Conjecture 1. In short, the "delusion" embedder F (where the opponent is an identical ADT agent) has argmax converge to going straight (because it will, in expectation, yield 0.81 utility, while other strategies yield less utility.) Further, if the agent uses the delusion embedder less than 10% of the time, then E(F(amF)) will start being greater than E(E(amE)) (where E is the embedder that plugs the chosen action into both your action and your opponent's action), so it will start using the delusion embedder more often. Therefore, 10% of the time, the ADT agent rams into itself because it uses the delusion embedder.

Now, the reason why this embedder can't be removed is interesting. We can consider the agent to be reasoning as follows: "I don't know whether I'll pick the delusion embedder or the copy embedder. There's a 90% chance that an ADT agent will pick the copy embedder E, in which case F(amF) will score highly, because amF will just go straight, and amE picks swerving with 90% probability. Also, E(F(ADT)) is around 0.729, because about 90% of the time, my foe picks the copy embedder and so do I, and about 10% of the time, my foe picks the delusion embedder and so do I, and we ram into each other."

However, note that E(F(amF)) (approximately 0.81) systematically deviates from E(F(ADT)|ADT=amF) (approximately 0). This is pretty much Abram's criterion for CDT and EDT diverging. Plugging an action into the embedder tends to give a different result than plugging yourself into the embedder and conditioning on you taking that action. The second quantity is a more accurate estimate of the utility you can expect to get. However, you can't just use it in place of E(F(amF)) because the conditional may give nonsensical answers if you almost certainly don't take the action, because conditionals on improbable events are less accurate. To rephrase this, using the conditional is vulnerable to the futarchy hack, where a trader with a lot of wealth bets on conditional contracts driving down the value of F(ADT)|ADT=amF, so the ADT agent doesn't use ...
What will the long-term future of employment look like?
by Richard_Ngo — 2018-10-24T19:58:09.320Z
Link: http://thinkingcomplete.blogspot.com/2018/02/what-will-long-term-future-of.html

Current debates over the automation of human jobs by AI tend to focus on how fast it will proceed: some jobs will be safe for the next decade, some for the next half-century, some for the foreseeable future. In this essay I want to take a different approach, by asking which (if any) jobs will continue to be done by humans even after the development of arbitrarily advanced technology.

[Here I've omitted a long section on why AI is capable of becoming arbitrarily competent; it was relevant to the original competition into which this essay was entered, but it's nothing new to anyone on this site. See my linked blog for the full post and references].

The fact that AI will eventually utterly outclass humans in performance on any given information-processing task does not mean that humans will necessarily be shut out from those industries: by Ricardian comparative advantage, it could still be profitable to employ humans even when they have no absolute advantage over AI. [6] However, since copying software is trivially easy, and the price of hardware continues to plummet, such roles will be very limited, especially if there is any sector of the economy in which humans have a comparative advantage over AI.

I claim that there is such a sector, consisting of what I shall call 'social jobs’. I define these as jobs where most or all of the value produced comes not from their outputs, but from the fact that they are being performed by other humans. The principle is most clearly illustrated by personal relationships: we want friends and partners who not only say and do the right things, but also really feel love and respect for us. In the economic sphere, there are particularly important social components of jobs in sales, management and leadership, and entertainment. To understand them, it's instructive to consider jobs whose value is almost entirely tied to the fact that humans are involved. Take chess, for instance. If we view the value of professional chess players as due to their production of high-quality games, then the advent of superhuman chess engines has rendered them entirely redundant. Yet professional chess not only still exists, but also features ever-growing prize pools. [7] So we are better to model chess players as entertainers, whose humanity is central to the existence of their jobs.

There are plenty more examples to consider. Fans of s...
Portland SSC Meetup Sunday, 11/11/18
by Noah Walton — 2018-10-24T17:46:53.710Z

Come join us for the Sunday, November 11th meetup. We're meeting at 12:30 PM -- email me for location details. (nwalton125@gmail.com)

We usually have a group discussion. Last time, we talked in one group for a while and then split into two. We discussed topics like:

-Open / closed / empty individualism

-Prediction markets

-Confusions about foundations of mathematics

-The possibility of using constant personal EEGs for introspection

And many others.
In praise of heuristics
by Bucky — 2018-10-24T15:44:47.771Z

We’ll get there in the end, bear with me.

Introduction to ZD strategies in IPD

Feel free to skip if you’re already familiar with ZD strategies.

In the iterated prisoner’s dilemma (IPD) a zero determinant (ZD) strategy is one which forces your opponent’s winnings to be a linear relation to your own winnings. These strategies can take either generous or extortionate forms.

Think of it as a modified version of tit-for-tat.

***

A Generous ZD strategy still always respond to C with C but will sometimes also respond to D with a C (it sometimes fails to punish). With "standard" PD utilities (T=0, R=-1, P=-2, S=-3) my opponent gains 1 utility by defecting. If I defect back in retaliation, I cost him 2 units of utility. If I defect back with probability 0.7, on average I cost him 1.4 units of utility. This still means that defecting is disadvantageous for my opponent (loss of 0.4 utility) but not quite as disadvantageous as it would be if I was playing pure tit-for-tat (loss of 1 utility).

This gets slightly more complex when you don't have constant gaps between T, R, P and S but the principle remains the same.

If he defects at all, my opponent will end up gaining more utility than me, but less than he would have got if he had co-operated throughout.

Advantages of GZD are:

1. Total utility isn’t damaged as much by accidental defections as it is in pure tit-for-tat.

2. It won’t get caught in endless C-D, D-C, C-D, D-C as tit-for-tat can.

***

On the other hand, Extortionate ZD always responds to D with D but also sometimes responds to C with D. Provided I don’t respond to C with D too often, it is still advantageous to my opponent to play C (in terms of their total utility).

If my opponent co-operates at all I'll end up with more utility than him. If he gives in and plays C all the time (to maximise his own utility) I can achieve a better utility than I would with C-C. .

The main disadvantage of EZD in evolutionary games is that it defects against itself.

For both EZD and GZD you can vary your probabilities to be more or less generous/extortionate, provided you always ensure your opponent gets the most utility by co-operating.

Different opinions on fairness

An extortionate ZD strategy is similar to an opponent who has different perceptions of what is fair. Maybe your opponent had to pay to play the game but you got in free so he wants a higher percentage of the ...
Starting Meditation
by Hazard — 2018-10-24T15:09:06.019Z

[Update] this intent has been put on hold. Check this comment for details. This note was put at the top, because whenever I see a post about "I'm going to do XYZ" my most burning question is always, "Well did they?"

I think since about last winter I've been convinced that there is something really interesting that comes from going deep down the path of meditation. Things like happiness is a chore, the little dragon is dead, Kensho, "My attempt to explain Looking", and Scott's review of Mastering the Core Teachings of the Buddha all were things that lead me to that conclusion.

I've had bouts of meditation practices in the past, the longest streak was perhaps a month, and I never got anywhere past, "Yeah, I do feel calmer throughout the day after having meditated, and sometimes I feel incredibly peaceful in the midst of meditation."

I am "trying to get enlightened", but as people have pointed out, it doesn't seem like that's a great frame of mind to be in throughout a practice. That's why I'm documenting my current goals and assumptions in this post, and then I'm going to forget about it for a while and look back in a few months and if things have changed.

Here's the plan:

30 min time for meditation each morning, everyday.
Using The Mind Illuminated as a guide, going through the various stages.
Finishing reading the book over the next week.
Each week when I do my weekly review, check if I can foresee days where it will be hard to do morning meditation, and make alternate plans.
I will document any days I miss in the comments of this post.
I will document any changes in attitudes towards this whole endeavor in the comments

Things I Want

Fewer moments of feeling like consciousness is a burden.
I've got a ton of strong curiosity about how minds actually work and what's really going on in my head, and I want to have a deeper felt/experiential map of my own mind and existance.
I want to suffer less.
I want to get in touch with lower levels of my experience, and more basic parts of my mind.

Both for epistemic clarity, and because I expect it will make me happier.

It sounds sorta like super powers and I've always wanted super powers.
I want to be kinder.
I want to be less terrified of dying.
I want to be less scared of being hurt by people.
I'm crazy excited by the ide

...