trentmkelly/LessWrong-43k huggingface dataset:
Can we use ideas from ecosystem management to cultivate a healthy rationality memespace?
by Bird Concept — 2019-06-13T12:38:42.809Z

Background: ecosystems management practices for improving community memespaces

One can model individual human minds, as well as a community of minds, as an “ecosystem” of “memes”. These memes might be things like:

Bayesian epistemology
a habit of checking Facebook when one wakes up
wiggling one’s fingers to indicate agreement with a statement
prefacing things one says with “My model of this is that...”
Doing calibration training
Referring to blog posts in conversation
Taking silent pauses to think mid-conversation

Etc. etc.

Calling this set an “ecosystem”, seems to me to be mechanistically very close to what’s actually going on. At least, this is because:

Memes mutate as they are transmitted between minds
Memes undergo selection pressure as they are transmitted
The underlying topology/geography of social, cultural and geographical networks of people influence their spread
Memes can be in equilibrium with other memes
Memes can act as “invasive species”

Now there is an emerging European rationality community, largely driven by efforts from the Prague rationalists. This community imports many memes from the Bay area rationality community. For a high-level, historical examination of this memespace, see

Julia Galef’s map of bay area memespace

.

At a recent CFAR workshop, we discussed how we can ensure that this interaction is successful.

Five of us, felt that the ecosystem model carried sufficient mechanistic similarity to the actual situation that it would be helpful to read up on things like: protocols for deliberate introductions of new species into new environments, invasive species regulations and protection programs, pest control, and more.

Collection of background notes

We spent 1h researching this. Now the outside view predicts that if we were to leave it at that, the 16-page Google doc would never be used again. Hence we’re experimenting with releasing our notes together with a LessWrong question, in order to allow others to benefit from and build on our progress.

You can find our notes

here

.

These notes are provided “as is”. I briefly went through them to make them more readable, but apart from that this should not be interpreted as something the authors endorse as being true, and despite originating at a CFAR workshop it is not official CFAR content.

Open questions

We’d be interested in using further research to answer questions such as:

What are warning signs of a m

...
What kind of thing is logic in an ontological sense?
by Chris_Leong — 2019-06-12T22:28:47.443Z

The existence of logic seems somewhat mysterious. It's this thing that seems to exist, but unlike other things that exist, it doesn't seem to exist anywhere in specific or in any tangible form. Further, while it is easy to mock Plato for mysticism when he posits perfect forms existing in some kind of mysterious Platonic Realm, that's actually uncomfortably close to a description of what logic is often seen as.
Let's talk about "Convergent Rationality"
by David Scott Krueger (formerly: capybaralet) — 2019-06-12T21:53:35.356Z

What this post is about: I'm outlining some thoughts on what I've been calling "convergent rationality". I think this is an important core concept for AI-Xrisk, and probably a big crux for a lot of disagreements.

It's going to be hand-wavy!

It also ended up being a lot longer than I anticipated.

Abstract: Natural and artificial intelligences tend to learn over time, becoming more intelligent with more experience and opportunity for reflection. Do they also tend to become more "rational" (i.e. "consequentialist", i.e. "agenty" in CFAR speak)? Steve Omohundro's classic 2008 paper argues that they will, and the "traditional AI safety view" and MIRI seem to agree. But I think this assumes an AI that already has a certain sufficient "level of rationality", and it's not clear that all AIs (e.g. supervised learning algorithms) will exhibit or develop a sufficient level of rationality. Deconfusion research around convergent rationality seems important, and we should strive to understand the conditions under which it is a concern as thoroughly as possible.

I'm writing this for at least these 3 reasons:

I think it'd be useful to have a term ("convergent rationality") for talking about this stuff.
I want to express, and clarify, (some of) my thoughts on the matter.
I think it's likely a crux for a lot of disagreements, and isn't widely or quickly recognized as such. Optimistically, I think this article might lead to significantly more clear and productive discussions about AI-Xrisk strategy and technical work.

Outline:

Characterizing convergent rationality
My impression of attitudes towards convergent rationality
Relation to capability control
Relevance of convergent rationality to AI-Xrisk
Conclusions, some arguments pro/con convergent rationality

Characterizing convergent rationality

Consider a supervised learner trying to maximize accuracy. The Bayes error rate is typically non-0, meaning it's not possible to get 100% test accuracy just by making better predictions. If, however, the test data(/data distribution) were modified, for example to only contain examples of a single class, the learner could achieve 100% accuracy. If the learner were a consequentialist with accuracy as its utility function, it would prefer to modify the test distribution i...
MLU: New Blog!
by  — 2019-06-12T04:20:37.499Z

I'm in the process of moving mindlevelup from Wordpress to a new static site hosted by Netlify. I like this because now I have more control over scripts and the visuals. It also finally puts into place my goal of having a place to have short/longform posts.

The plan is to slowly update posts and incrementally update the site.

The new site is here.

My short-form blog, Muse, has also been moved here.

RSS feeds for both main-line blog posts and short-form posts can be found the About page.
The Outsider and the Onlooker (on esoteric meaning in fiction)
by KyriakosCH — 2019-06-11T20:02:41.386Z

“Unhappy is he to whom the memories of childhood bring only fear and sadness. Wretched is he who looks back upon lone hours in vast and dismal chambers with brown hangings and maddening rows of antique books, or upon awed watches in twilight groves of grotesque, gigantic, and vine-encumbered trees that silently wave twisted branches far aloft. Such a lot the gods gave to me—to me, the dazed, the disappointed; the barren, the broken. And yet I am strangely content, and cling desperately to those sere memories, when my mind momentarily threatens to reach beyond to the other. ” H.P. Lovecraft, The Outsider

The Outsider is, in many ways, the most remarkable short story that H.P. Lovecraft wrote. Certainly he has produced quite a few other interesting and elegant very short stories (Dagon, The Statement of Randolph Carter and The Transition of Juan Romero come to mind), yet the Outsider follows a different, possibly unique structure.

Many readers focus on the ending, where the nature of the protagonist is revealed. To be sure it is memorable, although perhaps not entirely unexpected. What I always found far more striking, and what made me love this piece immediately, was what had been clarified half-way through the story: the whereabouts of the protagonist.

The Outsider had spent innumerable years in a desolate and morbid castle. He finally decides to risk climbing on the circular wall of the tallest tower, hoping that he may get to rise above the ominous forest he so despises, and for once in his life see the light of day… Lovecraft carefully prevents the reader from wondering whether this continuous rise to dizzying heights is somehow not what it was made out to be, so we indeed share the sense of wonder and surprise the Outsider has when we get to understand just what was above the terrible forest and the eternal night of the area with the castle. The aforementioned environment, with its imposingly tall and dense forest, the ancient fort and moat and the silent labyrinth of shadows beyond, was left behind by the hapless narrator – yet what he wished for came at the price of a horrible realization: His entire personal realm was not part of the actual world, but a subterranean, chthonic region.

At first I was impressed by the revelation itself. Later on, though, I did focus on what it connoted. It is known that Lovecraft wished to weave a narration of cosmic horro...
How much does neatly eating matter? What about other food manners? 
by sapphire — 2019-06-11T19:40:45.539Z

I just read this post about the importance of sandwhich eating skill. The author describes how his investment firm served very hard to eat food to potential clients.

During my internship, a Prestigious Private Equity Firm was looking to improve its stock price/shareholder base. So a delegation of higher-ups (the COO/CFO/Head of IR/Head of Legal) went around pitching their stock to institutions that they thought might be interested in buying and holding it for a long time. These sorts of meetings were a fairly common occurrence at my old firm—we’d have perhaps an average of three or four a week—and would sometimes be catered.

The meeting with PPEF was catered. The meal seemed like an almost-intentional[ii]selection of food items that are difficult to consume in a professional setting—sandwiches with way too much mayo, kettle-cooked potato chips (the extra crunchy kind), and chocolate chip cookies that crumbled if you bit them. There were napkins, but there were not enough napkins.

The people from my firm almost uniformly avoided the food. A few nibbled carefully on the cookies; only one—a portfolio manager with a fierce intellect and a lack of regard for what others might think of his presentation—dared eat a sandwich. Much like any normal human would[iii], he went through several napkins and looked rather undignified at times. (Though this was unimportant, because he was the one who would decide if PPEF would get the investment it wanted.) I of course ate nothing, because I was an intern focused on taking good notes and not appearing overly intimidated.

All four PPEF delegates ate every food item we provided—to do otherwise might have been rude. What’s more, they did it with a shocking amount of grace. Chips seemed not to crunch; any filling that threatened to escape a sandwich was carefully corralled. Napkins were almost unnecessary and were fastidiously refolded if used. In their manners and mannerisms, the PPEF delegates were precise and uniform. None of this appeared to take any attention. It all looked as natural as breathing. In fact, though food was surely being eaten, it almost seemed that they were not eating at all. When they later typed on their iPads—while making frequent eye contact with everyone across the table—their fingers did not so much as smudge the glass.

The author correctly points out that the executives must have been selected for sandwh...
Tal Yarkoni: No, it's not The Incentives—it's you
by Zack_M_Davis — 2019-06-11T07:09:16.405Z
Link: https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/

Neuroscientist Tal Yarkoni denounces many of his colleagues' tendency to appeal to publish-or-perish incentives as an excuse for sloppy science (October 2018, ~4600 words). Perhaps read as a complement to our recent discussion of Moral Mazes?
Long Term Future Fund applications open until June 28th

by habryka — 2019-06-10T20:39:58.183Z

The Long Term Future just reopened its applications. You can apply here:

Apply to the Long Term Future Fund

We will from now on have rolling applications, with a window of about 3-4 months between responses. The application window for the coming round will end on the 28th of June 2019. Any application received after that will receive a response around four months later during the next evaluation period (unless it indicates that it is urgent, though we are less likely to fund out-of-cycle applications).

We continue to be particularly interested in small teams and individuals that are trying to get projects off the ground, or that need less money than existing grant-making institutions are likely to give out (i.e. less than ~$100k, but more than $10k, since we can’t give grants below $10k). Here are some concrete examples:

To spend a few months (perhaps during the summer) to research an open problem in AI alignment or AI strategy and produce a few blog posts or videos on their ideas
To spend a few months building a web app with the potential to solve an operations bottleneck at x-risk organisations
To spend a few months up-skilling in a field to prepare for future work (e.g. microeconomics, functional programming, etc)
To spend a year testing an idea that has the potential to be built into an org

You are also likely to find reading the writeups of our past grant decisions valuable to help you decide whether your project is a good fit:

November 2018 writeup
April 2019 writeup

Apply Here

What kind of applications can we fund?

After last round, CEA clarified what kinds of grants we are likely able to make, which includes the vast majority of applications we have received in past rounds. In general you should err on the side of applying, since I think it is very likely we will be able to make something work. However, because of organizational overhead we are more likely to fund applications to registered charities and less likely to fund projects that require complicated arrangements to be compliant with charity law.

For grants to individuals, we can definitely fund the following types of grants:

Events/workshops
Scholarships
Self-study
Research project
Content creation
Product creation (eg: tool/resource that can be used by community)

We will likely not be able to make the following types of grants:

Grantees requesting funding for a list of possible projects

In this case, we would fund

...
Get Rich Real Slowly
by Jacob Falkovich — 2019-06-10T17:51:32.654Z

Cross-posted from Putanumonit.

Some of you have read Get Rich Slowly and thought: no, this is not slow enough. The dizzying pace of 6-7% annual return is too much. You want more safety and less volatility, something towards the bottom-left corner of the efficient frontier.

Concrete example: you have $10,000 today, and you may want to spend them sometime in the next 2-3 years on a car, a mini-retirement, or just an emergency. You would like there to be more than $10,000 when you need the money, but what you really want is to be confident that there wouldn’t be any less than that. Emerging market stock index funds can turn $10,000 to $15,000 in two years, or to $6,000. You just want there to be $10,500.

This post will briefly cover the basics of low-risk-low-return saving, with general principles and particular examples. With apologies to my international readers, the examples are all USA-specific. The general approach, however, should be easily transferrable – and you’ll know what scams to watch out for.

“Risk-free” rate

The benchmark for the return rate on low-risk investments is the federal funds rate, which is currently set by the Federal Reserve at 2.5% [1]. This is the rate at which big institutional banks borrow dollars [2] from each other and from the central bank overnight. This number also tracks very closely the rate at which the US government borrows money for one year.

The chance of any bank going bankrupt overnight or the US government defaulting on its debt within 1 year are both very close to zero, which is why the federal funds rate is often referred to as the “risk-free rate of return”. Of course, nothing is ever truly risk-free when finance is concerned. “Risk-free” is a euphemism for “this won’t blow up unless the entire rest of the financial system blows up as well, and at that point, you should care about your stocks canned food more than about your dollar savings.”

So, big banks can borrow and lend “safely” at 2.5%. Let’s see what normal schlubs like us can get when we go to the big banks ourselves.

Checking and Savings Accounts

There are roughly 70,000 retail bank branches in the United States. A third of them belong to the biggest 5 retail banks: Wells Fargo, JP Morgan Chase, Bank of America, US Bank, and PNC. Either one would be happy to open you a checking account – a simple account where your money is insured by the government,

...
Honors Fuel Achievement
by Samo Burja — 2019-06-10T16:10:01.002Z

This is an excerpt from the draft of

my upcoming book

on great founder theory. It was originally published on SamoBurja.com. You can

access the original here.

It is a cherished dream for many people to win a Nobel Prize, or an Oscar, or a knighthood, or whatever honor is most respected in the field they dedicate themselves to. These ritualized honors are very important to us, but do we fully understand them?

We usually think honors are about the recipient, but the giver of honors also gains. The giver and recipient collaborate to publicly assert that the recipient is worthy of prestige, and that the giver has the authority to grant it. Honors are thus acts of an alliance to mutually boost prestige.

This meaning is even codified in diplomatic protocol; representatives of countries often exchange honors for the explicit purpose of signalling alliance.

The audience also participates in this transaction of prestige. They either accept the whole affair and the implied claims of the giver and the recipient, or reject or ignore them. The honors only have meaning—and thus the primary parties only gain—if the onlookers take them seriously. The performance of honor-giving is a bid for that audience’s assent, both the literal immediate audience, as well as the broader public who will hear about the honors bestowed or see them televised.

The audience accepts the frame because they recognize the preexisting prestige of someone involved. Honors can be prestigious because prestigious people receive them, because prestigious people give them, or both.

Consider the Nobel Prize in science. Its purpose is to tell the public who the most notable experts in a field are. In other words, it makes the recipient’s standing within a given scientific community more visible to the rest of society, fortifying their standing within that particular scientific community in the process. This is a useful service to the scientific community and the public.

The Nobel Prize has different functions depending on the field in which it is awarded. In the case of the Literature and Peace Prizes, its function is at least partially to advance the political goals of the overseeing organization. Rather than making the existing distribution of prestige more legible, these prizes alter it by granting prestige to the proponents of preferred causes. Looking at a list of Nobel Peace Pri...
On pointless waiting
by Kaj_Sotala — 2019-06-10T08:58:56.018Z

I’ve often noticed in myself a tendency, if I am not doing something immediately engrossing, to find myself waiting.

Waiting, waiting, waiting, not really being present, just willing time to pass.

But the weird thing is, frequently there isn’t anything in particular that I’m waiting for. Getting out of that situation, yes, but I don’t have anything in particular that I’d want to do when I do get out.

I have a suspicion that this might have to do with mental habits ingrained in school.

In elementary school, there’s no real goal for your studies. Mostly it’s just coming there, doing the things that teachers want you to do, until the day is over and you get to go.

In that environment, every minute that passes means winning. Every minute takes you a bit closer to being out of there. That’s the real goal: getting out so you can finally do something fun.

During a lesson you are waiting for recess, during recess you are waiting for the end of the day. Outside school you are waiting for the weekend, on the weekend you are waiting for the bliss of the long summer leave.

Waiting, waiting, waiting.

So you learn to pay attention to the time. Human minds are tuned to feedback, things that let them know how well they are doing. And since each passing minute takes you closer to the goal, the passing of time becomes its own reward.

Time having passed means that you have achieved something. Time having passed means that you can feel a tiny bit of satisfaction.

And then that habit, diligently trained for a decade, can carry over to the rest of your life. Even as an adult, you find yourself waiting, waiting, waiting.

You don’t know what it is that you are waiting for, because you are not really waiting for anything in particular. Even if it would actually be more pleasant to stay engaged with the present moment, you keep tracking the time. Because waiting feels like winning, and every passing minute feels like it takes you closer to your goal.

Even if you don’t actually know what your goal is. Even if reaching your goal will only give you a new situation where you can again wait, so that you are never actually present.

Still, you keep waiting, waiting, waiting.

—

(

typical mind fallacy

employed for the sake of artistic license; I am describing my own experience, without claiming this to be a universal one)
Dissolving the zombie argument
by Chris_Leong — 2019-06-10T04:54:54.716Z

Update: Upon reflection, I'm not entirely satisfied with this post. I think I definitely managed to identify some of the confusion around these kinds of discussion, but a smaller proportion than I would have liked.

The Zombie argument (David Chalmer's website, Stanford Encyclopedia of Philosophy) is one of the most famous arguments against materialism, so I'll assume that you can find an explanation yourself if you aren't already familiar with it.

I always find it fascinating when you have two sides that can't seem to communicate with or understand one another. I think the root of the problem is that both sides have a different notion of what counts as a zombie. The Dualist Conception of consciousness involves qualia, so their conception of a philosophical zombie is an entity that lacks qualia. This is a notoriously hard term to define - some would say because it is meaningless - but all that matters here is that they have a stricter conception of consciousness that the Materialist. The Materialist Conception of consciousness involves certain information processing taking place, so a Materialist Conception of a zombie would involve these processes taking place, but also not taking place, which would be a contradiction.

Here's the confusion. If a someone were to claim that humans don't fit the Dualist Conception of a zombie and that Materialism is true, they'd be contradicting themselves, because Dualists have a wide conception of what counts as a zombie that all entities in a Materialist world would fit this definition. On the other hand, if someone were to claim that that Materialist Conception of a zombie were logically possible, which is merely to claim that they can posit this without contradiction, they would be mistaken since Materialist's have such a narrow conception of what would count as a zombie that this class is an empty set.

Once the definition of what counts as a zombie has been fixed, so too has the outcome of the argument. And this is really contingent on what counts as consciousness, so the Zombie argument isn't actually where the fundamental difference lies. This isn't a mere linguistic difference, it's a question of what natural structures exist that cry out to be given a label. Or as Richard Kennaway might frame it, an attempt to understand the nature of a phenomenon which we alrea...