trentmkelly/LessWrong-43k huggingface dataset:
On Internal Family Systems and multi-agent minds: a reply to PJ Eby
by Kaj_Sotala — 2019-10-29T14:56:19.590Z

Introduction

I recently had a conversation with PJ Eby in the comments of my article “

Building up to an Internal Family Systems model

”. My most recent reply to it started getting rather long, and is also more broadly relevant as it contains updates on how I view IFS and the multi-agent framework in general. As a result, I decided to post it as its own article.

pjeby’s comments that I’m replying to are

here

and

here

; to verify that I understood him correctly, I wrote a summary of what I took to be his core points (below). He mostly endorsed them as correct, while having a few clarifications; the below summary has incorporated those corrections. After listing all of his points, I will present my own replies.

The summary is divided into two parts. Earlier on, pjeby wondered why IFS was popular among rationalists; one of the things I said in response was that rationalists like reductionism, and IFS helps reduce the mind into smaller components. pjeby felt that IFS is not good reductionism. My response goes into detail about what kinds of claims I view IFS as making, how I interpret those in terms of my multi-agent minds series, and how I would now rephrase my original article differently. Here I feel like I broadly agree with pjeby, and feel that our disagreement has more to do with using terminology differently than it does with actual object-level issues.

The second part concerns the practical usefulness of IFS as a therapeutic model. After all, a model can be useful while still not being true. Here I have more disagreement, and feel that (regardless of how good it is as a literal description of the mind) IFS has considerable practical value.

This article assumes that one has read my later article about

Unlocking the Emotional Brain

, as I reference concepts from it.

My summary of pjeby’s positions

IFS as a reductionist model

Good reductionism involves breaking down complex things into simpler parts. In the case of doing reductionism on agents, the end result should be mechanisms without agency. IFS "breaks down" behavior into mini-people inside our heads, each mini-person still being just as agenty as a full person is. This isn't simplifying anything.
Talking about subagents/parts or using intentional language causes people to assign things properties that they actually don't have. If you say that a thermostat "wants" the temperature to be something in particular, or t

...
How do you assess the quality / reliability of a scientific study?
by Eli Tyre — 2019-10-29T14:52:57.904Z

When you look at a paper, what signs cause you to take it seriously? What signs cause you to discard the study as too poorly designed to be much evidence one way or the other?

I'm hoping to compile a repository of heuristics on study evaluation, and would love to hear people's tips and tricks, or their full evaluation-process.

I'm looking for things like...

"If the n (sample size) is below [some threshold value], I usually don't pay much attention."
"I'm mostly on the lookout for big effect sizes."
"I read the abstract, then I spend a few minutes thinking about how I would design the experiment, including which confounds I would have to control for, and how I could do that. Then I read the methods section, and see how their study design compares to my 1-3 minute sketch. Does their design seem sensible? Are they accounting for the first-order-obvious confounds?"
etc.
[Site Update] Subscriptions, Bookmarks, & Pingbacks
by Ruby — 2019-10-29T04:32:31.109Z

We've just launched several new features that we've been working on for the past few weeks.

Subscriptions Overhaul

The subscriptions system has been completely rebuilt to 1) support notifications on a much broader range of events, 2) make it possible to receive all notifications by email and/or on-site notifications, 3) allow for batching of notifications.

How to Subscribe

The triple-dot, drop-down menu on posts and comments now have multiple subscription options. You can now subscribe to all posts from a given user, all comments on a given post, all replies to a particular comment, and local groups, with some more options for subscriptions coming soon.

On desktop, hovering over posts in post lists causes the triple-dot menu to appear.

What the triple-dot menu looks like

Viewing subscription options in the triple-dot drop-down menu

Subscription Settings

Subscription settings can managed in the Notifications Panel of your User Settings

Managing Subscriptions

You can manage individual subscriptions by accessing your subscriptions page from your user page or via www.lesswrong.com/manageSubscriptions

Bookmarks

For your convenience we have added bookmark functionality to help keep track of those wonderful posts you might not have time for when you first encounter them. Using the triple dot menu, you can now find a bookmark option.

Viewing your bookmarks

By default, your bookmarks are displayed at the top of the home page in the Recommendations section. This can be configured via the gear icon

Clicking on "Bookmarks" will take you to your bookmarks page which lives at www.lesswrong.com/bookmarks. You can also reach your bookmarks page via the drop-down menu when you hover over your username in the top right.

Pingbacks [Experimental Opt-In Only]

When a post has been linked to by another post on LessWrong, the pingback feature will list any referencing posts at the bottom of the post page. "Pingback" is equivalent to "cited by".

This feature is currently only available to those with experimental features switched on since we are still actively iterating on it (see your account settings to opt-in to experimental features). Note that not all historically used LessWrong URL schemas currently work– this will be fixed before the general release of Pingbacks.

We hope you find these features useful!
Is it harder to become a MIRI mathematician in 2019 compared to in 2013?
by riceissa — 2019-10-29T03:28:52.949Z

Below, I make the distinction between "MIRI mathematician" and "MIRI engineer". (In my mind I tend to think of these as "researchers" and "engineers", respectively, but I think MIRI calls both of these classes of people "researchers" these days so I want to avoid using "researcher".) Basically I count anyone who has published a paper or post in agent foundations as a mathematician, and everyone else as an engineer. From the current team page, I would classify Nate, Eliezer, Benya, Scott, Sam, Abram, and Tsvi as "MIRI mathematician", and Jesse, Nick, Buck, Ben, James, and Edward as "MIRI engineer". I don't actually know if this is a reasonable classification given that MIRI's recent work isn't public.

As far as I can tell, MIRI has not made any new mathematician hires since mid-2017; see this table which I made, and this blog post which I believe is the last hiring update for mathematicians.

Assuming there are no unannounced hires, the lack of hires can be for two broad reasons:

A change in the demand: less need for new people, more selective about who to hire, etc.

A change in the supply: fewer people to pick from, drop in quality of people that come into contact, etc.

I think both changes have probably happened, although my guess would be that with the takeoff of the AI safety field, there are now more people to pick from, so the lack of hires is mostly due to a change in the demand.

One way I've been thinking about these changes is to consider the same person attempting to work at MIRI as a mathematician in 2013 vs 2019. In particular, I've been thinking about Nate Soares, who was hired as a research fellow in 2014. To give my own summary, Nate was a full-time Google engineer who had taken multi-variable calculus and real analysis in college (in addition to the mathematics appearing in his CS degree), but otherwise with no experience learning or doing research in math. He discovered LessWrong probably in early or mid 2013, and came around to MIRI's worldviews by July 2013. He then spent five months (August to December) learning basic category theory, set theory, model theory, computability and logic, and provability logic, then attended a MIRI workshop, studied some more math, and joined MIRI as a research associate (January 2014) and then as a research fellow (April 2014). Nate might be an outlier in his studying/research abilities, so might be a bad examp

...
Book Review:  The AI Does Not Hate You
by PeterMcCluskey — 2019-10-28T17:45:26.050Z
Link: http://www.bayesianinvestor.com/blog/index.php/2019/10/24/the-ai-does-not-hate-you/

Book review: The AI Does Not Hate You: Superintelligence, Rationality
and the Race to Save the World, by Tom
Chivers.

This book is a sympathetic portrayal of the rationalist movement by a
quasi-outsider. It includes a well-organized explanation of why some
people expect tha AI will create large risks sometime this century,
written in simple language that is suitable for a broad audience.

Caveat: I know many of the people who are described in the book. I've
had some sort of connection with the rationalist movement since before
it became distinct from transhumanism, and I've been mostly an insider
since 2012. I read this book mainly because I was interested in how the
rationalist movement looks to outsiders.

Chivers is a science writer. I normally avoid books by science writers,
due to an impression that they mostly focus on telling interesting
stories, without developing a deep understanding of the topics they
write about.

Chivers' understanding of the rationalist movement doesn't quite qualify
as deep, but he was surprisingly careful to read a lot about the
subject, and to write only things he did understand.

Many times I reacted to something he wrote with "that's close, but not
quite right". Usually when I reacted that way, Chivers did a good job of
describing the the rationalist message in question, and the main problem
was either that rationalists haven't figured out how to explain their
ideas in a way that a board audience can understand, or that
rationalists are confused. So the complaints I make in the rest of this
review are at most weakly directed in Chivers direction.

I saw two areas where Chivers overlooked something important.

Rationality

One involves CFAR.

Chivers wrote seven chapters on biases, and how rationalists view them,
ending with "the most important bias": knowing about biases can make
you more biased. (italics his).

I get the impression that Chivers is sweeping this problem under the rug
(Do we fight that bias by being aware of it? Didn't we just read that
that doesn't work?). That is roughly what happened with many people who
learned rationalism solely via written descriptions.

Then much later, when describing how he handled his conflicting
attitudes toward the risks from AI, he gives a really great description
of maybe 3% of what CFAR teaches (internal double crux), much like a
blind man giving a really clear description of the upper half of a

...
Why are people so bad at dating?
by Xodarap — 2019-10-28T14:32:47.410Z

I’m confused why people are so bad at dating. It seems to me like there are tons of $20 bills lying on the ground which no one picks up.

For example, we know that people systematically choose unattractive images for their dating profiles. Sites like PhotoFeeler cheaply (in some cases, freely) resolve this problem. Since photo quality is one of the strongest predictors of number of matches, you would think people would be clamoring to use these sites. And yet, not many people use them.

In the off-line dating world, it surprises me how few self-help books are about dating. Right now, zero of the top 10 Amazon best-selling self-help books are about dating. I see only two dating books in the top 50: The 5 Love Languages and Super Attractor. To the extent these books exist, they often have little to no empirical support; my guess is that horoscopes are the most frequently read source of dating advice. Evidence-based books like Mate are less widely read.

Possible Solution #1: Inadequate Equilibria

It might be that we are in an Inadequate Equilibrium. Eliezer proposes three general ways in which seeming inefficiencies can exist:

1. Cases where the decision lies in the hands of people who would gain little personally, or lose out personally, if they did what was necessary to help someone else;

This doesn’t seem very compelling in the case of online dating. Anyone could choose to use PhotoFeeler for themselves, for example.

2. Cases where decision-makers can’t reliably learn the information they need to make decisions, even though someone else has that information

Again, this isn’t compelling. PhotoFeeler clearly lets you know what other people think of your photos.

3. Systems that are broken in multiple places so that no one actor can make them better, even though, in principle, some magically coordinated action could move to a new stable state.

Regressions done by Hitsch et al., as well as common sense, indicate that improving your own photos, even if you do nothing else or nothing else changes about the world, does make a significant impact in your likelihood of finding a good partner. So again, this seems uncompelling.

Possible Solution #2: Free Energy

I’ve seen a number of novice rationalists committing what I shall term the Free Energy Fallacy, which is something along the lines of, “This system’s purpose is supposed to b

...
Internet Distance
by jefftk — 2019-10-28T13:10:03.716Z

I recently read a

blog
post on locating servers in Iceland that included:

If you draw a straight line from San Francisco to Amsterdam you will
cross Iceland. [We have the] most customers from the US and Europe, so
it makes sense to pick this geographical location.

I don't want to pick on them in particular, but this is a common
misconception about how the internet works. Internet traffic doesn't
follow great-circle
routes, but instead follows the cables. [1] In the case of connections
between the US, Iceland, and Europe, these are underwater
cables:

This is only a diagram, and doesn't show exact routes, but you can see
that Iceland has much better connectivity to the East than the West.
A packet from the US to Iceland is probably going to first cross the
Atlantic to mainland Europe, and then backtrack.

Running a traceroute from my RCN connection in Boston, this is what's
happening. I see packets going via
hge0-2-0-0.border1.bos.ma.rcn.net, then
10-2-3.bear2.boston1.level3.net, then
ae-2-3205.edge3.amsterdam1.level3.net, on their way to
185.112.146.187, an IP in Iceland.

We can also see this with ping times:

Paris to Iceland: 52ms

Boston to Paris: 108ms

Boston to Iceland: 136ms

If you want to place a server "close" to both the US and Europe,
trying to split the difference and put it in Iceland is worse than
just putting it in Europe.

(Separately, it's also not helpful to think of a domain as having to be
hosted from only one place. You can configure DNS so the domain will
resolve to different IP addresses in different parts of the world, and
you can handle user requests as closer to the user as possible.
Requests like "please give me script.js" or "please record
that someone visited example.com" can easily be handled by
multiple servers around the world.)

[1] This isn't completely true: there are non-cable links, but cables
carry the vast majority of internet traffic.

Comment via: facebook
Is requires ought
by jessicata — 2019-10-28T02:36:43.196Z
Link: https://unstableontology.com/2019/10/28/is-requires-ought/

The thesis of this post is: "Each 'is' claim relies implicitly or explicitly on at least one 'ought' claim."

I will walk through a series of arguments that suggest that this claim is true, and then flesh out the picture towards the end.

(note: I discovered after writing this post that my argument is similar to Cuneo's argument for moral realism; I present it anyway in the hope that it is additionally insightful)

Epistemic virtue

There are epistemic virtues, such as:

Try to have correct beliefs.

When you're not sure about something, see if there's a cheap way to test it.

Learn to distinguish between cases where you (or someone else) is rationalizing, versus when you/they are offering actual reasons for belief.

Notice logical inconsistencies in your beliefs and reflect on them.

Try to make your high-level beliefs accurately summarize low-level facts.

These are all phrased as commands, which are a type of ought claim. Yet, they all assist one following such commands to have more accurate beliefs.

Indeed, it is hard to imagine how someone who does not (explicitly or implicitly) follow rules like these could come to have accurate beliefs. There are many ways to end up in lala land, and guidelines are essential for staying on the path.

So, "is" claims that rely on the speaker of the claim having epistemic virtue to be taken seriously, rely on the "ought" claims of epistemic virtue itself.

Functionalist theory of mind

The functionalist theory of mind is "the doctrine that what makes something a mental state of a particular type does not depend on its internal constitution, but rather on the way it functions, or the role it plays, in the system of which it is a part." For example, according to functionalism, for myself to have a world-representing mind, part of my brain must be performing the function of representing the world.

I will not here argue for the functionalist theory of mind, and instead will assume it to be true.

Consider the following "is" claim: "There is a plate on my desk."

I believe this claim to be true. But why? I see a plate on my desk. But what does that mean?

Phenomenologically, I have the sense that there is a round object on my desk, and that this object is a plate. But it seems that we are now going in a loop.

Here's an attempt at a way out. "My visual system functions to present me with accurate information about the objects ar...
What if AI is “IT”, and we don’t know about this?
by kocmoca — 2019-10-27T23:32:24.820Z

I am using IT character from that "IT" movie as a metaphor for describing a problem in AGI research. AI can potentially learn (or have already learnt) about all our irresistible pleasures and agonizing fears, similar to what IT could do in the film.

AGI initially “knows” or can deduct from only what we provide to it. Ideally, we want AGI to know as much information as needed (no more, no less), so it can provide us with the most relevant inferences to inform our decisions. Importantly, we don’t want to provide AGI with all the information we have, as we want to preserve privacy and autonomy.

Our goals and values are not stable and change after new information is received. Here, I mean different levels of “our” —  goals and values of individuals, groups, organizations, nations and the society as a whole.

Question: Can we already understand “what we truly want” as individuals, groups and society, and what is best for us, by extracting the most relevant data we have and using appropriate machine learning tools? The same question goes for “what we truly fear” and what is actually the worst for us —is it possible to already extract from the data?
How to Improve Your Sleep
by lynettebye — 2019-10-27T23:28:44.228Z

This piece is cross-posted on my blog here.

Do you feel sleepy enough during the day that it impairs your ability to function? Do you fall asleep involuntarily during the day? Do you feel tired enough that you feel weak, have difficulty doing things, or feel unable to focus?

If you answered yes to any of these questions, here’s a set of guidelines to help you figure out what’s worth trying. I go over generally applicable advice for daytime sleepiness and fatigue, then cover signs that would indicate more rare causes you can investigate. If feeling tired is significantly impairing your productivity, the information value of trying these experiments is probably well worthwhile.

This infographic checks for the most common causes of fatigue and what you should do it for each. These common causes account for the majority of people with daytime sleepiness, so there’s a good chance they cover the most relevant information for you. If you’ve already read a bunch about sleep, you’ll find loads more detail in the eleven pages of research that follow.

_This guide was heavily influenced by UpToDate.com, an evidence-based clinical resource, and interviews with medical professionals. But, just so we’re clear, I’m not a medical professional and these suggestions aren’t medical advice. Listen to the person who actually went to medical school if they give you different advice. _

1. Are you fatigued?

If you can fall asleep during the day in less than 8 minutes, you're probably sleep-deprived and may have a sleep disorder. You might have fatigue (or excessive daytime sleepiness) if:

You feel sleepy enough during the day that it impairs your ability to function, or you fall asleep involuntarily during the day.

You feel tired enough that you feel weak, have difficulty doing things, or can’t focus because you're tired.

If you experience the above more than once a week or have some weeks where you’re just tired the whole time, it’s probably worthwhile for you to spend time trying to solve the issue. (It’s probably not fatigue if you just feel tired right after a trip when jet-lagged, or if you had to pull an all-nighter.)

What works to make you feel well-rested will probably be different from others, particularly people who always feel rested by default. So, you can approach your sleep as a hits-based experiment. It might take you some trial and error to find what works for you...
Prediction markets for internet points?
by paulfchristiano — 2019-10-27T19:30:00.898Z

Using real money in prediction markets is all-but-illegal, and dealing with payments is a pain. But using fake money in prediction markets seems tricky, because by default players have no skin in the game.

Here’s a simple proposal that I think might work reasonably well without being too hard to try:

Create a service that tracks Internet Points. Anyone can quickly sign up with a Facebook account and is initially given 1000 points. Points are non-transferrable and are permanently associated with your Facebook account.

Run prediction markets in which people wager points rather than dollars. You can initially copy questions and verdicts from existing markets and bookies. (Ideally you can just fork an existing market implementation.

Keep the point tracking simple and transparent. Allow anyone to observe anyone’s point total, and the history of bets that led to those totals. Avoid issuing new points. Try to create a sense of stability. No leaderboards, just observing your friends’ (or strangers’) point totals. (Maybe allow people to deactivate their account to hide their point total page, but if they reactivate they still have their old point total.)

Limit maximum exposure to 50% of net worth. (But people can “exit” a market by taking a bet on the other side—betting on a 5 year question doesn’t necessarily tie up your points indefinitely.) Encourage people to optimize their log returns rather than expected returns.

Make it easy to link to pages for particular markets so people can see the current spread on a given question; make it very easy to quickly create a new account and bet on that market (2 clicks—login with facebook, and allow service to access your profile info).

It may be that a close-enough service already exists, though I’m a bit skeptical.

If you are at all interested in doing this, you have my blessing, this is a pretty generic idea (similar to sites that have existed in the past, e.g. Foresight Exchange). I’d also be happy to pay a bounty for an implementation I think is reasonable, I’d pay at least $1k for something that seems OK and up to $10k for something I think is actually really good.

Why I think this would be good

The barriers to entry in these markets would be tiny compared to existing prediction markets—within 30 seconds of seeing someone mention a market on Facebook, I could have created an account and placed a bet. I would likely bet i...