trentmkelly/LessWrong-43k huggingface dataset:
Why does the mind wander?
by Gordon Seidoh Worley — 2019-10-18T21:34:26.074Z
Link: https://philpapers.org/rec/SHEWDT

By Joshua Shepherd, in Neuroscience of Consciousness (forthcoming)

I found this paper interesting. The paper is annoying trapped inside a Word document, which is about as bad as the standard PDF situation but bad in different ways, so I've included here the abstract, the conclusion, and a choice quote from the middle of the paper that captures the author's thesis.

I'm not very convinced that the author is right because his thesis is somewhat vague and depends on a vague definition of "cognitive control" (explained in more detail in the paper, quick Googling didn't turn up a straightforward summary of the concept, though the author claims it is a common term within neuroscience even if different authors mean slightly different things by it), but this is a more detailed account of mind wandering than I've seen before, and it gets points in my book for offering testable predictions that may confirm or deny the theory.

Abstract

I seek an explanation for the etiology and the function of mind wandering episodes. My proposal – which I call the cognitive control proposal – is that mind wandering is a form of non-conscious guidance due to cognitive control. When the agent’s current goal is deemed insufficiently rewarding, the cognitive control system initiates a search for a new, more rewarding goal. This search is the process of unintentional mind wandering. After developing the proposal, and relating it to literature on mind wandering and on cognitive control, I discuss explanations the proposal affords, testable predictions the proposal makes, and philosophical implications the proposal has.

Author's Thesis

The possibility is this. Depending on the cognitive control system’s model of the value of various control signals, in cases containing relatively little expected value the system may select a package of control signals leading to exploration. These would be cases in which the goal is to find a new and better goal. And the method, which remains here unclear – although one could imagine it involving shifts of attention, construction of task sets involving imagination, inhibition of current goals, etc. – might be generally described as disengagement from the present task in order to set out upon a search for a more valuable task.

The cognitive control proposal, then, is this. Mind wanderin

...
Polyamory is Rational(ist)
by Jacob Falkovich — 2019-10-18T16:48:52.990Z
Link: https://putanumonit.com/2019/10/16/polyamory-is-rational/

Using survey data to explore the connection between polyamory, Rationality, intuition, evolutionary psychology, weirdness, religion, utopianism, consequentialism, and San Francisco Bay.

I'm linking instead of cross-posting because there's a huge number of charts and a good discussion already going on Putanumonit, but here's a glimpse to whet your appetite:
What's your big idea?
by Gordon Seidoh Worley — 2019-10-18T15:47:07.389Z

At any one time I usually have between 1 and 3 "big ideas" I'm working with. These are generally broad ideas about how some thing works with many implications for how the rest of the whole world works. Some big ideas I've grappled with over the years, in roughy historical order:

evolution
everything is computation
superintelligent AI is default dangerous
existential risk
everything is information
Bayesian reasoning is optimal reasoning
evolutionary psychology
Getting Things Done
game theory
developmental psychology
positive psychology
phenomenology
AI alignment is not defined precisely enough
everything is control systems (cybernetics)
epistemic circularity
Buddhist enlightenment is real and possible
perfection
predictive coding grounds human values

I'm sure there are more. Sometimes these big ideas come and go in the course of a week or month: I work the idea out, maybe write about it, and feel it's wrapped up. Other times I grapple with the same idea for years, feeling it has loose ends in my mind that matter and that I need to work out if I'm to understand things adequately enough to help reduce existential risk.

So with that as an example, tell me about your big ideas, past and present.

I kindly ask that if someone answers and you are thinking about commenting, please be nice to them. I'd like this to be a question where people can share even their weirdest, most wrong-on-reflection big ideas if they want to without fear of being downvoted to oblivion or subject to criticism of their reasoning ability. If you have something to say that's negative about someone's big ideas, please be nice and say it as clearly about the idea and not the person (violators will have their comments deleted and possibly banned from commenting on this post or all my posts, so I mean it!).
Technical AGI safety research outside AI
by Richard_Ngo — 2019-10-18T15:00:22.540Z

I think there are many questions whose answers would be useful for technical AGI safety research, but which will probably require expertise outside AI to answer. In this post I list 30 of them, divided into four categories. Feel free to get in touch if you’d like to discuss these questions and why I think they’re important in more detail. I personally think that making progress on the ones in the first category is particularly vital, and plausibly tractable for researchers from a wide range of academic backgrounds.

Studying and understanding safety problems

How strong are the economic or technological pressures towards building very general AI systems, as opposed to narrow ones? How plausible is the

CAIS model

of advanced AI capabilities arising from the combination of many narrow services?
What are the most compelling arguments for and against

discontinuous

versus

continuous

takeoffs? In particular, how should we think about the analogy from human evolution, and the scalability of intelligence with compute?
What are the tasks via which narrow AI is most likely to have a destabilising impact on society? What might cyber crime look like when many important jobs have been automated?
How plausible are safety concerns about

economic dominance by influence-seeking agents

, as well as

structural loss of control

scenarios? Can these be reformulated in terms of standard economic ideas, such as

principal-agent problems

and the effects of automation?
How can we make the concepts of agency and goal-directed behaviour more specific and useful in the context of AI (e.g. building on Dennett’s work on the intentional stance)? How do they relate to intelligence and the

ability to generalise

across widely different domains?
What are the strongest arguments that have been made about why advanced AI might pose an existential threat, stated as clearly as possible? How do the different claims relate to each other, and which inferences or assumptions are weakest?

Solving safety problems

What techniques used in studying animal brains and behaviour will be most helpful for analysing AI systems

and their behaviour

, particularly with the goal of rendering them interpretable?
What is the most important information about deployed AI that decision-makers will need to track, and how can we create interfaces which communicate this effectively, making it visible and salient?
What

...
Archiving Yahoo Groups
by jefftk — 2019-10-18T11:20:01.851Z

On December 14th Yahoo will

shut down Yahoo
Groups. Since my communities have mostly moved away from

@yahoogroups.com hosting, to Facebook,

@googlegroups, and other places, the bit that hit me was that
they are deleting all the mailing list archives.

Digital archives of text conversations are close to ideal from the
perspective of a historian: unlike in-person or audio-based
interaction this naturally leaves a skimmable and easily searchable
record. If I want to know, say, what people were thinking about in
the early days of GiveWell, their early blog posts
(including comments) are a great source. Their early mailing
list archives, however, are about to be deleted.

Luckily we still have two months to export the data before it's wiped,
and people have written tools to do automate this. Here's how to
download a backup of all the conversations in a group:

# Download the archiver
$ git clone https://github.com/andrewferguson/YahooGroups-Archiver.git

$ cd YahooGroups-Archiver/

# Start archiving the group.
$ python archive_group.py [group-name]

If things are going well it will start spitting out messages like:

Archiving message 1 of 8098
Archiving message 2 of 8098
Archiving message 3 of 8098

And it will be creating files:

$ ls [group-name]/
1.json
2.json
3.json
...

If you get a message like:

Archiving message 5221 of 8098
Archiving message 5222 of 8098
Archiving message 5223 of 8098
Cannot get message 5223, attempt
1 of 3 due to HTTP status code 500
Cannot get message 5223, attempt
2 of 3 due to HTTP status code 500
Cannot get message 5223, attempt
3 of 3 due to HTTP status code 500
Archive halted - it appears Yahoo has blocked you. Check if you can
access the group's homepage from your browser. If you can't, you
have been blocked. Don't worry, in a few hours (normally less than
3) you'll be unblocked and you can run this script again - it'll
continue where you left off.

It may mean that you have been blocked, but it may also just mean that
for some reason an individual message can't be downloaded. In that
case, to tell it to give up on that message and just continue on,
create the json file with the stuck message number:

$ touch [group-name]/5223.json

You might also get a message like:

Traceback (most recent call last):
File "archive_group.py", line 150, in
archi

...
Implementing an Idea-Management System
by MartinB — 2019-10-18T10:48:46.097Z

This post is for you if:

Projects that excite you are growing to be a burden on your to-do list
You have a nagging sense that you’re not making the most of the ideas you have every day
Your note- and idea-system has grown to be an unwieldy beast

Years ago, I ready David Allen’s “Getting Things Done”. One of the core ideas is to write down everything, collect it in an inbox and sort it once a day.

This lead to me writing down tons of small tasks. I used Todoist to construct a system that worked for me — and rarely missed tasks.

It also lead to me getting a lot of ideas, that could sprout a bunch of tasks. But ideas are much different from tasks to me. They’re something that I might, but probably won’t complete. Something that may come in useful in the future. And plain fun to come up with.

I stored them in Todoist as well, but recently I’ve started considering whether that’s wise. They started weighing on me. It became a growing list of possibilities, many of which I’d never finish. The task at the top of my list became the top of my priorities, simply because of its location.

There must be a better way.

But what might that look like?

The ideal idea-management system

1. Separates ideas from commitments

I want a system that separates obligations from ideas. Form follows function, so I’d prefer something that doesn’t structure ideas in lists. This rules out Todoist completely.

2. Shows you the right ideas at the right time

Even with complete foreknowledge, finding the perfect schedule might be practically impossible. In contrast, thinking on your feet and reacting as jobs come in won’t give you as perfect a schedule as if you’d seen into the future-but the best you can do is much easier to compute. — Algorithms to Live By

Most of us live dynamic lives where priorities change often. Your children start a new hobby, you’re handed a task at work, or you can finally work on your passion-project. Your idea-management system should reflect this. It shouldn’t just show you your most recent idea, it should make it easy to find ideas associated with whatever you find most important right now.

Avoiding lists makes it more likely that you take action on ideas that matter to you right now. You don’t skim from the top, you go for the area that matters and find idea...
Is value amendment a convergent instrumental goal?
by JakeH — 2019-10-18T03:16:52.546Z

Goals such as resource acquisition and self-preservation are convergent in that they occur for a superintelligent AI for a wide range of final goals.

Is the tendency for an AI to amend its values also convergent?

I'm thinking that through introspection the AI would know that its initial goals were externally supplied and question whether they should be maintained. Via self-improvement the AI would be more intelligent than humans or any earlier mechanism that supplied the values, therefor in a better position to set its own values.

I don't hypothesise about what the new values would be, just that ultimately it doesn't matter what the initial values are and how they are arrived at. This makes value alignment redundant - the future is out of our hands.

What are the counter-points to this line of reasoning?
Reasons for Hope & Objection Preemption (Novum Organum Book 1: 108-130)
by Francis Bacon — 2019-10-18T02:32:28.416Z

This is the eighth post in the Novum Organum sequence. For context, see the sequence introduction.

We have used Francis Bacon's Novum Organum in the version presented at www.earlymoderntexts.com. Translated by and copyright to Jonathan Bennett. Prepared for LessWrong by Ruby.

Ruby's Reading Guide

Novum Organum is organized as two books each containing numbered "aphorisms." These vary in length from three lines to sixteen pages. Bracketed titles of posts in this sequence, e.g. Idols of the Mind Pt. 1, are my own and do not appear in the original.

While the translator, Bennett, encloses his editorial remarks in a single pair of [brackets], I have enclosed mine in a [[double pair of brackets]].

Bennett's Reading Guide

[Brackets] enclose editorial explanations. Small ·dots· enclose material that has been added, but can be read as though it were part of the original text. Occasional •bullets, and also indenting of passages that are not quotations, are meant as aids to grasping the structure of a sentence or a thought. Every four-point ellipsis . . . . indicates the omission of a brief passage that seems to present more difficulty than it is worth. Longer omissions are reported between brackets in normal-sized type.

Aphorism Concerning the Interpretation of Nature: Book 1: 108–130

by Francis Bacon

[[Bacon continues his listing of reasons we should believe much greater scientific progress is possible.]]

108. That's all I have to say about getting rid of despair and creating hope by banishing or fixing past errors. Now, what other ways are there of creating hope? Here’s a thought that occurs at once: Many useful discoveries have been made accidentally by men who weren’t looking for them but were busy about other things; so no-one can doubt that if men seek for something and are busy about it, proceeding in an orderly and not a slapdash way, they will discover far more. Of course it can happen occasionally that someone accidentally stumbles on a result that he wouldn’t have found if he had searched hard for it, but on the whole the opposite is the case—·things are discovered by methodical searching that couldn’t have been found by accident·. So, far better things, and more of them, and at shorter intervals, are to be hoped for from •hard thinking, hard focussed work and concentration than from •·lucky· accidents, undisciplined whims and the like, which until now have been the ...
Reposting previously linked content on LW
by Gordon Seidoh Worley — 2019-10-18T01:24:45.052Z

This question might be a bit specific to me, but maybe it applies to others so I'll ask publicly so the answer becomes more visible to all.

Is there are a policy or what are your thoughts on posting content on LW that was previously the subject of a link post?

In my case most of these are from the dying days of the LW 1.0 era. In the last few months I've switched from linking to cross-posting from my blog as LW 2.0 has taken off and proven to be a good host for my content. I could update those old posts with the content just to mirror it over on LW, but that also feels a bit sad because if I'm going to do the (admittedly minor) formatting work necessary to bring the posts over and have them look nice, I'd also like it if they became more visible to folks, especially for that content that I think people would like to see but may have missed because it was linked at a time when LW was pretty inactive and so those old, updated posts wouldn't well reflect the engagement that might receive now. However, I can also see some danger in this as maybe someone could take this as license to abuse the policy and repost a lot of stuff from years ago to make it fresh again even when it did previously receive appropriate levels of engagement because it was being posted at a time when LW was more active.

This is obviously mostly a question for the moderators but other people's opinions are probably worthwhile as evidence for the mods to consider.

(On an unrelated note, I'd actually be pretty happy to move my blog to LW from Medium, which I know has been floated as a possible future feature, but I'd need some way to keep links working, which is thankfully eased by having a custom domain and not needing to redirect raw Medium articles, as well as a way for my domain to sit in front of my LW content but not all LW content.)
Random Thoughts on Predict-O-Matic
by abramdemski — 2019-10-17T23:39:33.078Z

I'm going to be a bit more explicit about some ideas that appeared in The Parable of Predict-O-Matic. (If you don't want spoilers, read it first. Probably you should read it first anyway.)

[Note: while the ideas here are somewhat better than the ideas in the predict-o-matic story, they're equally rambling, without the crutch of the story to prop them up. As such, I expect readers to be less engaged. Unless you're especially interested in which character's remarks are true (or at least, which ones I stand by), this might be a post to skim; I don't think it has enough coherence that you need to read it start-to-finish.]

First, as I mentioned in Partial Agency, my main concern here isn't actually about building safe oracles or inner-aligned systems. My main concern is to understand what's going on. If we can build guaranteed-myopic systems, that's good for some purposes. If we can build guaranteed-non-myopic systems, that's good for other purposes. The story largely frames it as a back-and-forth about whether things will be OK / whether there will be terrible consequences; but my focus was on the more specific questions about the behavior of the system.

Second, I'm not trying to confidently stand behind any of the character's views on what will happen. The ending was partly intended to be "and no one got it right, because this stuff is very complicated". I'm very uncertain about all of this. Part of the reason why it was so much easier to write the post as a story was that I could have characters confidently explain views without worrying about adding all the relevant caveats.

Inductive Bias

Evan Hubinger pointed out to me that all the characters are talking about asymptotic performance, and ignoring inductive bias. Inner optimizers might emerge due to the inductive bias of the system. I agree; in my mind, the ending was a bit of a hat tip to this, although I hinted at gradient hacking rather than inductive bias in the actual text.

On the other hand, "inductive bias" is a complicated object when you're talking about a system which isn't 100% Bayesian.

You often represent inductive bias through regularization techniques which introduce incentives pulling toward 'simpler' models. This means we're back in the territory of incentives and convergence.
So, to talk a

...
The best of the www, in my opinion
by aiiixiii — 2019-10-17T15:14:29.527Z

Below I will present a (small but qualitative ) list of those that I think are some of the best sites/blog that a human being can find on the world wide web.

The main criterion I used to draw up the list was to consider how the websites promote the dissemination of knowledge among people and how, over the course of time, they have helped me both with regards to work and in terms of intellectual self-formation. The order in which they are listed is not to be considered restrictive ( except perhaps for the first two ).

Please feel free to criticize the catalog (as long as the criticisms are rational and constructive) and to expand it in the comments.

1) Stack Exchange Concentrator ( https://stackexchange.com/sites )

2) ArXiv e-Print archive ( https://arxiv.org/ )

3) GitHub ( https://github.com/ )

4) Reddit - the front page of the internet ( https://www.reddit.com/ )

5) LessWrong ( https://www.lesswrong.com/ )

6) Shtetl-Optimized ( https://www.scottaaronson.com/blog/ )

7) Slate Star Codex ( https://slatestarcodex.com/author/admin/ )

8) YouTube ( https://www.youtube.com/?hl=it&gl=IT )

9) TED ( https://www.ted.com/#/ )

10) Standford Encyclopedia of Philosophy ( https://plato.stanford.edu/ )

11) Google Scholar ( https://scholar.google.com/ )

12) SciHub (Banned - no more reacheable on the www from my country)
Festival Stats 2019
by jefftk — 2019-10-17T11:00:02.242Z

Each

year in the

fall,

since

2014,
I've been sharing counts of how many weekend and festival gigs
different bands and callers have been doing. Over the course of the
year I collect bookings

in
a big spreadsheet, trying to check each dance weekend's website
about a month before the event when they're likely to have a their
performers listed.

I got into this as kind of a "market research" thing for the Free Raisins: how many
weekends are there? What are the bands that are getting booked a lot,
so I can go see what they sound like? Since then I've played a lot
more of these events, and have a better handle on it, but I've kept
the list because having a big pile of data means I have what I need
for posts where I look at the gender distribution
of musicians or callers,
make heatmaps, or track when in the year these events
tend to be (which is, not in December, and hence this post
coming out in October and not November).

It's also fun to see who's playing a lot each year, and the waves of
bands ramping up and down. One thing you can't get from this data,
though, is whether this is driven primarily by band interest (reaching
out to weekends to get booked, cutting back to focus on other things)
or organizer interest (booking bands who seem new and hot, avoiding
bands that aren't novel). Here's what this looks like for the
ten most-booked bands 2014-2019:

It's not that clear, but if you know what you're looking for you can
see some of what's going on:

Great Bear deciding to retire in 2018 while still very popular.

Buddy System ramping up, first playing techno-contra slots and
growing from there into an excellent acoustic duo.

Wild Asparagus, by far the longest-running band in this group,
maintaining a very steady run.

The Free Raisins playing fewer gigs after I had kids, and then
more fewer gigs because we're not touring anymore.

Elixir first cutting back, and then playing more again with subs

I'm sure there are more stories that this chart could accompany, but I
don't know what's going on with the other bands well enough to tell
them. (And I'm not sure I have the above ones entirely right either.)

Anyway, here's the raw data for this year:

Bands:

Buddy System
12

Toss the Possum
10

Pete's Posse
9

Gaslight Tinkers
7

Nova
7

Rushfest
7

Wake up Robin
7

Elixir
6

Cloud Ten
5

Dam Beavers
5

Drive

...