trentmkelly/LessWrong-43k huggingface dataset:
[Link] What do conservatives know that liberals don't (and vice versa)? 
by defilippis — 2019-10-02T16:14:37.661Z

I am a PhD student currently conducting research on political polarization and persuasion. I am running an experiment that requires a database of trivia questions which conservatives are likely to get correct, and liberals are likely to get wrong (and vice versa). Our pilot testing has shown, for example, Democrats (but not Republicans) tend to overestimate the percentage of gun deaths that involve assault-style rifles, while Republicans (but not Democrats) tend overestimate the proportion of illegal immigrants who commit violent crimes. Similarly, Democrats (but not Republicans) tend to overestimate the risks associated with nuclear power, while Republicans (but not Democrats) underestimate the impact of race-based discrimination on hiring outcomes.

Actually designing these questions is challenging, however, because it’s difficult to know which of one’s political beliefs are most likely to be ill-informed. As such, I am running a crowdsourcing contest in which we will pay $100 for any high-quality trivia question submitted (see contest details here:

https://redbrainbluebrain.org/

). The only requirements are that participants submit a question text, four multiple choice answers, and a credible source. The deadline for submissions is October 15th, 2019 at 11:59 p.m.

My intuition is that the LessWrong community will be particularly good at generating these kinds of questions given their commitment to belief updating and rationality. If you don't have the time to participate in the contest, I welcome any ideas about potential topics that might be a fruitful source of these kinds of questions. Would also appreciate any clues as to what other communities may be particularly good at generating these kinds of questions.
What are we assuming about utility functions?
by Grue_Slinky — 2019-10-02T15:11:17.721Z

I often notice that in many (not all) discussions about utility functions, one side is "for" their relevance, while others tend to be "against" their usefulness, without explicitly saying what they mean. I don't think this is causing any deep confusions among researchers here, but I'd still like to take a stab at disambiguating some of this, if nothing else for my own sake. Here are some distinct (albeit related) ways that utility functions can come up in AI safety, in terms of what assumptions/hypotheses they give rise to:

AGI utility hypothesis

: The first AGI will behave as if it is maximizing some utility function

ASI utility hypothesis

: As AI capabilities improve well beyond human-level, it will behave more and more as if it is maximizing some utility function (or will have already reached that ideal earlier and stayed there)

Human utility hypothesis

: Even though in some experimental contexts humans seem to not even be particularly goal-directed, utility functions are often a useful model of human preferences to use in AI safety research

Coherent Extrapolated Volition

(CEV) hypothesis

: For a given human H, there exists some utility function V such that if H is given the appropriate time/resources for reflection, H's values would converge to V

Some points to be made:

The "Goals vs Utility Functions" chapter of Rohin's Value Learning sequence, and the resulting discussion focused on differing intuitions about the AGI and ASI utility hypotheses. Specifically, the main post there pointed out that seemingly anything can be trivially modeled as being a "utility maximizer" (further discussion here), whereas only some intelligent agents can be described as being "goal-directed" (as defined in this post), and the latter is a more useful concept for reasoning about AI safety.
AGI utility doesn't logically imply ASI utility, but I'd be surprised if anyone thinks it's very plausible for the former to be true while the latter fails. In particular, the coherence arguments and other pressures that move agents toward VNM seem to roughly scale with capabilities. A plausible stance could be that we should expect most ASIs to hew close to the VNM ideal, but these pressures aren't quite so overwhelming at the AGI level; in particular, humans are fairly goal-directed but only "partially" VNM, so the goal-directedness pressures on an AGI will likely be at this order of magnitude. De

...
Human instincts, symbol grounding, and the blank-slate neocortex
by Steven Byrnes — 2019-10-02T12:06:35.361Z

Intro: What is Common Cortical Algorithm (CCA) theory, and why does it matter for AGI?

As I discussed at Jeff Hawkins on neuromorphic AGI within 20 years, and was earlier discussed on LessWrong at The brain as a universal learning machine, there is a theory, due originally to Vernon Mountcastle in the 1970s, that the neocortex[1] (75% of the human brain by weight) consists of ~150,000 interconnected copies of a little module, the "cortical column", each of which implements the same algorithm. Following Jeff Hawkins, I'll call this the "common cortical algorithm" (CCA) theory. (I don't think that terminology is standard.)

So instead of saying that the human brain has a vision processing algorithm, motor control algorithm, language algorithm, planning algorithm, and so on, in CCA theory we say that (to a first approximation) we have a massive amount of "general-purpose neocortical tissue", and if you dump visual information into that tissue, it does visual processing, and if you connect that tissue to motor control pathways, it does motor control, etc.

Whether and to what extent CCA theory is true is, I think, very important for AGI forecasting, strategy, and both technical and non-technical safety research directions—see my answer here for more details.

Should we believe CCA theory?

CCA theory, as I'm using the term, is a simplified model. There are almost definitely a couple caveats to it:

There are sorta "hyperparameters" on the generic learning algorithm which seem to be set differently in different parts of the neocortex. For example, some areas of the cortex have higher or lower density of particular neuron types. There are other examples too.[2] I don't think this significantly undermines the usefulness or correctness of CCA theory, as long as these changes really are akin to hyperparameters, as opposed to specifying fundamentally different algorithms. So my reading of the evidence is that if you put, say, motor nerves coming out of visual cortex tissue, the tissue could do motor control, but it wouldn't do it quite as well as the motor cortex does.[3]

There is almost definitely a gross wiring diagram hardcoded in the genome—i.e., set of connections between different neocortical regions and each other, and other parts of the brain. These connections later get refined and edited during learning. Again, we can ask how much the existence of this innate gros

...
Toy model #6: Rationality and partial preferences
by Stuart_Armstrong — 2019-10-02T12:04:53.048Z

In my research agenda on synthesising human preferences, I didn't mention explicitly using human rationality to sort through conflicting partial preferences.

This was, in practice, deferred to the "meta-preferences about synthesis". In this view, rationality is just one way of resolving contradictory lower-level preferences, and we wouldn't need to talk about rationality, just observe that it existed - often - within the meta-preferences.

Nevertheless, I think we might gain by making rationality - and its issues - an explicit part of the process.

Defining rationality in preference resolution

Explicit choice

We can define rationality in this area by using the one-step hypotheticals. If there is a contradiction between lower-level preferences, then that contradiction is explained to the human subject, and they can render a verdict.

This process can, of course, result in different outcomes depending on how the question is phrased - especially if we allow the one-step hypothetical to escalate to a "hypothetical conversation" where more arguments and evidence is considered.

So the distribution of outcomes would be interesting. If, in cases where most of the relevant argument/evidence is mentioned, the human tends to come down on one side, then that is a strong contender for being their "true" rational resolution of the issues.

However, if in stead the human answers in many different ways, especially if the answer changes because of small changes in how the evidence is ordered, how long the human has to think, whether they get all the counter-evidence or not, and so on - then their preference seems to be much weaker.

For example, I expect that most people similar to me would converge on one answer on questions like "does expected lives save dominate most other considerations in medical interventions?", while having wildly divergent views on "what's the proper population ethics?".

This doesn't matter

Another use of rationality could be to ask the human explicitly whether certain aspects of their preferences should matter. Many human seem to have implicit biases, whether racial or other; many humans believe that it is wrong to have these biases, or at least wrong to let them affect their decisions[1].

Thus another approach for rationality is to query the subject as to whether some aspect should be affecting their decisions or not (because humans only consider a tiny space ...
Double Tongue Whistling
by jefftk — 2019-10-02T11:10:01.689Z

I can whistle about seven notes per second, which corresponds to a
reel at 105bpm. [1] While this isn't a problem for

whistling basslines it's slightly too slow
for melodies at contra dance speed (~110-122bpm). I want to figure
out how to whistle faster, and I know there are people who whistle
faster, but I don't know how it's usually done.

I see two main routes:

Do what I currently do, but faster.

Figure out how to do something else.

The former doesn't seem very promising: I've been playing around with
whistling for decades and I suspect I'm pretty close to a local
maximum with my current approach. On the other hand, I'm just trying
to get from seven notes per second to eight, which seems like it might
be possible?

The latter is pretty open, and probably involves doing something
that's slower at first but will eventually be faster. The main
problem is, how do I know that after I put in all that effort I'll
actually end up with something faster? Ideally there would be people
demonstrating on youtube or something, with "here's how to whistle
quickly" videos, but I'm not seeing that.

Still, it seems like some sort of double-tonguing should work.
Normally when I whistle I mark the notes with my glottis, the same as
the two glottal stops in "uh-oh". A different option, though, would
be to mark the notes by making a velar closure with my tongue, the
same as the two velar stops in "cook" ("k"). And then I could
alternate between them, which seems like it should let me get up to
twice the speed of the slower one. Here's what the three sound like:

I find velar stops harder, partly because it's not what I'm used to,
and partly because I'm already using my tongue to form the whistle.
Currently I can them 4-5 times per second. When alternating I can do
a little better, 5-6 times per second, but that's still less than the
8-10 you'd expect from doubling my velar-only speed. I can go
te-ke-te-ke with closures 10-11 times per second, so I am optimistic.

Has anyone learned to double-tongue their whistling successfully?
Does this work?

[1] Or a jig at 140bpm, since that's six notes per measure instead of
eight.

Comment via: facebook
Does the US nuclear policy still target cities?
by Jeffrey Ladish — 2019-10-02T00:18:10.261Z

The history of nuclear strategic bombing

Daniel Ellsberg’s

The Doomsday Machine

brought my attention to a horrifying fact about early US nuclear targeting policy. In 1961, the US had only one nuclear war plan, and it called for the destruction of every major Soviet city and military target. That is not surprising. However, the plan also called for the destruction of every major Chinese city and military target, even if China had not provoked the United States. In other words, the US nuclear war plan called for the destruction of the major population centers of the most populous country in the world, even in circumstances where that country had not attacked the United States or its allies. Ellsberg points out that at the time, people at RAND and presumably other parts of the US defense establishment understood that the Chinese and the Soviets were beginning to diverge in strategic interests and thus should not be treated as one bloc. Nevertheless, the top levels of the US command, including President Eisenhower, were committed to the utter destruction of both Chinese and Soviet targets in the event of a war with either country.

The policy of destroying cities is a legacy left over from strategic bombing in World War II. The destruction of Hiroshima and Nagasaki are the most famous, but the fire bombings of Japanese and Germany cities destroyed far more infrastructure and killed far more people than the two atomic bombs. The given rational for strategic bombing was to destroy the ability of the enemy states to continue to make war. If a state can no longer produce airplanes and tanks, either because the factories have been destroyed or because there are no longer people to work in the factories, then its ability to resist is diminished.

Given the level of technology and development in WWII, strategic bombing had a chance at achieving military objectives, because the conflict was to carry on for multiple years. On the timescale of years, a country’s capacity to build armaments and resupply armies in the field can be crucial to victory.

Nuclear war changes this calculus. In a modern nuclear war involving SLBMs (Submarine launched ballistic missiles), ICBMs (Intercontinental ballistic missiles), strategic bombers, and other weapon systems, the majority of an adversary’s military, industrial and population centers could be destroyed in a matter of...
Survival and Flourishing Fund Applications closing in 3 days
by habryka — 2019-10-02T00:12:21.287Z

The grant round we announced a month ago for the new Survival and Flourishing Fund is closing in 3 days. We haven't gotten that many applications, so I would recommend applying.

https://www.lesswrong.com/posts/CcAnPPKmPmbXxtWME/survival-and-flourishing-fund-grant-applications-open-until

From the original announcement post:

The plan is to make a total of $1MM-$2MM in grants to organizations working on the long term flourishing and survival of humanity.

At this point in time SFF can only make grants to charities and not individuals, so if you have a project or organization that you want to get funding for, you will have to either already be part of an established charity, found one, or be sponsored by an existing charity.
Open & Welcome Thread - October 2019
by Ruby — 2019-10-01T23:10:57.782Z

If it’s worth saying, but not worth its own post, here's a place to put it.

You can also make a shortform post.

And, if you are new to LessWrong, here's the place to introduce yourself.

Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.

The Open Thread sequence is here.
LW Team Updates - October 2019
by Ruby — 2019-10-01T23:08:18.283Z

Like last month, this is a once-monthly updates for LessWrong team activities and announcements.

Please also feel free to use the comments section on this post as a Schelling point to give feedback, file bug reports, or ask questions you have about the site. (You can also email us, ask a question, use Intercom, or message us via our FB page.)

Recent Features

Link Previews

We successfully shipped Link Previews in September. Now when you hover of an embedded link you get a pop-up, for internal links to LessWrong posts you get a preview. See the full announcement here.

Improvements to the Community Map

In the course of supporting SSC Meetups Everywhere, we've made a few improvements to the community map on our Community page.

In addition to Groups and Events, now users can also add themselves to the map together with some info about what they're looking for or willing to do, e.g. someone can list themselves as willing to arrange local meetups if there are enough people in their area.
You can sign up for event and group notifications, as well as notifications for once there are N people within a given radius from you.
The UI has been improved.
The map is now more performant.

For more information about the LessWrong community section, including the different event types (LW/SSC/EA/MIRIx), see the relevant section of our FAQ.

Upcoming Features

Although we hoped to deliver them in September, the subscriptions overhaul and new editor are still under works. Unless things go terribly wrong, they should be released this month. Convert Comments to Posts is currently blocked behind some tech debt.

We might announce plans for additional near-term features once we've completed our Q4 planning.

General Updates

Petrov Day

On September 26th, 1983, Stanislav Petrov, a Russian lieutenant colonel, decided not to report an [in fact false] alarm of incoming missiles from the US, thereby likely preventing nuclear war. September 26 has since become a Rationalist community holiday celebrating the calamity that did not occur that day and reminding us of the dangers present in a world powerful dual-use technologies.

LessWrong commemorated Petrov Day this year from the unilateralist curse angle and turned the site into a live exercise and experiment. A big red button was placed on the homepage and 125 users were given launch codes with the power to bring down the frontpage for 24 hours.

The LessWrong fron...
World State is the Wrong Abstraction for Impact
by TurnTrout — 2019-10-01T21:03:40.153Z

These existential crises also muddle our impact algorithm. This isn't what you'd see if impact were primarily about the world state.

Appendix: We Asked a Wrong Question

How did we go wrong?

When you are faced with an unanswerable question—a question to which it seems impossible to even imagine an answer—there is a simple trick that can turn the question solvable.

Asking “Why do I have free will?” or “Do I have free will?” sends you off thinking about tiny details of the laws of physics, so distant from the macroscopic level that you couldn’t begin to see them with the naked eye. And you’re asking “Why is X the case?” where X may not be coherent, let alone the case.

“Why do I think I have free will?,” in contrast, is guaranteed answerable. You do, in fact, believe you have free will. This belief seems far more solid and graspable than the ephemerality of free will. And there is, in fact, some nice solid chain of cognitive cause and effect leading up to this belief.

~ Righting a Wrong Question

I think what gets you is asking the question "what things are impactful?" instead of "why do I think things are impactful?". Then, you substitute the easier-feeling question of "how different are these world states?". Your fate is sealed; you've anchored yourself on a Wrong Question.

At least, that's what I did.

Exercise: someoneme, early last year says that impact is closely related to change in object identities.

Find at least two scenarios which score as low impact by this rule but as high impact by your intuition, or vice versa.

You have 3 minutes.

Gee, let's see... Losing your keys, the torture of humans on Iniron, being locked in a room, flunking a critical test in college, losing a significant portion of your episodic memory, ingesting a pill which makes you think murder is OK, changing your discounting to be completely myopic, having your heart broken, getting really dizzy, losing your sight.

That's three minutes for me, at least (its length reflects how long I spent coming up with ways I had been wrong).

Appendix: Avoiding Side Effects

Some plans feel like they have unnecessary side effects:

Go to the store.versusGo to the store and run over a potted plant.

We talk about side effects when they affect ou...
Welcome to EA Latvia [Edit With Your Details]
by Andis_Draguns — 2019-10-01T21:01:51.040Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Welcome to Baltimore LessWrong [Edit With Your Details]
by bookinchwrm — 2019-10-01T18:52:33.097Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
What funding sources exist for technical AI safety research?
by johnswentworth — 2019-10-01T15:30:08.149Z

What funding sources exist? Who are they aimed at - people within academia, independent researchers, early-career researchers, established researchers? What sort of research are they aimed at - MIRI-style deconfusion, ML-style approaches, more theoretical, less theoretical? What quirks do they have, what specific things do they target?

For purposes of this question, I am not interested in funding sources aimed at strategy/infrastructure/coordination/etc, only direct technical safety research.
Communication and Waking Hours
by jefftk — 2019-10-01T11:00:01.416Z

One of the ways technology interacts with culture is about who's job
it is to avoid contacting people at bad times. With older
technologies there were clear conventions based on their limitations:

Mail: you can send the letter whenever, and the person will deal
with it when convenient.

Phone: don't call someone when they might be asleep, but if you
work nights unplug your phone during the day.

It's more complicated now, though, because the new ways of contacting
people all boil down to "a message goes from your phone to theirs" and
their phone may or may not be configured to let them know about the
message. I think the rules basically come down to:

If the default configuration is either always silent or has a
nightly "quiet period" then you can send whenever and not worry about
waking people.

Otherwise treat it as if it will wake people up unless you
happen to know that this particular person has configured their app
differently.

In the first category we have email, Slack, and probably some other
apps. In the second we have phone, SMS, Whatsapp, FB Messenger, Hangouts,
Signal, and almost everything else.

Overall this is a bad equillibrium: it would be better to at least
have the option to send your message whenever, and if the person is
asleep they'll get it in the morning. The receiver has a much better
idea of when are ok times for notifications than the sender does . But
there's currently no way to know how someone has their device
configured, so you generally need to stay on the safe side and wait
for a reasonable hour before sending.

If we did successfully move from "sender guesses do-not-disturb" to
"receiver's device tracks do-not-disturb", however, it would be good
to have some sort of override. For example, if you called me in the
middle of the night ideally you would get an automated message saying
that I'm asleep, but offering you the option to press a button to get
reach me if it's urgent enough that you want to wake me up. Systems
where calls from trusted contacts bypass do-not-disturb don't do this
well: not every late-night call from my sister is intended to wake me
up, and there are people who may need to contact me urgently that I
won't have thought to configure in my phone.