trentmkelly/LessWrong-43k huggingface dataset:
Press Your Luck (2/3)
by Douglas_Reay — 2018-03-08T15:58:58.817Z

This is the second in a three part sequence, examining the scenario in which software development organisations might knowingly take a significant risk of unintentionally launching an AI program into inadequately supervised or controlled self-improvement, by comparing the scenario to the 'press your luck' game mechanic.

This second part lays out the problem caused for humanity by the risk inherent in having software development being carried out by multiple organisations with competitive motivations.

Milestones

We can conceptualise the progress towards the launch of a general purpose AI that can out-think humans as a series of milestones, analogous to the alert states used by the US military.

DEFCON 5 : A computer program designed to become an expert in a single pre-determined domain, such as chess, that can only learn and improve from examples within that domain, as its learning algorithm has built in domain-specific assumptions.

DEFCON 4 : A computer program with a generic learning algorithm, that can improve at identifying tanks, or riding a unicycle, depending on the examples it is trained with, and the dependencies it has already learned.

DEFCON 3 : A computer program that has learned sufficient dependencies, that it can learn to program computers, at a level where it can comprehend an existing computer program, and then produce an improved next generation of it by devising, implementing and testing multiple new algorithms or approaches.

DEFCON 2 : A computer program that looks ahead more than one generation, and tries to produce an improved program to play chess, by first designing an improved version of a chess program designing program.

DEFCON 1 : A computer program which considers the full environment in which a program runs (including hardware, networking and human dictated programming or monitoring constraints) as factors which affect the capability of a program to run efficiently, and treats altering that environment as a problem to solve.

Somewhere between DEFCON 2 and DEFCON 1, humanity enters a game of "Press Your Luck".

Example

Suppose there are two companies, MarchCorp and AprilCorp, that are competing for market share in an area of business where the use of efficient programs is a significant factor. If MarchCorp steps closer to the edge, and starts taking AprilCorp's market share because MarchCorp took some risks and e...
Making yourself small
by Helen — 2018-03-08T14:26:31.465Z

Disclaimers:

Epistemic status: trying to share a simplified model of a thing to make it easier to talk about; confident there’s something there, but not confident that my read of it or this attempt at simplification is good.
This post is a rewrite of a talk I gave at a CFAR event that seemed well-received; a couple of people who weren’t there heard about it and asked if I’d explain the thing. I tried to write this relatively quickly and keep it relatively short, which may mean it’s less clear than ideal - happy to hash things out in the comments if so.
The thing is much easier to describe if I occasionally use some woo-y language like “aura” and “energy” to gesture in the direction of what I mean. I’ll put these words in quotes so you know I know I’m being woo-y; feel free to mentally insert “something-you-might-call-” before each instance, if that helps.

Rationalists

love

talking

about

status

. And that’s great - it’s a useful idea, for sure.

But I think in our eagerness to notice and apply the concept of status, we end up conflating it with a related-but-different thing. I also think the different thing is super useful in its own right, and important to understand, and I hope sharing my relatively basic thoughts on it will also let others build off that beginning.

So this post is my attempt to explain that different thing. I’m going to call it “making yourself big” and “making yourself small”.

This post:

Horses, goats, bulls
A framework
What to do with it

1. Horses, goats, bulls

Let’s start with an animal showing how it’s done.

This video

popped up in my feed recently, and is an amazing example of an animal (the goat) “making himself big”. To us, it’s obvious that the bull could flatten the goat in any real contest. But the bull doesn’t know that! He’s not reasoning about relative mass or propulsive power; he’s responding purely to how “big” the goat is making his “aura”.

Watch the video and see if you can notice specifically how the goat is doing this.

0:07 - where the goat rears up - is an obvious moment, but I claim 0:30-0:36 is even better, where the bull feints forward a couple of times while the goat stands firm, using his posture to “project” his “energy” irrepressibly forward. The bull is simply unable to continue toward him.

The next two examples are of a human working with horses, to show examples of it looks like for a human to be big or small.

(Full discl...
The Steampunk Aesthetic
by Raemon — 2018-03-08T09:06:40.436Z

Epistemic Status: Poetry. More confident about Linux than I have a right to be.

Last year, some friends wanted to buy a boat to live on, because the Bay Area is hella expensive. They acquired a tugboat, in Alaska. I was among a few people who helped them pilot it south.

I currently do not have enough information to know whether the boat turned out to be a good idea, but in the meanwhile I gained what you might call (if you'll pardon the phrase) an intuitive, gears level understanding* of the Steampunk Aesthetic.

(My dad tried to teach me this when I was little. Sorry I didn't get it at the time, Dad)

i. Aesthetics of Abstraction

The Apple Aesthetic is clean, smooth, and magical.

You hold a glowing rectangle in your hand, formed as solid and opaque as the monolith from 2001: A Space Odyssey. You touch the rectangle, and things happen. You can learn to communicate with a little homunculus inside named Siri and ask her for favors, and if you phrase the words just right, perhaps she might grant your wish.

The rectangle has secrets, but you are not meant to pry into them too deeply, and if you try, forces from beyond will thwart you in ways subtle and unsubtle.

If your rectangle breaks, you cannot fix it. You must take it to the Apple Priests in their white halls of power, lit by shining icons. They will secret your rectangle away, work their arcane power on it and return it to you (or perhaps leave you another rectangle in its place).

The Linux Aesthetic is magical too, but of a different sort.

There are arcane languages you can learn, but they do not hold your hand. You either know the words of power, or you do not. Most ways of speaking the words of power are so wrong that nothing happens. Worse is to be almost right, and yet so wrong that you undo entire chapters of you life.

You may borrow scrolls from wizards more powerful than you, but god help you if you speak their words without understanding what exactly they mean. Read one scroll blindly, and fate might spare you. Read three scrolls that you stole from different masters, and reality may twist itself into such knots that you can never undo them.

But, get the words exactly right, and you can shape reality to your will.

Apple and Linux have their respective virtues, but we are not here to talk about them – we are here to contrast them with Steampunk.

The steampunk world is one of exposed wires...
My attempt to explain Looking, insight meditation, and enlightenment in non-mysterious terms
by Kaj_Sotala — 2018-03-08T07:37:54.532Z

Epistemic status: pretty confident. Based on several years of meditation experience combined with various pieces of Buddhist theory as popularized in various sources, including but not limited to books like The Mind Illuminated, Mastering the Core Teachings of the Buddha, and The Seeing That Frees; also discussions with other people who have practiced meditation, and scatterings of cognitive psychology papers that relate to the topic. The part that I’m the least confident of is the long-term nature of enlightenment; I’m speculating on what comes next based on what I’ve experienced, but have not actually had a full enlightenment. I also suspect that different kinds of traditions and practices may produce different kinds of enlightenment states.

While I liked Valentine’s recent post on kensho and its follow-ups a lot, one thing that I was annoyed by were the comments that the whole thing can’t be explained from a reductionist, third-person perspective. I agree that such an explanation can’t produce the necessary mental changes that the explanation is talking about. But it seemed wrong to me to claim that all of this would be somehow intrinsically mysterious and impossible to explain on such a level that would give people at least an intellectual understanding of what Looking and enlightenment and all that are. Especially not after I spoke to Val and realized that hey, I actually do know how to Look, and that thing he’s calling kensho, that’s happened to me too.

(Note however that kensho is a Zen term and I'm unfamiliar with Zen; I don't want to use a term which might imply that I was going with whatever theoretical assumptions Zen might have, so I will just talk about “my experience” when it comes up.)

So here is my attempt to give an explanation. I don’t know if I’ve succeeded, but here goes anyway.

----

One of my key concepts is going to be cognitive fusion.

Cognitive fusion is a term from

Acceptance and Commitment Therapy

(ACT), which refers to a person “fusing together” with the content of a thought or emotion, so that the content is experienced as an objective fact about the world rather than as a mental construct. The most obvious example of this might be if you get really upset with someone else and become convinced that something was all their fault (...
Murphy’s Quest Ch 8: False Pentachotomy
by alkjash — 2018-03-08T05:30:00.786Z

There’s a powerful spell employed by the leaders of my old world called the Dichotomy. It goes something like this:

Pick two unfortunate phenomena.

Make an Either/Or statement as if the two phenomena are mutually exclusive.

Famous examples include:

“Either the One Percent are greedy bastards with way too much money Or funneling money to the poor has no net effect on poverty.”

“Either America spends too much money on healthcare Or due to lack of proper healthcare, thousands of babies die every year of easily preventable causes.”

“Either systemic racism throws an unfair number of minorities in prison Or minorities commit more crime than the average citizen.”

The Principle of Dichotomy is:

Although bad things happen, two bad things cannot happen at once.

By this Principle, the Dichotomy spell forces reality to pick a side, reducing the number of tragedies in the world by one.

Earlier tonight, I laid out five mutually exclusive ways I might break out of jail.

Unfortunately, the Principle of Dichotomy doesn’t hold in this world.

—

(A) Plun rallies a rag-tag but loveable rescue squad to break me out of jail.

I tiptoe down hall after hall, looking for the way out of the inner cloister.

There’s a commotion in the front courtyard.

Are they onto me?

The clash of steel. Spells being cast. Someone’s fighting.

My curiosity takes over and I sneak towards the noise.

Four hooded figures stand in the center of the courtyard, surrounded by twenty-odd priests and initiates in various states of undress. The fight is going about as well for the four as one can expect against a team of healbots.

“We need to retreat! Rogue, cast smoke bomb!”

It’s Plun’s voice!

“I can’t, I’m silenced!”

I identify two of the other intruders as Tess and Vant, but the last – a hooded shadow the size of a five-year-old – I don’t recognize.

The voice of a young girl emanates from the cloak, casting powerful Dark magic.

Soon, the clergy knock out Plun, Tess, and Vant, leaving only the child shadow, who makes a valiant effort. From her miniature staff she casts a continuous torrent of Shadow Bolts.

Footsteps ring in the corridor behind me.

—-

(B) Demons invade Beltine and I’m freed in the ensuing chaos.

“Finally, your majesssty, after yearsss of ssstudy, I have completed the teleportasssion devissse.”

Evil laughter emanates from behind the obsidian throne.

“If I may, perhapsss we might tessst it out firss...
Ideological Turing Test: Playtest report
by Joshua_Fox — 2018-03-07T18:22:17.471Z

We enjoyed an Ideological

Ideological Turing Test

game at LessWrong Tel Aviv and then again at LessWrong Jerusalem in February.

ITT has been played very rarely before. The Columbus Rationality group once did it. Their approach was to vote on the actual stance of authors of essays written by non-group members, gathered before the meeting.

We refined the game as we played it in Tel Aviv. If you play it, please let me know how you refined the rules.

Our rules:

A list of topics is written on the board. People volunteer to go up and speak.
They can choose from the board or find their own issues.
They speak 90 seconds pro, 90 seconds con.
In each case they take 20 seconds to prep and to avoid tipping their hand as to which they prefer.
Count votes from the audience:

Was the speaker convincing for pro? for con?
What is the speaker's actual stance?

The author then reveals their actual stance.

Example topics:

Veganism
Drugs legalization
Han Unification
Post-Modernism
Homeschooling
Scientology
AI Foom

Improvements:

1. Gamification:

To get a true game, you might give points for those who fool the audience about their actual stance; or you could award points, ice-skating style, for good form.

2. Convincing vs role-playing

Are speakers trying to be convincing, or to accurately simulate the position of someone who believes in the position?

Most LWers, with their rationalist bent, prefer the former. For this, you will need well-reasoned arguments.

The latter is the original ITT, as described by Bryan Caplan. I prefer this version: I want to learn the Dark Arts.

For example, if you were arguing pro post-modernism, to be convincing you would explain, like

Scott Alexander

, that agendas shape understanding of truth. To simulate a true believer, you would mix in some babble. But sound sincere. Poe’s Law aside, if you go overboard, you will fool no one!

3. Knowing the speaker’s opinions

A topic where the audience knows the speaker’s position, say, Scientology, makes a fun challenge for the speaker, but it becomes impossible to gamify the guess about the speaker's position.

On the other hand, if you do not know the speaker’s position, you can have fun guessing, but such topics are usually less controversial and so less interesting.
How to Dissolve It
by TurnTrout — 2018-03-07T06:19:22.923Z

In the last month and a half, I've had more (of what I believe to be) profound, creative insights to technical problems than in the five years prior. For example, I independently came up with the core insight behind DenseNets during the second lecture on convolutional neural nets in my Deep Learning class. I've noticed that these insights occur as byproducts of good processes and having a conducive mindset and physiology at that point in time. I'm going to focus on the former in this post.

End in Mind

Often, when I'm confused about a technical problem, I realize that I don't even know what a solution would look like. Sure, I could verify a solution if one were handed to me on a platter, but in these situations, I generally lack a detailed mental model of how I'd want a solution to behave. This may sound rather obvious once brought to your attention, but I'd like to emphasize how easy it is to spend hours in murky, muddled, pattern-matching gradient descent, mindlessly tweaking the first ideas that came to you.

Incredibly, generating a detailed solution model is often enough to quickly resolve (or at least reduce) even formidable-seeming problems.

Backward Chaining

The aforementioned process is backward chaining1 (as opposed to forward chaining, which is what you're doing with the work-from-what-I-have-now approach). Take the time to actually visualize what a solution to the problem would look like. How would it behave in response to different inputs? Are you offloading all of the work onto a weasel word (like emergence)?

When you're done, map-territory confusions should be ironed out, it should be clear whether current approaches truly tackle the core issue or just something similar-looking, and black-box terms should be disintegrated. If you can't do this, you probably can't solve the problem yet - you should spend more time working on your understanding. I encourage you to read this multiple times - the bulk of this post's value resides in this section.

For example, yesterday I thought about a smaller problem for about 15 minutes, and had nothing to show for it. I almost resigned to asking a more mathematically-experienced friend if he had any ideas, but instead forced myself to go through the above process. The answer was immediately obvious.

In my experience, proper solution formulations are accompan... ,,,
Murphy’s Quest Ch 7: Outside the Box
by alkjash — 2018-03-07T05:50:00.807Z

“I need to pee.”

…

“I need to pee.”

…

“I really need to pee!”

Father Penitence prays louder to drown out my requests, “Goddess above, lead this forsaken soul back to the fold.”

“Father, I really need to pee!”

He replies through gritted teeth, “You went ten minutes ago.”

“I’m sorry, I drink water obsessively when I’m bored!”

Father Penitence heads upstairs, leaving me alone in the darkness.

“Hey wait! I really need to pee!”

A half hour later, he brings me a chamber pot.

—

The number on my wrist has been decreasing steadily since The Incident. As far as I can tell, it ticks down by one every minute.

FTH: -936

There it goes. It’s pretty useful for keeping time down here.

I wonder when it’ll stop.

FTH: -937

I wonder if it’ll ever stop.

—

Nobody ever talks about how boring jail is. Thankfully, I have Father Penitence to keep me company. He’s not much of a conversationalist, but he has a way with words.

“Dear Goddess, grant this wayward youth the wisdom to repent.”

I’ve never heard one man express the same sentiment in so many different ways.

“Mother in Heaven, may your infinite grace …”

FTH: -2144

—

When you hit rock bottom, there’s nowhere to go but up.

My strength and motivation are returning to me. A boy my age can only mope around for so long. The way I see it, there’s only three ways this can go:

(A) Plun rallies a rag-tag but loveable rescue squad to break me out of jail.

(B) Demons successfully invade Beltine and I’m freed in the ensuing chaos.

(C) I’m hauled in front of an impartial jury of holy folk and convince them to set me free with an impassioned speech.

“Father, may I have pen and paper?”

I might as well prepare my speech for option C.

Respected Elders, let me pose one simple question. The word ‘sin’ derives from the Greek word ‘hamartia,’ which means ‘to miss the mark.’ I stand before you to beg forgiveness for my sins. Yet what is sin but the unfailing sign of aiming high? Should a man be punished for missing the mark because in his devotion to the Goddess he sets his sights too high? My fate is …

FTH: -4723

—

Another day, another catastrophe.

Last night I knocked over the chamber pot whilst getting up to pee. The shards of ceramic on the ground bring to mind two options that I missed:

(D) Hide a shard, kill Father Penitence as he approaches and steal his key.

(S) (for Shawshank.) Dig for twenty years.

I try to block them out of my mind, but the damage has been done. By noticing these...
Murphy’s Quest Ch 6: Perverse Incentives
by alkjash — 2018-03-07T03:50:01.374Z

How did I end up in this position?

Let’s review the facts.

First.

From Class Choice to ROT13, from Harry Potter villains to this chastity cult, from XP sharing to judgy old men, this world is out to get me.

Second.

I have despaired of living the Good Life and decided to take revenge.

Third.

There is a little-known law of biology which states that the male gaze, rapacious as it normally is, doubles in rapaciousness for each day ungratified. By my count, this makes me 32 times as susceptible to the female form as the average 16 year old male.

Fourth.

The only remotely feminine thing in this church is a life-sized Goddess statue in the chapel.

It is for these reasons that I find myself, just before the stroke of midnight on the 17th of February of the 2014th year of our Lord and Savior Jesus Christ, standing half-naked on the preacher’s podium in said chapel, facing said statue, relieving said ungratified urge.
Person-moment affecting views
by KatjaGrace — 2018-03-07T02:30:00.392Z

[Epistemic status: sloppy thoughts not informed by the literature. Hoping actual population ethicists might show up and correct me or point me to whoever has already thought about something like this better.]

Person-affecting views say that when you are summing up the value in different possible worlds, you should ignore people who only exist in one of those worlds. This is based on something like the following intuitions:

World A can only be better than world B insofar as it is better for someone.
World A can’t be better than world B for Alice, if Alice exists in world A but not world B.

The further-fact view says that after learning all physical facts about Alice and Alice’—such as whether Alice’ was the physical result of Alice waiting for five seconds, or is a brain upload of Alice, or is what came out of a replicating machine on Mars after Alice walked in on Earth, or remembers being Alice—there is still a further meaningful question of whether Alice and Alice’ are the same person.

I take the further-fact view to be wrong (or at least Derek Parfit does, and I think we agree the differences between Derek Parfit and I have been overstated). Thinking that the further-fact view is wrong seems to be a common position among intellectuals (e.g. 87% among philosophers).

If the further-fact view is wrong, then the what we have is a whole lot of different person-moments, with various relationships to one another, which for pragmatic reasons we like to group into clusters called ‘people’. There are different ways we could define the people, and no real answer to which definition is right. This works out pretty well in our world, but you can imagine other worlds (or futures of our world) where the clusters are much more ambiguous, and different definitions of ‘person’ make a big difference, or where the concept is not actually useful.

Person-affecting views seem to make pretty central use of the concept ‘person’. If we don’t accept the further-fact view, and do want to accept a person-affecting view, what would that mean? I can think of several options:

How good different worlds are depends strongly on which definition of ‘person’ you choose (which person moments you choose to cluster together), but this is a somewhat arbitrary pragmatic choice
There is some correct definition of ‘person’ for the purpose of ethics (i.e. there is some relation between person moments that mak

...
[defunct] Moderation Action List (warnings and bans)
by Ben Pace — 2018-03-06T19:18:44.226Z

Update: This hasn't been updated since 2021, and we're no longer updating it.

This post is a (continually updated) list of moderation bans and warnings of LWers, with a link to the comment thread where they happened.

(Added: Note that the vast majority of bans are for brand new users who are norm-violating in their first comments or otherwise seem like bad user growth for their site, and they are not listed here. This list is for users who have had a fair bit of activity on the site.)

Dr. Jamchie is banned for three months for reasons.
Tempus is banned for being Eugine Nier.
diegocaleiro is banned for reasons.
PDV has had a public and private warning.
Steve Whetstone is banned for spamming lots of really long, crackpot-looking comments
Said Achmiz has had a public warning.
Elo has had a public warning.
ialdaboath is banned for reasons.
Gleb_Tsipursky is banned for a long history of scammy behaviour, including evidence that he paid MTurkers to upvote his LW comments (discussed in this thread, which offers this recorded evidence).
Curi is banned for reasons
Periergo is banned for reasons
IlyaShpitser has had a public warning.