trentmkelly/LessWrong-43k huggingface dataset:
Is there a definitive intro to punishing non-punishers?
by pjeby — 2019-10-31T20:20:30.653Z

Hey LW. I've just written a draft of an article, that I'm trying to edit to link to useful prior notes on various concepts. One of those concepts is the idea that we evolved to "punish the non-punishers", in order to ensure the costs of social punishment are shared by everyone.

The thing is, I vividly remember reading an article from the rationalsphere that introduced me to this idea many years ago, but I cannot seem to find it anywhere. The idea is mentioned many places where I could have seen the original article, including on Overcoming Bias, Slate Star Codex, and Less Wrong, but all my searches seem to find passing mentions in comments, or throwaway mentions in articles assuming you already know what it means.

Since my article will otherwise continue this apparent tradition of assuming everyone knows what it means and not linking to an introduction, can anyone point me to the article I'm recalling, or perhaps suggest another good non-technical introduction to the idea (i.e. not this paper), so readers of the article I'm writing who aren't yet familiar with the idea can learn about it? Thanks!
Rohin Shah on reasons for AI optimism
by abergal — 2019-10-31T12:10:02.018Z

Rohin Shah

I along with several AI Impacts researchers recently talked to talked to Rohin Shah about why he is relatively optimistic about AI systems being developed safely. Rohin Shah is a 5th year PhD student at the Center for Human-Compatible AI (CHAI) at Berkeley, and a prominent member of the Effective Altruism community.

Rohin reported an unusually large (90%) chance that AI systems will be safe without additional intervention. His optimism was largely based on his belief that AI development will be relatively gradual and AI researchers will correct safety issues that come up.

He reported two other beliefs that I found unusual: He thinks that as AI systems get more powerful, they will actually become more interpretable because they will use features that humans also tend to use. He also said that intuitions from AI/ML make him skeptical of claims that evolution baked a lot into the human brain, and he thinks there’s a ~50% chance that we will get AGI within two decades via a broad training process that mimics the way human babies learn.

A full transcript of our conversation, lightly edited for concision and clarity, can be found here.

By Asya Bergal
Welcome to Rationality [Edit With Your Details]
by sairjy — 2019-10-31T11:04:03.352Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Halloween
by KatjaGrace — 2019-10-31T00:20:01.525Z

Leonard Cohen on his poetic voice:

“As I grew older, I understood that instructions came with this voice. What were these instructions? The instructions were never to lament casually. And if one is to express the great inevitable defeat that awaits us all, it must be done within the strict confines of dignity and beauty.”

This is apparently exactly the opposite of how most people in the American world feel at this time of year. The great inevitable defeat should be expressed with electronically jeering plastic skeletons, humorous fake corpses, faux-threatening gravestones, and—let’s not forget the violence and disease responsible for so many deaths—generous fake blood, vomit, and slime. The celebration should be not only frivolous and ugly to the max, but really go hard on minimizing dignity! Don’t just dress as a fun ugly corpse, make it a sexually depraved fun ugly corpse!

Isn’t this just a bit weird? For instance, how is this good?

I’ve heard a couple of times something like: “Halloween is nice because it is rebellious, and a relief from all that seriousness. People usually treat these things with sacredness and somberness, and it’s a lot to deal with.”

If that’s what it is, would it also be cool if we incorporated more second-rate stand up comedy routines and pranks into funerals? Or had more fun smallpox themed parties?

I don’t think people do actually contend with death or horror in a more comfortable way via Halloween, for the most part. My guess is that they basically don’t think about the content of what they are doing, and just get used to hunting for bargain plastic corpse babies to put on their lawns, and laughing with each other about how realistic and gruesome they are. Which seems more like desensitization than coming to terms with a thing. Which I doubt is a good kind of relief from seriousness.

Also, if we are going to have a change of mood break around the potential horrors of life, the alternate feelings and purposes Halloween suggests just don’t seem very good. ‘Trick or treat?’ Cheap malice or cheap hedonism? Quick, find a huge mound of junk food, I forgot because I’m very competitive about drunkenly parading my buttocks in a way that makes me seem clever..

I’m obviously missing something, and I don’t actually blame anyone much within reason for getting into the holidays of their culture, but come on culture, what is this?
[AN #71]: Avoiding reward tampering through current-RF optimization
by Rohin Shah — 2019-10-30T17:10:02.211Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

Designing agent incentives to avoid reward tampering

(Tom Everitt, Ramana Kumar, and Marcus Hutter) (summarized by Flo): Reward tampering occurs when a reinforcement learning agent actively changes its reward function. The post uses

Causal Influence Diagrams

(

AN #61

) to analyze the problem in a simple grid world where an agent can easily change the definition of its reward. The proposed solution is current-RF optimization: Instead of maximizing the sum of rewards that would be given after each action (where the reward signal can dynamically change over time), the agent searches for and executes a plan of actions that would maximize the current, unchanged reward signal. The agent would then not be incentivized to tamper with the reward function since the current reward is not maximized by such tampering. There are two different flavours to this: time-inconsistency-aware agents account for future changes in their own behaviour due to modified reward signals, while TI-unaware agents ignore this in their planning. TI-aware agents have an incentive to preserve their reward signal and are therefore potentially incorrigible.

Flo's opinion: I enjoyed this application of causal diagrams and think that similar detailed analyses of the interactions between failure modes like wireheading, instrumental goals like reward preservation and the specific implementation of an agent would be quite valuable. That said, I am less excited about the feasibility of the proposed solution since it seems to require detailed knowledge of the agent about counterfactual rewards. Also, I expect the distinction between changes in the reward signal and changes in the state that happen to also affect the reward to be very fuzzy in real problems and current-RF optimization seems to require a very sharp boundary.

Rohin's opinion: I agree with Flo's opinion above, and I think the example in the blog post shows how the concept "what affects the reward" is fuzzy: in their gridworld inspired by the game Baba Is You, they say that moving the word "Reward" down to make ...
Rugby & Regression Towards the Mean
by Bucky — 2019-10-30T16:36:00.287Z

With the rugby world cup final approaching this Saturday I figured I'd publish this which I wrote towards the end of last year but never submitted.

Introduction

A couple of weeks ago England played New Zealand in Rugby Union. For those who don’t follow the sport New Zealand (the All Blacks) are comfortably the best team in the world. They have been top of the world rankings since 2009. They win 92% of their matches. With a population of 4.8m, it’s fair to say that New Zealand’s national rugby affiliation must be off the charts.

At half time England were 15-10 up. An interesting statistic was rolled out. Since their World Cup win in 2011 they have only lost 4 out of 20 of the matches in which they were behind at half time. At the end of the match the All Blacks had turned it around again, winning 16-15, making it 4/21 losses.

This sounds very impressive but I wondered how much of this effect could be put down to regression towards the mean. After all, a team that wins 92% of their matches might well be good enough to win 81% of matches when they are losing at half time without postulating an extra performance boost in such circumstances.

I thought it would be fun to investigate.

Method

I gathered all of their results and corresponding half time scores since the 2011 World Cup. I only included data from teams who could realistically win a half against the All Blacks in order not to bias the results with high scores against weaker teams. I included Argentina, Australia, England, France, Ireland, Scotland, South Africa and Wales. This gave 85 results.

If anything, this will be biased towards having more wins when losing at half time. Losing at half time would imply that the opposition is more likely to be towards the top end of performance. However the predicted second half performance is based on all matches involving the relevant teams. So the calculation isn’t perfect but I couldn’t think of a better way to do it without reducing my dataset which would reduce experimental power.

The dataset isn’t big enough to allow for lowering my p-value threshold much by multiple hypothesis testing. I have two formal p-values:

Do New Zealand generally perform better in the second half than the first?

Do New Zealand perform better than normal in the second half when they are losing at half time?

I’m looking for p<0.025 for these 2. I calculate other p-values they are labell...
A new kind of Hermeneutics
by aiiixiii — 2019-10-30T12:42:59.274Z

Hermeneutics [həːmɪˈnjuːtɪks]
-The branch of knowledge that deals with interpretation, especially of the Bible or literary texts-

Oxford dictionary

The limits of my language mean the limits of my world.

Ludwig Wittgenstein

Gottfried Wilhelm von Leibniz, (probably) the greatest mathematicians and natural philosopher who ever walked fifteenth century continental Europe, dreamed of an all-embracing formal language, able to express the highest concepts that the human mind can explore: science, mathematics, metaphysics. He called this vocabulary characteristica universalis. He was also a declared rationalist, although the definition of "rationalist" was a little different from the modern one. Leibniz never went so far into the technical details of building such a language, to the point that many historians suspect that it never went beyond a simple unattainable aspiration, one of those cases in which the idea turns out to be bigger than the man who gave birth to it. On the other hand, Kurt Gödel was among the heavyweights who were convinced that Leibniz's dream was not just a feverish delirium. Yes, the same Kurt Gödel of the incompleteness theorem. I have no intention of expressing whether characteristica universalis can be possible or not (although a conversation i had with a linguist persuaded me that it is practically infeasible), I prefer to focus on the importance that formal languages ​​have had in the transmutation of human society.

Scholars define prehistory as the set of events that occurred before the birth of written language but why is written language so important to the point of being the fundamental marker of our maturity? The gravity of this innovation lies in the fact that it allowed us to note dates, witnesses, thoughts and this facilitated the work of posterity in reconstructing what came before, what generated the world in which they live, in which we live. Writing has allowed historians the luxury of looking into the past simply by observing words, turning them into scholars of texts. Texts are powerful tools, through them it is possible to share experiences, secret informations, knowledge and cooking recipes. They are also fundamental for the perpetuation of religious cults. Imagine yourself thousands of years in the past, in the hot and muggy Indian subcon...
The Technique Taboo
by lsusr — 2019-10-30T11:22:47.184Z

For a strange few decades that may just be starting to end, if you went to art school you'd be ostracised by your teachers for trying to draw good representational art. "Representational art" means pictures that look like real things. Art school actively discouraged students from getting better at drawing.

"Getting better at drawing" is off-topic at my weekly local drawing club too. I've literally never heard it discussed.

This taboo extends far beyond art. My nearest gym forbids weightlifters from using electronic systems to log their progress. I'm friends with programmers who can't touch type. None of them use Vim macros.

"I have sometimes suspected that the quickest way to get worried looks from many modern Western meditation teachers is to talk about practice in a way that implies the attempt to actually master anything." — Daniel M. Ingram

In the part of the United States where I live, the subject of skill is often taboo. Not just relative differences in skill level between specific present individuals (which would make sense). The implicit acknowledgement of skill as a trainable attribute is taboo.

Not all professions have this issue. Math is still math. Biology is still biology. One can politely discuss a cook's cooking. Magicians respect coin manipulation like it's 1904.

But when traditional colleges supply the labor force for a professional trade outside of academia, that's when discussion of skill (especially rote learning) becomes taboo[1]. College students learn everything about their trade except how to do it. Then we maintain a collective silence concerning technique.

A Chinese major teaches you how to talk about Chinese, not how to read it.

An English major teaches you how to talk about novels, not how to write one.

An art major teaches you how to talk about masterpieces, not how to create one.

A Computer Science Engineering major…well, you get the idea.

That's a partial explanation, but it doesn't explain why skill differences in weightlifting and meditation are also taboo.

Societies make taboo exactly those topics whose mere discussion threatens the precarious dominance of those at the top of the social order by drawing attention to the system's internal contradictions.

I think my society is hiding something from itself.

Medical school is an exception to this pattern. This may be because medical school considers itself a form of technical t

...
Theater Tickets, Sleeping Pills, and the Idiosyncrasies of Delegated Risk Management
by Davidmanheim — 2019-10-30T10:33:16.240Z

Risk management is difficult, but even when it’s easy, companies and policymakers often do something other than optimal risk mitigation. This isn’t puzzling, once we realize that the incentives in place give the decisionmakers the leeway, or even positive incentives, to behave sub-optimally. There are three types that seem most relevant, along with a few (anonymized) stories from when I was working in reinsurance of how they play out in practice.

Sleeping Pills

Occasionally, a small insurance company would purchase reinsurance for things that didn’t make business sense for their company. I might have seen a home insurance company in Ohio that had fifty million dollars in reserve, then would buy reinsurance for hurricanes that covered all losses greater than ten and less than twenty five million dollars. Yes, there are hurricanes that impact Ohio, such as Xenia in 1974 and Ike in 2008, but they weren’t large enough to even hit the minimum for this type of policy. Not only that, but the company had money in reserves to cover this incredibly unlikely loss, and buying reinsurance isn't cheap. Let’s say that between brokers, transaction costs, and everything else, it cost them a hundred thousand dollars to cover an expected loss of ten thousand dollars.

Noticing this, I asked why they wanted this policy. My boss told me it was a sleeping pill. He explained that the CEO of the insurance company would get really nervous and unhappy every time a hurricane was approaching the US, and decided this CEO didn’t want to worry anymore. That isn’t unreasonable - most people who buy travel insurance to cover their $1,000 vacation could just accept the risk, but they prefer not to worry.

In general, buying risk mitigation isn’t worth the cost in expected value terms, but they are worthwhile because they can buy off the worry. In this case, it’s less innocuous, since the CEO was using company money to buy what amounted to personal sleeping pills. It happens because the CEO won’t ever be blamed for hedging the risk, and it cost the company a hundred thousand dollars per year, which was not enough for shareholders to notice.

Theater Tickets

Once, we received a request from a client to train them in using the terrorism risk model they licensed. That’s not unusual, but they said that they wanted us to fi...
Prospecting for Conceptual Holes
by lsusr — 2019-10-30T08:34:52.769Z

Imagine you wanted to explore the number line. You start at 1 and then you explore 2, 3, 4 and so on. No matter how long you do this you'll never discover −1, 12, 2π, ∞, i, or a single noncomputable number. That's because the most interesting numbers are precisely those located in directions conceptually orthogonal to whatever numbers you've already explored.

The easiest concepts to learn are the ideas ahead of you on a path you're already taking. If you know classical mechanics then it's straightforward to learn relativity. You could crack turbulence without ever changing fields.

The second-easiest things to learn are those outside your specialization but inside of your culture's intellectual tradition. If you grew up in the West then you know there are such things as physics and painting even if you never learned how to do them.

The second-hardest things to learn are those outside your society's intellectual tradition. Consider for example kenshō[1]. It's possible to live an entire lifetime outside of Japan without ever discovering that such a concept exists. Westerners don't choose not to experience kenshō. This choice is made invisibly.

The hardest things to learn are those belonging to domains of knowledge which haven't been (openly) conceptualized anywhere, such as photography before the invention of the camera. This category includes zero-day exploits.

We can number each kind of conceptual hole from least to most orthogonal.

Type 1: Concepts you are aware of but do not understand.

Type 2: Concepts you are not aware of, but which belong to a field you are aware of.

Type 3: Fields of knowledge you do not know exist, but which exist.

Type 4: Fields of knowledge that remain genuinely secret or have not been invented.

Expertise is about mastering one field, but general intelligence is how prepared you are for something new. The more different kinds of concepts you understand the better you will be at solving new kinds of problems. This is behind the principle of learning that breadth of knowledge equals breadth of transference.

Filling holes of Type 1 is how you build expertise in a field. Holes of Types 2-4 are better for building broader (and therefore more transferable) knowledge. In other words, holes of greater-numbered types are better for increasing your general intelligence. If this is your goal then holes of Type 2 and 3 are the most valuable.

I find h...
Learning from other people's experiences/mistakes
by alkay — 2019-10-30T00:43:08.326Z

One of the fastest way to learn is to learn from someone else's mistakes and experiences. This short-cuts a lot of unnecessary trial and error and can save significant time. However, one is sorely tempted to repeat the experiences/mistakes of others. One may think, that they are smarter/luckier than the others who made those mistakes. One may not trust that the right lessons were learnt by others in their experiences. One may think there is loss of agency in just following along a path someone else prescribed. What are some guidelines do you use to learn from others experiences? How do you judge their lessons are worth following? How do you stop yourself from attempting to make those mistakes yourself.
When do you start looking for a Boston apartment?
by jefftk — 2019-10-29T16:30:01.716Z

A huge fraction of Boston-area apartments are on a September 1st to
September 1st lease cycle. This is driven by college student rentals,
but the whole city has ended up on the same pattern. So you might
expect to see the most apartment listings in August or maybe July?
Nope:

The peak is in late spring, May-Jun, about a hundred days out. It's
not as dramatic as what I saw with one year's data in 2013, but people are
planning way ahead. I think this might be driven by students lining
up housing before they leave for the summer?

This matches my experience looking for apartments and lining up
housemates: if you have strong preferences or want an especially good
fit you need to start looking in the late Spring.
Fibromyalgia, Pain & Depression.  How much is due to physical misalignment?
by leggi — 2019-10-29T14:58:17.652Z

I used to think the human 'machine' was a poor design.

I was wrong.

It was poor usage that had made life so painful.

This is my explanation for chronic pain and the many pain-related syndromes currently classified as idiopathic (the cause is unknown) i.e. fibromyalgia, restless legs, chondritis, plantar fasciitis, shin splints, many "IBS" issues, frozen shoulder, neck cricks, tension headaches etc. A long list of symptoms and conditions that plague the modern world.

This is based on:

My recovery from nearly 20 years of severe depression and a lifetime of pain.
Anatomical facts. Logic starts here: midline anatomy & the median plane.
My education and experience in handling mammalian tissues as a veterinary surgeon (surgery, autopsies, butchery).

Key Points:

Myalgia of physical imbalance. If you are not using the "right muscles" you are using the wrong muscles. I did not use my main muscles of movement. I had no connection to my Base-Line muscles - the body's core pillar of strength. Without a connection to my Base-Line I had no inner reference to reset back to 'baseline healthy' and a body that is physically balanced and aligned.

Physical restrictions form in our body-wide web of connective tissues in a response to inflammation (triggered by injury, infection etc.) which induces cross-linking of collagen fibers. Some easy notes here: Connective Tissue Response.

Restrictions reduce range of movement adding to the body's imbalance and misalignment.
Physical restrictions apply tensions throughout the body, along "threads" that run from head to fingers to toes.
These tensions generate pain & weird sensations. The body adjusts to avoid pain, adding to the misalignment.
This physical restrictions are 'stored trauma', creating an 'individual trauma imprint' and with it a unique collection of symptoms for each person. Generalised patterns exist, with grouping of symptoms along affected threads.

Physical imbalance in the usage of the muscular system and restricted connective tissues cause:

Unexplained pain. (often diagnosed as fibromyalgia these days). Pain related symptoms that increase in number and intensity over time.
Self-doubt. Stress. Hypochondria.
Anxiety. Depression.

My Story.

I was too young to voice my pain when it started and grew up thinking 'stiff and sore' was normal - that the ...