trentmkelly/LessWrong-43k huggingface dataset:
Nonlinear perception of happiness
by Jan_Kulveit — 2018-01-08T09:04:15.314Z

Epistemic status: Speculative.

tl;dr: Perception of hapiness is related to some "raw" happiness by an equivalent of a psychophysics law. The "raw" quantity should be used when aggregating. Far-reaching implications for utility calculation would follow.

A body of research seeks to understand happiness and measure it quantitatively. Often the measurements use tools such as Oxford Happiness Inventory, Subjective Happiness Scale, Panas Scale, etc. What these instruments have in common is they measure a perception, and the scales used are linear.

A proposal: let's make a distinction between the perception of happiness, which is measured in this way, and a hypothetical raw happiness. While we cannot measure such quantity in practice, we can at least imagine how it would be measured in a thought experiment – e.g., by an outside observer who has complete access to the mental states of beings, and has some algorithmic way how to determine happiness of mental states.

Given this distinction, we may then ask, how would a human perception of happiness be related to such raw quantity?

Conjecture: Human perception of happiness has a nonlinear form, that is, with a linear increase of the raw happiness there is nonlinear increase of the perceived happiness.

One candidate for the relation can be the widely known psychophysics law, Weber–Fechner law, stating the subjective sensations are proportional to the logarithm of the stimuli intensity. This models light intensity and the perceived difference in weight. It was also proposed the logarithmic perception applies to more indirect senses, e.g. sense of time intervals. It seems plausible it would hold for the perception of quantities like wealth: if we measure the perception of wealth by asking people to rate their wealth on a scale from 0 to 10, we would get log(monetary value).

Now - what if this holds also for the sense of happiness, as used in philosophy and utilitarian calculus? Specifically, we may propose the percieved happiness pH related to the raw hapiness H as

pH=k.loga(H)

where k is an unknown proportionality constant.
(While I chose happiness, the argument would be the same for related or similar quantities, such as well-being.)

Implications

It may seem such logarithmic rescaling is just an irrelevant change of scale. However, when we aggregate a quantity over many people, there are significant differences between u...
In Defence of Meta
by Chris_Leong — 2018-01-08T00:43:40.399Z

The current mods of the Less Wrong community seem to believe that meta (I'm using meta in a broad sense to mean meta + community) discussion and community discussion are both things that need to be heavily discouraged. In part, we can see this inspired by Elizier's post Less Meta and a general tendency for the regular inside internet communities to gradually end up spend more and more of their time in meta, instead of in the site itself. It's generally considered rude to unilaterally end a discussion, so it's not easy for someone to say, "Everyone, everyone, this issue really isn't that important, let's just move onto things that are more productive". This is definitely a dynamic to be aware of.

However, Less Meta came at the end of entire sequence of posts about how he thought that the community should develop. Elizier wasn't denying the value of meta discussion and, as I am arguing, neither should we. Instead, I think we should be asking two questions:

Is our meta discussion important?
Is our meta discussion productive?

1 asks whether resolving the question would significantly benefit the community. 2 asks whether further discussion is likely to push the community forward on that issue. If the community seems unable to make progress on a particular topic, it may be best wait in the hope that the community changes and resolution becomes possible in the future. If we let this guide us, then I believe we can find the middle path that ensures necessary discussion occurs without it becoming a time sink.

Three reasons why meta discussion is important at this stage:

Meaningful meta discussion is now possible again for the first time in a number of years. I seem to have been getting the impression that for a while, online meta discussion was sub-ordinate to the in person discussions happening in the bay area. Many key people within the community stopped following LW. Discussions on LW had no impact because the key players weren't on LW. Now that LW is actually being used again, discussions which were unproductive before, may be productive now. In fact, there's likely a bit of a backlog that has developed of important topics we need to discuss as a community.
A lot has happened in the community since Elizier wrote the Craft and the Community. It seems like we might need a new vision for our community and understanding

...
Scientist Fiction
by Jacob Falkovich — 2018-01-07T19:50:00.366Z

Rabbit fiction is always about rabbits, but science fiction isn’t always about science. The best science fiction, however, usually is. Science fiction transports the reader to strange and novel worlds; figuring out how those worlds work is often as important to the story as resolving the plot. And the best approach we have to figuring out how a world works is, in fact, science.

Harry Potter is a pure fantasy book, but Harry Potter and the Methods of Rationality is very much science fiction. The protagonist of the latter desperately applies the scientific method to understanding Hogwarts, rather than calmly accepting that reality is broken and stuffing his face full of ginger newts.

It’s an interesting exercise to categorize science fiction based on the science at the heart of each story. This can be anything from microeconomics, to theoretical computer science, to political philosophy. The two books I want to discuss today: Liu Cixin’s The Three-Body Problem goes with the classic combination of math and astrophysics, and Jeff Vandermeer’s Annihilation is a rare example of ecology fiction.

Both books aren’t leaving the reader to do the science all by themselves, they each are centered around scientist main characters. Three-Body Problem focuses on two Chinese physicists: astrophysicist Ye Wenjie and Wang Miao, a nanotechnology researcher. The main character of Annihilation is “the biologist”, because, well:

There were four of us: a biologist, an anthropologist, a surveyor and a psychologist. I was the biologist. All of us were women this time, chosen as part of the complex set of variables that governed sending the expeditions.
[…]
I would tell you the names of the other three, if it mattered, but only the surveyor would last more than the next day or two.
– Annihilation, page 4.

However, the two books have radically different takes on a core question of what makes a scientist? One of the books’ answers I found deeply unsatisfying, while the other created a memorable character that may have altered my own life more than any work of fiction ever has.

Continue reading on Putanumonit.com
Argument As Mental Arm-Wrestling: Who Has the Best BS?
by Bound_up — 2018-01-07T18:55:02.130Z

Once upon a time, I delivered devestatingly precise deconstructions of points, complete with on-the-spot productions of analogies and the contstruction of crystal-clear absract descriptions of the important principles of reasoning most relevant to the discussion at hand.

I wasn't one to try and aggrandize myself thusly; it's just that I was right, you see, and considered my correct understanding to be something worth sharing and also worth receiving. At the same time, I didn't want my interlocutor to feel bad or anything, so I tried to compensate for the fact that they were wrong with various social graces. I pointed out all the ways my interlocutor was right; I presented my rebuttal in an even, non-triumphant tone; I used the language of uncertainty as much as I could without becoming actively dishonest.

The expected result: My interlocutor traces the lines of my thoughts, notes that they are correct, and adopts them as their own, in great part because they feel respected and appreciate how politely their errors were revealed to them.

The real result: They thought they were right and I was wrong.

For years I banged my head on this brick wall by supposing that, strange though it seemed, I just...must not have been clear enough. Ever more precise, more devestating rebuttals did I prepare, ever so politely expressed, gifted to my interlocutor with a bow and ornamented in the wrapping paper of deference.

Right, so here's the thing. Most of the time, people weren't even listening to my explanations, not really. It didn't matter how clear the explanations were to them, any more than it matters how clearly you speak English to one who speaks no English. They're speaking another language, you see? And they thought I was speaking their language as well; they heard me speaking their language, garbled, weak, and pathetic as it was. That bit about being polite and using even tones? As far as they were concerned, that was the message. For me, it was just packaging; for them, the packaging was the message. The medium was the message; the presentation was the content.

And my "content" was weak. I thought I was balancing things out: strong points, weak presentation. But, since they weren't really listening to my points, all they heard was the weak presentation. In their language, this kind of presentation indicated my lack of con...
Demon Threads
by Raemon — 2018-01-07T03:48:09.310Z

tldr: a Demon Thread is a discussion where everything is subtly warping towards aggression and confusion (i.e. as if people are under demonic influence), even if people are well intentioned and on the same 'side.' You can see a demon thread coming in advance, but it's still hard to do anything about.

("Flame Wars" are similar but I felt the connotation was more like "everything has already gone to hell, and people aren't even pretending to be on the same side")

I kept wanting to reference this post when discussing internet discussion policy, and I kept forgetting that nobody has written it yet. So here it is.

Suggested Background Reading:

Politics is Hard Mode (Rob Bensinger)

Civility is Never Neutral

(Ozy)

Writing that Provokes Comments

(me)

Musings on Double Crux and Productive Disagreement

(me)

If someone in the future linked you to this post, it's probably because a giant sprawling mess of angry, confused comments is happening - or is about to happen - and it's going to waste a lot of time, make people upset, and probably less likely to listen to each other about whatever the conversation ostensibly is about.

I have some ideas on what to do instead, which I discuss in this followup post.

But for now, this post is meant to open a discussion, explore the mechanics of how demon threads work, and then in the comments brainstorm solutions about how to handle them.

Wrong On the Internet

I find “Someone Is Wrong On the Internet” to be a weird, specific feeling.

It's distinct from someone being factually wrong - people can be wrong, point it out, and hash out their disagreements without a problem. But a common pattern I’ve witnessed (and experienced) is to notice someone being wrong in a way that feels distinctly bad, like if you don’t correct them, something precious will get trampled over.

This is when people seem most prone to jump into the comments, and it’s when I think people should be most careful.

Sometimes there actually is an important thing at stake.

There usually isn’t.

It often feels like there is, because our social intuitions were honed for tribes of a hundred or two, instead of a world of 7 billion. We live in a different world now. If you actually want to have an impact on society, yelling at each other on the internet is almost certainly not the best way to do so.

When there actually is something important at stake, I think there are usually better plans than...
Roleplaying As Yourself
by orthonormal — 2018-01-06T06:48:03.510Z

(This is a basic intuition pump I've found helpful in making decisions, and maybe you'll like it too.)

For all its shortcomings, I think there was something quite useful about the "What Would Jesus Do?" meme within the Christian framework. Of course it's not a very sophisticated ethical guide, and it comes with all kinds of biases; but asking it does put the believer into a frame of mind that emphasizes things like compassion and duty, and it sometimes helps the believer generate options that weren't in their default solution space.

Is there a version of this handy tool for the consequentialist, with our muddled mixture of selfish and altruistic goals and impulses, and the added difficulty that we're looking to actually optimize rather hard?

The one that works best for me is a double roleplay.

Jernau Gurgeh, champion of strategic games across the galaxy, sits down to a nice futuristic immersive roleplaying game: The Orthonormal Experience. Gurgeh will be controlling a denizen of the early 21st century on Earth, someone with the online name of Orthonormal. Getting Orthonormal to do well by Orthonormal's own standards is Gurgeh's objective.

Just as our roleplaying games have game masters who can call out uncharacteristic plans, so too does Gurgeh's game. He can't simply use his superior vantage point to calculate the right stocks for Orthonormal to buy today and sell tomorrow, because Orthonormal couldn't do that except by luck. He can't even have Orthonormal think at peak performance on some days- there are character attributes (penalties like Anxiety Disorder) he has to play around.

But Gurgeh is able to think patiently, and strategically, about the various obstacles blocking Orthonormal's progress, and to guide Orthonormal's thoughts in plausible ways to work on these. There's a lot of points out there to be scored: better states to reach in Orthonormal's relationships, career, inner life, and more.

What would Gurgeh do?

One last note: a couple of the bugs with this approach can be confronted within the approach itself. If I decide that Gurgeh might do X, and I try X and fail, it can be tempting to get frustrated with myself. But this isn't what Gurgeh would do next! He'd take my failure as more data about what this character's current attributes are, and look for wa...
LW Update 1/5/2018 – Comment Styling
by Raemon — 2018-01-06T02:11:45.433Z

We've just pushed an update with a few stylistic tweaks, a few small bug fixes, and most notably, an overhaul on the comment styling.

The intent here was to make it easier to skim comments and keep track of branching threads. This is a bit of an experiment, and the current approach is that we're treating the comment section as conceptually different from the main post section.

This post has moderately deep comment trees that can showcase it in action.

Bugs that we could easily remember fixing:

KarmaThreshold for RSS feed should work now
Auto-styled links no longer lie to you
Meta posts show up in the Daily feed
Magical Sorting on the front page now favors Front Page posts and Curated posts slightly (with +10 and +20 effective karma,
Rationality: Abridged
by Quaerendo — 2018-01-06T00:20:04.919Z

This was originally planned for release around Christmas, but our old friend Mr. Planning Fallacy said no. The best time to plant an oak tree is twenty years ago; the second-best time is today.

I present to you: Rationality Abridged -- a 120-page nearly 50,000-word summary of "Rationality: From AI to Zombies". Yes, it's almost a short book. But it is also true that it's less than 1/10th the length of the original. That should give you some perspective on how massively long R:AZ actually is.

As I note in the Preface, part of what motivated me to write this was the fact that the existing summaries out there (like the ones on the LW Wiki, or the Whirlwind Tour) are too short, are incomplete (e.g. not summarizing the "interludes"), and lack illustrations or a glossary. As such, they are mainly useful for those who have already read the articles to quickly glance at what it was about so they could refresh their memory. My aim was to serve that same purpose, while also being somewhat more detailed/extensive and including more examples from the articles in the summaries, so that they could also be used by newcomers to the rationality community to understand the key points. Thus, it is essentially a heavily abridged version of R:AZ.

Here is the link to the document. It is a PDF file (size 2.80MB), although if someone wants to convert it to .epub or .mobi format and share it here, you're welcome to.

There is also a text copy at my brand new blog: perpetualcanon.blogspot.com/p/rationality.html

I hope you enjoy it.

(By the way, this is my first post. I've been lurking around for a while.)
Biased reward-learning in CIRL
by Stuart_Armstrong — 2018-01-05T18:12:27.570Z

In Cooperative Inverse Reinforcement Learning (CIRL), a human H and a robot R cooperate in order to best fullfil the human's preferences. This is modeled as a Markov game M=⟨S,{AH,AR},T(⋅|⋅,⋅,⋅),{Θ,R(⋅,⋅,⋅;⋅)},P0(⋅,⋅),γ⟩.

This setup is not as complicated as it seems. There is a set S of states, and in any state, the human and robot take simultaenous actions, chosen from AH and AR respectively. The transition function T takes this state and the two actions, and gives the probability of the next state. The γ is the discount factor of the reward.

What is this reward? Well, the idea is that the reward is parameterised by a θ∈Θ, which only the human sees. Then R takes this parameter, the state, and the actions of both parties, and computes a reward; this is R(s,aH,aR;θ) for a state s and actions aH and aR by the human and robot respectively. Note that the robot will never observe this reward, it will simply compute it. The P0 is a joint probability distribution over the initial state s0, and the θ that will be observed by the human.

Behaviour in a CIRL game is defined by a pair of policies (πH,πR), that determine the action selection for H and R respectively. Each agent gets to observe the past actions of the other agent, so in general these policies could be arbitrary functions of their observation histories: πH:[AH×AR×S]∗×θ→AH and πR:[AH×AR×S]∗→AR.

The optimal joint policy is the policy that maximises value, which is the expected sum of discounted rewards. This optimal is the best H and R can do if they coordinate perfectly before H observes θ. It turns out that there exist optimal policies that depend only on the current state and R's belief about θ.

Manipulation actions

My informal critique of CIRL is that it assume two untrue facts: that H knows θ (ie knows their own values) and that H is perfectly rational (or noisly rational in a specific way).

Since I've been developing more machinery in this area, I can now try and state this more formally.

Assume that M always starts in a fixed state s0, that the reward is always zero in this initial state (so R(s0,⋅,⋅;⋅)=0), and that transitions from this initial state are independent of the agent's actions (so T(s|s0,⋅,⋅) is defined indendently of the actions). This makes R's initial action aR0 irrelevant (since R has no private information to transmit).

Then let πH be the optimal policy for θ, and (πH... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Have you felt exiert yet?
by Stuart_Armstrong — 2018-01-05T17:03:51.029Z

Pre-adolescent children haven't felt strong lust yet. Those of us who've avoided strong pain are also missing an experience of that affect. Nostalgia can come up very early, but does require a bit of living first. Depression can strike at any age.

So, in general, there are emotions and feelings that people are capable of feeling, in the right circumstances, but that some people have not yet felt.

As a thought experiment, I'd thus like to introduce the emotion of exiert, which no human being has yet felt, because it's only triggered by (very) unusual experiences. Maybe it's something that can only be felt after a decade of weightlessness, or after the human brain has adapted to the living for a long time within the atmosphere of a gas giant.

Let's assume that exiert has some survival uses - it's helpful in navigating the gas giant's ever-changing geography, say - and it can load onto both positive and negative affect (just as confusion and surprise can). Assume also that experiencing exiert can have impact on people's aethetic preferences - it can cause them to like certain colour schemes, or open them to different types of pacing and tone in films or theatre productions.

In the spirit of trying to deduce what human values really are, the question is then: is the AI overriding human preferences in any of the following situations:

The AI premeptively precludes the experience of exiert, so that we won't experience this emotion even if we are put in the unusual circumstances.
Suppose that we wouldn't "naturally" experience exiert, but the AI acts to ensure that we would (in those unusual circumstances).
The AI acts to ensure that some human that had not yet experienced strong lust, nostalgia, or extreme pain, could never experience that emotion.
Suppose some human would not "naturally" experience strong lust, nostalgia, or extreme pain, but the AI acts to ensure that they would.
The AI acts to ensure that some human experiences exiert, but that this is triggered by normal circumstances, rather than unusual ones.
Global online debate on the governance of AI
by CarolineJ — 2018-01-05T15:31:29.091Z

Hi guys,

For background, I’m a French EA, attended a CFAR workshop, and recently decided to work on AI policy as it is a pressing and neglected issue. I’ve been working for The Future Society for a few weeks already and would like to share with you this opportunity to impact policy-making.

The Future Society

is a Harvard Kennedy School-incubated think tank dedicated to the governance of emerging advanced technologies. It has partnerships with the Future of Life Institute and the Centre for the Study of Existential Risk.

The think-tank provides an participatory debate platform to people all around the world

The objective is to craft actionable and ethical policies that will be delivered in a White Paper, to the White House, the OECD, the European Union and other policymaking institutions that the think-tank is working with.

Because we know AI policy is hard, the idea is to use collective intelligence to provide innovative and reasonable policies. The debate is hosted on an open source collective intelligence software resulting from a research project funded by the European Commission, technologically supported by MIT. It’s based on research on collective intelligence, going from open and exploratory questions to more in-depth discussions. Right now, we are in the “Ideation” phase, which is very open. You can make constructive answers and debate with other people who are also interested in crafting AI Policies with instant translation.

The platform is like an online forum articulated around several issues, both short-term and long-term oriented. You have six themes, including “AI Safety and Security”, “Reinvent Man & Machine Relationship” and “Governance Framework”.

So far, most of the answers have been very constructive. But with you guys… it can be even better.

Because you are Rationalists, I really wanted to pick your brains to think rationally and critically about AI governance.

It would be great if you guys could participate, on the topic you’re most interested in, knowing that a) it will be impactful b) you will be able to challenge your thoughts with other people passionate about AI social impacts. Of course, you don’t have to talk about AI safety if you’d rather focus on other topics.

Don't hesitate to post short (yet insightful) comments to start, just to fuel the debate.

So please connect on the debate, and participate.

The debate is here