trentmkelly/LessWrong-43k huggingface dataset:
Agency and Sphexishness: A Second Glance
by Ruby — 2019-04-16T01:25:57.634Z

This post is backported from my old personal blog, originally published 11 October, 2014. It's similar in theme but very different in focus than my recent post, On the Nature of Agency, making for a good complement.

Epistemic Status: Though I wrote this over four years ago and now use different models/lenses/ontologies, what I wrote then still seems largely correct. The exception is the endorsement of noticing which I haven't thought about much in recent years and therefore can't endorse, though I do now want to think about it more.

.

Agency
Agency is the property of agents. An agent has explicit goals which they strive to accomplish by planning and executing appropriate actions. Non-agents unreflectively act out default behaviours without considering whether these actions achieve their goals.

Sphexishness
Coined by Douglas Hofstader in reference to the sphex wasp, sphexishness is the execution of seemingly intelligent behaviour by following a rigid algorithm. Sphexish behaviours are repeated automatically, out of habit, without checking for their effectiveness at achieving desired goals.

When I first learnt the concepts of agency and sphexishness, I didn’t give them much thought. I unreflectively assumed that you were either an agent or you weren’t; you were either sphexish or you weren’t. If you were the kind of person who thought about their goals, read Less Wrong, went to CFAR workshops, then clearly you were an agent – you actually cared about your goals. I, of course, was such as person. It’s the general population that is sphexish, they don’t notice when things don’t get them what they wanted. And it all maps well on the controversial metaphor of PCs and NPCs.

But you don’t just get agent status and that’s it. Rather, as I have learnt, there is a constant slide into sphexishness which you must work damn hard to avoid.

Looking at definition given, sphexishness has two components:

Unreflective, automatic, default behaviours.
These behaviours fail to achieve goals.

Really, it’s 2) which matters. Firstly, not only can you have unreflective, automatic behaviours which are not sphexish, you absolutely need unreflective, automatic behaviours. Attention and cognitive resources are limited and you cannot be constantly using System 2 to examine how you do things to achieve your goals. Instead you must have put in place default behaviours which do the job....
Literature Review: Distributed Teams
by Elizabeth — 2019-04-16T01:19:27.307Z

Introduction

Context: Oliver Habryka commissioned me to study and summarize the literature on distributed teams, with the goal of improving altruistic organizations. We wanted this to be rigorous as possible; unfortunately the rigor ceiling was low, for reasons discussed below. To fill in the gaps and especially to create a unified model instead of a series of isolated facts, I relied heavily on my own experience on a variety of team types (the favorite of which was an entirely remote company).

This document consists of five parts:

Summary
A series of specific questions Oliver asked, with supporting points and citations. My full, disorganized notes will be published as a comment.

My overall model of worker productivity is as follows:

Highlights and embellishments:

Distribution decreases bandwidth and trust (although you can make up for a surprising amount of this with well timed visits).
Semi-distributed teams are worse than fully remote or fully co-located teams on basically every metric. The politics are worse because geography becomes a fault line for factions, and information is lost because people incorrectly count on proximity to distribute information.
You can get co-location benefits for about as many people as you can fit in a hallway: after that you’re paying the costs of co-location while benefits decrease.
No paper even attempted to examine the increase in worker quality/fit you can get from fully remote teams.

Sources of difficulty:

Business science research is generally crap.
Much of the research was quite old, and I expect technology to improve results from distribution every year.
Numerical rigor trades off against nuance. This was especially detrimental when it comes to forming a model of how co-location affects politics, where much that happens is subtle and unseen. The most largest studies are generally survey data, which can only use crude correlations. The most interesting studies involved researchers reading all of a team’s correspondence over months and conducting in-depth interviews, which can only be done for a handful of teams per paper.

How does distribution affect information flow?

“Co-location” can mean two things: actually working together side by side on the same task, or working in parallel on different tasks near each other. The former has an information bandwidth that technology cannot yet duplicate. The latter can lead to serendipit...
IRL 7/8: Generalizing human-robot cooperation: Cooperative IRL
by RAISE — 2019-04-15T10:13:32.565Z
Link: https://app.grasple.com/#/level/1606

Every Monday for 8 weeks, we will be posting lessons about Inverse Reinforcement Learning. This is lesson 7.

Note that access to the lessons requires creating an account here.

Have a nice day!
Conspiracy World is missing.
by Elias — 2019-04-15T07:58:15.163Z

Can one of you tell me what happened to /tag/conspiracy_world ? I could not find it anywhere I looked online, as the only links lead to that page, which says that it couldn't find the content.

I came to search it because of Eliezers "Final Words", 27th April, 2009. I would like to read the rest of it.
Scrying for outcomes where the problem of deepfakes has been solved
by mako yass — 2019-04-15T04:45:18.558Z

(Prompted by the post: On Media Synthesis: An Essay on The Next 15 Years of Creative Automation, where Yuli comments "Deepfakes exist as the tip of the warhead that will end our trust-based society")

There are answers to the problem of deepfakes. I thought of one, very soon after first hearing about the problem. I later found that David Brin spoke of the same thing 20 years ago in The Transparent Society. The idea seems not to have surfaced or propagated at all in any of the deepfake discourse, and I find that a little bit disturbing. There is a cartoon Robin Hanson that sits on my shoulder who's wryly whispering "Fearmongering is not about preparation" and "News is not about informing". I hope it isn't true. Anyway.

In short, if we want to stay sane, we will start building cameras with tamperproof seals that sign the data they produce with a manufacturer's RSA signature to verify that the footage comes directly from a real camera, and we will require all news providers to provide a checked (for artifacts of doctoring and generation), verified, signed (unedited) online copy of any footage they air. If we want to be extra thorough (and we should), we will also allocate public funding to the production of disturbing, surreal, inflammatory, but socially mostly harmless deepfakes to exercise the public's epistemic immune system, ensuring that they remain vigilant enough to check the national library of evidence for signed raws before acting on any new interesting video. I'm sure you'll find many talented directors who'd jump at the chance to produce these vaccinating works, and I think the tradition will find plenty of popular support, if properly implemented. The works could be great entertainment, as will the ensuing identification of dangerously credulous fools.

Technical thoughts about those sealed cameras

The camera's seal should be fragile. When it's broken (~ when there is any slight shift in gas pressure or membrane conductivity, when the components move, when the unpredictable, randomly chosen build parameters fall out of calibration), the camera's registered private key will be thoroughly destroyed, with a flash of UV, current, and, ideally, magnesium fire, so that it cannot be extracted and used to produce false signatures. It may be common for these cameras to fail spontaneou...
Can coherent extrapolated volition be estimated with Inverse Reinforcement Learning?
by Jade Bishop — 2019-04-15T03:23:30.967Z

Given the following conditions, is it possible to approximate the coherent extrapolated value of humanity to a "good enough" level?:

Some form of reward/cost function estimation is used, such as inverse reinforcement learning or inverse optimal control. The details of the specific IRL/IOC algorithm in question are not important, just the fact that the reward/cost function is estimated. For the unfamiliar, IRL is essentially the opposite of traditional reinforcement learning in that, given a set of observations and actions, it tries to determine the reward, utility, or value function (all interchangeable) of the agent(s) that generated the set.

An agent is able to observe another (presumably human) agent's behaviour and update their estimate of the reward function based on that, without having direct sensory input from them. This is essentially what mirror neurons do. Technologically-speaking, this is probably the most difficult part to achieve, but not too important for the purpose of this question.

Here is my reasoning to believe that this approximation will in fact work:

First, we assume that all these constraints are true.

The estimated reward function is continuously updated with data from every individual it meets, using some form of weighted experience replay system so as to not overwrite previously-learned information.

Given that IRL/IOC can already estimate the reward function of one agent, or even a specific class of agents such as streaked shearwater birds¹, with a sufficiently complex system this algorithm should be able to extend to complex (read: human) agents.

As the number of observations n approaches infinity (or some sufficiently large number), the reward function should approach a reward function that is a "good enough" approximation of the coherent extrapolated value of humanity.

Note that there does not need to exist some actual reward function that is natively used by real humans, evaluated by their brain. As long as human behaviour can be sufficiently approximated by a neural network, this will hold; given the wide abilities of neural networks, from classifiers to learning agents to machine translation, I don't see this as too much of a stretch.

However, I do anticipate certain objections to this explanation. Let me run through a few of them.

Humans are too complex to have their behaviours estimated by inverse reinforcement learning.

This

...
The Cacophony Hypothesis:
Simulation (If It is Possible At All) Cannot Call New Consciousnesses Into Existence
by TheTripleAffirmative — 2019-04-14T21:20:43.522Z

Epistemic Status: The following seems plausible to me, but it's complex enough that I might have made some mistakes. Moreover, it goes against the beliefs of many people much smarter than myself. Thus caution is advised, and commentary is appreciated.
I.

In this post, I aim to make a philosophical argument that we (or anyone) cannot use simulation to create new consciousnesses (or, for that matter, to copy existing people's consciousnesses so as to give them simulated pleasure or pain). I here make a distinction between "something that acts like it is conscious," (e.g. what is commonly known as a 'p-zombie') and "something that experiences qualia." Only the latter is relevant to what I mean when I say something is 'conscious' throughout this post. In other words, consciousness here refers to the quality of 'having the lights on inside', and as a result it relates as well to whether or not an entity is a moral patient (i.e. can it feel pain? Can it feel pleasure? If so, it is important that we treat it right).

If my argument holds, then this would be a so-named 'crucial consideration' to those who are concerned about simulation. It would mean that no one can make the threat of hurting us in some simulation, nor can one promise to reward us in such a virtual space. However, we ourselves might still exist in some higher world's simulation (in a manner similar to what is described in SlateStarCodex's 'The View from the Ground Level'). Finally, since one consequence of my conclusion is that there is no moral downside to simulating beings that suffer, one might prefer to level a Pascal's Wager-like argument against me and say that under conditions of empirical and moral uncertainty, the moral consequences of accepting this argument (i.e. treating simulated minds as not capable of suffering) would be extreme, whereas granting simulated minds too much respect has fewer downsides.

Without further ado...

II.

Let us first distinguish two possible worlds. In the first, simulating consciousnesses [in any non-natural state] is simply impossible. That is to say, the only level on which consciousnesses may exist, is the real, physical level that we see around us. No other realms may be said to 'exist'; all other spaces are mere information -- they are fiction, not real. Nature...
A Numerical Model of View Clusters: Results
by Shmi — 2019-04-14T04:21:00.947Z
Link: https://quantitativephilosophy.wordpress.com/2019/04/13/separation-and-clustering-of-views-playing-around/

As promised in the last post, here are some results of the simulation.

So now that the hard work is done, let’s see what the model gives us. I cheated a bit and adjusted the defaults to show something reasonable. You can see the hand-fudged coefficients in the code, if you feel like.

You can also play with the live model yourself here.

So, let’s have a look:

50 people, views starting uniformly distributed 20 points apart,
equilibrium separation is about 220 points.

To recap the original setup, we have a single dimension of views, starting with the uniform distribution. This corresponds to the horizontal axis on the top two graphs (“potential” and “force”), and to the vertical axis on the bottom graph (the time evolution of clustering and alienation of views). Note that each time step on the graph corresponds to 1000 steps in the code, i.e. the whole length of the simulation is 100,000 steps.

The basic results are not very surprising:

People with the views closest to each other converge, with those on the fence taking awhile to pick a side, but eventually being pulled (and pushed) toward one of the view clusters.
The separation between prevalent views corresponds to the equilibrium between repulsion of an outgroup and attraction of what one may thing as universal values.
The number of view clusters roughly corresponds to the initial separation spread (1000 in this example) divided by the equilibrium distance.

Here is another couple of run with the same parameters:

It is interesting to notice that, while we eventually end up with just 4 clusters, sometimes there is an intermediate metastable equilibrium with 5 view clusters, and eventually the closest two merge together.

Let’s play around with the parameters a little more. Upping the gain models how strong the attraction/repulsion feelings are. Here are a couple of runs with the “feelings” being twice as intense:

It looks like the clustering happens faster when the feelings are more intense, not very surprising. The number of clusters does not seem to be affected, though. What if instead of changing the intensity, we play with the “width”, meaning how quick the repulsive feelings subside after enough separation. Why should they subside? Scott Alexander discussed it in I Can Tolerate Anything Except The Outgroup:

You wouldn’t celebrate Osama’s death, only Thatcher’s. And you wouldn’t call ISIS savages, only Fox News. Fox is t

...
Quantitative Philosophy: Why Simulate Ideas Numerically?
by Shmi — 2019-04-14T03:53:11.926Z
Link: https://quantitativephilosophy.wordpress.com/

Adapted from my blog. I argue that numerical simulations are an effective yet underused tool in philosophy (and rationality) and give a concrete example of a numerical toy model, from assumptions to design to implementations to results, including the source code and a live site to play with.

It used to be that logic was king. A convincing argument was all that was necessary to get one’s ideas taken seriously and often accepted. After all, if it makes sense, it must be right, right? Right? The thought of actually checking the ideas experimentally is not very old, and, while firmly entrenched in the scientific method, sometimes earnestly, sometimes as a lip service, it is still not the commonly accepted practice in many “softer” sciences.

This forum is a site devoted to rationality, and has plenty of interesting insights, but very few of them have been actually tested. There is a good reason for it: thinking ideas up is much easier than checking them! Eliezer Yudkowsky, the original contributor, set this tone of using reasoning as an argument, without ever closing the feedback loop of checking the conclusions with numbers. There were a few exceptions to this pattern, such as the Iterated Prisoner’s Dilemma bot tournaments, but in general the focus is on reasoning and sometimes mathematical proofs, where possible and warranted, not on making testable predictions and actually testing them.

This leaves out one of the most powerful tools of checking validity of an idea: numerical simulation. If the idea is any good, one ought to be able to formalize it to the degree where its conclusions can be tested by creating simulations and studying their behavior.

The situation is somewhat better at Slate Star Codex: Scott Alexander has lots of wonderful ideas, and he is more aware of the need to check them by something other than more logic. But his approach for testing them is generally literature search or polls/surveys. Those are useful, but they focus on the ideas as black boxes, rather than on their internal mechanics.

Some examples of interesting ideas ripe for numerical modeling:

Inadequate Equilibria
Motte and Bailey
Moloch
Outgroup
Evaporative Cooling of Group Beliefs

In the following I focus on one idea, similar to the outgroup one listed above:

PEOPLE FEEL MORE AFFINITY TO THOSE WHOSE VIEWS ARE CLOSE TO THEIR OWN, AND ARE OFTEN REPULSED BY THOSE WHOSE VIEWS DIVERGE A LOT FRO

...
A Case for Taking Over the World--Or Not.
by Flange Finnegan — 2019-04-14T01:34:45.306Z

As my education progresses, I'm seeing more and more paralells, through some fictional but generally nonfictional accounts, that sugget that the world is broken in a way that causes suffering to be an emergent property, if not intrinsic. Not just HPMoR (and Significant Digits after it), but in FDR's State of the Union Speech, The Grapes of Wrath, Jared Diamond's Guns, Germs, and Steel, among other works. People have been aware for most of human history that, because of humanity's rules, people must suffer.

Knowing that this whole site is more or less dedicated to defeating unconscious ays of thinking and holds the mental enlightening of the human race paramount, I would like to pose this question:

What would we have to do to save the world?

Before breaking this question and this intent down, I'd like to clarify some things:

I am solely concerned with the practicalities, not with what people would or should do. Anybody who's seen enough of the world and how it works have an idea of the immensity of it, but humans made the current state of events what they are today (barring other undiscovered factors not covered by my priors), with the majority of them being largely redundant to the process in one way or another. People have demonstrated repeatedly that a group of people can have an impact disproportionate to their individual means.

What would have to occur, in the current political, economic, social etc. climate and onwards?

Would it have to be conspiracy? Or something else?

You can ask any other bounding questions that you would like, such as "What is the minimum amount of manpower and resources required to accomplish X through the most expedient and readily available means?" At the end of the day (so to speak) we should be able to shortly arrive at some sort of operational plan. There's no sense in taking this knowledge and not using it to further our cause.
Where to Draw the Boundaries?
by Zack_M_Davis — 2019-04-13T21:34:30.129Z

Followup to: Where to Draw the Boundary?

Figuring where to cut reality in order to carve along the joints—figuring which things are similar to each other, which things are clustered together: this is the problem worthy of a rationalist. It is what people should be trying to do, when they set out in search of the floating essence of a word.

Once upon a time it was thought that the word "fish" included dolphins ...

The one comes to you and says:

The list: {salmon, guppies, sharks, dolphins, trout} is just a list—you can't say that a list is wrong. You draw category boundaries in specific ways to capture tradeoffs you care about: sailors in the ancient world wanted a word to describe the swimming finned creatures that they saw in the sea, which included salmon, guppies, sharks—and dolphins. That grouping may not be the one favored by modern evolutionary biologists, but an alternative categorization system is not an error, and borders are not objectively true or false. You're not standing in defense of truth if you insist on a word, brought explicitly into question, being used with some particular meaning. So my definition of fish cannot possibly be 'wrong,' as you claim. I can define a word any way I want—in accordance with my values!

So, there is a legitimate complaint here. It's true that sailors in the ancient world had a legitimate reason to want a word in their language whose extension was {salmon, guppies, sharks, dolphins, ...}. (And modern scholars writing a translation for present-day English speakers might even translate that word as fish, because most members of that category are what we would call fish.) It indeed would not necessarily be helping the sailors to tell them that they need to exclude dolphins from the extension of that word, and instead include dolphins in the extension of their word for {monkeys, squirrels, horses ...}. Likewise, most modern biologists have little use for a word that groups dolphins and guppies together.

When rationalists say that definitions can be wrong, we don't mean that there's a unique category boundary that is the True floating essence of a word, and that all other possible boundaries are wrong. We mean that in order for a proposed category boundary to not be wrong, it needs to capture some statistical structure in reality, even if reality is surprisingly detailed and there can be more than one such structure.

The

...
Why is multi worlds not a good explanation for abiogenesis
by solomon alon — 2019-04-12T18:53:58.908Z

I'm not a expert in the multi world theory. So this question could very well be extremely stupid. However, given the assumption that there are nearly infinite amount of worlds that are slightly different than each other, nearly every possible event would happen. This includes the formation of life. Now what are the odds that we would be witnessing that world, as far as I can tell 100 percent.

Now I'm not clear exactly how often quantum events lead to a slightly different world but even at the rate of 1 quantum event a year in the entire universe. should lead to a near infinite explosion of completely different universes.

Now I'm not claiming that this is the explanation for abiogenesis or that abiogenesis is proof of multi worlds because that would be multi worlds of the gap fallacy however I'm not clear why I have never even seen this explanation even once for abiogenesis.

I also suspect that mathematically many worlds would usually be the wrong explanation for nearly everything because it runs into serious odds problems and in 99.99999 percent of cases there is a better explanation. however it should at least be considered

COULD SOMEONE EXPLAIN TO ME EXACTLY WHERE I WENT WRONG