trentmkelly/LessWrong-43k huggingface dataset:
Notes From an Apocalypse
by Toggle — 2017-09-22T05:10:21.792Z

(This is a loose adaptation of a talk I sometimes give on the Cambrian Explosion, smoothed a bit for popular consumption. That talk, in turn, draws heavily from a 2006 paper by Charles Marshall, titled “Explaining the Cambrian ‘Explosion’ of Animals”. It can be read in full here. I’m mostly just trying to test out the new site with an essay I had lying around, be moderately entertaining, and maybe try to suggest this event as a topic of interest for those who care about intelligence explosions and cognitive emergence. If I succeed in all three, then I encourage you to start with Marshall for the more technical, thorough, and correct analysis.)

A.

The Cambrian Explosion is our name for an event that took place about 540 million years ago, one that accounts for the sudden appearance of advanced animal life. Emphasis on “sudden”; it’s like somebody flipped a light switch. What follows is just a story (fragments of a story, really), but it’s a pretty good one, and I don’t think it’ll lead you too far astray.

Let’s start 540 million years ago. The planet is still recovering from a series of catastrophic “snowball Earth” phases, ice ages so severe that the oceans had frozen right across the equator; the tropical glaciers wound their way through loose sediment to leave an enduring footprint. Now that the last of these has thawed, it’s finally starting to warm up in a more permanent way and get a little more hospitable. There are two major continents at the moment, but one of them is starting to break up into smaller and smaller chunks. As with our rising temperatures, this helps create a more habitable planet, since it gets you more high-nutrient coastlines per unit land area. Importantly, the last hundred million years have provided us with oxygen concentrations in the atmosphere at basically 21st-century levels. To the unprepared organisms that dominate the first half of Earth’s history, oxygen can act as a hazardous toxin, but if you harness it correctly you can really kick your metabolism into a higher gear. A good time to be alive, really.

The land is mostly barren. A little bit of pond scum holding on in shallow pools and other moist areas, maybe, but seeds and placentas won’t exist for a long age yet, and so far every reproductive system requires the presence of standing water. The ocean, though, the ocean is a different story, full of life

...
Strategic Goal Pursuit and Daily Schedules
by Rossin — 2017-09-22T04:23:23.590Z

In the post Humans Are Not Automatically Strategic, Anna Salamon writes:

there are clearly also heuristics that would be useful to goal-achievement (or that would be part of what it means to “have goals” at all) that we do not automatically carry out. We do not automatically:

(a) Ask ourselves what we’re trying to achieve;

(b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;

(c) Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;

(d) Gather that information (e.g., by asking as how folks commonly achieve our goal, or similar goals, or by tallying which strategies have and haven’t worked for us in the past);

(e) Systematically test many different conjectures for how to achieve the goals, including methods that aren’t habitual for us, while tracking which ones do and don’t work;

(f) Focus most of the energy that *isn’t* going into systematic exploration, on the methods that work best;

(g) Make sure that our "goal" is really our goal, that we coherently want it and are not constrained by fears or by uncertainty as to whether it is worth the effort, and that we have thought through any questions and decisions in advance so they won't continually sap our energies;

(h) Use environmental cues and social contexts to bolster our motivation, so we can keep working effectively in the face of intermittent frustrations, or temptations based in hyperbolic discounting;

When I read this, I was feeling quite unsatisfied about the way I pursued my goals. So the obvious thing to try, it seemed to me, was to ask myself how I could actually do all these things. I started by writing down all the major goals I have I could think of (a). Then I attempted to determine whether each goal was consistent with my other beliefs, whether I was sure it was something I really wanted, and was worth the effort(g).

For example, I saw that my desire to be a novelist was more motivated by the idea of how cool it would feel to be able to have that be part of my self-image, rather than a desire to actually write a novel. Maybe I’ll try to write a novel again one day, but if that becomes a goal sometime in the future it will be because there is something I really want to write about, not because I would like to be a writer.

Once I narrowed my goals

...
Motivating a Semantics of Logical Counterfactuals
by Sam_A_Barnett — 2017-09-22T01:10:27.759Z

(Disclaimer: This post was written as part of the CFAR/MIRI AI Summer Fellows Program, and as a result is a vocalisation of my own thought process rather than a concrete or novel proposal. However, it is an area of research I shall be pursuing in the coming months, and so ought to lead to the germination of more ideas later down the line. Credit goes to Abram Demski for inspiring this particular post, and to Scott Garrabrant for encouraging the AISFP participants to actually post something.)

When reading Soares' and Levinstein's recent publication on decision theory, Death in Damascus, I was struck by one particular remark:

Unfortunately for us, there is as yet no full theory of counterlogicals [...], and for [functional decision theory (FDT)] to be successful, a more worked out theory is necessary, unless some other specification is forthcoming.

For some background: an FDT agent must consider counterfactuals about what would happen if its decision algorithm on a given input were to output a particular action. If the algorithm is deterministic (or the action under consideration is merely outside of the range of possible outputs of the agent), then it is a logical impossibility that the algorithm produces a different output. Hence the initial relevance of logical counterfactuals or counterlogicals: counterfactuals whose antecedents represent a logical impossibility.

My immediate thought about how to tackle the problem of finding a full theory of counterlogicals was to find the right logical system that captures counterlogical inference in a way adequate to our demands. For example, Soares and Fallenstein show in Toward Idealized Decision Theory that the principle of explosion (from a contradiction, anything can be proved) leads to problematic results for an FDT solution. Why not simply posit that our decision theory uses paraconsistent logic in order to block some inferences?

Instead, and to my surprise, Soares and Levinstein appear to be more concerned about finding a semantics for logical counterfactuals - loosely speaking, they are looking for a uniform interpretation of such statements which shows us what they really mean. This is what I infer from reading the papers it references on theories of counterlogicals such as Bjerring's Counterpossibles, which tries to extend Lewis' possible-worlds semantics for counterfactuals to cases of logical

...
Splitting Decision Theories
by somni — 2017-09-22T00:47:47.367Z

[epistemic status: maybe wrong; thinking aloud, would like people to yell at me]

There is a repeated motion that occurs when deciding what an AI should do:

(1) Create a decision theory

(2) Create a thought experiment in which an agent with *DT makes a choice which fails to fulfill its utility function (e.g. Oh no! It loses all its money to blackmail!)

(3) Create a DT which does well against problems which the core difficulty which allowed the previous decision theory to lose all its money

If decision theories are as precisely imagined as mathematical structures. For every two distinct decision theories, there exists in mathematical reality a set of "thought experiments" such that the two theories decide differently on them.

This seems weird and difficult now because there isn't a shared logical notation between different "thought experiments". As of now characterizing the class of splitting decision problems for two decision theories is pretheoretic. However, for every pair of decision theories DT_1 and DT_2 the object split(DT_1, DT_2) actually exists. Current notational limits make it currently difficult to simply and completely characterize the class of choice problems on which two DTs give different answers.

But it feels like this sort of problem occupies a similar status as “algorithms” did before the first Universal Turing Machine was constructed.

-

Questions:

In fun games (like prisoner’s dilemma) we have agents (like fairbot) that fight each other. The source code for these agents is entangled with their decision theory. Does examining bots engaged in modal combat make this problem more tractable?

This process repeats like clockwork (it feels like a new decision theory comes out every year or so?) in hopes of giving their baby AI a good way of making good choices and not losing all its money. What if I built an AI that formalized and internalized this process and just ... gave itself good advice? Within logical inductors traders bet on which theorems would be best and traders which make bad bets lose their money. If we can formalize split(DT_1, DT_2) we can look at how well agents fulfill their utility functions in this space. Can we use this to establish a kind of poset of decision theories?
Marketing Failure
by edoarad — 2017-09-21T20:02:13.424Z
Link: https://edoarad.wordpress.com/2017/09/21/marketing-failure/

The amount of money spent worldwide on advertisements is astronomical, reaching about 550 billion dollars annually and is expected to continue growing rapidly. In 2016, the top 20 companies with the biggest annual advertising budget spend between 2.7 to 8.3 billion dollars annually.

Is this a result of a race-to-the-bottom-type market failure? If so, can we solve it?

A nice story I heard was that when the united states acted to ban cigarette advertising, which was an enormous industry, the cigarette companies were actually in favor! The reason for that is that the cost of advertisement is huge and that the main motivation for advertising was to get more market share but that the advertisements did not cause more people to smoke more, so they were stuck in a prisoner's dilemma situation. Sadly, this story seems to be wrong, but it is meant to show that a big share of the money spent on marketing seems to go toward convincing that your product is better then the alternatives. Even if 1% of the marketing cost could be reduced without intrinsically harming the economy, it would still amount to a total of 1 billion dollars which could be spent on R&D, higher salaries etc.

By the way, it is very interesting to see that on average about 10% of revenue goes into sales and marketing, which for some sectors is more then the percentage of revenue that goes into R&D. This is a good indicator that it is not enough to have a good product, or even the best product, if you want people to buy it. More precisely, it means that this is what these companies believe. Perhaps some sectors are in a marketing bubble, and that is not actually the case, as I suspect might be the case in some industries. For example, in the laptop industry a crux that would imply whether or not we are in such a bubble is whether people today choose their laptop solely based on how much this laptop fits their needs (quality, price, size,..) as opposed to buying a computer based on emotional connection to the brand (macbook, dell,..). It would be a good thing, if we are in such a bubble, as it means that it will eventually burst and more money will be spent on improving the products as a way to generate more consumers. An alternative formulation of this process is how much does a company adapt the product to fit the consumer as opposed to influencing the consumers to fit the product.

...
Why Attitudes Matter
by ozymandias — 2017-09-21T15:07:44.687Z

Sometimes when I am giving ethical advice to people I say things like "it's important to think of yourself and your partner as being on the same team" or "just remember that women in short skirts are almost certainly not wearing short skirts to arouse you in particular" or "cultivate your curiosity and desire to know what's actually going on."

I get pushback on this. After all, I am a consequentialist. Why am I talking about people's attitudes instead of their actions? It doesn't matter what I think of the woman in the short skirt, as long as I refrain from being a dick to her because of her clothing choices.

An emphasis on attitudes can be really bad for some people. Some people, having been given the advice that they should cultivate their curiosity, will spend a lot of time navel-gazing about whether they're really curious and whether this curiosity counts as curiosity and maybe they are self-deceiving and actually just want to prove themselves right Not only is this really unpleasant, but if you're spending all your time navel-gazing about whether you're sufficiently curious you're never actually going to go buy a book about the Abbasid empire. It completely fails to achieve the original goal. If this is a problem you're prone to, I think my attitude-based advice is probably not going to be helpful, although I can't give any other advice; I personally get as much navel-gazing as I can stand trying to keep my obviously shitty attitudes in check, and don't have any introspective energy left over for anything else.

Nevertheless, I think an attitude emphasis can be really important, for two reasons.

First, for any remotely complicated situation, it would be impossible to completely list out all the things which are okay or not okay. For instance, think about turning my "think of yourself and your partner as being on the same team" advice into a series of actions. You might say "it is wrong to insult your partner during disagreements." But for some people, insults are part of resolving disagreements. Saying "I am not sure you've really thought this through" rather than "that is the stupidest fucking idea I've ever heard" feels artificial to them, like they're walking on eggshells. For them, intimacy requires the ability to sa

...
Value Arbitrage
by rayalez — 2017-09-21T04:23:25.955Z

This is an extremely useful idea that I've learned just recently. In fincance, "arbitrage" means buying and selling things in different markets to take the advantage of difference in price. Like buying a toy in India for $15, and then selling it for $25 in the US.

Turns out, you can arbitrage not just products, but also information, knowledge, skill, or even human relationships.

For example, let's say you have met two amazing people who don't know each other. When you introduce them to each other, you generate a massive amount of value for both of them essentially out of nothing - you have given each of them a gift of knowing someone great.

Another example, is arbitraging information. Eliezer Yudkowsky have learned a lot about rationality and human biases from the books he has read, and then parlayed it into his blog, and later a Harry Potter fanfiction - and that has generated a huge amount of value to the people on the internet, who would never have been exposed to these ideas otherwise.

You can also arbitrage skills. That simply means applying good ideas you have learned in one field to a completely different one. I'm constantly applying the skills and ideas I've learned as a 3D artist to my programming process, and, surprisingly, the ideas I've learned from programming are making me a much better artist.

Finally, an unexpected idea that I have discovered, is that you can arbitrage fiction tropes. You can take a trope you have seen in one story, apply it to a completely different genre, and it will create a new and interesting idea. You can take a SciFi trope and set it in a fantasy world, or you can take a plot structure from a dramatic TV show and use it in your comedy script. In fact, this is how a lot of movie ideas are created.

Smart ideas are often universal. Learn a mental pattern in one field - and apply it elsewhere. Now that I got pretty good at it, every time I have an epiphany in one of my crafts, I intentionally go through each of the other fields I'm interested in and apply it there. This has helped me to generate a lot of very interesting ideas.
Common vs Expert Jargon 
by Raemon — 2017-09-21T03:05:31.203Z

tldr: Jargon always has a complexity cost, but you can put effort into making a concept more accessible, and it's especially valuable to put that effort in for terms that you'd like to be used by layfolk, or that you expect to be used a lot in spaces where you expect lots of layfolk to be reading/participating.

I. Lessons from Game Design

Magic the Gathering deals a lot with complexity. Each year, new abilities and rules are added to the game. This gives experienced players the chance to constantly discover new things, but it comes with some issues.

First, it makes the game harder for new players (the game kept growing more complex over time, raising the amount of information a new player had to process at once)

And second, even for experienced players: each instance of complexity is a cost. Players (both new and old) can only handle so much, and some forms of complexity are less fun than others. (For example, forcing players to do a lot of book-keeping, rather than letting them make interesting strategic decisions)

Six years ago, their creative director wrote about a new paradigm of Magic design. One of their solutions was to pay careful attention to how they spent complexity points in ways that affected new players.

Three examples:

1. Common Cards

In Magic, when you buy a new pack, 11 cards are "common", 3 are "uncommon" and one is "rare". Experienced players buy lots of cards and can have access to lots of rares, but new players generally just buy a few cards, so most of their cards are common. Therefore, the complexity of the cards at common determines how much complexity newcomers have to deal with.

2. Keywords

One way to reduce "effective complexity" is to bundle concepts together in a keyword. Instead of saying "this creature deals damage to each of the creatures blocking it and then deals the remainder of its damage to the player", it just says "Trample". There's an initial cost of learning what Trample means, but afterwards, every time you see the word "Trample" on a creature it works the same way.

Trample has some neat things going for it: it sounds evocative, and gets to build off of existing ideas in your brain. You already know what a big animal looks like. You can imagine a small creature getting in the way of the elephant, and it slowing the elephant down slightly but n

...
Against EA PR
by ozymandias — 2017-09-21T01:23:00.502Z

Scott Alexander recently wrote about weird effective altruism. Many people (mostly, but not entirely, people who aren't effective altruists) offered the opinion that weird effective altruists should be banned from EA, or at least shouldn't be allowed to give talks at EA Global and have blog posts written about them. Weird effective altruist causes are (sort of by definition) off-putting to most people; therefore, if you want people to donate to global poverty relief, you should kick out all of those people concerned about farmed animal welfare/AI risk/wild animal welfare/psychedelics research/suffering in fundamental physics, lest we scare the normies.

There are many reasonable critiques of this point of view, including that it's not remotely clear that any of those claims are more frightening to normal people than "it is morally obligatory to make personal sacrifices in order to help poor, faraway black people." But ultimately I reject the entire premise.

I'd like to be clear about what I'm not saying in this post. I am not saying all "weird effective altruism" causes are effective; I believe some are and some aren't. I think many effective altruists are not taking seriously enough the difficulty of figuring out how effective highly speculative causes are, and that unless we seriously address this we're going to waste potentially millions of dollars on boondoggles. And I suspect a lot of weird effective altruism tends to over-explore certain cause areas (for example, things you think of if you read too many science fiction novels) and underexplore other cause areas (for example, boring things). I don't intend this post to be a whole-hearted defense of weird effective altruism, but simply a criticism of a single narrow argument too often wielded against it.

So the question arises: why is effective altruism a thing at all?

Most people care about charity effectiveness, at least a little bit. They look up their charities on Charity Navigator before donating; they object to money being spent on big CEO salaries or on overhead instead of on services; they circulate criticisms of the Susan G Komen Foundation and PETA. And yet not only do most social programs not work, for the vast majority of programs we simply haven't collected the information to see whether it works or not. This isn't a "no o

...
LW 2.0 Open Beta Live
by Vaniver — 2017-09-21T01:15:53.341Z

The LW 2.0 Open Beta is now live; this means you can create an account, start reading and posting, and tell us what you think.

Four points:

1) In case you're just tuning in, I took up the mantle of revitalizing LW through improving its codebase some time ago, and only made small amounts of progress until Oliver Habryka joined the project and put full-time engineering effort into it. He deserves the credit for the new design, and you can read about his strategic approach here.

2) If you want to use your current LW account on LW2.0, we didn't import the old passwords, and so you'll have to use the reset password functionality. If your LW account isn't tied to a current email, send a PM to habryka on lesswrong and he'll update the user account details on lesserwrong. He's also working on improving the site and sleeping and things like that, so don't expect an immediate response.

3) During the open beta there will be a green message in the bottom right hand corner of the screen. This is called Intercom, and is how you can tell us about issues with the site and ask other questions.

4) The open beta will end with a vote of users with over a thousand karma on whether we should switch the lesswrong.com URL to point to the new code and database. If this succeeds, all the activity from the open beta and the live site will be merged together. If the vote fails, we expect to archive LW until another team comes along to revive it. We currently don't have a date set, but this will be announced a week in advance.