trentmkelly/LessWrong-43k huggingface dataset:
We need a standard set of community advice for how to financially prepare for AGI
by GeneSmith — 2021-06-07T07:24:03.271Z

Earlier today I was reading this post about the rationalist community's limited success betting on bitcoin and thinking about how the singularity is going to be the ultimate test of the rationalist community's ability to translate their unusual perspectives into wealth and influence.

There needs to be some default community advice here for people who believe that we're likely to create AGI in our lifetimes but don't know how to prepare for it. I think it would an absolute shame if we missed opportunities to invest in the singularity the same way we missed opportunities to invest in Bitcoin (even though this community was clued in to crypto from a very early stage). I don't want to read some retrospective about how only 9% of readers made $1000 or more from the most important even in human history even though we were clued in to the promise and peril of AGI decades before the rest of the world.

John_Maxwell made a post about this last year along the same lines. but i'd like to expand on what he wrote.

Why is this important?

In addition to the obvious benefit of everyone getting rich, I think there are several other reasons coming up with a standard set of community advice is important.

Betting on the eventual takeover of the entire world economy by AI is not yet a fashionable bet. But like Bitcoin, betting on AGI will inevitably become a very fashionable bet in the next few decades as first early adopters buy in, and then it becomes standard part of financial advice given out by investment professionals.

In these early days, I think there is an opportunity for us to set the standard for how this type of investment is done. This should include not just a clear idea of how to invest in AGI's creation (via certain companies, ETFs, AI focused SPACs etc), but also what should NOT be done.

For example, the community advice should probably advise against investing in companies without a strong AI alignment team, as capitalizing such companies will increase the likelihood that AI will destroy you and everything you love. We may also want to discourage investment in companies that don't have a clause on how they plan to deal with race conditions that could compromise safety. There are probably other considerations we should make that I am not thinking of. OpenAI's charter seems like a pretty well thought-out set of guidelines for AGI creation. This site has a very healthy ...
Often, enemies really are innately evil.
by Andrew Vlahos — 2021-06-07T06:42:14.177Z

Part 1: What evil is:

I'm not just talking about a "mere" clinical psychopath or sociopath who doesn't feel guilt and isn't restrained by social norms. First of all, people who don't feel the emotions can decide to be good anyway, or at least not pointlessly cruel for no benefit. Second, everyone partly fulfils the definition of psychopath (see this TED talk for details). Saying that psychopaths are the problem just doesn't work.

I'm also not talking about people who don't know any better, like medieval soldiers who are taught honorable fealty. I'm not even talking about people who theoretically could do otherwise but Moloch would ruin them if they tried. I don't mean victims of circumstance.

What I mean are the many people who know that some things are damaging to someone else, gain no tangible value from doing them (or even expect that their life would be worse off!), know it is not a virtuous act, and do the harmful acts anyway without expecting future good to come from it.

People commonly have a terminal value of dragging other people down. Consider a game theory study by J. Sayer Minas, Alvin Scodel, David Marlowe, and Harve Rawson. The study itself is behind a paywall, but it's described in the book Prisoner's Dilemma by William Poundstone. People would often reduce their own prize if it means that their opponent's is reduced more.

Don't think this study is big enough to be representative? Neither do I. That's fine, though, because there are many, many, MANY more pieces of evidence from (almost) every internet troll, bully, and rapist, and many other criminals too. I'm not including on this list domestic abusers, since they often get rewarded with things like the spouse's money, or dinner being consistently made what and when they want. Remember, I'm not just claiming that some people are evil when it's convenient for them, or intrinsically apathetic. Many people would, when making a choice between harming a stranger or doing nothing, would harm the stranger. My point is that people often are innately evil, not a majority but not a negligible percentage, and saying that everyone is good inside is very, very wrong.

There is some good news. People really can change their behavior and values. Someone being evil now doesn't mean they always will be. But, if you aren't convinced yet, let me tell you specifically the harm done from thinking that people are good in...
Are"ingroups" vs. "outgroups" as commonly used typically based on self-identification or is it not sufficient?
by orthogenesis — 2021-06-07T01:03:24.912Z

Wikipedia says:

"In sociology and social psychology, an in-group is a social group to which a person psychologically identifies as being a member. By contrast, an out-group is a social group with which an individual does not identify."
Implying it is about psychological identification. This makes sense for groups like libertarians, socialists, rationalists and political ideologies or cultural leanings where individual choice and agency matters, perhaps more than for "immutable" characteristics bestowed upon people by birth. Even so, there is fuzziness (e.g. what if you call yourself one, but most people who call themselves one don't recognize you as one?).

But there are usages where it's unclear if ingroup and outgroup are about self-identification or categorization by others, or even some "objective standard" (all people with trait X are part of an ingroup and you can't join if you lack X). I don't think everyone uses the Wikipedia definition based on "psychological self-identification" when they speak of the terms. I have heard the term used for situations where the person is ascribed to a group and other members of the group are called the person's "ingroup" without knowing anything about psychological state of self-identification. Is this a misuse of the term?

Imagine a person that dislikes the country they are born in and never left -- they reject the label of (insert demonym of nation) to describe themselves -- perhaps they aspire to be another country's citizen (but can't accomplish that for practical reasons) or maybe even dislike the idea of nationalism or patriotism and identify as "citizen of the world". They do not favor their own country and support other countries economically or even politically.
Would this person be considered (1) disloyal member of ingroup by fellow members of the ingroup or perhaps outgroup sympathizer (2) outgroup member by fellow citizens?

How does the ingroup/outgroup distinction handle situations where the individual and other members of a group disagree on whether they are members of the same group?
Or does the ingroup/outgroup distinction only apply where self-identification and broader consensus agree on what group a person belongs to, which becomes the person's ingroup?
Holidaying and purpose
by KatjaGrace — 2021-06-06T19:30:12.433Z

I’m on holiday. A basic issue with holidays is that it feels more satisfying and meaningful to do purposeful things, but for a thing to actually serve a purpose, it often needs to pass a higher bar than a less purposeful thing does. In particular, you often have to finish a thing and do it well in order for it to achieve its purpose. And finishing things well is generally harder and less fun than starting them, and so in other ways contrary to holidaying.

This isn’t a perfect relationship though, so a natural way to mitigate the trade-off is to just look harder until you find things that serve a worthy purpose while being non-committal and consistently non-arduous. For instance, you can exercise or learn about history or practice guitar or write half-assed blog posts without real conclusions or narrative consistency.

There is also probably good holidaying to be done that doesn’t seem obviously purposeful, and maybe that is more in the spirit of holidaying. Perhaps one should avoid too much purpose, lest one end up not holidaying?

Today I travelled by rowing boat across a lake and back, with my boyfriend and some of his family.

Now we are going to the zoo.
What are good resources for gears models of joint health?
by Randomized, Controlled — 2021-06-06T17:39:50.303Z

Painscience.com and Hargrove's "A Guide To Better Movement" are pretty good for a model of predictive processing and the roll of the nervous system in chronic pain and movement. I still don't feel like I have a good model of bone and joint health in general, however. Eg, I'm currently nursing a flare up of patelo-femoral pain in my left knee. I've done a number of things over the past few months to deal with it, with some success, including buying and reading Painscience's book length patelo-femoral tutorial. Recently I've had a bit of pain in my foot, possibly in the tibiocalcaneal or tibionavicular tendons. I find that even though I now know a fair amount about PFS and the way the nervous system processes pain, these models don't generalize well to sporadic, idiopathic pain in another joint.

Possibly the answer is: "lol that model doesn't exist", or "lol wanna get a phd?" but if there are good resources, I'd be an eager consumer.

A sub-question that I'm particularly interested in is: what, if anything, is know about the relationship between base line muscle tone and joint issues? I have good reason to think my baseline muscle tone is higher than average.
Looking for reasoned discussion on Geert Vanden Bossche's ideas?
by megabranflake — 2021-06-06T17:07:20.987Z

Considering the current covid hysiteria and censorship on social media platforms I'm finding it hard to find reasoned discussion of Geert Vanden Bossche's ideas. They seem to be centered around vaccine immune escape and original antigenic sin. If you have anything to contribute please do. Thanks
Unrefined thoughts on some things rationalism is missing vs religions
by Srdjan Miletic — 2021-06-06T17:04:51.358Z

Rationalism seems somewhat like a proto-religion to me. It has norms of behavior very different from society at large. It has apocalyptic prophecies which it's members strive to stave off. Still, what's missing? I think a few things:

Rituals

Symbols

A community

Institutions around which to build a rationalist life

The normalization of rationalism as a central identity rather than a peripheral one or a non-identity

Rituals

Rituals come in many shapes and sizes. Some rituals are recurring and universal. Passover, Petrov Day or Eid all happen once a year at the same time for all believers. Others are singular and specific to the individual. A Bar-mitzvah, Hindu marriage or Irish wake all happen at different times for different individuals and mark transitions from one part of life to another. Some are happy. Others are sad.. Some are for the whole community. Think of Haj. Others are for friends and familly, think of a wedding. Others are for family alone.

I think a major purpose rituals serve is to create/reinforce identity. I'm not sure how they do this, but I have a few plausible mechanisms in mind:

Getting members of the same community/faith to meet and spend time together

Giving people time to think about important questions and their faith

Triggering irrational mechanisms which make our brains recognize each other as being part of the same tribe. (similar to how marching in time or wearing a uniform in the army works)

Raemon's ritual sequence has some good thoughts on what makes rituals work and failure modes of rituals., Ruby's wedding ceremony has some of the best speeches/rituals I've read.

While certain rationalists do celebrate certain rituals such as Solstice or Petrov Day, these rituals are not widely celebrated (most rationalists I know don't celebrate them) and are few in number. Importantly, they're also only one kind of ritual. There's no rationalist ritual to mark the transition from childhood to adulthood. From life to death. From an individual to a family. I think most pieces of the puzzle are missing.

Symbols

A cross means Christianity. The Stars and Stripes means America. Every identity has it's symbols. I think symbols are important. I'm not sure why I believe this. I know people in history Maytered themselves over symbols. Old faithers dying for the two fingers instead of three. Maybe symbols are a coordination mechanism. A way to stak...
Review of "Learning Normativity: A Research Agenda"
by Gyrodiot — 2021-06-06T13:33:28.371Z

Introduction

We (Adam Shimi, Joe Collman & myself) are trying to emulate peer review feedback for Alignment Forum posts. This is the second review in the series. The

first’s introduction

sums up our motivation and approach rather well, we will not duplicate it here.

Instead, let’s dive into today’s reviewed work:

Learning Normativity: A Research Agenda

by Abram Demski. We’ll follow the same structure as before: summarize the work, locate its hypotheses, and examine its relevance to the field.

This post was written by Jérémy; as such, his perspective will likely bias its content, even if both Adam and Joe approve of it.

Summary

The post describes a conceptual target for AI alignment, normativity, that differs in significant ways from other approaches such as

value learning

and

imitation learning

.

Pointing at norms instead of a specific set of values has several interesting features, especially how it handles the uncertainty of feedback from humans. Norms are reflected imperfectly in human behavior, approval from humans regarding norm-following is sparse and imperfect, yet norms roughly convey what a machine should do.

Abram points at language learning as a major motivating example. We don’t learn English by knowing all the rules, we don’t even know all of them, those we can articulate don’t fit the data perfectly. Nevertheless, we can succeed, by human standards, at language acquisition.

Going beyond those standards, Abram mentions superhuman performance in language based on various properties of texts, with the implicit statement (if we interpret the post correctly) that this performance stems from a better adherence to the underlying norms of language.

This example shows that learning should be possible even in the absence of a gold standard, an ideal reference to which the output of an agent may be compared for performance. No feedback can be fully trusted; a system should eventually learn to distrust entire types of feedback, be robust to ontology shifts in feedback, and more generally be able to reinterpret any kind of feedback.

This creates a hierarchy in feedback, which is mirrored in the value specification problem: if one cannot specify values directly, one may try to specify how they are learned, and barring that one may try to learn how to learn, etc. Abram suggests there are ways to learn all relevant levels at once, generalizing over all of them. This ...
What to optimize for in life?
by Henrik Karlsson — 2021-06-06T07:35:30.709Z

I listened to an interview with Patrick Collison were he claimed that when coding one should always optimize for speed - even when speed is not an issue. (Presumably because it leads to good coding practices, clean code, less build up of unnecessary functionality etc.)
Assuming that is correct - and I think there is something to it - it makes me wonder: is there something similar that one could optimize for in life? Life is such a multivariate thing that it can at times be hard to know what to prioritize.
What parameter is a candidate for having most positive side effects on your life when optimized?
The dumbest kid in the world (joke)
by CronoDAS — 2021-06-06T02:57:05.230Z

"The dumbest kid in the world"

A young boy enters a barber shop and the barber whispers to his customer, “This is the dumbest kid in the world. Watch while I prove it to you.”

The barber puts a dollar bill in one hand and two quarters in the other, then calls the boy over and asks, “Which do you want, son?”

The boy takes the quarters and leaves.

“What did I tell you?” said the barber. “That kid never learns!” Later, when the customer leaves, he sees the same young boy coming out of the ice cream store.

“Hey, son! May I ask you a question? Why did you take the quarters instead of the dollar bill?”

The boy licked his cone and replied, “Because the day I take the dollar, the game is over!”
Anthropic Paradoxes and Self Reference 
by dadadarren — 2021-06-06T02:52:12.132Z

In this post I will explain how anthropic paradoxes are connected with self-reference.

Sleeping Beauty Problem

The contention is how to treat the fact that "I am awake now/today". To briefly summarize the debate at the cost of oversimplification: SIA suggests treating today as a random sample of all days. While SSA suggests treating today as a random sample of all awakening days in the experiment. This debate can be considered as the dispute over the correct way of interpreting/defining today.

It should be noted the direct use of the word "today" is not necessary. For example, in Technicolor Beauty due to Titelbaum, the experimenter would randomly pick a day and paint the room blue and paint the room red the other day. Say I wake up and sees the room is blue. How should I consider the fact that "The blue day has an awakening?". Here the direct reference is averted yet the problem remains the same. i.e. blue day is being asked only because that's what I'm seeing now.

Doomsday Argument

The doomsday argument essentially says, considering myself as a random sample from all humans (SSA), then I am more likely to have my current birth rank if the total number of humans is low. This should shift my forecast of the species' future towards "doom soon". The SIA counter-argument suggests I shall regard myself to be a random sample from all observers that could potentially exist. Then I actually do exist (as a human) is evidence that favors a higher number of humans. It exactly cancels out the pessimistic shift due to my birth rank.

The debate is about the correct way to define myself. Again, a direct reference to "myself" is not necessary. Often the argument is presented with wordings such as "us", "the current generation", or even "people in 2021". Yet the argument remains the same because those wordings are chosen because that's what you and I (the people discussing this problem) are.

Fine-Tuning Argument

Fine-tuning is the idea that the existence of life in this universe seems rather surprising. E.g. even if a few free parameters in the standard model have slightly different values, then life as we know it would not be possible. Fine-tuning has been used to promote two types of claims. One is the teleological argument that the universe is indeed fine-tuned by some unknown actor to specifically support life. The other is the multiverse claim tha...
Open and Welcome Thread – June 2021
by habryka — 2021-06-06T02:20:22.421Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Search-in-Territory vs Search-in-Map
by johnswentworth — 2021-06-05T23:22:35.773Z

Let’s split out two different types of optimization. The first type includes things like Google Maps’ pathfinder, a human making a plan, or Amazon’s logistical algorithms. The second type includes things like a thermostat, a heat-seeking missile, or water flowing downhill.

The distinction I want to point to here is where the things-we’re-optimizing-over live. For the first type, optimization is over things-in-a-model; a search algorithm runs on a map. For the second type, optimization is over things-in-the-world; a search algorithm runs on the territory directly. Search-in-map vs search-in-territory.

Same Algorithm, Different Inputs

A toy example: suppose we have a big pile of rocks, and we want to find the rock whose mass is closest to the mass of a reference weight (without going over).

A search-in-territory algorithm might use one of those old-school balance-scales to compare masses pairwise. We could pull each rock out of the pile one-by-one, and:

First, compare the rock to the reference weight. If it’s heavier, throw it away and move on to the next rock.
Second, compare it to the best rock found thus far, and replace the best rock with this one if it’s heavier.

At the end, we’ll have chosen the best rock.

A balance-scale. (

Source

)

A search-in-map algorithm might instead start by weighing the reference weight and each rock on a scale, and entering all the masses into a spreadsheet. But then, it could proceed exactly like the search-in-territory algorithm. It would run through the list of rock-masses one-by-one, and:

First, compare the rock-mass to the reference mass. If the rock-mass is larger, move on to the next one.
Second, compare the rock-mass to the best rock-mass found thus far, and replace the best rock-mass with this one if it’s larger.

At the end, there is one extra step: we have to go back out into the world and find the actual rock with the best mass. Note that this could be nontrivial, if we didn’t label or organize our rocks in the process of weighing them.

Key point of this example: these two types of optimization need not involve different algorithms. In practice they often do - things we typically think of as “search algorithms” tend to be used more for search-in-map, and things we typically think of as “controllers” tend to be used more for search-in-territory. But in principle, one can often apply the algorithms opposite their usual context.

When Sh

...