trentmkelly/LessWrong-43k huggingface dataset:
The Great Karma Reckoning
by habryka — 2021-01-15T05:19:32.447Z

Today the LessWrong team ran a script that recalculated the scores and karma of all posts, comments, tags and users based on the complete vote history. This has resulted in very substantial changes to all the karma scores on the site. Here are the biggest changes:

We removed the historical 10x multiplier for posts that were promoted to main on LW 1.0
This reversed the effect of a lot of voting-related bugs we had over the past few years
This reverts the period in early 2018 where all votes were as strong as current strong-votes are, which resulted in very large karma inflation during that time
All votes from before the new voting system with weak- and strong-votes are treated as weak-votes, but do now scale proportional to your karma (so a good number of 1-strength votes were converted to 2-strength votes)

You are welcome to strong-upvote posts that you voted on before strong-votes existed, if you want to give them a bit of a boost.

We've been planning on doing this for a long time, mostly to make historical comparisons of the scores of different posts easier and also have more confidence that current karma balances are not the result of bugs we fixed many months ago.

For the purpose of the 2019 review, you will be counted in the 1000+ vote bracket if you either had 1000+ karma before we ran todays migration, or if you just hit that target after today's migration. So nobody should suddenly be negatively surprised about their ability to participate in the vote.
Grey Goo Requires AI
by harsimony — 2021-01-15T04:45:35.905Z
Link: https://harsimony.wordpress.com/2020/10/05/grey-goo-requires-ai/

Summary: Risks from self-replicating machines or nanotechnology rely on the presence of a powerful artificial intelligence within the machines in order to overcome human control and the logistics of self-assembling in many domains.

The grey goo scenario posits that developing self-replicating machines could present an existential risk for society. These replicators would transform all matter on earth into copies of themselves, turning the planet into a swarming, inert mass of identical machines.

I think this scenario is unlikely compared to other existential risks. To see why, lets look that the components of a self-replicating machine.

Energy Source: Because you can’t do anything without a consistent source of energy.

Locomotion: Of course, our machine needs to move from place to place gathering new resources, otherwise it will eventually run out of materials in its local environment. The amount of mobility it has determines how much stuff it can transform into copies. If the machine has wheels, it could plausibly convert an entire continent. With a boat, it could convert the entire earth. With rockets, not even the stars would be safe from our little machine.

Elemental Analysis: Knowing what resources you have nearby is important. Possibilities for what you can build depend heavily on the available elements. A general purpose tool for elemental analysis is needed.

Excavation: Our machine can move to a location, and determine which elements are available. Now it needs to actually pull them out of the ground and convert them into a form which can be processed.

Processing: The raw materials our machine finds are rarely ready to be made into parts directly. Ore needs to be smelted into metal, small organics need to be converted into plastics, and so on.

Subcomponent Assembly: The purified metals and organics can now be converted into machine parts. This is best achieved by having specialized machines for different components. For example, one part of the machine might print plastic housing, another part builds motors, while a third part makes computer chips.

Global Assembly: With all of our subcomponents built, the parent machine needs to assemble everything into a fully functional copy.

Copies of Blueprint: Much like DNA, each copy of the machine must contain a blueprint of the entire structure. Without this, it will not be able to make another copy of itself.

Decision ...
RationalWiki on face masks
by Viliam — 2021-01-15T01:55:49.836Z

Thinking about freedom of speech and the latest "purges" on social networks, my thoughts are like this: I prefer freedom of speech even for people like homeopaths and anti-vaxers, not because I consider their opinions to be inherently valuable, but because a decision algorithm that would ban them, would probably also have banned Ignác Semmelweis two centuries ago.

Then I thought again and realized I don't actually need such an old example. A decision algorithm that would today ban people who say "COVID-19 is just a flu" would have one year ago banned people who advised wearing face masks, wouldn't it?

And then I was like: I wonder how RationalWiki approached the topic of face masks... you know, a year ago. Clicking on their history and searching for "mask" -- between February 24th and April 21st, the only mentions of "mask" on the "COVID-19 pandemic" page were:

A "DON'T PANIC" button at the top of the page, subtitled: "Helpful advice on social media stories about COVID-19 from Douglas Adams… and from professional epidemiologists." with reference linking an article called: How to prepare for coronavirus in the U.S. (Spoiler: Not sick? No need to wear a mask.).
A paragraph on Hal Turner's exaggeration of COVID-19 impact on China, ending with: "For believers of this nonsense, Hal is happy to direct you to vendors for surgical masks (that in reality will not be helpful against this virus) [note 4] and MREs.". The footnote said: "Most experts believe that common surgical masks will not do any good for preventing this particular virus. The virus may even be too small in size for heavier duty N95 respirators to filter it out effectively." and linked an article called: People are racing to buy face masks amid the coronavirus outbreak, but they probably won't protect you from illness.

The following edit updated the section containing criticism of Brazilian president Jair Bolsonaro, concluding with: "Journalists noted that he neither wore a face mask (even though he was seen coughing on several occasions) nor gloves, precautions which other world leaders have taken."

Not sure if that implies that masks are okay when you are already infected, or that masks are bad for plebs but you are supposed to wear them if you are a world leader, or perhaps that if you are a bad guy then either wearing or not wearing the mask is a thing to be condemned.

On May 19th, a short ed...
Any rationalist judges?
by kjz — 2021-01-14T23:31:13.246Z

Are there any rationalist judges, or anyone studying with the intention to become a judge? It seems like an area where rationalist thinking could have a broader impact. For example, judges often have some discretion when deciding prison sentences and other punishments. They also have the ability to give detailed explanations of their judgements. Could a judge, through wise application of rationalist principles, influence the judicial system in a positive direction?
Covid 1/14: To Launch a Thousand Shipments
by Zvi — 2021-01-14T17:30:01.468Z

Pardon me while I make my way to the rooftops.

So I’m sure it’s not that simple especially because of regulatory issues, but… did you hear the one where humanity could have produced enough mRNA vaccine for the entire world by early this year, and could still decide to do it by the end of this year, but decided we would rather save between four and twelve billion dollars?

If not, there’s a section on that.

Meanwhile, we also can’t figure out how to put the vaccine doses we already have into people’s arms in any reasonable fashion. New policies are helping with that, and we are seeing signs that things are accelerating, but wow is this a huge disaster.

We took some steps this week towards sane policy. Everyone over 65 is eligible in most places due to new CDC guidance. All doses we have will be distributed rather than reserved. Distribution will be based on the speed at which existing doses are put into arms. Mass vaccination centers are coming online. One can be hopeful for the path of future policy. At a minimum, it’s a start.

In the near term, deaths are way up, the majority of which is almost certainly a real increase, and they probably won’t peak for another week or two. Positive test rates are down a little, but that’s explained by the thankfully rising test counts, so we have not yet meaningfully turned the corner after Christmas.

We likely won’t have that much time to do so before things get worse again. The English strain is definitely coming. It looks like it won’t be as bad as it looked like it might be, so we’ll have some more time and the end result won’t be as severe, and it looks like England has been able to use sufficiently extreme measures to contain it in the last few days, but it’s definitely here, definitely spreading and definitely going to make things a lot worse.

Let’s run the numbers.

The Numbers

Predictions

Prediction last week: 17.0% positive rate on 9.5 million tests, and an average of 2,800 deaths. The holidays are over, there will be some fallout, with things getting slightly worse, but with the main boost in deaths from Christmas mostly coming later.

Results: 15.2% positive rate on 11 million tests, and an average of 3,325 deaths.

This guess turned out to be backwards. We got a lot more rise in the death rate than I guessed. The positive test rate improved as we cleared past the holidays and got more testing online, ...
Thoughts on Iason Gabriel’s Artificial Intelligence, Values, and Alignment
by Alex Flint — 2021-01-14T12:58:37.256Z

Iason Gabriel’s 2020 article Artificial Intelligence, Values, and Alignment is a philosophical perspective on what the goal of alignment actually is, and how we might accomplish it. In the best spirit of modern philosophy, it provides a helpful framework for organizing what has already been written about levels at which we might align AI systems, and also provides a neat set of connections between concepts in AI alignment and concepts in modern philosophy.

Goals of alignment

Gabriel identifies six levels at which we might define what it means to align AI with something:

Instructions: the agent does what I instruct it to do

Expressed intentions: the agent does what I intend it to do.

Revealed preferences: the agent does what my behaviour reveals I prefer.

Informed preferences or desires: the agent does what I would want it to do if I were rational and informed.

Interest or well-being: the agent does what is in my interest, or what is best for me, objectively speaking.

Values: the agent does what it morally ought to do, as defined by the individual or society.

Schemas like this are helpful because they can "pop us out" from our unexamined paradigms. If we are, for example, having a discussion about building AI from inside the "revealed preferences" paradigm, it is good to know that we are having a discussion from inside that paradigm. It is a virtue of modern philosophy to always be asking what unexamined paradigm we are inside, and to be pushing us to at least see that that we are inside such-and-such a paradigm, in order that we can examine it and decide whether to keep working within it.

In that spirit, I would like to offer a conceptualization of the paradigm that I think all six of these levels are within, in order that we might examine that paradigm and decide whether we wish to keep working within it. It seems to me that we presuppose that when we deploy AI, we will pass agency away from humans and into the hands of the AI, at least for as long as it takes the AI to execute our instructions, intentions, preferences, interest, or values. We imagine our future AIs as assistants, genies, or agents with which we are going to have some initial period of contact, followed by a period during which these powerful agents go off and do our bidding, followed perhaps by later iterations in which these agents come back for further instructions, intentio...
Genetic Engineering: How would you change your DNA?
by Felix Karg — 2021-01-14T11:43:35.160Z

Recently, genetic engineering and related disciplines gained fame through capacities granted by gene therapy. It appears that it should be possible to fully and freely modify our own genome at some point, maybe even within the next ten years. So I was wondering — if we could change our genome, what would we want to change?

Question: If we had the ability to perfectly genetically engineer ourselves, with no chance of failure:

What would you change?
Why would you change it? (if not apparent)
What would you never change?
What do you think others will do?

Please refrain from discussions about ethics. Broad answers suffice.
Discussion on the choice of concepts
by KatjaGrace — 2021-01-14T04:00:13.283Z

“The reason that you can currently make toast without doing great damage is just that your toaster is stupid.”

“Can ‘stupid’ be correctly applied to toasters?”

“Yes”

“What if I say no?”

“Well, if you have a conception of stupidity that can’t be applied to toasters, and one that can, why would you choose the one that can’t?”

“But I don’t have two—I’m talking about the actual concept”

“There isn’t an actual concept, there are a bajillion concepts, and you can use whichever one you want.”

“There’s one that people mean”

“Not really—each person has a slightly different usage, and probably hasn’t pinned it down. For instance if you ask them if toasters are stupid, they might be unsure.”

“Yes! They are unsure because they are trying to guess what the real concept is, from their limited collection of exposures to it. If it were about making one up, why would they be uncertain?”

“They might be uncertain which one they want to make up”

“You’re saying when people say words, they are ascribing meanings to them that they just made up, according to which definition they like most?”

“Well, they like definitions that fit with other people’s usage a lot more than other ones.”

“I think they are just guessing what the real meaning is”

“There isn’t a real meaning”

“Ok, what the consensus meaning is”

“There isn’t a consensus”

“Yeah but they believe there is one”

“You’re like a word meaning nihilist—you want only these ‘real’ word meanings or at least these word meanings of consensus, yet you know they don’t exist. That seems sad.”

“Maybe, but that doesn’t make it wrong. And also, I was talking about what other people do.”

“What does it matter what other people do? You can use whatever meanings you want.”

“That seems unfriendly somehow”

“What if you do it in a friendly way? For instance, where a meaning is ambiguous, if you choose the best one. For instance, if you say toasters can be stupid?”

“It’s more a vibe of do-it-alone responsibility for everything, thinking of others as machinery that happens to be near you, that rings my alarm bells. Leaving the common experience of word usage to stand outside the system, as it were, and push the common stock of concepts in the way that you calculate best. At least it seems somehow lonely and cold”

“That’s a bit dramatic - I think the odd nudge in a good direction is well within the normal human experience of word usage. Plus, often people clearly redefine word...
Some recent survey papers on (mostly near-term) AI safety, security, and assurance
by Aryeh Englander — 2021-01-13T21:50:40.089Z

As part of a project I am doing at work, I took a look around to find recent overview / survey / literature review papers on several topics related to AI safety, security, and assurance. The focus was primarily on nearer-term issues, but with an eye towards longer-term risks as well. Several of the papers I found will be familiar to many people here, but others might not be, so I am sharing this list in case it is helpful to anybody else.

Note that this is not intended at all to be a comprehensive list even for the topics covered. I tried to find between 1 and 4 overview or survey papers for each topic, but no more than that - I and/or someone on my team will need to actually read through all these, after all, so I didn't want too many papers. I did not include blog posts or book-length sources, such as Russell's Human Compatible or Christian's The Alignment Problem. I also mostly did not include papers from earlier than a year or two ago, which is why it might look a bit odd that I haven't included some of the more well-known earlier papers.

The specific topics in this list are related to the project I'm doing at work, so I only focused on trying to find papers related to these topics. However, I am also very interested in recent survey / lit review papers on other related topics, so if you know of any good ones please link to them in the comments. And of course, if you know of other good survey / lit review papers on these specific topics, please let me know about those.

General

Toreini, et al (July 2020), Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context (mainly focused on ethics but also including safety & security)
Huang, et al (2018), A Survey of Safety and Trustworthiness of Deep Neural Networks: Verification, Testing, Adversarial Attack and Defence, and Interpretability
Osoba, et al (RAND, Feb 2020), Steps Towards Value-Aligned Systems
On ML in smart cities: Ahmad, et al (Dec 2020), Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges

Side effects

Saisubramanian, et al (Aug 2020), Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems

Out of distribution (OOD) robustness

Koh, et al (Dec 2020), WILDS: A Benchmark of in-the-Wild Distribution Shifts (benchmark paper, but with several sections that provide an overview)
Related Work se

...
What is the currency of the future? 5 suggestions.
by MrThink — 2021-01-13T21:10:34.005Z

(Disclaimer: I am not an expert, and this article is for entertainment purposes only, it is not financial advice.)

This post describes five types of currencies that I believe we might see in the future: debt based, commodity/service based, bonds based, stock based, brand-based.

The goal of the post is not to argue which types are the best, the goal is simply to describe the different types.

1. Brand based currency (Currencies based on nothing)

Brand based currencies (not an established term) is money that is not based on anything, it exists, and people might use it, but the value is entirely based on people believing it. In a way the money has a brand that people believe is trustworthy, and will uphold a certain value. Many cryptocurrencies like Bitcoin and Litecoin fall into this category. These currencies could be replicated, but still some people seem to believe in the brands, that they will uphold a certain value.

It could be argued that gold is a brand based currency, it is mostly the belief that gold will keep its value that makes it such a common investment. (Gold will also be discussed in commodity based currencies)

2. Debt based currencies (ex Fiat)

Debt based currencies are currencies like US dollars, euro and yen. These currencies have been created by an actor (often Central banks like the Federal Reserve) issuing loans to other actors like governments and banks. The currency in itself is not tied to anything with intrinsic value, but since the actors who took the loans need to repay them, there is an interest in accumulating it.

For example, if all people suddenly decided that a currency was useless, still the banks who had loans in that currency would need to repay the debt, or break the law and face bankruptcy.

3. Commodity/service based currencies

These are currencies backed by commodities (like corn, oil, batteries, GPUs, Land) or services (like GPU time, leased real estate, or file storage).

Gold based currencies would fall under this category as well.

Often these types of “currencies” are in the form of futures, where there is a contract to buy a certain commodity or service at a certain price. Meaning if you buy an oil future, you are promised to get that oil at a specified date in the future. Since it works as a loan that is repaid in commodities, futures “should” increase in value over time.

Real estate can fall under either commodities o...
Notes on Gratitude
by David Gross — 2021-01-13T20:37:30.004Z

This post examines the virtues of appreciation, gratitude, and reciprocity. It is meant mostly as an exploration of what other people have learned about these virtues, rather than as me expressing my own opinions about them, though I’ve been selective about what I found interesting or credible, according to my own inclinations. I wrote this not as an expert on the topic, but as someone who wants to learn more about it. I hope it will be helpful to people who want to know more about these virtues and how to nurture them.

What are these virtues?

The sort of “appreciation” I want to explore here is appreciation-of-others specifically (not, for example, aesthetic appreciation). When you appreciate someone for something they have done that benefits you, we sometimes call this sort of appreciation “gratitude.” However, you can appreciate someone without necessarily feeling gratitude (for example, you might appreciate someone’s sense of humor but not think that it makes sense to thank them for it).

In practice, there’s a lot of overlap between gratitude and appreciation, and the border between them isn’t very well-defined. In this post, I’ll use one or the other of “appreciation” or “gratitude” if the distinction seems to matter, and I’ll use either of them if it doesn’t seem to matter. If that’s confusing, leave a comment and I’ll try to clarify.

When we return a favor, we are doing a specific sort of gratitude-adjacent action that goes by the name “reciprocity.”

To rise to the level of virtues, appreciation, gratitude, and reciprocity should be habitual: in other words, you have the virtue of appreciation if it is characteristic for you to notice opportunities for appreciation, to then feel appreciative, and to follow through by skillfully performing appreciative actions.

Related virtues & vices

There are some other virtues that have to do with recognizing others, including respect, remembrance, honor, consideration, recognition, and solidarity. The ability to notice opportunities for appreciation requires attention. The skill of “savoring” can also be part of a good appreciative sense. People are more likely to express gratitude for benevolent acts they regard as intentional and not shallowly self-interested, and it may require some empathy to discern these things.

The skill of accepting appreciation or gratitude gracefully (rather than with deflection or false-mode...