trentmkelly/LessWrong-43k huggingface dataset:
D&D.Sci 4th Edition: League of Defenders of the Storm
by aphyer — 2021-09-28T23:19:43.916Z

STORY (skippable)

When you graduated top of your class from Data Science School, you didn't care where you ended up working, you just wanted to find the highest-paid job possible. (Your student loans may have had an impact on this decision).

You were expecting a job on Wall Street, or perhaps some Silicon Valley firm. You were not expecting for Goldman Sachs to be outbid at the last minute by a South Korean e-sports team, looking for a 'data specialist' to assist them in winning tournaments of the entirely original new game 'League of Defenders of the Storm©.'

Your new employers at Cloud Liquid Gaming seem friendly enough. They show you their database of games, and tell you they're looking for assistance in selecting their team for an upcoming tournament.

Then you make the mistake of asking them how the game works. All of them start talking at once:

"Okay, so the first thing you need to understand is that if your HK carry isn't able to scale to swamp fights you won't be able to contest Count Shorna's Curse-"

"No, first you need to explain how itemization works, and how ever since Mike's Malign Maul got nerfed to deal with the Grapeshot Gloom build it's been-"

"You have to start from the beginning! You are all Callers, resolving your disputes by summoning spirits to do battle for-"

"Oh come on, why do we need to care about the lore, it's-"

Things do not get more useful from there. Half an hour later, with very little new information and a gigantic headache, you excuse yourself to look at the data. Perhaps you can get something useful out of that without having to listen to them explain every detail of the game. Apparently they have an important game tomorrow they're looking for advice on, and you're interested to make a good impression on your new employers.

DATA & OBJECTIVES

You've managed to learn a few basic things about how the game works:

Each team simultaneously selects 5 characters.
The same character can be selected by both teams.
The same character cannot be selected twice by the same team.
Your employers are confident they know which characters their opponents will play at their next game:

Dire Druid
Greenery Giant
Phoenix Paladin
Quartz Questant
Tidehollow Tyrant

They would like you to help them pick out a set of 5 characters that maximizes their chances of winning against that enemy team. You may select any 5 o

...
Cryosleep
by harsimony — 2021-09-28T22:33:34.817Z
Link: https://harsimony.wordpress.com/2021/08/04/cryosleep/

(Cross posted from my blog)

Epistemic Status: Unfounded speculation about a far future technology

Would time travel by any other name smell just as sweet? Apparently not, since few people are interested in developing cryonics technology (funny enough, antifreeze does smell sweet). Fundamentally, successful cryonics would offer the ability to travel forward in time, a fact which is often overlooked.

It’s a fascinating idea which would open up a lot of new opportunities. Though there is plenty of debate about whether or not it makes sense to sign up for cryonics, virtually nobody (even the cryonics companies!) discusses the benefits to society if it became viable.

For the purposes of this post, I will define “cryosleep” as both the successful preservation and resurrection of a person. Cryonics today focuses mostly on preservation, and further work is needed to successfully resurrect people.

Imagine if being successfully preserved and resurrected was as safe as driving a car. It would change everything.

Cryosleep would allow people to travel into the future with ease. This is already an amazing benefit, and it is fun to consider how far into the future you might choose to travel.

The other benefits of cryosleep follow from freeing people of the constraint that they must live within a particular time. For example, cryosleep opens up the possibility of interstellar travel without requiring extreme speeds. A person can simply set a course for a distant world, preserve themselves, and wake up when they arrive.

Even on earth, there would be several huge benefits. For one, individuals with terminal illnesses can wait for a life saving therapy to be developed. Anyone can await the development of life extension technologies postponing their death until immortality is invented [1].

People can also wait for better living conditions. Individuals with particular skills can sleep until those skills are in high demand. They can also wait for exciting opportunities to work. One might imagine a great artist waiting to wake up until they are commissioned for a large project. This has the effect of reducing the relative the cost of living, as preservation is cheap and individuals only work when their wages are high [2].

This self-selection has interesting effects on culture. The option to leave your current time period would concentrate productive, happy, and altruistic people in the pres...
You can talk to EA Funds before applying
by evhub — 2021-09-28T20:39:57.427Z

Cross-posted to the EA Forum.

One thing I have realized as a fund manager for the EA Long-Term Future Fund is that there are a lot of grants that I would like to make that never cross our desk, just because potential applicants are too intimidated by us or don’t realize that their idea is one we’d be willing to fund if only they applied.

To try to remedy that problem, I’m going to start offering the following service: if you have any idea of any way in which you think you could use money to help the long-term future, but aren’t currently planning on applying for a grant from any grant-making organization, I want to hear about it. Feel free to send me a private message on the EA Forum or LessWrong. I promise I’m not that intimidating :)

Not only that, but having talked about this with some of the other EA Funds managers, many of them were willing to extend the same offer as well:

For the Animal Welfare Fund, just send an email to Kieran Greig.

For the EA Infrastructure Fund, both Michelle Hutchinson (via EA Forum or email) and Michael Aird (currently a guest manager; can be contacted via EA Forum, LessWrong, or email) would be happy to chat.

For example, here are some of the sorts of grants I’m often excited about but that I rarely see anyone apply for:

“I want to transition to a career in something longtermist, but that transition would be difficult for me financially and I’d like to have some extra financial reserves to make it easier.”

“I think I would be more productive in my longtermist job if I had more money to spend on things that would save me time.”

“I have an idea for a longtermist project I want to work on, but I don’t want to commit to definitely working on that project for the duration of a long grant and want freedom to change my mind and switch to a different project if I want.”

“I have an ambitious idea for a project that I think would benefit the long-term future, but I think it would take a lot of money, more than what I normally see LTFF grants being given out for.”

Really, though, I don’t want to anchor anybody too much on these specific ideas—if you have any idea of any way in which you think you could use money to help the long-term future, I want to hear about it.
Collection of arguments to expect (outer and inner) alignment failure?
by Sam Clarke — 2021-09-28T16:55:28.385Z

Various arguments have been made for why advanced AI systems will plausibly not have the goals their operators intended them to have (due to either outer or inner alignment failure).

I would really like a distilled collection of the strongest arguments.

Does anyone know if this has been done?

If not, I might try to make it. So, any replies pointing me to resources with arguments that I've missed (in my own answer) would also be much appreciated!

Clarification: I'm most interested in arguments that alignment failure is plausible, rather than merely that it is possible (there are already examples that establish the possibility of outer and inner alignment failure for current ML systems, which probably implies we can't rule it out for more advanced versions of these systems either).
Problems with using approval voting to elect to a multi-individual body?
by RyanCarey — 2021-09-28T11:03:19.282Z

Suppose a nation is electing k individuals, to populate a senate or something. Now approval voting has some good properties, so suppose the protocol is to let each of the m citizens vote for as many candidates as they want. We then award a seat to the candidate with the most votes, and deduct m/k from their vote tally, or thereabouts. And then we repeat the process using the new tallies, until we have k elected candidates. It seems like a problem with this approach is that if the most popular candidate could copy themselves k-1 times, then their copies would all have the same score as themselves, and so they would completely monopolise the senate. In a very real, practical sense, this protocol would mean that one political party could dominate the senate. This would undercut the ability of the senate to combine disparate views, and to represent the diversity of opinions of the population, as well as reducing its robustness to takeover by one group.

My questions are: 1) is this how approval voting is proposed to be used for multi-individual body, and 2) is there any suitable proposal that would retain desirable properties of approval voting (simple interface, and attaining high utility) while avoiding this problem?
Sunscreen: much more than you wanted to know
by Pierre Vandenberghe — 2021-09-28T10:42:53.948Z
Link: https://pierreavdb.substack.com/p/sunscreen-much-more-than-you-wanted

I’ve seen so many different takes on sunscreen that I had to do something about it. Here is my best effort to summarize the current research.

What does sunscreen do?

Without getting into technical details, sunscreen reflects or scatters UV radiation (mainly from the sun), offering protection against sunburn (Norval & Wulf 2009).

Consequently, sunscreen also reduces “the severity of solar elastosis, the development of solar keratoses, new naevi, SCCs and cutaneous photodamage, and the reactivation of herpes labialis,” all of which are pretty nasty things caused by overexposure to the sun.

With that out of the way, let's move on to a bigger question.

Is sunscreen really effective against skin cancer?

There are different types of skin cancer. In this section, we'll focus on malignant melanoma (“the skin cancer with increased risk of death” Lindqvist & Olsson 2016). The current consensus is that overexposure and episodic sunburns are the leading causes of melanoma. Sunscreen seems to be effective in preventing melanoma in white children (Gallagher et al., 2000). However, as the author puts it, melanoma is rare and that makes it hard to study.

So, sometimes, we see strange findings.

There's one paper that everyone keeps talking about. Between 1988 and 1990, Westerdahl et al. compared 400 melanoma patients to 640 healthy “controls.” The researchers asked them about their sunscreen use and personal factors. They found that not only was sunscreen ineffective against melanoma but there was also a positive link between them! Could sunscreen be harmful and cause skin cancer?

Well, not really.

Westerdahl and his colleagues provide several hypotheses for how to explain these surprising results:

They tried to control for sun exposure (people who use more sunscreen are more in the sun generally and thus more at risk), but they admit their method was not perfect. More on this later.

Sunscreen usually protects only one kind of UV radiation, and other types might be dangerous. More on this later.

Maybe sunscreen contains carcinogenic agents? More on this later.

"Sunscreen may increase melanoma risk by interfering with vitamin D synthesis." More on this later.

First off, it sounds logical that people who use more sunscreen are more in the sun. You don't apply sunscreen before locking yourself in your basement to play D&D. So, people wh...
Selection Theorems: A Program For Understanding Agents
by johnswentworth — 2021-09-28T05:03:19.316Z

What’s the type signature of an agent?

For instance, what kind-of-thing is a “goal”? What data structures can represent “goals”? Utility functions are a common choice among theorists, but they

don’t seem quite right

. And what are the inputs to “goals”? Even when using utility functions, different models use different inputs -

Coherence Theorems

imply that utilities take in predefined “bet outcomes”, whereas AI researchers often define utilities over “world states” or “world state trajectories”, and

human goals seem to be over latent variables in humans’ world models

.

And that’s just goals. What about “world models”? Or “agents” in general? What data structures can represent these things, how do they interface with each other and the world, and how do they

embed

in their low-level world? These are all questions about the type signatures of agents.

One general strategy for answering these sorts of questions is to look for what I’ll call Selection Theorems. Roughly speaking, a Selection Theorem tells us something about what agent type signatures will be selected for (by e.g. natural selection or ML training or economic profitability) in some broad class of environments. In inner/outer agency terms, it tells us what kind of inner agents will be selected by outer optimization processes.

We already have many Selection Theorems:

Coherence and Dutch Book theorems

,

Good Regulator

and

Gooder Regulator

, the

Kelly Criterion

, etc. These theorems generally seem to point in a similar direction - suggesting deep unifying principles exist - but they have various holes and don’t answer all the questions we want. We need better Selection Theorems if they are to be a foundation for understanding human values, inner agents, value drift, and other core issues of AI alignment.

The quest for better Selection Theorems has a lot of “surface area” - lots of different angles for different researchers to make progress, within a unified framework, but without redundancy. It also requires relatively little ramp-up; I don’t think someone needs to read the entire giant corpus of work on alignment to contribute useful new Selection Theorems. At the same time, better Selection Theorems directly tackle the core conceptual problems of alignment and agency; I expect sufficiently-good Selection Theorems would get us most of the way to solving the hardest parts of alignment. Overall, I think they’re a ...
Beware Superficial Plausibility
by Celer — 2021-09-28T03:58:08.961Z

Andrew Wakefield was funded by some people suing vaccine manufacturers to publish some extremely damaging research. With a sample size of twelve people, he “found” that the Measles-Mumps-Rubella vaccine

caused autism

, kicking off the modern

anti-vax movement

.

In most places where I spend time, anti-vaxxers are an easy target. They’re not just wrong: they’re harming people. And when we desist, it’s because we realize that spending time mocking people who are obviously wrong is bad for us (

here’s Scott for an example

).

“You need to do your research”: Vaccines, contestable science, and maternal epistemology is a fascinating article because it’s a sympathetic view of the “critical, postmodern, and feminist perspectives on science and knowledge production” by anti-vax mothers. The author has a Ph.D. in Communications and an MA in English Lit, Critical Theory, and Women's Studies: she's sympathetic to their concerns and, when she talks about multiple sources of knowledge and contesting scientific explanations of the world, is familiar with a compatible intellectual tradition. And with quotes from her article, I’m going to try to demonstrate just how divorced reasoning processes can be from outcomes by showing how people can land on very familiar views that I expect many of my readers to share, while also drawing from painfully bad epistemology.

Epistemic status: this is a qualitative study with a sample size of 50 people. Worse, it’s someone sympathetically reporting these mothers’ framings to an audience, instead of giving us access to the raw data. Take everything here with several grains of salt.

Participants in this study lacked confidence in the quality of much published research on this topic in light of traditional scientific standards.

Arguments maintained the ideal of expert knowledge but challenged particular claims to expertise

Arguments from participants were not based in a contention that participants had the right to be included in the technical sphere but rather suggested that the standards of the technical sphere had become too lax. This distinction is important because it challenges the commonplace assumption that vaccine refusers believe in pseudoscience as definitive. Instead, participants in this study lacked confidence in the quality of much published research on this topic in light of traditional scientific standards.

“My big thing is when, yo

...
Inner Belt
by jefftk — 2021-09-28T03:50:06.954Z

A friend recently wrote in a local group that "the disastrous Inner
Belt 'redevelopment' removed thousands of homes", and if you look at
the Inner Belt area today it does look like it could have been a
victim of urban renewal:

While highways (93, the Central
Artery, McGrath)
and urban renewal (West End)
destroyed many nearby residential areas, and Inner
Belt is named after a cancelled
highway project, that's not the history of this particular area.

Colonial Boston was surrounded by tidal mudflats and wetlands, with
far less dry land than we have today:

Wheeler, 1775

The area we now call Inner Belt was part of the estuary that separated
Charlestown from Cambridge and Boston. You can see the first step
toward filling it in with the 1833 Hales
Map showing the Boston
and Lowell Railroad crossing the Charles, just east of Craigie
Bridge:

Over the next few decades more railroads cut through:

Boston
and Northwestern Junction, 1879

By 1897 the areas between the tracks had been thoroughly filled in to
build out a railyard for the Boston
and Maine Railroad, which had grown to include the Boston and
Lowell:

(This is around when McLean, then
called the 'Asylum for the Insane', moved from the northernmost part
of this area out to Belmont.)

In 1938 we get the first good aerial photos of metro Boston, and you
can see that Inner Belt, and the area just South, are full of tracks:

1955 is similar, if a bit higher resolution:

By 1969, large parts of the northern part of the yard, today's Inner
Belt, have been replaced with trucking-focused distribution centers:

This is essentially what's there today. This was never a residential
area, and the idea that thousands of homes were destroyed to make way
for these buildings is a myth.
Blood Is Thicker Than Water 🐬
by Zack_M_Davis — 2021-09-28T03:21:53.997Z

Followup to: Where to Draw the Boundaries?

Without denying the obvious similarities that motivated the initial categorization {salmon, guppies, sharks, dolphins, trout, ...}, there is more structure in the world: to maximize the probability your world-model assigns to your observations of dolphins, you need to take into consideration the many aspects of reality in which the grouping {monkeys, squirrels, dolphins, horses ...} makes more sense.

The old category might have been "good enough" for the purposes of the sailors of yore, but as humanity has learned more, as our model of Thingspace has expanded with more dimensions and more details, we can see the ways in which the original map failed to carve reality at the joints ...

So the one comes to you—a-gain—and says:

Hold on. In what sense did the original map fail to carve reality at the joints? You don't deny the obvious similarities between dolphins and fish—between dolphins and other fish. That's a cluster in configuration space! The observation that dolphins are evolutionarily related to mammals may be an interesting fact that specialized professional evolutionary biologists care about for some inscrutable specialist reason. But I'm not a professional biologist. Choosing to define categories around evolutionary relatedness rather than macroscopic human-relevant features seems like an arbitrary æsthetic whim. Why should I care about phylogenetics, at all?

This one is going to take a few paragraphs.

Focusing on evolutionary relatedness is not an arbitrary æsthetic whim because evolution actually happened. Evolution isn't just a story that our Society's specialists happen to have chosen because they liked it; they chose it because it predicts what we see in the world. You can't choose a substantively different theory and make the same predictions about the real world. (At most, you'd end up with an isomorphic theory with additional epiphenominal elements, asserting that an allele rose in frequency "because" the angels willed it, without an account of why the angels' will happens to line up with what would have transpired if there were no angels.) Similarly, category definitions represent hidden probabilistic inferences; you can't "redraw" the "boundaries" of the categories your mind actually uses and still make the same predictions about the real world. Accordingly, it shouldn't be surprising that our knowled...
Is progress in ML-assisted theorem-proving beneficial?
by mako yass — 2021-09-28T01:54:37.820Z

I want to believe that the outputs of the recently founded Hoskinson Center for Formal Mathematics (led by Jeremy Avigad, researcher of mathematical logic, automated reasoning and philosophy of mathematics), will live up to Mr Hoskinson's aura of technical responsibility.

One of the stated goals of the HCFM is to make mathematics more legible (both between mathematicians, and to non-mathematicians), and since mathematics is itself a motor of legibility in software systems and physical engineering, maybe new techniques in automated proving will make problems in AI motivation more legible too. That would be really great.

On the other hand, I don't know how to place bounds on the set of problems that could be solved with superhuman mathematical reasoning. HCFM is likely to produce a broad set of unexpected capabilities, and I can't see a way to exclude the problem of real-world, agentic utility optimization from that set.

One of the limitations of neural models right now is that they have only very limited kind of memory, they're mostly just reflexes, very wise ones, evolved over the course of, often the equivalent of tens of thousands of years of condensed human experience, but none of these wise reflexes so far can evolve a set of coherent thoughts over time, pretty much everything they do is an instant reaction to the current state of their environment.
GPT3 was startling because words are a type of memory, and our writing often depicts evolving thoughts, so sometimes it verges on evolving thoughts, but it was a limited kind of thinking because words are imprecise and most of the training data was rambly and inconsequential. That said, I don't know what GPT3's limits would turn out to be after, for instance, being subjected fully to Anthropic's inspection technique, and the tuning technique that will grow from that.
Well, mathematical claims constitute an extremely precise and robust format for the encoding of memories. In the training data, formal proofs, reference correlates closely with salience; the components are cited, new and interesting fruits always follow from them. To evolve the ability to predict the conclusions that follow the assumptions is equivalent to evolving the mind of a mathematician, in their prime.
Each mathematical result is a kind of evolution of a kind of thought, older thoughts evolving creatively and usefully into new ones. For me, ther...