trentmkelly/LessWrong-43k huggingface dataset:
Is there a way to test your personal COVID vaccine response? How do corticosteroids impact response? 
by Sameerishere — 2021-06-03T05:08:14.776Z

Anyone have insight on (1) Whether there is a way to test how effective an immune response you had to the vaccine or (2) the extent to which inhaled glucocorticoids (specifically Symbicort) impact the effectiveness of the mRNA COVID vaccines (specifically Pfizer)?

On (1) there doesn't seem to be any useful test commercially available.

This 3/29/21 article from MD Anderson says:

"The Centers for Disease Control and Prevention (CDC) discourages antibody testing

for assessing immunity

after getting the vaccine. A vaccinated person is very likely to get a negative result from a serology test, even if the vaccine was successful and protective. That’s because different serology tests detect antibodies to different parts of the virus. Some tests detect antibodies to the spike protein of the virus, which are produced in response to viral infection or the vaccine."
(It's unclear why they / the CDC then refrain from suggesting that people use those latter tests.)

This April 2021 article from Cleveland.com (citing an expert at Cleveland Clinic) says:

"There are tests that can provide some information about someone’s immune response to the vaccine, but those tests are conducted by labs and are not available for anyone to buy, said Dr. James Fernandez, an allergy and immunology expert at the Cleveland Clinic.The lab tests can detect the level of Immunoglobulin G (IgG), the most common type of antibody that protects against infection, in someone’s blood. The problem is that experts don’t know what level of IgG is needed to constitute a “protective level,” Fernandez said.“We can’t faithfully or accurately or actively say what that means to you, [whether] you’re protected or not protected,” Fernandez said. “It’s just a number.”"

Apparently there are solutions to this under development, but not commercially available (or even past peer review):

https://www.npr.org/sections/health-shots/2021/04/28/991651498/new-blood-tests-should-show-how-long-a-covid-19-vaccine-will-protect-you

https://news.northwestern.edu/stories/2021/03/home-blood-test-for-covid-19-antibodies/

On (2): It seems inhaled corticosteroids should be fine:

The AAAAI emphatically says: "No, there is no impact on an individual’s ability to respond to the vaccine and control of asthma is essential! There is no data to suggest that inhaled corticosteroids and/or leukotriene receptor antagonists impact...
Summaries of uncertain priors
by skybrian — 2021-06-03T02:43:15.726Z

In Bayesian jargon, people talk about having a "flat prior" when they mean being very uncertain about a question. But more often you just give a probability that you think a question is true. We don't normally draw probability distributions to share with each other.

I'm wondering if there's a natural way to talk not just about the probability you think something is true, but an estimate of your confidence, in some quantitative way? And what would it mean to be "well-calibrated" in your uncertainty?
For example, I might be quite confident that a particular coin is fair and will come up heads 50% of the time (because I've gathered a lot of data), while being much less confident about another 50% bet even though I think it's as likely as not.
Attributions, Karma and better discoverability for wiki/tag features
by jimrandomh — 2021-06-02T23:47:03.604Z

LessWrong has an associated wiki, tightly integrated with the tagging system. Today, we have just shipped a set of new features to make the LessWrong wiki better:

1. Voting and karma for wiki edits

You can now vote on wiki-edits on the tag-history page, in the Recent Discussion section, and on the All-Posts page:

This now properly provides karma incentives for improving the wiki.

2. Wiki/tag edits appear on the All Posts page, as do comments on wiki/tag discussion pages

The All-Posts page now also shows wiki edits on the daily page! This now allows you to much more easily keep track of the wiki-editing activity on the site:

You can click on them to expand and see the full diff, as well as vote on any edits.

3. Wiki/tag pages have a table of contents, like post pages.

Wiki and tag pages now have a ToC like post pages. This should make it much easier to navigate long wiki pages, like the Rationality tag:

4. Attributions and contributors on tag pages

I am particularly excited about this one. Just below the ToC on tag pages you can see a list of all contributors to a given tag/wiki page:

The number on the left is the total karma these authors have received for their contributions to this tag page, plus their small-vote strength (this also determines the order of the list of contributors).

But more importantly, when you hover over the author, you get to see which parts of the current tag page where written by them!

The goal is to make it more engaging to produce timeless content, make contributing to the wiki more motivating, and to make it easier to decide which wiki pages are worth reading.

(Looking for some of that wiki-edit karma? Check out the FAQ and the Wiki-Tag Dashboard).
Thoughts on the Alignment Implications of Scaling Language Models
by leogao — 2021-06-02T21:32:08.555Z

[Epistemic status: slightly rambly, mostly personal intuition and opinion that will probably be experimentally proven wrong within a year considering how fast stuff moves in this field]

This post is also available on my personal blog.

Thanks to Gwern Branwen, Steven Byrnes, Dan Hendrycks, Connor Leahy, Adam Shimi, Kyle and Laria for the insightful discussions and feedback.

Background

By now, most of you have probably heard about GPT-3 and what it does. There’s been a bunch of different opinions on what it means for alignment, and this post is yet another opinion from a slightly different perspective.

Some background: I'm a part of EleutherAI, a decentralized research collective (read: glorified discord server - come join us on

Discord

for ML, alignment, and dank memes). We're best known for our ongoing effort to create a GPT-3-like large language model, and so we have a lot of experience working with transformer models and looking at scaling laws, but we also take alignment very seriously and spend a lot of time thinking about it (see here for an explanation of why we believe releasing a large language model is good for safety). The inspiration for writing this document came out of the realization that there's a lot of tacit knowledge and intuitions about scaling and LMs that's being siloed in our minds that other alignment people might not know about, and so we should try to get that out there. (That being said, the contents of this post are of course only my personal intuitions at this particular moment in time and are definitely not representative of the views of all EleutherAI members.) I also want to lay out some potential topics for future research that might be fruitful.

By the way, I did consider that the scaling laws implications might be an infohazard, but I think that ship sailed the moment the GPT-3 paper went live, and since we’ve already been in a race for parameters for some time (see:

Megatron-LM

,

Turing-NLG

,

Switch Transformer

,

PanGu-α/盘古α

,

HyperCLOVA

,

Wudao/悟道 2.0

, among others), I don’t really think this post is causing any non-negligible amount of desire for scaling.

Why scaling LMs might lead to Transformative AI

Why natural language as a medium

First, we need to look at why a perfect LM could in theory be Transformative AI. Language is an extremely good medium for representing complex, abstract concepts compactly and with little noise. Natural l...
What question would you like to collaborate on?
by weathersystems — 2021-06-02T19:31:11.374Z

In Why don't long running conversations happen on LessWrong? adamzerner writes:

Here is how things currently work:

Someone writes a post.
It lingers around the front page for a few days. During this time, conversations emerge in the comments section.
After a few days, the post no longer persists on the front page and conversations largely fizzle out.

I'd like to try and have a longer polymath project style collaboration focused on answering a question together. Instead of each person working to give their individual answers to the question, we'd come up with an answer together through an extended discussion.
When/if we've reached some sort of milestone by either answering the question, or making some interesting progress towards answering the question, we show the results of our collaboration to lesswrong in the form of a new post.

As a starting point, here are my proposed rules (up for negotiation):

Each top level answer to this post should be its own question. Please state the question at the top of your comment. Then you can give background information or try to convince us that your question would be a good one to pick for this exercise. The question can be one already asked on lesswrong.
Each person can submit a maximum of 3 questions.
We have 5 days to submit questions and receive feedback or requests for clarification.
Please only submit questions that you yourself are willing to work on.
Once the 4 days are up, all the top level (non-meta) questions will be compiled into a new post to be voted on. (Hopefully that will help with the problem of the earliest submitted questions getting more upvotes).
We work on whichever question gets the most votes.
We can sort out the logistics of the collaboration (where should discussion take place? should we use a wiki or google doc? how long should we commit to working?etc) once we've sorted out what question we're working on and who is willing to contribute.
If you'd like to comment on this idea itself without submitting a question for the group's consideration, use a "comment" instead of an "answer."

I have no idea how many people would be interested in doing this. But I'd guess it only takes 2 people for this thing to work. And at least one person is interested (me). So worst case scenario the person with the top voted question will have at least one other person to work with.
I'm willing to commit to working ...
Selection Has A Quality Ceiling
by johnswentworth — 2021-06-02T18:25:54.432Z

Suppose we’re working on some delightfully Hard problem - genetically engineering a manticore, or terraforming Mars, or aligning random ML models. We need very top tier collaborators - people who are very good at a whole bunch of different things. The more they’re good at, and the better they are, the better the chances of success for the whole project.

There’s two main ways to end up with collaborators with outstanding skill/knowledge/talent in many things: selection or training. Selection is how most job recruitment works: test people to see if they already have (some of) the skills we’re looking for. Training instead starts with people who don’t have (all of) the skills, and installs them de novo.

Key point of this post: selection does not scale well with the level of people we’re looking for. As we increase the number of skills-we-want in our collaborators, the fraction-of-people with all those skills shrinks exponentially, so the number-we-need-to-test grows exponentially. Training has much better asymptotic behavior: as the number of skills-we-want grows, the amount of training needed to install them grows only linearly - assuming we’re able to train them at all.

Bits Of Search

Suppose I have some test or criterion, and only half the population passes it - for instance, maybe I want someone with above-median math skills. That’s one bit of search: it eliminates half the possibilities.

If I want above-median math skills and above-median writing skills, that’s (approximately) two bits, and I expect (approximately) one-in-four people to pass both tests. (Really, math and writing skills are correlated, so it will be somewhat more than one-in-four and thus somewhat less than two bits of search.) As more skills are added to the list of requirements, adding more “bits of search”, the number of people who pass all requirements will fall exponentially. With k bits of search, only 1-in-2^k people will pass, so I’ll need to search over ~2^k people just to find one potential collaborator.

In practice, skills are not independent, but the correlation is weak enough that exponentials still kick in. (Indeed, the only way exponentials won’t kick in is if correlation increases rapidly as we add more skills.)

I also sometimes want more-than-one bit of search in just one skill. For instance, if I want someone in the top 1/32 of writing skill, then that’s 5 bits of search. In prac...
Can you improve your intelligence with these types of exercises?
by icompetetowin — 2021-06-01T21:13:48.656Z

Hi, I write and find exercises on biases to help myself and others think better.
For example:
Bob is an opera fan who enjoys touring art museums when on holiday.
Growing up, he enjoyed playing chess with family members and friends.

Which situation is more likely?

Bob plays trumpet for a major symphony orchestra.
Bob is a farmer.

My question to the LessWrong community:
Does it make sense to learn like this?
Answer to the example (and other exercises):
https://newsletter.decisionschool.org/p/decision-making-bias-base-rate-fallacy
"Existential risk from AI" survey results
by Rob Bensinger — 2021-06-01T20:02:05.688Z

I sent a two-question survey to ~117 people working on long-term AI risk, asking about the level of existential risk from "humanity not doing enough technical AI safety research" and from "AI systems not doing/optimizing what the people deploying them wanted/intended".

44 people responded (~38% response rate). In all cases, these represent the views of specific individuals, not an official view of any organization. Since some people's views may have made them more/less likely to respond, I suggest caution in drawing strong conclusions from the results below. Another reason for caution is that respondents added a lot of caveats to their responses (see the anonymized spreadsheet),1 which the aggregate numbers don't capture.

I don’t plan to do any analysis on this data, just share it; anyone who wants to analyze it is of course welcome to.

If you'd like to make your own predictions before seeing the data, I made a separate spoiler-free post for that.

Methods

You can find a copy of the survey here. The main questions (including clarifying notes) were:2

1. How likely do you think it is that the overall value of the future will be drastically less than it could have been, as a result of humanity not doing enough technical AI safety research?

2. How likely do you think it is that the overall value of the future will be drastically less than it could have been, as a result of AI systems not doing/optimizing what the people deploying them wanted/intended?

_________________________________________

Note A: "Technical AI safety research" here means good-quality technical research aimed at figuring out how to get highly capable AI systems to produce long-term outcomes that are reliably beneficial.

Note B: The intent of question 1 is something like "How likely is it that our future will be drastically worse than the future of an (otherwise maximally similar) world where we put a huge civilizational effort into technical AI safety?" (For concreteness, we might imagine that human whole-brain emulation tech lets you gather ten thousand well-managed/coordinated top researchers to collaborate on technical AI safety for 200 subjective years well before the advent of AGI; and somehow this tech doesn't cause any other changes to the world.)

The intent of question 1 *isn't* "How likely is it that our future will be astronomically worse than the future of a world where Go

...
If You Want to Find Truth You Need to Step Into Cringe
by Writer — 2021-06-01T19:03:56.956Z
Link: https://youtu.be/yhQreMUO7LE

This is the script of the Rational Animations video linked above, with a few minor edits and additions. I really like how the animations came out in this one, so if you are curious follow the link. If you only care about the arguments you can just read. Most of the images here are taken from the video.

If you honestly seek truth, and if you decide to tell the truth, at some point, you will accept to appear cringe to the eyes of most people. Why is that? Simply because truth may be cringe means that at some point, you will encounter a truth that other people are disgusted by, and if you decide to tell it, you will be associated with cringe.

Is it a necessity for at least some truths to be cringe, though? If that was not the case, the laws of the scientific method and of rationality would prescribe to avoid disgusting hypotheses. But our best guess on how to approach truth is to ask reality itself, not our social norms.

Truth doesn't seem to correlate much with cringe, especially if we consider truths that are difficult to attain. In fact, there is an exception for those truths that are necessary for survival: I expect those truths not to be cringe in basically every culture.

The picture looks more or less like this: truths necessary for survival are a subset of non-cringe statements, while other truths intersect with both cringe and non-cringe statements similarly.

When seeking truth, you shouldn't worry about if the truths you are discovering would be laughed at and scoffed at if said aloud. You would be like a swordsman who keeps glancing away to see if anyone might be laughing at him. Doing so makes you waste time, and it makes you move or withdraw for the wrong reasons.

What happens if you look away

The acceptance of cringe is one of the marks of a truth seeker. Keeping this fact at the forefront of your mind is helpful for not straying from the truth.

The good news is that there are probably good ways to mitigate social repercussions without relinquishing truth-telling. For example, if you act and state your beliefs confidently, other people will usually see you in a more positive light. You act as if you were high… on your social hierarchy. And this confidence will positively affect the image other people will form about you. I'm not saying that this matters in any way to the actual truth, though. It just matters to people.

Being cringe doesn't only happen when ...
Power dynamics as a blind spot or blurry spot in our collective world-modeling,  especially around AI
by Andrew_Critch — 2021-06-01T18:45:39.820Z

Where I’m coming from

***Epistemic status: personal experience***

In a number of prior posts, and in

ARCHES

, I’ve argued that more existential safety consideration is needed on the topic of multi-principal/multi-agent (multi/multi) dynamics among powerful AI systems.

In general, I have found it much more difficult to convince thinkers within and around LessWrong’s readership base to attend to multi/multi dynamics, as opposed to, say, convincing generally morally conscious AI researchers who are not (yet) closely associated with the effective altruism or rationality communities.

Because EA/rationality discourse is particularly concerned with maintaining good epistemic processes, I think it would be easy to conclude from this state of affairs that

multi/multi dynamics are not important (because communities with great concern for epistemic process do not care about them much), and
AI researchers who do care about multi/multi dynamics have “bad epistemics” (e.g., because they have been biased by institutionalized trends).

In fact, more than one LessWrong reader has taken these positions with me in private conversation, in good faith (I’m almost certain).

In this post, I wish to share an opposing concern: that the EA and rationality communities have become systematically biased to ignore multi/multi dynamics, and power dynamics more generally.

A history of systemic avoidance

***Epistemic status: self-evidently important considerations based on somewhat-publicly verifiable facts/trends.***

Our neglect of multi/multi dynamics has not been coincidental. For a time, influential thinkers in the rationality community intentionally avoided discussions of multi/multi dynamics, so as to avoid contributing to the sentiment that the development and use of AI technology would be driven by competitive (imperfectly cooperative) motives. (FWIW, I also did this sometimes.) The idea was that we — the rationality community — should avoid developing narratives that could provoke businesses and state leaders into worrying about whose values would be most represented in powerful AI systems, because that might lead them to go to war with each other, ideologically or physically.

Indeed, there was a time when this community — particularly the Singularity Institute — represented a significant share of public discourse on the future of AI technolo...
Forecasting Newsletter: May 2021
by NunoSempere — 2021-06-01T15:51:26.463Z

Highlights

Misha Yagudin creates a webpage to get one’s calibration chart for Good Judgment Open and CSET-Foretell
Hypermind experiments with new methods of eliciting, incentivizing and scoring long-range forecasts.
Augur launches Augur Turbo on Polygon, but gets very little trade volume

Index

Prediction Markets & Forecasting Platforms
In The News
Blog Posts
Papers
Hard to Categorize
Long Content

Sign up

here

or browse past newsletters

here

.

Prediction Markets & Forecasting Platforms

Augur

Augur has launched Augur Turbo on Polygon, the same second-layer chain which also hosts Polymarket. The website to access it is hosted on IPFS, hence the long address when one accesses it through a portal.

To unpack the tech stack:

Augur is a decentralized prediction market protocol. Its strength lies on its robust decentralized question resolution mechanism: resolutions can be disputed until a consensus is reached.
Polygon (previously Matic) is a parallel blockchain for Ethereum. It offers contract compatibility, so that contracts written for the main Ethereum blockchain can also run on Polygon. Moving assets from the main Ethereum blockchain to Polygon is initially somewhat cumbersome. But once inside Polygon, transaction fees are much cheaper.
Chainlink is an oracle service. Most of the time, it will work and it will work fast. The rest of the time, resolution can be disputed and Augur's protocol can be used instead. Chainlink claims to be decentralized, but appears not to fully be so.
IPFS is a censor-proof, decentralized alternative to the http protocol. To a first approximation, files in the network are hashed using the SHA-256 algorithm, and accessed using their hash. However, this turns out to be more complicated in practice.

So far, Augur Turbo merely has NBA markets, with low volume, and low liquidity.

Augur Turbo webpage. Screenshot taken the 1st of June, 2021. Notice that the market with the highest total volume has a mere $100 in trade volume.

From Augur’s early history, Is Augur Being Gamed explains what exactly Poyo did to profit from the creation of invalid markets. Back in the day, Augur didn’t have a tradable “invalid” resolution. Instead, invalid or ambiguous markets were resolved 50-50. This allowed Poyo to profit by creating markets he knew would resolve as invalid, and then buying the cheaper side.

CSET-Foretell

CSET-Foretell has added the ability for use...