trentmkelly/LessWrong-43k huggingface dataset:
Basics of Rationalist Discourse
by Duncan Sabien (Inactive) — 2023-01-27T02:40:52.739Z

Introduction

This post is meant to be a linkable resource. Its core is a short list of guidelines (you can link directly to the list) that are intended to be fairly straightforward and uncontroversial, for the purpose of nurturing and strengthening a culture of clear thinking, clear communication, and collaborative truth-seeking.

"Alas," said Dumbledore, "we all know that what should be, and what is, are two different things. Thank you for keeping this in mind."

There is also (for those who want to read more than the simple list) substantial expansion/clarification of each specific guideline, along with justification for the overall philosophy behind the set.

Prelude: On Shorthand

Once someone has a deep, rich understanding of a complex topic, they are often able to refer to that topic with short, simple sentences that correctly convey the intended meaning to other people with similar context and expertise.

However, those same short, simple sentences are often dangerously misleading, in the hands of a novice who lacks the proper background. Dangerous precisely because they seem straightforward and comprehensible, and thus the novice will confidently extrapolate outward from them in what feel like perfectly reasonable ways, unaware the whole time that the concept in their head bears little or no resemblance to the concept that lives in the expert's head.

Good shorthand in the hands of an experienced user need only be an accurate fit for the already-existing concept it refers to—it doesn't need the additional property of being an unmistakeable non-fit for other nearby attractors. It doesn't need to contain complexity or nuance—it just needs to remind the listener of the complexity already contained in their mental model. It's doing its job if it efficiently evokes the understanding that already exists, independent of itself.

This is important, because what follows this introduction is a list of short, simple sentences comprising the basics of rationalist discourse. Each of those sentences is a solid fit for the more-complicated concept it's gesturing at, provided you already understand that concept. The short sentences are mnemonics, reminders, hyperlinks.

They are not sufficient, on their own, to reliably cause a beginner to construct the proper concepts from the ground up, and they do not, by themselves, rule out all likely misunde...
The recent banality of rationality (and effective altruism)
by CraigMichael — 2023-01-27T01:19:00.643Z

[Neurotic status: Until recently, I've felt entirely comfortable telling friends, family, and co-workers that I'm a rationalist (or an aspiring rationalist)—that it wouldn't out-group me among them, or if it did, it was probably worth being out-grouped over. I will happily continue to use Bayesian reasoning, but I can't rep the label "IRL" right now. The distal reasons for this change are (1) I believe rationalism and effective altruism as movements are losing their ability to self-correct, (2) I'm not in a position to help fix (1). I hope this is temporary or that I'm wrong. I started writing last week, and there may have been development since then that I missed. I love you all like a milkshake.

I avoided using easily searchable terms below to keep out the riffraff.]

I started rewatching Deadwood last week. I previously finished it sometime around 2008. It was the right amount of time to wait. I've forgotten enough that it's entertaining again.

Spoilers ahead

I was more impressed than I remembered with

the first trial of Jack McCall

as portrayed in the series. In real life, there was a

Jack McCall

who killed Wild Bill Hickok while he was playing poker, and there is some evidence that

Wild Bill was holding "Aces and Eights"

when Jack shot him in Deadwood.

Doing some cursory searches, the portrayal of Jack McCall's first trial in the series isn't entirely historically accurate, and the series, in general, elides uncomfortable truths about "settler isopolities" (as both

IMDB trivia

and

scholarly literature

attest to). But the gist and broad strokes appear correct enough for the scope of the point I'm trying to make.

At the time of this shooting, Deadwood was an illegal settlement

on land designated for the Lakota

. Meaning there was no formal legal authority for adjudicating such crimes.

Forming pools of people to be selected for jurors and officers of the court.

In the absence of proper authority, the inhabitants of the illegal settlement (camp) self-organize and form a court to conduct a trial. They create a process to select jurors, a judge, a defense lawyer, and a prosecution lawyer. They decide on a venue and date for the trial. Then the trial happens. Voilà!

The Deadwooders (

Deadwoodans? Deadwoodites? Deadwoodganders?

) found themselves in an unprecedented circumstance. Yet, they devised a viable pla...
11 heuristics for choosing (alignment) research projects
by Orpheus16 — 2023-01-27T00:36:08.742Z

I recently spoke with Dane Sherburn about some of the most valuable things he learned as a SERI-MATS scholar.

Here are 11 heuristics he uses to prioritize between research projects:

Impact: Can I actually tell myself a believable story in which this project reduces AI x-risk? (Or better yet; can I make a guesstimate model that helps me estimate the microdooms averted from this project?)
Clarity of research question: Can I easily explain my core research question in a few sentences?
Relevance of research approach: Will my research project actually help me reduce uncertainty on my research question? When I imagine the possible results, are there scenarios where I actually update? Or do I already know (with high probability) what I’m likely to learn?
Mentorship: Would my mentor be able to give me meaningful guidance on this project? If not, would I be able to find one who could?
Feedback loops: Will I be able to get feedback within the first week? First day? Will I have to wait several weeks or months before I know if things are working?
Efficiency: How efficiently will I be able to collect information or run experiments? Will I need to spend a lot of time fine-tuning models? Is there a way to do something similar with pretrained models, so I can run experiments 10-100X more quickly?
Resources: WilI this project need datasets? Large models? Compute? Money? How likely is it that I’ll get the resources I need, and how long will it take?
Excitement: How much does the project subjectively excite me? Do I feel energized about the project?
Timespan: How long would it take to do this project? Would it fit into a window of time that I’m actually willing to devote to it?
Downsides/capabilities externalities: To what extent does the project have capabilities externalities? Could it increase x-risk?
Leaveability: How easy would it be to leave this project if I realize it’s not working out, or I find something better?
A different observation of Vavilov Day
by Elizabeth — 2023-01-26T21:50:01.571Z

At this time last year, I was in the middle of a 36 hour fast in honor of Nikolai Vavilov and his team, who starved themselves to preserve a seed bank that went on to dominate Russian agriculture.* One reason I did that was to honor the team and their sacrifices, but another was to test and develop my own ability to do hard things when necessary. It was a great experiment, I did better than I thought I would but also the costs took longer to repay than I thought, and all of that knowledge was really valuable to me.

I went back and forth about doing the fast this year. The sense of continuity and retesting myself felt valuable and I’m sad about missing out on them. But I’m currently doing hard things and my capacity to deal with that kind of pain is a limiting reagent right now. Fasting for Vavilov Day would come at the expense of an actual project that matters to me. Delaying real work for a symbolic sacrifice would not only be stupid, it would be bad symbolism. I can’t honor a sacrifice for a cause by sacrificing a cause for symbolism.

So this year I’m not fasting. I do think I’ll want another fast at some point to see how my medical miracle affected things, but it will wait.

*My one sentence here is already simplified from the story as fully known in the West, read the blog post for more. I also suspect a lot of details haven’t made it into English. Last year I looked into hiring a Russian researcher to investigate and even found someone in mid-Fubrary, but they put it off one week for a root canal and then had some other excuse the next week so seems like they’re not going to come through.
All AGI Safety questions welcome (especially basic ones) [~monthly thread]
by mwatkins — 2023-01-26T21:01:57.920Z

tl;dr: Ask questions about AGI Safety as comments on this post, including ones you might otherwise worry seem dumb!

Asking beginner-level questions can be intimidating, but everyone starts out not knowing anything. If we want more people in the world who understand AGI safety, we need a place where it's accepted and encouraged to ask about the basics.

We'll be putting up monthly FAQ posts as a safe space for people to ask all the possibly-dumb questions that may have been bothering them about the whole AGI Safety discussion, but which until now they didn't feel able to ask.

It's okay to ask uninformed questions, and not worry about having done a careful search before asking.

AISafety.info - Interactive FAQ

Additionally, this will serve as a way to spread the project Rob Miles' volunteer team[1] has been working on: Stampy and his professional-looking face aisafety.info. Once we've got considerably more content[2] this will provide a single point of access into AI Safety, in the form of a comprehensive interactive FAQ with lots of links to the ecosystem. We'll be using questions and answers from this thread for Stampy (under these copyright rules), so please only post if you're okay with that! You can help by adding other people's questions and answers or getting involved in other ways!

We're not at the "send this to all your friends" stage yet, we're just ready to onboard a bunch of editors who will help us get to that stage :)

Stampy - Here to help everyone learn about stamp maximization AGI Safety!

We welcome feedback[3] and questions on the UI/UX, policies, etc. around Stampy, as well as pull requests to his codebase. You are encouraged to add other people's answers from this thread to Stampy if you think they're good, and collaboratively improve the content that's already on our wiki.

We've got a lot more to write before he's ready for prime time, but we think Stampy can become an excellent resource for everyone from skeptical newcomers, through people who want to learn more, right up to people who are convinced and want to know how they can best help with their skillsets.

Guidelines for Questioners:

No previous knowledge of AGI safety is required. If you want to watch a few of the Rob Miles videos, read either the WaitButWhy posts, or the The Most Important Century summary from OpenPhil's co-CEO first that's great, but it's not a prerequisite t

...
Exquisite Oracle: A Dadaist-Inspired Literary Game for Many Friends (or 1 AI)
by Yitz — 2023-01-26T18:26:14.559Z

NOTE: You may divide all numbers in this document by 10, for a significantly shorter game.

Setup

Roles: With a single exception, everyone (human or otherwise) is given the role of “Player”, and an additional individual (or AI system) is designated as the “Oracle”.

The goal of the game: write a coherent 20,000 word paper based on a one-sentence prompt provided by the Oracle. The catch? Nobody but the Oracle (who cannot write to the paper itself) is allowed to type more than 2,000 words, and communication between players is only possible by consulting the Oracle.

Format: See footnote for technical details.[1]

Rules:

Players can only write 2,000 words in total before their turn is up. Any text written after that point will be discarded.
Players are not allowed to communicate with other players or directly read what has already been written before their turn started. However, they are allowed to ask the Oracle arbitrary questions about what has already been written, as well as give the Oracle notes which may be passed on to future Players.
All Players will be given a prompt by the Oracle before the game begins. This prompt is the same for everyone, and cannot be changed until the game is completed.
The text of a Player’s question, and the answer given by the Oracle will be included in the Player’s total word count (but not in the essay’s total word count). Accordingly, Players should attempt to keep their questions short, efficient, and focused.
If all Players have taken a turn, and the game has not yet ended, turns will be repeated, starting from the first Player.
The game ends once the total word count for the paper (not including communication with the Oracle) reaches 20,000 words, or once the Oracle declares the paper to be complete.

The Oracle has full access to all text already written, and should try to honestly and concisely answer all questions asked by Players. The Oracle can only speak when spoken to (or “consulted”) by a Player, though if they think it would be helpful, they may include unasked-for advice in their responses to Players. The Oracle should try to be as concise as possible, as any additional turns caused by being excessively verbose will inflict frustration. The ultimate goal of the Oracle is to help Players make the final paper coherent, consistent, and high quality.

Good luck, and have fun!

^

Technical formatting specifications (for formalized play)

P

...
AI Risk Management Framework | NIST
by DragonGod — 2023-01-26T15:27:19.807Z
Link: https://www.nist.gov/itl/ai-risk-management-framework

On January 26, 2023, NIST

released

the AI Risk Management Framework (AI RMF 1.0) along with a companion

NIST AI RMF Playbook

, AI RMF Explainer Video, an

AI RMF Roadmap

,

AI RMF Crosswalk

, and various

Perspectives

. Watch the event

here

.

In collaboration with the private and public sectors, NIST has developed a framework to better manage risks to individuals, organizations, and society associated with artificial intelligence (AI). The

NIST AI Risk Management Framework (AI RMF)

is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.

Released on January 26, 2023, the Framework was developed through a consensus-driven, open, transparent, and collaborative process that included a Request for Information, several draft versions for public comments, multiple

workshops

, and other opportunities to provide input. It is intended to build on, align with, and support AI risk management efforts by others.

A companion

NIST AI RMF Playbook

also has been published by NIST along with an

AI RMF Roadmap

,

AI RMF Crosswalk

, and various

Perspectives

. In addition, NIST is making available a video explainer about the AI RMF.

FLI also released a statement on NIST's framework:

﻿
FUTURE OF LIFE INSTITUTE

Statement

The Future of Life Institute applauds NIST for spearheading a multiyear and stakeholder initiative to improve the management of risks in the form of the Artificial Intelligence Risk Management Framework (AI RMF). As an active participant in its development process, we view the AI RMF as a crucial step in fostering a culture of risk identification and mitigation in the US and abroad.
With this launch, NIST has created a global public good. The AI RMF decreases barriers to examining the implications of Al on individuals, communities, and the planet by organizations charged with designing, developing, deploying, or using this technology. Moreover, we believe that this effort represents a critical opportunity for institutional leadership to establish clear boundaries around acceptable outcomes for Al usage. Many firms have already set limitations on the development of weapons and on activities that lead to clear physical or psychological harm, among others.

The release of version 1.0 of the AI RMF is no

...
"How to Escape from the Simulation" - Seeds of Science call for reviewers
by rogersbacon — 2023-01-26T15:11:33.733Z

Seeds of Science is a journal (funded through Scott Alexander's ACX grants program) that publishes speculative or non-traditional articles on scientific topics. Peer review is conducted through community-based voting and commenting by a diverse network of reviewers (or "gardeners" as we call them); top comments are published after the main text of the manuscript.

We have just sent out an article for review - "How to Escape from the Simulation" - that may be of interest to some in the LessWrong community, so I wanted to see if anyone would be interested in joining us a gardener to review the article. It is free to join and anyone is welcome (we currently have gardeners from all levels of academia and outside of it). Participation is entirely voluntary - we send you submitted articles and you can choose to vote/comment or abstain without notification (so it's no worries if you don't plan on reviewing very often but just want to take a look here and there at the articles people are submitting).

To register, you can fill out this google form. From there, it's pretty self-explanatory - I will add you to the mailing list and send you an email that includes the manuscript, our publication criteria, and a simple review form for recording votes/comments. If you would like to just take a look at this article without being added to the mailing list, then just reach out (info@theseedsofscience.org) and say so.

Happy to answer any questions about the journal through email or in the comments below. Here is the abstract for the article.

How to Escape From the Simulation

Many researchers have conjectured that the humankind is simulated along with the rest of the physical universe – a Simulation Hypothesis. In this paper, we do not evaluate evidence for or against such claim, but instead ask a computer science question, namely: Can we hack the simulation? More formally the question could be phrased as: Could generally intelligent agents placed in virtual environments find a way to jailbreak out of them? Given that the state-of-the-art literature on AI containment answers in the affirmative (AI is uncontainable in the long-term), we conclude that it should be possible to escape from the simulation, at least with the help of superintelligent AI. By contraposition, if escape from the simulation is not possible, containment of AI should be, an important theore...
Loom: Why and How to use it
by brook — 2023-01-26T14:34:50.435Z

Loom

is a desktop tool and chrome extension for screen capture and audio/video recording app with easy sharing features.

It's worth noting: Loom is probably the tool here I'm most excited about! I used it a lot during this project (to make all the videos, and to talk to Chana), and I think basically everybody who's doing video or text messaging in a professional capacity should consider using Loom. It's very straightforward, and it just works.

When can Loom be Useful?

Summarising forum posts you think are important (to save people time)
Walking through scientific papers
Explaining software tools
Talking through a model you made
Cutting a zoom meeting with a supervisor down to a 5 minute video explaining how your work is going

Other Tutorials

Loom have a 'getting started' page which has (unsurprisingly) good video introductions, but leans sales-pitch-y. The tool is quite intuitive, so it would also be reasonable to just download and start using it to get maybe 70% of the value.

Text Tutorial

Loom is available as a desktop app or chrome extension. The key interface has a number of recording settings; you can use loom to record your screen (full screen or a specific window), video, and audio.

Once you start recording, this interface will disappear and loom will become a smaller interface at the left of your screen. If you hover over each button a keyboard shortcut for that command will show.

When you click 'stop recording', loom automatically opens a webpage with your video on it. By default, loom plays videos at 1.2x speed.

Note that some features (videos >5min, screen drawing) require an $8/mo subscription.

Examples

"Summaries are underrated" forum post summary
"A Mathematical Framework for Transformer Circuits" walkthrough of a scientific paper
Guesstimate tutorial
Explanation of a squiggle model estimating $ value of changing the forum karma system

Personal Experience

I found loom really useful as a tool, and I can see myself using it a lot when I'm trying to explain things quickly, remotely, & asynchronously; I used it during this project to communicate with colleagues!

It's pretty intuitive, and the workflow is very smooth. It's mildly irritating that you can't (so far as I could spot) make videos save locally by default, and therefore need an active internet connection to record. It's also worth noting that the automated transcriber ...
Covid 1/26/23: Case Count Crash
by Zvi — 2023-01-26T12:50:00.709Z

A large decline in cases, right at the end of January.

This week’s fun exercise is the clash between a call for the CDC to focus on being a center for disease control and offload its quixotic side quests, versus a former head of the CDC insisting that people will die (die I tell you!) if such dangerous changes are implemented.

Executive Summary

Cases down by 27%.

Former FDA head calls for CDC to be CDC, former CDC head returns fire.

Jury duty? School? Covid positive? No problem, come on down.

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 320k cases (-5%) and 3,600 deaths (-8%).

Results: 245k cases (-27%) and 3,456 deaths (-13%)

Predictions for Next Week: 223k cases (-10%) and 3,300 deaths (-4%).

That is a huge case decline. Illinois did not report, although Indiana did – if Illinois does this a third time I’ll start correcting it the way I do Florida, for now I am not doing so. Most of the decline seems genuine, or a reflection of the end of some sort of backlog being filled perhaps. I am going to mostly buy it, and say that once again the seasonal peak has been reached. Things should improve from here.

Deaths also declined, which gives weight to the reporting hypothesis. They should start declining in earnest soon.

Deaths

Cases

How many cases are we missing? A lot. Here’s a fascinating chart of which cases are less likely to be reported by age and gender.

Physical World Modeling

Paper with details about the evolution of Covid variants. Has lots of technical detail, but nothing that changes my overall big picture views.

Bob Wachter sees conditions improving, is (somewhat) loosening up his rules.

Another booster analysis.

Simplified Booster

FDA is proposing an annual Covid booster shot matched to current strains. If ‘current’ can actually be current, then this seems like a plausible best we can do in practice. It would be better to do this in response to important new strains but I will admit, essentially, good luck with that on multiple levels. Once a year might be doable, and should maximize total engagement. All the talk over the evidence required for new boosters makes me think all boosters will forever be out of date, so it will be a question of who wants an annual booster anyway.

There is also mention of moving to a single-dose initial vaccine, wh...
How are you currently modeling COVID contagiousness?
by CounterBlunder — 2023-01-26T04:46:13.319Z

My long-distance partner has COVID; her symptoms first started ~10 days ago. She is now symptomless (except fatigue), but is still testing strongly positive on rapids. If she comes to visit me, what are the odds she's still contagious? More generally, how much weight do you place on "symptomless and 10+ days from onset" versus "testing positive on rapids" in modeling contagiousness?