trentmkelly/LessWrong-43k huggingface dataset:
Distillation Experiment: Chunk-Knitting
by DirectedEvolution — 2022-11-07T19:56:39.905Z

Summary

Here, I show the output of a protocol to break down information that exceeds typical working memory limits into chunks. The goal is to enhance understanding at first reading, both for the person breaking down the information and for the person reading the result. The original text, an explanation of how RNA interference works, can be found under "Text Of The Original Description." Readers who want to skip straight to the result should go to the section "Result of Chunk-Knitting Procedure." The goal is to produce a systematic way of building a composite understanding from complex atomic information. I call this procedure "Chunk-Knitting."

Note: I also expand some acronyms and bold key terms the first time they're introduced in the Chunk-Knitting output, but not in the original text.

Introduction and epistemic status

How can we deal with working memory limitations when presenting complex topics? Often, there are too many individual parts to remember easily. Yet the fascination of understanding how they fit together depends on first mastering these atomic parts. It is hard to maintain focus. It seems that learning may be bottlenecked by the student's ability to get through this period of internalizing atomic parts and their individual relationships. The payoff comes at the end, when the student understands how they all fit together into a powerful integrated whole. This should motivate us to find new ways to present information that minimize the memory and attention limitations that students face at the beginning of this process.

We know that spaced repetition, breaking information into chunks, and building connections between atomic concepts are all useful ways to improve memory and integrative understanding. Chunk-Knitting is an attempt to leverage these three tools to systematically create pedagogical text that is easier to understand the first time the reader encounters the ideas it is conveying. This post is a first attempt to apply Chunk-Knitting. The technique has not been experimentally studied, and there is no data supporting the hypothesis that Chunk-Knitting makes information easier to understand.

RNA interference is a naturally occurring method of gene regulation that is also widely used as an experimental technique and has therapeutic potential. Although the basic mechanism can be compressed to the space of a paragraph, the number of details is l...
Thinking About Mastodon
by jefftk — 2022-11-07T19:40:04.285Z

Social networks are normally very sticky—you want to be wherever
the people you want to interact with are—so it's hard for
alternatives to succeed. With the upheaval around Twitter, however, a
lot of people are considering moving away from it, and rethinking
social media choices in general, which makes this an unusual
opportunity for community migration. I'm especially seeing a lot of
suggestions that people move to Mastodon.

Instead of being run by a single company that builds the software and
runs the servers (Facebook, Twitter, etc) Mastodon is more like email.
You sign up for an account with some provider ("server"), but can talk
to people regardless of which provider they're signed up with
("federation"). There's an open protocol (ActivityPub),
open-source server software (Mastodon, Pleroma, etc),
and many servers you
could join.

Overall I'm pretty pessimistic about Mastodon, even if we only imagine
futures in which lots of people move to it, because of spam. Handling
spam on a centralized platform is hard but manageable; federation
changes this dramatically for the worse. Imagine you're running a
server, and you get an incoming message (comment, like, DM, etc) from
an account on another server. Today the default is to let it through,
but as Mastodon gets larger there will be more and more money to be
made in spamming there, and the larger a fraction of those incoming
messages would be spam. Many of the signals centralized platforms use
to detect spammers (local activity, account age, IP addresses, etc)
are not available in a federated context, leaving server admins (and
the software they delegate to) very little information for identifying
and blocking spam. It's at least as hard a problem as email spam
filtering, probably harder because of shorter messages, and I expect
this makes it very hard to run a server that doesn't drown its
users in incoming spam and reliably gets its messages out to other
servers. Maybe we get to the equivalent of everyone using
Gmail/Hotmail/Yahoo and anyone with a different provider has a very
frustrating time?

Still, I'm happy to give it a try, and if it does collapse in a pile
of spam, oh well. Does anyone have a server they'd recommend?
I'm not that excited about the idea of joining a server at random
because I'm trusting the server admin not to impersonate me, and also
because apparently being on a...
Ideas for tiny research projects related to rationality?
by Frej — 2022-11-07T18:45:44.682Z

As part of a course I have to write a small research paper in about a month. I would greatly appreciate any inspiration for a rationality-adjacent research questions for this paper. By course requirements, I have to justify that the research question is at least partly about communication.

I will only have a couple of weeks to both conduct any experiments and write the paper. The following points are just my own musings on what to work on. I would love any criticism, elaboration or inspiration for completely different ideas.

I have a hunch that often when people refer to some knowledge they hold (e.g. referencing the name of a researcher. The usage technical terminology...), they are attempting to signal something like intelligence. If this is true, I would like to see if there are any interventions that can move the conversation to be more constructive or "scout-mindsetted".
I'm very curious in, but completely unfamiliar with language models. I have been wondering if they can be used to improve or generate educational materials somehow. I'm assuming the scope of the paper could at most be constructing a proof of concept.
Loss of control of AI is not a likely source of AI x-risk
by squek — 2022-11-07T18:44:30.720Z

This post attempts to set out my views on why I don't find technical alignment to be a pressing issue, compared to more mundane AI risks like human misuse. It represents my personal views only, as someone who is not in AI research. I would appreciate any and all feedback.

Summary of views

A profound economic transformation through AI is a lot more likely to happen through various implementations of narrow AI, rather than artificial general intelligence (AGI).
An alternative way to frame this view is that if both are possible, then narrow AI transformation happens earlier in the timeline, with AGI in the distant long-term. In this scenario, risks like robustness, bias, and human safety problems are more likely and immediate sources of AI x-risk compared to AI alignment, and should accordingly receive more focus.
I believe development of AGI will take longer than the estimates of most people in this community. It is common to conflate cognition and execution in a lot of the AGI arguments that I’ve read, which I don’t think is an obvious conclusion. I think that developing an AI that can complete any action at least as well as a human is a hard problem that will require a breakthrough in the way we architect AIs.
Cutting-edge AI research that advances AI capabilities (not simply implement existing techniques) is the domain of wealthy organizations and governments, who choose to allocate time and resources to this problem as they find it useful to achieving their goals. If developing general AI is hard, then it would be much more time- and cost-effective to develop multiple ‘narrow’ AI with a bounded, defined set of actions that is still capable of achieving those goals.
Consequently, the default and most likely scenario for AI x-risk is one where AI has a bounded set of actions, but is still capable of leading to x-risk through intentional misuse or unintended consequences of deploying this AI.

1. There is no prototype for an AI that can execute general human tasks.

From the various arguments I’ve read which discuss the AI alignment problem, I often see an implicit assumption that a general AI that has an intelligent and accurate view of the world, enough to generate plans and predictions, would naturally be capable of executing those plans through an unbounded set of actions.

That doesn’t strike me as true. Let’s use an advanced ‘generalis...
AI Safety Unconference NeurIPS 2022
by Orpheus — 2022-11-07T15:39:57.231Z
Link: https://aisafetyevents.org/events/aisuneurips2022/

The AI Safety Unconference brings together persons interested in aspects of AI safety, from technical AI safety problems to issues of governance of AI. As an unconference, it aims to foster valuable social interactions between participants, through moderated discussion groups, one-on-ones, lightning talks, free-form interactions.

Date: Monday November 28 from 9:00 to 16:00, in New Orleans, alongside NeurIPS 2022.

Location: Near the Convention Center - exact location to be communicated directly with registered participants.

Participate

Fill the Application form.

The event is private, free, with a maximum of 100 participants.

Join the chat room on Matrix, to discuss online before or during the event.

Contribute facilitated discussion (30 min) or a lightning talk (10min). We also welcome other ideas of activities - feel free reach out to the organizers about that.

Agenda

09:00-09:30 - Event opening, breakfast is served

09:30-12:00 - Facilitated discussions and 1:1s

12:00-13:30 - Lightning talks, lunch is served

13:30-15:30 - Facilitated discussions and 1:1s

15:30-16:00 - Event closing

Vegan breakfast and lunch are provided, along with all-day drinks and snacks.

The Swapcard app is used for scheduling 1:1 meetings with other participants and registering to facilitated discussions.

We have confirmed participants from the following organizations: Mila, Uni. of Stanford, Anthropic, OpenAI, UC Berkeley, Uni. of Toronto, ETH & Max Planck Institute, Uni. of Cambridge, Vector Institute, NYU, ETH Zurich, DeepMind, Oxford, MIT, and more.

Further information

Testimonial of past events (2018, 2019):

A great way to meet the best people in the area and propel daring ideas forward. — Stuart Armstrong

The event was a great place to meet others with shared research interests. I particularly enjoyed the small discussion groups that exposed me to new perspectives. — Adam Gleave

Background readings:

Center for AI Safety's About AI Risk

Krakovna's AI safety resources

Alignment newsletter

Superintelligence: Paths, Dangers, Strategies by Nick Bostrom

Acknowledgements

The event is organized in partnership with the Center for AI Safety.

Organizers:

Orpheus Lummis

Mauricio H. Luduena

Thanks Nisan Stiennon for funding the event.

For any questions or feedback, reach out to info@aisafetyevents.org.
Hacker-AI – Does it already exist?
by Erland Wittkotter — 2022-11-07T14:01:23.915Z

In its simplest version: Hacker-AI (Type I) uses AI for hacking arbitrary IT systems, i.e., finding vulnerabilities automatically, e.g., elevating privileges/user roles (sys-admin or system), stealing encryption keys, hiding attacking tools from being detected, or creating code that is irremovable on (once) occupied systems. It would create exploits automatically. In an advanced version, Hacker-AI (Type II) would be actively operating malware on computer systems with sys-admin rights, trying to stay hidden from detection, and making itself irremovable from any later deployed software.

In my post “Hacker-AI and Digital Ghosts – Pre-AGI”, I argued that Hacker-AI and Digital Ghosts are feasible and a serious threat to our civilization. In my follow-up post, “Improved Security to Prevent Hacker-AI and Digital Ghosts”, I proposed several solutions to eliminate this threat, e.g., via a separate layer of standardized security-related operations detached from regular tasks in which we ignore all security-related (regular) results.

In this post, I will discuss methods of how hackers and Hacker-AI gain the upper hand in our current cyber-security ecosystem – despite the potential use of AI in cyber defense.

I don’t expect this post to surprise or educate hackers (criminal or governmental) or anyone knowledgeable enough to be a hacker -- they know already about Stuxnet, or Pegasus from the NSO-Group. The relative scarcity of Zero-Day Vulnerabilities could tell us that hacking is still not done systematically, but I am not sure about the cyberwar capabilities of governments. Moreover, many (enhanced) tools are required to be effective as a hacker. Some dumps of tools from hacker/government groups were published – but too little is publicly known, so I don’t dare to have a (public) opinion about that. I still assume it is not easy for lonely malicious actors or, more specifically, smart system developers, all trying to damage our (cyber-) world, to quickly turn into professional/efficient hackers. But the broad availability of these (more enhanced) software features could change quickly.

As a side note: I have had a question. In what world would we live if the physics of building nuclear weapons were easy enough to enable students to build nukes in school experiments? This world would likely be much different (less free, more surveillance), or most of us would no...
What’s the Deal with Elon Musk and Twitter?
by Zvi — 2022-11-07T13:50:00.991Z

At the end of long saga well-covered in hilarious fashion by Matt Levine, Elon Musk has purchased Twitter.

He then began doing things.

One of them was to tweet ‘fresh baked bread and pastries are some of the great joys of life.’ On that I hope we can all agree.

His other decisions were less obviously great. More of a mixed bag.

The response by many news sources and also individuals has been that everything is The Worst and a giant dumpster fire and what he has done will destroy Twitter, and is so much worse than we expected, oh no.

While it is true that the details have not exactly covered Musk in glory, it all also seems like the kind of thing those same sources would say no matter what. One can assume that everything that happens is being intentionally framed to make it look as bad as possible for Twitter and Musk. Any blame that can be assigned will be, etc.

While looking at what is happening, one can also usefully look at the coverage of what is happening. Notice how media sources describe events. That tells you how they describe events in these spots, and how you should expect them to in the future.

So I decided to gather up the things I’ve seen since Elon bought the place, and my thoughts on them, and try to put them into a semi-coherent format. Here you go.

Oh Elon

One of Musk’s first actions was to fire a bunch of top Twitter executives ‘for cause’ which if it stuck (which it won’t) would allow him to avoid tens of millions in severance payments. And by some weird coincidence Musk finished the transaction exactly in time to fire a bunch of Twitter employees right before a bunch of their stock options trigger. If anyone involved was surprised by these moves, that is on them.

By now, it should be clear that Musk has zero interest in honoring laws or norms or contracts when dealing with people unless he likes them, and that his approach to the law is ‘make me.’

It also should be clear Musk has zero interest in stopping, before showing zero interest in honoring norms or his contracts or laws, to ask what would happen next, or wonder whether he will inevitably lose the resulting lawsuits.

He did not ‘think things through’ when trying to get out of buying Twitter. He is not about to start now. That is not how this type of thinking works.

People are rightly drawing parallels to Trump’s ‘blatantly violate contracts and no-pay people all the time and dare ...
How to Make Easy Decisions 
by lynettebye — 2022-11-07T13:17:10.075Z

Note: I’m think I remember a similar post to this one existing somewhere, but I can’t find it and I don’t think it was illustrated with The Good Place memes….so enjoy. This was crossposted on my blog.

I respect how carefully the effective altruists around me try to use reason and logic to make good decisions. However, I sometimes see these people spending way too much effort to make careful decisions in situations where the outcome doesn’t matter.

You might not literally be destroying your life, but you’re probably flushing a good chunk of time down the toilet if you’re spending hours googling which lamp to buy.

You might spend time on a decision because you’re afraid of making a mistake or regretting your decision. Or you might not have considered the cost of your time – it can feel intuitive to keep researching until you’ve found the best decision, without noticing that the return on your time is miniscule.

Alternatively, you could just make the simpler decision and have all of that time back. Your time is probably incredibly valuable. Learning to respect that is a key productivity breakthrough to accomplishing more.

Caveats:

This advice is mainly aimed at people who have the privilege to value their time more than getting the best deal.
It’s even better advice for people who can absorb a mistake or two (like forgetting to mail back an Amazon return or giving away a new-but-ill-fitting shirt).
This advice is NOT aimed at people making important and hard decisions, like “what career should I pick?” These big questions can have huge differences in how impactful one decision vs the other is, so you often want to spend a good chunk of time making a careful decision.

Because some decisions matter and others don’t, I have a couple questions that help decide how important a decision is to get right:

1. Is the decision easily reversed or undone?

One heuristic I use: Can you replace it for less than thirty dollars in less than thirty minutes? (Or more precisely, can you replace/undo it for less than ½ as much as you value an hour, in less than thirty minutes?)

Examples: Amazon returns, buying a cheap replacement for a forgotten item while traveling, tickets with a flexible change policy.

If so, you shouldn’t spend more than half an hour total researching it. There’s a good chance you should spend much less time on it.

2. Are the options about equally good?

Is it difficult t...
Opportunities that surprised us during our Clearer Thinking Regrants program
by spencerg — 2022-11-07T13:09:14.523Z

This post was written by a subset of Clearer Thinking team members, and not all of the team members involved with the regranting necessarily agree with everything said here.

Update: the same week that we posted this, we were devastated to learn of the events surrounding FTX and to learn that the Future Fund team

resigned

. We slightly updated the conclusion, but we have not changed the body of the post (because the focus of it was to share information that we learned during the regranting program, and we still think it serves that purpose). Our thoughts are with the many customers and others who have been affected by the devastating events at FTX.

As part of the

Clearer Thinking Regrants program

, we evaluated over 630 project proposals and ended up with 37 finalists. In the many hours that we spent evaluating these finalists, we learned some things that surprised us and saw many potential opportunities to help the world that we hadn’t considered before. Of course, if you’re an expert in any of these areas, what surprised us non-experts may not surprise you.

Our aim in this post is to share object-level information that we did not know before the regranting program and that we think readers may find interesting or useful. In several cases, we are sharing the fact that specific organizations or projects have significantly more room for funding than we would have guessed, even after accounting for the

outcomes

of our regrants program.[1] We hope that readers find this information useful. By highlighting some organizations that have room for more funding, we hope that this will lead to impactful giving opportunities.[2]

Summary:

We were surprised that there hasn't been more work to quantify the risks from large-magnitude volcanic eruptions (considering the impact that such eruptions could have).
We were surprised that the

Rethink Priorities

Surveys Team

has significant room for funding for their own project ideas (since most of their work is research/consulting for existing orgs).
We hadn’t previously considered the extent to which the use of boiling as a water treatment method in low- and middle-income countries contributes to indoor air pollution (and, therefore, to potentially negative health outcomes).
We were surprised to learn that the

Happier Lives Institute

(HLI) has substantial room for additional

...
4 Key Assumptions in AI Safety
by Prometheus — 2022-11-07T10:50:40.211Z

AGI is already a very difficult topic, with a large amount of uncertainty, and not enough research being done for its core problems. However, I think even with this understanding, its vastness in scope of possibilities might still be severely under-realized. I’ve listed four major assumptions I think many make about what transformative AI will look like. Many of these are assumptions I have made in the past, as well as others. Some in the field have voiced their uncertainty, and it’s hard for me to determine just how many are working under these assumptions. I will also note that while I view these to be assumptions, that does not mean they are incorrect assumptions, only that acknowledging them as such might help us steer toward avenues currently under researched. My views are also continuously evolving, so I welcome any feedback if I’m somehow missing something crucial. I also expect to find a number of comments explaining that some actually are working on research outside of these assumptions, and welcome them to point me to the work they are doing.

AGI will happen before other dangerous, potentially catastrophic AIs

This is the biggest assumption, and I think the most probable to be wrong. What we are currently seeing is narrow AIs becoming more and more general, and more and more powerful. Alpha Go Zero was more general and powerful and Alpha Go, Alpha Zero more so than Go Zero, and

Mu Zero more powerful and general

than any of them. A few years ago, Natural Language Processing was limited to word-to-vector and markov chains. Now we have extremely sophisticated NLP trained on enormous word corpuses, and math transformers that can can

generalize out of distribution

. If this trend continues, we will develop very powerful AIs that are capable of posing an existential risk before we arrive at full AGI. I think these other Transformative AIs can be called Intermediate AIs.

There seems to be an assumption that AGI will occur with a breakthrough that is radically different from what we currently have. But if things continue to scale, I do not believe any breakthrough is needed to create very dangerous and powerful systems. With the rate of scale, we will likely see GPT-3s at

100x scale

in just a few years.

What makes this very dangerous, is that we might cross a dangerous point of no return without even realizing it...