trentmkelly/LessWrong-43k huggingface dataset:
"Map of AI Futures" - An interactive flowchart
by swante — 2024-11-27T21:31:40.269Z
Link: https://swantescholz.github.io/aifutures

Note: The linked site is a personal project, and all views expressed here are my own.

TL;DR

I created an interactive flowchart about various scenarios how the future of AI might play out.
By setting various conditional probabilities you can see charts of what the resulting estimated probabilities for good, ambiguous, and existentially bad outcomes are.
Use the site as a conversation starter and tool for reflection. You can share your results either as images of the resulting outcome charts, or just by sharing your personalized URL (which contains your estimates).
Feel free to clone, expand, and modify the site. I'd love to see how other people would structure their flowchart!

Additional context (copied from the website itself)

Background

Many experts and CEOs of the leading AI companies think advanced AI could pose catastrophic, or even existential risks to humanity.

I think we should take these concerns extremely seriously, and work hard to ensure that AI is developed in a way that is safe and beneficial for humanity.

This map is an attempt to visualize some of the key questions and uncertainties that could determine the future of AI, and to explore how different assumptions about these questions could lead to different outcomes.

How to use

White nodes are questions; grey nodes are intermediate states; red/yellow/green nodes are bad/ambiguous/good outcomes.
Adjust the sliders to set the conditional probabilities for each question.
The map and charts will update in real-time to show the probabilities for each node and outcome category.
Adjust the settings to make more likely paths bolder, or less likely paths transparent. By setting the minimum opacity to zero, impossible branches of the map will become fully transparent.
Click on a node to set it as the root node for the probability calculations, i.e. "what happens if we assume we have reached this state?". Click the same node again, or the "START HERE" node to reset the probability mass to the start.
For clicked and hovered-over nodes in the map, the corresponding sliders in the sidebar are highlighted in the same color, and vice-versa.
The number in each node represents the total probability (in %) to reach this node.
The numbers on edges represents the the chosen conditional probabilities for each question.

Caveats

The future is notoriously difficult to predict. I'm not an AI expert, and I'm sure this map is flawed in variou

...
How to solve the misuse problem assuming that in 10 years the default scenario is that AGI agents are capable of synthetizing pathogens
by jeremtti — 2024-11-27T21:17:56.687Z

Introduction

As artificial intelligence continues to revolutionize scientific research, the world is facing an unprecedented challenge in biosecurity. While AI's ability to democratize expert knowledge represents a remarkable achievement, it also lowers the barriers for malicious actors to create bioweapons. This threat operates across multiple scales - from individual actors with basic lab skills to organized terrorist groups and authoritarian states. Unlike traditional bioterrorism risks, which already required extensive expertise and resources, AI-assisted bioweapon development could potentially be pursued by anyone with access to standard lab equipment and the right language models. This post explores a three-layered defense strategy, examining how we can implement security measures at the conception, production, and diffusion stages of potential pathogens. Through a combination of traditional biosecurity measures, robust AI system design, and defensive AI applications, we might be able to mitigate these emerging risks without having to sacrifice the beneficial aspects of AI-assisted scientific research.

I) Designing

Current LLMs capabilities

Today, LLMs are already good and reliable guides to point to the sources you need and explain difficult concepts in simple words. With general-purpose AIs models like ChatGPT or Llama 2 being open source models, you can easily fine-tune or find fine-tuned versions of those models that haven’t been censored. In that case, such models can enable you to synthesize expert knowledge about the deadliest known pathogens, such as influenza and smallpox [1].

That is exactly what a group of students from MIT have done to prove this point [2]. During a hackathon, participants were asked to discover how to obtain and release 1918 Spanish flu by feeding both Llama 2 and an uncensored “Spicy” version with malicious prompts. While Llama 2 systematically denied answers, the Spicy model easily provided a step by step guide revealing key information to obtain the virus.

Studies haven’t already been designed to evaluate precisely how much uplift AI systems can provide in such misuses compared to using the internet only. However, given the time that the non-scientific students of the MIT took to complete the entire malicious process we can reasonably think that the uplift is sufficient to raise...
ARENA 4.0 Impact Report
by Chloe Li — 2024-11-27T20:51:54.844Z

If you're interested in helping to run the ARENA program, note that we're currently hiring for an Operations Lead! For more details, and to apply, see here.

Summary

The purpose of this report is to evaluate ARENA 4.0’s impact according to our four success criteria:

Source high-quality participants
Upskill these talented participants in ML skills for AI safety work
Integrate participants with the existing AI safety community and legitimise AI safety as a compelling field to work in
Accelerate participants’ career transition into AI safety

Overall, this iteration of ARENA was successful according to our success criteria.

We are happy that our 33 in-person programme participants rated their overall enjoyment of the ARENA programme at 9.1/10.

Criteria 1: Our participants were of high calibre, with ten having more than 4 years of experience as software engineers at top companies and four having a PhD-level academic qualification or higher.
Criteria 2: Our in-person programme lasts 4 weeks. The majority of participants felt they achieved the challenging concrete learning goals for each topic. Participants estimated the counterfactual time needed to learn the full ARENA content outside the programme as 10.2 weeks (even if they had access to ARENA’s material). We were particularly impressed with the capstone projects completed in the programme’s final week, two of which can be found here: capstone 1, capstone 2. We expect to see even more great work in the future!
Criteria 3: Participants rated the value of being in the LISA environment as 8.9/10. The top cited “most valuable gain” from the programme was meeting talented and like-minded peers in the AI safety community, emphasising the importance of running the programme in person at an AI safety hub. We are also glad that many participants commented on now feeling confident to take on ML safety research projects independently.
Criteria 4: Four participants had full-time AI safety roles at the end of the programme, and a further 24/33 participants planned to or were actively applying to AI safety roles at the end of the programme.

The structure of the report is as follows:

ARENA 4.0’s programme details
Method used for analysis
Criteria 1 - Source high-quality participants
Criteria 2 - Upskilling
Criteria 3 - Integration
Criteria 4 - Career Acceleration
Overall Programme Experience
Improvements for ARENA’s future in-p

...
On AI Detectors Regarding College Applications 
by Kaustubh Kislay — 2024-11-27T20:25:48.151Z

What are AI Detectors

You've probably already used them before, websites like GPTZero, ZeroGPT, Grammarly, Quilbot, all have their own AI Detectors. AI Detectors can be a combination of Pre-trained LLMS, Statistical Models, and Ml models using NLP(Natural Language Processing). The model will analyze linguistic patterns, sentence structures, and statistical measures like perplexity, the predictability of text, and burstiness, the sentence variety, to distinguish AI-generated content.

The Flaws

False Positives: Human-written text can be misclassified as AI-generated. This generally happens when there is minimal perplexity and burstiness. This translates to a simple and monotonous text which although AI like, might not have been written by AI.
False Negatives: Models like GPT-4 can produce text that passes as human-written, leading to undetected AI content. Recently Anthropic released Claude 3.5 Sonnet, but additionally and more recently they released their writing style function where you can add a certain writing style based on sets of text which it can replicate. AI will continue to improve its writing capabilities especially as scaling grows, so false negatives will be more apparent as time goes on.
Over-Reliance on Metrics: Measures like perplexity and burstiness can misclassify creative or technical writing. Like I mentioned before these are two of the main metrics all AI detectors use to distinguish AI writing. Text with low or consistent perplexity and burstiness are commonly marked as AI writing. But almost all academic writing also happens to fit that criteria, which is why academic research papers can easily be marked as AI when they are clearly not.
Transparency Issues: Many detectors lack clear explanations for flagged content, reducing trust and usability. It is important for us to understand that AI/ML models are essentially black boxes that are not interpretable to us humans whatsoever. In fact there are entire topics dedicated to making AI responses more interpretable such as Mechanistic Interoperability and Chain of Thought Reasoning(I won't go in detail but feel free to check them out they are fascinating). All in all not even the developers know with %100 certainty why their model says what they say.

How should you use AI Detectors

do not use AI to write full essays in the first place(for now). If you only sprinkle i

...
When the Scientific Method Doesn't Really Help...
by casualphysicsenjoyer — 2024-11-27T19:52:30.023Z
Link: https://chillphysicsenjoyer.substack.com/p/when-scientific-method-fails

Frameworks for living better

I’ve been reflecting a lot on how to live life better. Whilst there are many ways to do this effectively, I now have some conviction that the best method is by making better decisions. I guess most people on lesswrong will agree. Decisions seem to be the main channel to living better, since fairly motivated people generally have the same amount of working hours in the day. And so the only real differentiator you have versus peers (with a similar background / environment that you had) on getting better impact is via choices you make. I enjoy figuring out ways to have more money, do better research, live a more comfortable life, and have better health and relationships. But how does one make better decisions? I think the answer to that is by improving your decision making framework.

And if the title isn’t already a hint enough, this post is going to be about some errors in my own decision making framework in the past.

The Scientific Method

As someone who does science research, and as someone who has been trained in math and physics, I’ve had some prior ideas about what makes a good framework for making decisions. Initially I thought that the scientific method was a universally good way to guide oneself to making better decisions. The scientific method, to first order, is a recipe book that was originally used to arrive at truths about nature. To first order, the steps go like this:

make or guess a hypothesis about what is true, to come up with an updated ‘theory’ about the world. Example - diseases are caused by things called ‘germs’, that are really really small entities we can’t see but can be killed.
make sure that what is new in the model fits into your current existing model of what else is true in the world. Example - Since germs are really small, we can’t expect to see them with the naked eye, and so there is nothing obviously wrong with this germ theory.
figure out the novel thing that your theory predicts. Example - if we sanitise the rooms, we can kill the germs that cause disease!
if you’ve done 1., 2. and 3. correctly, congratulations! Now you have to verify that your new theory could be correct by testing the novel thing your theory predicts. Example - lets see what happens when we do surgery in dirty rooms versus clean rooms.

If your experiment turns out to be false, go back to step 1.

And I think sometimes, thinkin...
Causal inference for the home gardener
by braces — 2024-11-27T17:55:52.629Z

Note: This is meant to be an accessible introduction to causal inference. Comments appreciated.

Let’s say you buy a basil plant and put it on the counter in your kitchen. Unfortunately, it dies in a week.

So the next week you buy another basil plant and feed it a special powder, Vitality Plus. This second plant lives. Does that mean Vitality Plus worked?

Not necessarily! Maybe the second week was a lot sunnier, you were better about watering, or you didn’t grab a few leaves for a pasta. In other words, it wasn’t a controlled experiment. If some other variable like sun, water, or pasta is driving the results you’re seeing, your study is confounded, and you’ve fallen prey to a core issue in science.

When someone says “correlation is not causation,” they’re usually talking about confounding. Here are some examples:

A 2019 study found that

student test scores correlate with the number of books

in the home. But this isn’t a reason to start sending (lots of really thin!) books to everyone. It could simply be that books are a proxy for parental intelligence, or that parents with lots of books also hire tutors. Both associations would mean that the kids in book-heavy houses are smarter—without the books playing a causal role. Educated parents do a lot of things (e.g.,

piano lessons

) that get imitated but have dubious benefits.
Prison inmates tend to live shorter lives. Advocates and researchers have long blamed prison for this, citing unsafe and unsanitary conditions. But a recent careful study found that

convicted offenders sent to prison live

longer

, not shorter, than defendants who dodged a prison sentence. Deaths from car crashes, drug overdoses, and even heart attacks were lower in prison. The confound in the previous poorly-controlled studies was essentially lifestyle. People living on the margins of incarceration are already at much higher risk of death—even if they never get caught and sentenced. While there’s a correlation between prison time and an early death, it’s actually not due to prison. Prison increases lifespan for people unfortunate enough to be standing trial for serious offenses.
Cardiologists used to think that Vitamin E reduced heart attack risk because of several studies tracking health outcomes and diet. But years later, a

more reliable trial

(where subjects were randomized to receive Vitamin E suppleme

...
Repeal the Jones Act of 1920
by Zvi — 2024-11-27T15:00:06.801Z

Balsa Policy Institute chose as its first mission to lay groundwork for the potential repeal, or partial repeal, of section 27 of the Jones Act of 1920. I believe that this is an important cause both for its practical and symbolic impacts.

The Jones Act is the ultimate embodiment of our failures as a nation.

After 100 years, we do almost no trade between our ports via the oceans, and we build almost no oceangoing ships.

Everything the Jones Act supposedly set out to protect, it has destroyed.

Table of Contents

What is the Jones Act?

Why Work to Repeal the Jones Act?

Why Was the Jones Act Introduced?

What is the Effect of the Jones Act?

What Else Happens When We Ship More Goods Between Ports?

Emergency Case Study: Salt Shipment to NJ in the Winter of 2013-2014.

Why no Emergency Exceptions?

What Are Some Specific Non-Emergency Impacts?

What Are Some Specific Impacts on Regions?

What About the Study Claiming Big Benefits?

What About the Need to ‘Protect’ American Shipbuilding?

The Opposing Arguments Are Disingenuous and Terrible.

What Alternatives to Repeal Do We Have?

What Might Be a Decent Instinctive Counterfactual?

What About Our Other Protectionist and Cabotage Laws?

What About Potential Marine Highways, or Short Sea Shipping?

What Happened to All Our Offshore Wind?

What Estimates Are There of Overall Cost?

What Are the Costs of Being American Flagged?

What Are the Costs of Being American Made?

What are the Consequences of Being American Crewed?

What Would Happen in a Real War?

Cruise Ship Sanity Partially Restored.

The Jones Act Enforcer.

Who Benefits?

Others Make the Case.

An Argument That We Were Always Uncompetitive.

What About John Arnold’s Case That the Jones Act Can’t Be Killed?

What About the Foreign Dredge Act of 1906?

Fun Stories.

What is the Jones Act?

For a hundred years, we have required ships carrying cargo between two American ports to be American built, American owned (75% of stock owned by Americans), American manned and American flagged.

The Jones Act has since been continuously expanded [p.223-224] to ensure that there is no escape.

While protectionist laws around shipping are common, the severity of the Jones Act is unique. It is absolute. It is the most restrictive cabotage law among all OECD countries. Our domestic oceangoing trade is almost entirely unable to survive it – there are fewer than 100 oceangoing ships left in...
Long Live the Usurper
by pleiotroth — 2024-11-27T12:10:51.025Z

The notion of boxing –particularly failed boxing– is much more ubiquitous than people seem to give it credit for. It appears to me to be a common shape in systems which attempt to implement some sort of shape-agnostic competence-search at all. Intentionally or not. It is a relation between two games, where one of them only exists in order to produce desirable results for the players of the other one.

Fake Games

For any game (fray), there is a meta-game (throne) which allows for this game to be played. One of the moves within this meta-game was the decision that the rules of this fray matter, that its outcomes are allowed to have some sort of effect in the greater world. The next move of the throne can be to change this ruling. The meta game can also play the move of merely pretending that the fray matters, without ever actually heeding its rules. I can make a 100$ bet about a coin toss with you. You believe that this game really is law. If the coin comes up heads and I win then I will happily take my 100$, you will unhappily chalk it up to luck, believing that you are net neutral across branch-space and the illusion of the game mattering will be maintained. If it comes up tails however, I will not give you your money. I will point a gun at your head and take 100 bucks by force. There never was a gamble, only the illusion of one. The game we were really playing was a robbery with the pleasant (for me) side-effect that you will only go to the authorities half of the time.

With the Nixon Shock, Europe and Japan were rather abruptly confronted with the fact that “keep exchange rates stable to avoid crashes and have all economies indirectly, through foreign held dollars, backed by gold” was not in fact the game they were playing. The framework of Bretton Woods had made it look this way, but only because they had spent some time in a world in which this game looked remarkably similar to the real game of “make the dollar powerful”.

Dubiously fake games

The distinction is not nearly so clear cut though. In most cases the throne does not pick in advance whether it is faking or really playing by a certain set of rules. It will merely propose and permit actions which are beneficial in the moment without making any decision or commitment on whether the rules of all those little games beneath them are real. All parameters are free until they become relevant. So long as no con...
Hope to live or fear to die?
by Knight Lee — 2024-11-27T10:42:37.070Z

When fate is uncertain, be hopeful. Life is an insane gift and death is merely its absence. Look at the little bear cubs play-wrestling in the meadow, ever so carefree. You ask them, "do you not know 5/6 bear cubs will die before adulthood? Why are you taking life so lightheartedly?"

"Do you not know your futures are riddled with adversity? That you might not be strong enough? Why are you treating life so playfully?"

The bear cubs will ignore you. But if you could speak, they will explain. They'll explain that they are rugged little machines. Built exactly for this harsh game of life, where the outlook is so uncertain and perhaps bleak. But as it turns out, the optimal strategy to survive in such a harsh game of life... is actually... surprisingly... when there is so little you know and so little you can do about the frightening prospects. You should just be carefree. And happy. And silly. And play. And freaking love the game. And as natural selection has found, that is actually your best long term strategy.

(Credits to Patrice Schoefolt for bear photo)

PS: I'm sort of referring to AI risk. Some people think it is very high and are a little bit unhappy about it. This can affect the smartest/most rational people—just search for "Nina lost a tooth" in this post by Eliezer Yudkowsky.

If you aren't Eliezer Yudkowsky—in which case you're unlikely to convince anybody with your fears—but you also feel the risk is very high. Remember the bear cubs :)!

I'm not saying that there's nothing we can do about AI risk—I have a lot of hope that my idea can help a lot haha. I just feel that people who aren't not working on reducing the risk, don't need to bad about it. Even if you are, don't feel bad about it :)
The Queen’s Dilemma: A Paradox of Control
by Daniel Murfet — 2024-11-27T10:40:14.346Z

Our large learning machines find patterns in the world and use them to predict. When these machines exceed us and become superhuman, one of those patterns will be relative human incompetence. How comfortable are we with the incorporation of this pattern into their predictions, when those predictions become the actions that shape the world?

My thanks to Matthew Farrugia-Roberts for feedback and discussion.

As artificially intelligent systems improve at pattern recognition and prediction, one of the most prominent patterns that they'll encounter in the world is human incompetence relative to their own abilities. This raises a question: how comfortable are we with these systems incorporating our relative inadequacy into their world-shaping decisions?

To illustrate the core dynamic at play, consider a chess match where White is played by an AI, while Black is controlled by a team consisting of a human and an AI working in tandem. The human, restricted to moving only the queen, gets to play whenever they roll a six on a die; otherwise, their AI partner makes the move. The human can choose to pass, rather than move the queen. The AI on the Black team can play any piece at any time, including the queen.[1]

If the human aims to win, and instructs their AI teammate to prioritise winning above all else, it might develop strategies that minimise the impact of human "interference" – perhaps by positioning pieces to restrict the queen's movement. As the performance gap between the human and the AI on Black widens, this tension between achieving performance on the task and preserving meaningful human agency becomes more pronounced.

The challenge isn't about explicit control – the human can still make any legal move with the queen when given the chance. Rather, it's about the subtle erosion of effective control. The AI, making more moves and possessing superior strategic understanding, could systematically diminish the practical significance of human input while maintaining the appearance of cooperation. This distinction between de jure and de facto control becomes critical. We might accept if our queen becomes accidentally boxed in during normal play, but we bristle at the thought of our AI partner deliberately engineering such situations to mitigate our "unreliability."

The broader point is that even if AIs are completely aligned with human values, the very mec...