trentmkelly/LessWrong-43k huggingface dataset:
AISN #22: The Landscape of US AI Legislation - 
Hearings, Frameworks, Bills, and Laws
by Dan H — 2023-09-19T14:44:22.945Z
Link: https://newsletter.safe.ai/p/the-landscape-of-us-ai-legislation

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

This week we’re looking closely at AI legislative efforts in the United States, including:

Senator Schumer’s AI Insight Forum
The Blumenthal-Hawley framework for AI governance
Agencies proposed to govern digital platforms
State and local laws against AI surveillance
The National AI Research Resource (NAIRR)

Subscribe here to receive future versions.

Senator Schumer’s AI Insight Forum

The CEOs of more than a dozen major AI companies gathered in Washington on Wednesday for a hearing with the Senate. Organized by Democratic Majority Leader Chuck Schumer and a bipartisan group of Senators, this was the first of many hearings in their AI Insight Forum.

After the hearing, Senator Schumer said, “I asked everyone in the room, ‘Is government needed to play a role in regulating AI?’ and every single person raised their hands.” Elon Musk, CEO of xAI, called the hearings “a great service to humanity.”

Senator Josh Hawley raised concerns that despite the hearings, “nothing is advancing” in terms of legislation. Below, we’ll discuss several bills on AI policy which have been introduced to Congress, none of which have come to a vote.

The Blumenthal-Hawley Framework

Senator Hawley recently introduced a framework for AI legislation alongside Senator Richard Blumenthal. The pair lead the Senate Judiciary Subcommittee on Privacy, Technology and the Law, and have hosted three hearings on AI policy over the last five months.

The Blumenthal-Hawley framework recommends:

Licensing. General purpose AI systems and AIs used in high-risk situations should be required to obtain a license from an independent oversight body. There should be strong rules against conflicts of interest to prevent regulatory capture.
Legal Liability. Congress should clarify that Section 230 does not apply to AI (Sens. Blumenthal and Hawley have introduced a bill that would accomplish this), and hold AI companies liable for emerging harms such as deepfakes and election interference.
Transparency. Companies should be required to publicly report details about their training data, and label AI outputs with easily identifiable watermarks. The federal AI oversight body should maintain a database of AI harms.
Human in the loop. Users should be notified when they are

...
Compilation of Profit for Good Redteaming and Responses
by Brad West  — 2023-09-19T13:34:34.675Z

TLDR: Document compiling redteaming on Profit for Good available, with public comments invited and appreciated. This thread further includes two argument headings against Profit for Good to be discussed in the comments.

As one might suspect given that I have started a nonprofit to advance it, I believe Profit for Good, the philanthropic use of businesses with charities in vast majority shareholder position, is an immensely powerful tool. To get a sense of the scale of good that Profit for Good could do, see 14:22 of Cargill’s TED Talk for what 3.5T could do, which would be about 3.5 years of 10% of global net profits. Here is a good place to start regarding why we think it is promising and here is a longer reading list.

However, throughout time, many people have brought up criticisms and red-teaming. It is important to realize that Profit for Good, especially in initial stages, would likely require the use of philanthropic funds for launching, acquiring, and accelerating businesses that counterfactually could be used to directly fund extremely impactful charities that we know can do good. For this reason, it is critical that all of the potential reasons that Profit for Good might not succeed, not be the best use of funds, and/or have unintended negative consequences be considered. For this reason, I am in the process of aggregating red-teaming from wherever I can find it, organizing such redteaming, according to argument heading, verbatim where possible, and doing the same for the responses to such red-teaming. I am in the process of forming my own syntheses of the criticisms and responses. I am also continuing to compile new arguments and formulations/evidence regarding former arguments.

If you are interested in supplying new arguments under existing headings, your own syntheses of existing materials, or new argument headings, you may feel free to add a comment on the existing document or email me at brad@consumerpowerinitiative.org and I will integrate it verbatim into the document (please indicate if you would like such argument to be anonymous for any reason, or I will credit you with the contribution.

I will include in this piece the first two argument headings,...
[Link post] Michael Nielsen's "Notes on Existential Risk from Artificial Superintelligence"
by Joel Becker — 2023-09-19T13:31:02.298Z
Link: https://michaelnotebook.com/xrisk/index.html

Summary

From the piece:

Earlier this year I decided to take a few weeks to figure out what I think about the existential risk from Artificial Superintelligence (ASI xrisk). It turned out to be much more difficult than I thought. After several months of reading, thinking, and talking with people, what follows is a discussion of a few observations arising during this exploration, including:

Three ASI xrisk persuasion paradoxes, which make it intrinsically difficult to present strong evidence either for or against ASI xrisk. The lack of such compelling evidence is part of the reason there is such strong disagreement about ASI xrisk, with people often (understandably) relying instead on prior beliefs, self-interest, and tribal reasoning to decide their opinions.
The alignment dilemma: should someone concerned with xrisk contribute to concrete alignment work, since it's the only way we can hope to build safe systems; or should they refuse to do such work, as contributing to accelerating a bad outcome? Part of a broader discussion of the accelerationist character of much AI alignment work, so capabilities / alignment is a false dichotomy.
The doomsday question: are there recipes for ruin -- simple, easily executed, immensely destructive recipes that could end humanity, or wreak catastrophic world-changing damage?
What bottlenecks are there on ASI speeding up scientific discovery? And, in particular: is it possible for ASI to discover new levels of emergent phenomena, latent in existing theories?

Excerpts

Here are the passages I thought were interesting enough to tweet about:

"So, what's your probability of doom?" I think the concept is badly misleading. The outcomes humanity gets depend on choices we can make. We can make choices that make doom almost inevitable, on a timescale of decades – indeed, we don't need ASI for that, we can likely4 arrange it in other ways (nukes, engineered viruses, …). We can also make choices that make doom extremely unlikely. The trick is to figure out what's likely to lead to flourishing, and to do those things. The term "probability of doom" began frustrating me after starting to routinely hear people at AI companies use it fatalistically, ignoring the fact that their choices can change the outcomes. "Probability of doom" is an example of a conceptual hazard5 – a case where merely using the concept may lead to mistakes in your thinking. Its

...
Do LLMs Implement NLP Algorithms for Better Next Token Predictions?
by simeon_c — 2023-09-19T12:28:45.660Z

Do you think that base LLMs implement forms of "meta" algorithms like TF-IDF to predict the next token better?

Intuitively, it would be pretty smart to proceed that way because a text tends to be coherent in terms of style and in terms of vocabulary, so it would be very useful for an LLM to implement internally some NLP algorithms that help a ton in guessing the next words.

It's a question I'm interested in because if there's any "meta" algorithm implemented internally whose efficiency depends on the training setup the LLM is in, it increases the chances that situational awareness would arise.
On martingales
by Joey Marcellino — 2023-09-19T11:39:22.454Z

Suppose we’re in Monaco (though not Monacan citizens) and itching for some roulette. We roll up to the Monte Carlo and bet $10 on black, but the ball stops on red. “No problem,” we think, “we just got unlucky.” We go to bet another $10, but pause for some quick math. We’re currently $10 in the hole; if we bet another $10, we’ll be down $20, and winning that bet will only bring us back to even. We realize that, in order to meet our original goal and leave the table $10 richer, we need to bet enough to recoup our losses, plus an extra $10, so instead we bet $20. Success! Congratulating ourselves on our brilliant strategy, we decide to spend the rest of the day milking it for all it’s worth, and by the evening we’re broke on the curb.

This sort of strategy is called a martingale, and it’s usually a bad idea, for a few reasons. For one, the Infinite Monkey theorem guarantees that, for any finite-size bankroll, we’ll eventually encounter a string of losses long enough that we can’t continue betting. For another, the size of our required bet grows at least exponentially in the number of consecutive losses, so the number of losses required to bankrupt us can be much lower than what one might expect. That said, the idea of the martingale has always kind of appealed to me (fortunately, I don’t live near Monaco), so I wanted to think about it a bit more quantitatively. Specifically, I wanted to know:

Given some level of risk tolerance and some betting strategy, how much profit can we make playing European roulette (which has slightly better odds than American), betting on black at 1:1 payout, using a martingale, assuming we don’t go bankrupt before hitting our risk threshold?

Let’s say we have N dollars in our bank account, and we adopt the following strategy. On the first spin, we bet some fraction f of our account size Nf. If we lose, we bet to recoup our losses, plus an additional Nf. We repeat this process until we win, in which case the bet resets to Nf, or run out of money.

I’ll make a couple of simplifying assumptions, namely:

we neglect the growth of N as we win
when the required bet size exceeds N, we lose everything

Our bet size after n losses is

bn=n−1∑i=0bi+Nf=Nf(2n+n).

We’d like to know the number of consecutive spins we can lose before going bankrupt nmax, so we set this equation equal to ...
Luck based medicine: angry eldritch sugar gods edition
by Elizabeth — 2023-09-19T04:40:06.334Z

Introduction

Epistemic status: everything is stupid. I’m pretty sure I’m directionally right but this post is in large part correcting previous statements of mine, and there’s no reason to believe this is the last correction. Even if I am right, metabolism is highly individual and who knows how much of this applies to anyone else.

This is going to get really in the weeds, so let me give you some highlights

1-2 pounds of watermelon/day kills my desire for processed desserts, but it takes several weeks to kick in.

It is probably a microbiome thing. I have no idea if this works for other people. If you test it let me know.

I still eating a fair amount of sugar, including processed sugar in savory food. The effect is only obvious and total with desserts.

This leads to weight loss, although maybe that also requires potatoes? Or the potatoes are a red herring, it just takes a while to kick in?

Boswellia is probably necessary for this to work in me, but that’s probably correcting an uncommon underlying defect so this is unlikely to apply widely.

Stevia-sweetened soda creates desire for sugar in me, even though it doesn’t affect my blood sugar. This overrides the watermelon effect, even when I’m careful to only drink the soda with food.

My protein shakes + bars also have zero-calorie sweeteners and the watermelon effect survives them. Unclear if it’s about the kind of sweetener, amount, or something else.

Covid also makes me crave sugar and this definitely has a physiological basis.

Metabolism is a terrifying eldritch god we can barely hope to appease, much less understand.

Why do I believe these things? *Deep breath* this is going to take a while. I’ve separated sections by conclusion for comprehensibility, but the discovery was messy and interconnected and I couldn’t abstract that out.

Boswellia

Last October I told my story of luck based medicine, in which a single pill, taken almost at random, radically fixed lifelong digestion issues. Now’s as good a time as any to give an update on that.

The two biggest effects last year were doubling my protein consumption, and cratering sugar consumption. I’m still confident Boswelia is necessary for protein digestion, because if I go off it food slowly starts to feel gross and I become unable to digest protein. I’m confident this isn’t a placebo becau...
Don't Think About the Thing Behind the Curtain.
by keltan — 2023-09-19T02:07:39.853Z

Foreword

This is a submissions to the rational animation writing contest this script is Halloween themed and inspired by their most recent video (Please imagine Yann LeCun in his Pjs/HJ). I wanted to get it out ASAP.

I have left out any types of directions for animation, I wasn't sure if it was expected.

Thank you to Rational Animations for leaving the gates open for scripts. I’m quite new to LW and rationality. But I’m excited to have this opportunity.

Any and all feedback on the script, it's clarity, the moral, corrections, my writing quality are welcomed. Please don't feel like you need to be nice or hold back with your criticisms.

Recommended listening:

Soundtrack from the movie "The Babadook"

The Thing Behind the Curtain

Daddy lays you down in bed, time to rest that filled up head. Brimming with talk of creature unseen, one that might rip out your spleen. Teeth like blades and drool that shines green.

Mommy puts on your night clothes, and the long hanging drapes your daddy does close.

Through a cracked door and a smile, "a word of advice" Daddy does bring, "Don't open your eyes or pay attention to... The Thing. It may dance, it may sing. But if you do not look it can not get in"

And with those words your parents are gone, time for them to go watch whatever is on.

On your back, looking up at luminous plastic stars, a lava lamp, a cardboard model of mars, light scratches from the ant farm beside your head, jars of “collected fungi” under the bed.

“I am a scientist”, you say to yourself. “I must eschew childish belief, leave it up on the shelf. No matter what I think I feel, I’ve got to remember Sagan said, demons aren't real.”

And then you heard the curtains move. The game was a foot, it would annoy and you would not look.

The first gambit The Thing Behind the curtains tried, was making noise while continuing to hide.

Natural to begin, they made you question your mind, a slight tug at the rails the vails hang upon. A gust of wind? Was the window open?

Then pebbles that tap against the window, be not mistaken this is no friend but a foe. Ignore the sounds, to sleep you must go.

You are no fool, monsters do not exist. You know this... yet, your mind and the sounds persist. To remedy this fallacy, that is what it is, now you see. In your mind you return to happy memory, with your parents by the sea, the time when Daddy came to get you when you climbed too high in the tr...
Panel with Israeli Prime Minister on existential risk from AI
by Michaël Trazzi — 2023-09-18T23:16:39.965Z
Link: https://x.com/netanyahu/status/1703823935060451628?s=20

Israeli Prime Minister, Musk, Tegmark and Brockman discuss existential risk from AI.

Nothing truly revolutionary was said. I think the most interesting bits are that the Prime Minister seems to be taking AI risk seriously, has in mind exponential progress, wants to prevent monopolies and thinks that we roughly have 6 years before things change drastically.

Some quotes from the prime minister:

I had a conversation with one of your colleagues, Peter Thiel, and he said to me, "Oh, it's all scale advantages. It's all monopolies." I said, well, yeah, I believe that, but we have to stop it at a certain point because we don't want to depress competition.

AI is producing, you know, this wall [hands movement gesture some exponential progress wall]. And you have these trillion dollar companies that are produced what, overnight? And they concentrate enormous wealth and power with smaller and smaller number of people. And the question is, what do you do about that monopoly power?

With such power comes enormous responsibility. That's the crux of what we're talking about here, is how do we inject a measure of responsibility and ethics into this, into this exponentially changing development?

Max [Tegmark]'s book takes you to the existential question of whether, you know, you project basically machine intelligence or human intelligence into the cosmos. Human intelligence turned into machine intelligence, into the cosmos and so on. That's a big philosophical question. I'd like to think we have about six years for that.

I think we have to conduct a robust discussion with the other powers of the world based on their self-interest as you began to do. And I think that's a pioneering work. And I think we have a shot maybe at getting to some degree of control over our future, which could be amazing.
Some reasons why I frequently prefer communicating via text
by Adam Zerner — 2023-09-18T21:50:48.620Z

I often prefer communication via text to chatting in person, on the phone, or via a video call. Let's call this latter grouping "oral communication" and the former "textual communication".

This preference applies to various contexts: social, personal, work, intellectual. And I don't think I personally know anyone who has a stronger preference for this than I do, although I suspect that there are at least a handful of people on LessWrong with stronger preferences than mine.

Here are some reasons that stick out to me for why I frequently prefer textual communication to oral communication, ordered very roughly from most to least important.[1]

I find that, with oral communication, especially in groups, you frequently just end up in a "tangent frenzy" instead of discussing any one thing to something that at least vaguely resembles completion.
It's easier/possible to discuss multiple threads in parallel.
Being asynchronous, I, as well as the person(s) I'm talking with, can take my/their time and think before responding. This helps in figuring out your thoughts as well as with expressing them more clearly.
Personally I find both interruptions and inverted interruptions a good amount more unpleasant than others do.
The conversation can last a lot longer. With in person conversations, you usually will need to end them for practical reasons such as "it's 11pm and time to head home". But with textual communication, there's never really these sorts of obstacles.
I think there is something about textual communication that is more meritocratic. It better holds people accountable. If you say something dumb, someone else can easily call it out in a thread, and if you don't respond to that thread, well, it kinda just sits there and makes you look bad. Sort of. On the other hand, in oral communication, you can say something dumb, the conversation moves on, and people don't get a proper chance to call you out for the dumb thing.
It's easy to share the conversation with others.[2]
For various reasons, sometimes it's nice to be able to reference the conversation in the future.
I like being able to quote things. Sometimes quoting something the other person said to make it clear that I am responding to it. Other times quoting something someone who isn't part of the conversation said, but that I think is relevant.
If I have something to say that has a dependency, I can say it, link to the depe

...
Forecasting for Policy (FORPOL) - Main takeaways, practical learnings & report
by janklenha — 2023-09-18T17:44:16.305Z

Tl;dr: In Czech Priorities, a think tank based in Prague, together with Metaculus, we recently ran a complex forecasting tournament funded by EAIF to support Czech policymaking. Today, we are publishing a

write-up

of our findings from the tournament, as well as supplementary materials to help any groups of forecasters looking to do the same.

The article was also cross-posted on EA Forum.
______

From October 2022 to March 2023, we ran a forecasting tournament with a total of 54 questions. In March, we have discussed some of our preliminary findings in

this post

.

Almost all of our forecasting questions were developed in cooperation with 16 different public institutions and ministerial departments. Each institution or department defined its most useful forecasting topics, participated in a workshop to define specific questions with us, and was later provided with the results. This was intended as a proof of concept of one possible approach to incorporating forecasting in public decision-making.

Once defined, our forecasting questions were then posted on a private Metaculus sub-domain (in Czech language), where an average of 72 forecasters had the opportunity to address them as they would any other question on Metaculus (median of 18 predictions per user). Throughout the tournament, we produced 16 reports detailing the rationales and forecasts, to be used by the cooperating institutions. The institutions and their topics were listed in our

previous post

.

This approach, in combination with multiple media appearances, also allowed us to strengthen our position as one of the leading Czech institutions with expertise in foresight methods.

We've created a write-up detailing our steps in both talking to institutions and managing the tournament with its specifics. Here are our five overarching takeaways for groups of forecasters who want to make an impact through policymaking, and some specific lessons learned:

General takeaways

Develop partnerships with policymakers across various policy areas. Finding policymakers eager to try analytical innovations is usually not the main bottleneck.
Diversify your portfolio of foresight methods. Particularly in the EU, policymakers are becoming increasingly aware of the importance of foresight. Their familiarity with methods such as scenario planning or horizon scanning can grant more inroads into the policy process for f

...