trentmkelly/LessWrong-43k huggingface dataset:
Timelines to Transformative AI: an investigation
by Zershaaneh Qureshi — 2024-03-26T18:28:50.408Z

Cross-posted on the EA Forum.
This post is part of a series by Convergence Analysis’ AI Clarity team.

Justin Bullock and Elliot Mckernon have

recently motivated

AI Clarity’s focus on the notion of transformative AI (TAI). In

an earlier post

, Corin Katzke introduced a framework for applying scenario planning methods to AI safety, including a discussion of strategic parameters involved in AI existential risk. In this post, I focus on a specific parameter: the timeline to TAI. Subsequent posts will explore ‘short’ timelines to transformative AI in more detail.

Feedback and discussion are welcome.

Summary

In this post, I gather, compare, and investigate a range of notable recent predictions of the timeline to transformative AI (TAI).

Over the first three sections, I map out a bird’s eye view of the current landscape of predictions, highlight common assumptions about scaling which influence many of the surveyed views, then zoom in closer to examine two specific examples of quantitative forecast models for the arrival of TAI (from Ajeya Cotra and Epoch).

Over the final three sections, I find that:

A majority of recent median predictions for the arrival of TAI fall within the next 10-40 years. This is a notable result given the vast possible space of timelines, but rough similarities between forecasts should be treated with some epistemic caution in light of phenomena such as Platt’s Law and information cascades.
In the last few years, people generally seem to be updating their beliefs in the direction of shorter timelines to TAI. There are important questions over how the significance of this very recent trend should be interpreted within the wider historical context of AI timeline predictions, which have been quite variable over time and across sources. Despite difficulties in obtaining a clean overall picture here, each individual example of belief updates still has some evidentiary weight in its own right.
There is also some conceptual support in favour of TAI timelines which fall on the shorter end of the spectrum. This comes partly in the form of the plausible assumption that the scaling hypothesis will continue to hold. However, there are several possible flaws in reasoning which may underlie prevalent beliefs about TAI timelines, and we should therefore take care to avoid being overconfident in our pre

...
Summer Program for High-Schoolers to start working on impactful projects
by nonplus — 2024-03-26T18:26:40.303Z
Link: https://forum.effectivealtruism.org/posts/d8mFsHDhDvChHEiwF/summer-program-for-high-schoolers-guest-speakers-yoshua

Please share this opportunity with high schoolers you know – we’d be grateful for your help spreading the word!

About Non-Trivial

The

Non-Trivial Fellowship

is now accepting applications.

It’s an online summer program for high school students aged 14-20 to start an impactful research or policy project.

Accepted fellows get access to:

A $500 scholarship
Up to $15,000 in funding
Guidance from facilitators at Oxford, Cambridge, and Stanford Universities
Guest speaker sessions, including Turing Award winner

Yoshua Bengio

and philosopher

Peter Singer

The summer cohort is running from July 8th – August 30th, 2024 and we will accept 200 people. The application has sections on probability, brainteasers, and game theory which many enjoy.

Apply by March 31st

, for increased interview chances and an early admissions decision.

How you could help

1. Let high school students you may know about Non-Trivial (e.g. your old school, gifted programs you used to be a part of).

2. Share on social media

Instagram:

https://www.instagram.com/p/C4-9dHWo3_1/

Click to tweet:

nntrvl.org/tweet
Modern Transformers are AGI, and Human-Level
by abramdemski — 2024-03-26T17:46:19.373Z

This is my personal opinion, and in particular, does not represent anything like a MIRI consensus; I've gotten push-back from almost everyone I've spoken with about this, although in most cases I believe I eventually convinced them of the narrow terminological point I'm making.

In the AI x-risk community, I think there is a tendency to ask people to estimate "time to AGI" when what is meant is really something more like "time to doom" (or, better, point-of-no-return). For about a year, I've been answering this question "zero" when asked.

This strikes some people as absurd or at best misleading. I disagree.

The term "Artificial General Intelligence" (AGI) was coined in the early 00s, to contrast with the prevalent paradigm of Narrow AI. I was getting my undergraduate computer science education in the 00s; I experienced a deeply-held conviction in my professors that the correct response to any talk of "intelligence" was "intelligence for what task?" -- to pursue intelligence in any kind of generality was unscientific, whereas trying to play chess really well or automatically detect cancer in medical scans was OK.

I think this was a reaction to the AI winter of the 1990s. The grand ambitions of the AI field, to create intelligent machines, had been discredited. Automating narrow tasks still seemed promising. "AGI" was a fringe movement.

As such, I do not think it is legitimate for the AI risk community to use the term AGI to mean 'the scary thing' -- the term AGI belongs to the AGI community, who use it specifically to contrast with narrow AI.

Modern Transformers[1] are definitely not narrow AI.

It may have still been plausible in, say, 2019. You might then have argued: "Language models are only language models! They're OK at writing, but you can't use them for anything else." It had been argued for many years that language was an AI complete task; if you can solve natural-language processing (NLP) sufficiently well, you can solve anything. However, in 2019 it might still be possible to dismiss this. Basically any narrow-AI subfield had people who will argue that that specific subfield is the best route to AGI, or the best benchmark for AGI.

The NLP people turned out to be correct. Modern NLP systems can do most things you would want an AI to do, at some basic level of competence. Critically, if you come up with a new task[2], one which the model has n...
My Interview With Cade Metz on His Reporting About Slate Star Codex
by Zack_M_Davis — 2024-03-26T17:18:05.114Z

On 16 March 2024, I sat down to chat with New York Times technology reporter Cade Metz! In part of our conversation, transcribed below, we discussed his February 2021 article "Silicon Valley's Safe Space", covering Scott Alexander's Slate Star Codex blog and the surrounding community.

The transcript has been significantly edited for clarity. (It turns out that real-time conversation transcribed completely verbatim is full of filler words, false starts, crosstalk, "uh huh"s, "yeah"s, pauses while one party picks up their coffee order, &c. that do not seem particularly substantive.)

ZMD: I actually have some questions for you.

CM: Great, let's start with that.

ZMD: They're critical questions, but one of the secret-lore-of-rationality things is that a lot of people think criticism is bad, because if someone criticizes you, it hurts your reputation. But I think criticism is good, because if I write a bad blog post, and someone tells me it was bad, I can learn from that, and do better next time.

So, when we met at the Pause AI protest on February 12th, I mentioned that people in my social circles would say, "Don't talk to journalists." Actually, I want to amend that, because when I later mentioned meeting you, some people were more specific: "No, talking to journalists makes sense; don't talk to Cade Metz specifically, who is unusually hostile and untrustworthy."

CM: What's their rationale?

ZMD: Looking at "Silicon Valley's Safe Space", I don't think it was a good article. Specifically, you wrote,

In one post, [Alexander] aligned himself with Charles Murray, who proposed a link between race and I.Q. in "The Bell Curve." In another, he pointed out that Mr. Murray believes Black people "are genetically less intelligent than white people."

End quote. So, the problem with this is that the specific post in which Alexander aligned himself with Murray was not talking about race. It was specifically talking about whether specific programs to alleviate poverty will actually work or not.

This seems like a pretty sleazy guilt-by-association attempt. I'm wondering—as a writer, are you not familiar with the idea that it's possible to quote a writer about one thing without agreeing with all their other views? Did they not teach that at Duke?

CM: That's definitely true. It's also true that what I wrote was true. There are different ways of interpreting it. You're welcome to inte...
Barefoot FAQ
by dkl9 — 2024-03-26T16:29:48.314Z
Link: https://dkl9.net/essays/barefoot_faq.html

Disclaimer: the post is provided as-is, without warranty of any kind, express or implied.

I am often barefoot, including in public. People seeing this give many of the same responses repeatedly. Here are my answers.

Why are you barefoot?

Sith it's funny.

But really, why?[1]

Walking barefoot lets me experience what's around me in more detail by exposing my senses to the texture of the ground, as opposed to the uniform isolation of shoes. This effect is much stronger from going barefoot compared to other clothing-removals, sith feet are the only body part reliably in contact with solid matter. Enjoying this may be a peculiarity of my mind.

Most shoes expect (and thereby promote) a narrower, pointier foot-shape and more rigid movement than is natural. Letting the feet spread and bend naturally may have musculoskeletal benefits.

Going barefoot is an act of experimental minimalism. By acclimating to a lack of shoes, I may be able to live without any, thereby reducing what I need as to broaden what I tolerate.

Most people I meet are used to everyone wearing shoes (or equivalent) most of the time. Approaching them barefoot shocks them. When their responses bothered me, persisting expanded my comfort zone and helped me develop audacity. Once I got used to it, these reactions came to amuse me. Once they get used to it, I will have finished opening their minds.

That's illegal.

It's usually legal.

I want to follow rules somewhat, so if a place I visit really does require footwear, and authorities/officials tell me they care, I'll put on shoes (if I have any with me) or leave.

You'll step on broken glass, or some other sharp object.

I would worry about that if sharp objects, pointing upward, were often found on the ground. In my experience, such things are rare. I should and do, for safety, look around a bit at where I walk.

You'll get an infection.

Only if I walk in mud or continually wet places, or leave a sole-wound open. I avoid those circumstances, so this concern is exaggerated.

Diabetics should always have footwear on.

Correct, and there are other health conditions with the same conclusion. My pancreas is perfectly fine, but readers may need to consider this.

Your toes will freeze off, or you could burn your feet.

This is the most credible concern I've heard so far. I got second-degree burns on my soles from walking barefoot on tarmac in summer. I should and do wear shoes in winter...
What's Your Best AI Safety "Quip"?
by False Name — 2024-03-26T15:35:33.178Z

Motivated by thinking gay rights were advanced by asking "When did you choose to be straight?" Which emphasised that what isn't a choice and doesn't harm others shouldn't be proscribed. Here, we're seeking a memetic way of framing the fact that the alignment problem is unsolved.

Author's "null quip": "Can you get a 5-year-old to do what you want? The way you want them to do it? Every time, guaranteed? What about your parents/partner/strangers, et al.? No? So there's no general way to get even a five year old to go your way - and for a computer that's able to do most of what we can (and you've seen that it's getting to be that way...?), and so much faster than we can, we're going to have much more trouble than a toddler can give us getting that to go our way, without a lot of time and thinking about that problem, no?"

Then introduce the petition, candidate's name, funding bucket, etc. Or, could just use it to defuse an optimist's reason-free optimism in a debate (as argument by analogy from one intelligent being to another, this doesn't seem fallacious).

The Question: can you think of anything as common-sensical and dispassionate as this[1], but more concise, more memetic? And do you know any way to "implement like hell"?

(Though, as Nvidia's "Blackwell" development attests, the Invisible Hand has determined to slash its own wrists, willing to risk it, anyway - and now it has a blade long enough for everyone else's too - and nor quips nor revolutions on history have impeded whatever Mama Money wants - Mama gets...)

^

For which reasons the "doomer" tag must go: doomer is doom-sayer, a definitionally marginal figure, best ignored. An AI "Safety Advocate" can't be poo-pooed without looking like/being an idiot inclusive-or asshole who hates being safe and alive.
What is the nature of humans general intelligence and it's implications for AGI?
by  — 2024-03-26T15:20:39.860Z

Humans seems to have some form of generality. We seem capable of a solving a large range of problems and the people that are capable on one aspect seem more capable in general. However the nature of this generality is important. There are at least two options that I've thought of.
1)A general intelligence is intrinsically better at solving problems
2) A general intelligence is better at solving problems in general because it is capable of absorbing social information about problems. And society has information about solving lots of different problems.
Option 2 is the one I lean towards as it fits with the evidence. Humans spent a long time in the stone age, with the same general architecture, but now can solve a much larger set of problems because of education and general access to information.
The difference is important because it has implications for the solving of novel problems (not solved by society today). If the form of generality we can make is all about absorbing social information there are no guarantees about it being able to go beyond the social knowledge in a principled way. Conceptual leaps to new understanding might require immense amounts of luck and so be slow to accumulate. ASIs might be the equivalent of us stuck in the stone age, at least to start with.
Are people thinking about these kinds of issues when considering time lines?
Economics Roundup #1
by Zvi — 2024-03-26T14:00:06.332Z

I call the section ‘Money Stuff’ but as a column name that is rather taken. There has been lots to write about on this front that didn’t fall neatly into other categories. It clearly benefited a lot from being better organized into subsections, and the monthly roundups could benefit from being shorter, so this will probably become a regular thing.

They Took Our Jobs

Quite the opposite, actually. Jobs situation remains excellent.

Whatever else you think of the economy, layoffs are still at very low levels, the last three years are the lowest levels on record, do note that the bottom of this chart is 15,000 rather than zero even without adjusting for population size.

Ford says it is reexamining where to make cars after the UAW strikes. The union responded by saying, essentially, ‘f*** you, pay me’:

“Maybe Ford doesn’t need to move factories to find the cheapest labor on Earth,” he said. “Maybe it needs to recommit to American workers and find a CEO who’s interested in the future of this country’s auto industry,” Fain said.

Which is, of course, what both of them would always say no matter what. I do presume that now that the UAW has raised the price and expected future price of dealing with them, Ford is now placing a higher priority in getting new factories and jobs outside the reach of the UAW.

Company Formations Seem Permanently Higher

Something rather remarkable happened here in 2020.

High-propensity applications means businesses likely to hire employees with payroll. The number was holding steady for a decade at 100k such applications per month, then it jumped and is now stable at closer to 150k, with other applications a similar percentage above previous trend. This really is a big game.

Whoops

So, this happened:

Tom Gara: NY Mag’s personal finance columnist was convinced by a cold caller claiming to be a CIA agent in Langley that she needed to empty her bank account, put the money in a shoebox and give it to a guy coming to meet her on the street.

Falling for this scam was unbelievably stupid. I do not understand how they pulled this off and she fell for it. However writing the details of this up is a public service, as is admitting that this happened to you, so we thank her for that.

Critical Bureaucracy Theory: A lot of people dunking on this, but she’s doing society a service by describing it in d

...
Enhancing biosecurity with language models: defining research directions
by mic — 2024-03-26T12:30:35.215Z
Link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4772574

This report explores the potential of large language models (LLMs) to enhance biosecurity. We conducted interviews with nine biosecurity experts to understand their daily tasks, and how LLMs could be more useful for their work. Our findings indicate that approximately 50% of our interviewees’ biosecurity-related tasks, such as gathering information from papers and reports, reviewing safety forms, and writing memos and summaries, have high potential for automation with LLMs. Skills critical for biosecurity work, like processing information and communicating effectively, could also be augmented by LLMs. However, current LLMs have limitations, such as often providing shallow or incorrect information. We provide suggestions for LLM-based tools that could significantly advance biosecurity efforts and list field-specific datasets to facilitate their development.

Authors: Michael Chen, Martin Holub, Cameron Tice

Illustrative image generated by DALL-E.

Introduction

The COVID-19 pandemic brought with it a staunch reminder that not only individuals, but also societies at large, are subject to threats from natural pathogens. While it caught the majority of us by surprise, the occurrence of a global pandemic was, and continues to be, something we must learn to expect. In the last 500 years, there have been at least 15 epidemics with a death toll in excess of 1 million (that is one every 33 years on average), and historically, pandemics were able to nearly wipe out whole civilizations. Even if the risk of a global pandemic occurring in any given year is small, it is not zero or negligible, resulting in a substantial threat to the future of humanity in the long run. In fact, there are other contributors to the gravity of this risk. While in the past we may have had to deal with natural pathogens only, the technological developments in biological engineering we have seen over past decades create a tangible risk of malicious actors designing and building synthetic pathogens. AI has been seen as an enabling technology across a range of domains, and it is a matter of concern to which extent it can contribute to biological risk.

Recent empirical evaluations of large language models (LLMs) have found that AI assistance can improve human attempts to plan biological attacks, at least to some extent (OpenAI 2024, Soice et al. 2023), although other reports have claim...
Legality as a Career Harm Assessment Heuristic
by jefftk — 2024-03-26T11:20:05.428Z

A question many people in the effective altruism movement have
struggled with around earning to give is

how
to handle

potentially
harmful careers. It's obviously self-defeating if you cause more
harm in earning your money than the good it does when you donate it,
but we want a higher threshold than that. As humans we need to have
approaches that account for our self-serving biases, where we tend to
underestimate the harm we cause and overestimate the good we do.
Additionally, some kinds of harm (ex: murder) do not seem like the
kind of thing you ought to be able to "cancel out" through donation,
even if the donation clearly has larger benefits (ex: saves vastly
many lives).

Unfortunately for most jobs, even questionable ones, the social impact
is very hard to work out. Consider someone deciding to go into the oil
industry: how much would they contribute to carbon emissions, after
considering the oil company's elasticity of labor and the elasticity
of production? Does cheaper oil displace even more carbon-intensive
coal? How likely are extreme climate outcomes? Is the benefit of
cheaper energy in lifting people out of poverty enough to make it
positive on its own? Making a high-quality impact estimate for a
career is a huge amount of work, and there are a lot of potential
careers, especially when you consider that some roles in the oil
industry might be far more replaceable than others.

What should we do in cases where the benefits seem much larger than
the harms, but the harms are still significant? A potential rule I've
been kicking
around is, "don't do work that is illegal, or that would be
illegal if the public knew what you were really doing." The idea is,
we have a system for declaring profitable activities with negative
externalities off limits, one that is intended for the more common
case when someone is keeping what they earn for their own benefit. But
we can't just use "don't do work that is illegal" because our
legislative system can be slow to react to changes in the world or
information that isn't yet widely available. For example, if most
people understood the cost-benefit tradeoffs in research to assess
the pandemic potential of viruses or create very powerful AI systems
I expect both would be prohibited.

It is, however, only a heuristic. For example, it gives the wrong
answer in cases where:

Crafting a law
proh

...
Orthogonality Thesis seems wrong
by Donatas Lučiūnas — 2024-03-26T07:33:02.985Z

Orthogonality Thesis (as well as Fact–value distinction) is based on an assumption that objective norms / values do not exist. In my opinion AGI would not make this assumption, it is a logical fallacy, specifically argument from ignorance. As black swan theory says - there are unknown unknowns. Which in this context means that objective norms / values may exist, maybe they are not discovered yet. Why Orthogonality Thesis has so much recognition?