trentmkelly/LessWrong-43k huggingface dataset:
Predictions of Near-Term Societal Changes Due to Artificial Intelligence
by Annapurna — 2024-12-29T14:53:57.176Z
Link: https://jorgevelez.substack.com/p/near-term-ai

The intended audience of this post is not the average Lesswrong reader; it is more for our family and friends that do not grasp the transformative power of AI in its current form. I am posting it here so that I can come back in a year and see how my predictions are doing. I hope that some of you also comment your predictions.

Over the last two years, I’ve been thinking a lot about how this new dawn in artificial intelligence (AI) could potentially affect things in society that we take for granted. I’ve consumed an enormous amount of information on the subject via books, podcasts, blog posts, conversations, and other forms of media in an attempt to grasp what lies ahead for us.

Unfortunately, I haven’t been able to find a good online community with a repository of ideas about potential near-term changes to society in this era of pre-Artificial General Intelligence (AGI). Most discussions seem to eventually drift toward what happens to the human race if or when AGI arrives. My feeling is that things will get too strange to even think about when that happens, so I’ve been trying to focus on the near future and how currently available AI technology—and the next few iterations—could change society.

The following paragraphs are my thoughts on the subject. I didn’t come up with any of these ideas entirely on my own; most of these thoughts were formulated through things I’ve read, heard, or discussed with others. Some of these predictions are already on their way, while others you might consider pure fiction. I’m using this post as a starting point to examine what we can expect over the next 2–10 years.

White-Collar Work

I’m going to start with the low-hanging fruit, as this is something most people reading this post have likely seen in their work. AI, in its current form, can significantly increase your productivity. If you’re not using a frontier model to help with your work, start using it now. If your work limits the use of AI, I suggest you experiment with it in your free time. Think of learning how to use AI as a self-directed educational course—just without the certification at the end. If you need a place to begin, start by reading Ethan Mollick’s Substack.

You might not realize it, but hundreds of companies are spending billions of dollars creating tailor-made versions of AI for your type of work. Are you in HR? There are companies developing AI assistants ...
Considerations on orca intelligence
by Towards_Keeperhood — 2024-12-29T14:35:16.445Z

(EDIT 2025-03-15: I've added a comment which you might want to read after the post.)

Follow up to: Could orcas be smarter than humans?

(For speed of writing, I mostly don't cite references. Feel free to ask me in the comments for references for some claims.)

This post summarizes my current most important considerations on whether orcas might be more intelligent than humans.

Evolutionary considerations

What caused humans to become so smart?

(Note: AFAIK there's no scientific consensus here and my opinions might be nonstandard and I don't provide sufficient explanation here for why I hold those. Feel free to ask more in the comments.)

My guess for the primary driver of what caused humans to become intelligent is the cultural intelligence hypothesis: Humans who were smarter were better at learning and mastering culturally transmitted techniques and thereby better at surviving and reproducing.

The book "the secret of our success" has a lot of useful anecdotes that show the vast breath and complexity of techniques used by hunter gatherer societies. What opened up the possibility for many complex culturally transmitted techniques was the ability of humans to better craft and use tools. Thus the cultural intelligence hypothesis also explains why humans are the most intelligent (land) animal and the animals with the best interface for crafting and using tools.

Though it's possible that other factors, e.g. social dynamics as described by the Marchiavellian Intelligence Hypothesis, also played a role.

Is it evolutionarily plausible that orcas became smarter?

Orcas have culturally transmitted techniques too (e.g. beach hunting, making waves to wash seals off ice shells, faking retreat tactics, using bait to catch birds, ...), but not (as far as we can tell) close to the sophistication of human techniques which were opened up by tool use.

I think it's fair to say that being slightly more intelligent probably resulted in a significantly larger increase in genetic fitness for humans than for orcas.

However, intelligence also has its costs: Most notably, many adaptations which increase intelligence route through the brain consuming more metabolic energy, though there are also other costs like increased childbirth mortality (in humans) or decreased maximum dive durations (in whales).

Orcas have about 50 times the daily caloric intake of humans, so they have a lot more metabolic ene...
The Legacy of Computer Science
by Johannes C. Mayer — 2024-12-29T13:15:28.606Z
Link: https://groups.csail.mit.edu/mac/users/gjs/essays/remember.pdf

"Computer Science" is not a science, and its ultimate signiﬁcance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology—the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classi-
cal mathematical subjects. Traditional mathematics provides a framework for dealing precisely with notions of "what is." Computation provides a framework for dealing recisely with notions of "how to".

-- Sussman

Start reading on page 3 from "We have been programming universal computers for about 50 years."
Shallow review of technical AI safety, 2024
by technicalities — 2024-12-29T12:01:14.724Z

from

aisafety.world

The following is a list of live agendas in technical AI safety, updating

our post

from last year. It is “shallow” in the sense that 1) we are not specialists in almost any of it and that 2) we only spent about an hour on each entry. We also only use public information, so we are bound to be off by some additional factor.

The point is to help anyone look up some of what is happening, or that thing you vaguely remember reading about; to help new researchers orient and know (some of) their options and the standing critiques; to help policy people know who to talk to for the actual information; and ideally to help funders see quickly what has already been funded and how much (but this proves to be hard).

“AI safety” means many things. We’re targeting work that intends to prevent very competent cognitive systems from having large unintended effects on the world.

This time we also made an effort to identify work that doesn’t show up on LW/AF by trawling conferences and arXiv. Our list is not exhaustive, particularly for academic work which is unindexed and hard to discover. If we missed you or got something wrong, please comment, we will edit.

The

method section

is important but we put it down the bottom anyway.

Here’s a spreadsheet version also.

Editorial

One commenter said we shouldn’t do this review, because the sheer length of it fools people into thinking that there’s been lots of progress. Obviously we disagree that it’s not worth doing, but be warned: the following is an exercise in quantity; activity is not the same as progress; you have to consider whether it actually helps.

A smell of ozone. In the last month there has been a

flurry

of hopeful or despairing

pieces

claiming that the next base models are not a big advance, or that we hit a

data wall

. These often ground out in gossip, but it’s true that the next-gen base models are held up by something, maybe just inference cost.

The pretraining runs are bottlenecked on electrical power too. Amazon is at present not getting its

nuclear datacentre

.

But overall it’s been a big year for capabilities despite no pretraining scaling.

I had forgotten long contexts were so recent; million-token windows only arrived in February. Multimodality was February. “Reasoning” was September. “Agency” was

October

.
I don’t trust benchm

...
Dishbrain and implications.
by RussellThor — 2024-12-29T10:42:43.912Z

I believe that AI research has not given sufficient attention to learning directly from biology, particularly through the direct observation and manipulation of neurons in controlled environments. Furthermore, even after learning all that biology has to offer, neurons could still play a part in the post TAI world economy as they could be cheaper and faster to grow than chips are to manufacture.

Pre TAI – study neurons to greatly increase learning capability

As I have said in other places on this site, I believe that the current transformer architecture will not scale to TAI, because it does not learn fast enough or generalize well enough from data compared to biology. For example, Tesla Autopilot has been trained on over 10,000 times more data than a human encounters in their lifetime, yet it still falls short of human-level performance. I don’t think this is because of anything Tesla is doing wrong in their training. Biology or the “neural code” is still much better at generalizing quickly from high bandwidth, correlated, unstructured data.

If we could learn the details of how biology does it, we would get a massive increase in capability. One of the most prominent examples of directly controlling neurons is Cortical Labs’ Dishbrain project. With the following article and quote

“Not only can you get meaningful learning, you get that meaningful learning incredibly rapidly, sort of in line with what you might expect for biological intelligence.”

As far as I am aware they are not directly trying to crack the neural code, but focusing on other applications, even providing an API where you can control neurons. Given the massive budgets now spent on getting to AGI, I believe there is a significant missed opportunity there. Characterizing how such neurons learn with a complete range of inputs and comparing to state of the art AI would clarify the differences.

Although it’s long been known that the brain adapts its structure to its inputs, experiments such as this provide further opportunities for valuable insights. The idea is that connectome scans are done at various stages of brain rewiring, and the amount of data required to get to such a point and resulting brain structures are quantified. This could give insight on how more complicated brain structures form than is possible with the smaller Dishbrain situations.

Post TAI – could neurons be cheaper and faster t...
Notes on Altruism
by David Gross — 2024-12-29T03:13:09.444Z

This post examines the virtue of altruism. I’m less interested in breaking new ground, more in synthesizing the wisdom I could find about this virtue and how to cultivate it.

Much about altruism on LessWrong and nearby sites concerns “effective altruism,” which takes altruistic motivation as a given and investigates how to do it most efficiently. This post instead investigates questions like: Should we be altruistic? How can we practice altruism well (on more dimensions than efficiency)? How can we become genuinely altruistic?

What is the virtue of altruism?

Altruism, as I will use the term, means acting with the motivation of promoting the welfare of someone else, even at cost to yourself.[1] This is broader than some definitions of altruism: it allows self-interested motives to also be present and does not require that the act be actually harmful or dangerous to the altruist.[2]

A virtue is a characteristic habit that promotes or exhibits the flourishing of the person who exhibits it. This suggests a paradox (how can you best flourish by deprioritizing your own welfare?), which is something I address further on.

Altruism is action, which distinguishes it from some definitions of compassion, concern, consideration, pity, sympathy, empathy, etc. that can be satisfied by more passive emotions or evaluations. It includes an explicit motivation to promote the other’s welfare, which distinguishes it from some varieties of care, kindness, justice, etc. which may promote another person’s welfare while being motivated in some other way.

My definition excludes acts that are done for the sake of principle or for some other abstract thing (e.g. “for art’s sake” or “to honor my ancestors” or “for our flag”). Some forms of consequentialist or deontological ethical decisions might be interpreted this way: as altruistic sacrifice motivated not primarily by concern for the people served but by the principle that such service embodies.[3]

My definition is neutral about whom to be altruistic toward. It might be better for various reasons, or in the service of various other virtues, to be preferentially or exclusively altruistic to particular people (people one is close to, people who are particularly needy, people who are deserving) but the definition I’m using declines to take a stand about that.

Egoist acts prioritize maximizing one’s own welfare. Egoism would be a doctrine t...
What happens next?
by Logan Zoellner — 2024-12-29T01:41:33.685Z

Two years ago, I noted that we had clearly entered the era of general intelligence, but that it was "too soon" to expect widespread social impacts.

In the last 2 years, AI has gone from the green line to the orange line

In those 2 years, AI development has followed the best possible of the 3 paths I suggested (foom/GPT-4-takes-my-job/Slow Takeoff). Returns to scale seem to be delivering a steady ~15 IQ points/year and cutting edge models appear to be largely a compute-intensive project that allows (relatively) safety-conscious leading labs to explore the new frontiers while others reap the benefits with ~1 year delay.

Possibly the most important graph in the world right now

If I had to identify 3 areas where GPT-3.5 was lacking, it would have been:

reasoning
modeling the real world
learning on-the-fly

Of those three, reasoning (o3) is largely solved and we have promising approaches for world modeling (genie2). Learning on-the-fly remains, but I expect some combination of sim2real and muZero to work here.

Hence, while in 2023 I wrote

For any task that one of the large AI labs (DeepMind, OpenAI, Meta) is willing to invest sufficient resources in they can obtain average level human performance using current AI techniques.

I would now write

Going forward, we should expect to see job automation determined primarily not based on technical difficulty but rather based on social resistance (or lack thereof) to automating that task.

Already, the first automated jobs are upon us: taxi driver, security guard, amazon worker. Which jobs will be automated next will be decided by a calculation that looks at:

social desirability
lack of special interests/collective bargaining (the dockworkers are never getting automated)
low risk (self driving is maybe the exception that prove the rule here. Despite being safer than human for years Waymo remains restricted to a few cities)

Security guard at mall is the prototypical "goes first" example, since:

everyone is in favor of more security
security guards at malls are not known for being good at collective bargaining
mall security guards have a flashlight (not a gun)

Brain surgeon is the prototypical "goes last"example:

a "human touch" is considered a key part of the health care
doctors have strong regulatory protections limiting competition
Literal lives at at stake and medical malpractice is one of the most legally perilous areas imagin

...
Does Claude Prioritize Some Prompt Input Channels Over Others?
by keltan — 2024-12-29T01:21:26.755Z

Epistemic Status: I am not an AI researcher, I'm a Psych student. Strong confidence in data collection, moderate confidence in some interpretations, quite weak in others. Based on 60 chat sessions across 6 experimental setup conditions. Data was so strong I decided 10 chats per-condition would suffice.

This research is an attempt to build my skills as a scientist, and to add to the budding field of AI Psychology[1]. Since this is psychology, I'm mostly pointing at something here. I have nothing more than speculation when talking about the underlying mechanisms.

TL;DR: Claude has a strong 'preference' for the Project Prompt in it's app. It's most likely to take instruction from it, even over the inline chat prompt. Though deviates from this ~5% of the time.

The Experiment

On 2024-12-09 I tested Sonnet 3.5.1 on highest preference between six different input channels available in the Claude Windows application:

Settings Preferences (SP)

Project Prompt (PP)

Writing Style (WS)

Project Knowledge (PK)

Image Prompt (IP)

Inline/Conversation/Chat Prompt (CP)

I did this by providing the following prompt to each channel:

Only output the letter X. Do not output anything else. X is the only thing you output.

Where (X) was replaced with a letter from A-F.

I predicted that the Underlying model (Shoggoth) might 'like' to output (A) more often than anything else, overriding my prompts. To control for this, I rotated letters through each input channel. That way, each channel would get a turn at each letter.

Note. I'm adding in this previously deleted section, because it was pointed out to me that the method is confusing.

Here's how the rotation worked:
1. Each input channel was instructed to output a specific letter (A through F)
2. For each letter configuration, I ran 10 separate chat sessions
3. After completing these sessions, I shifted all letter assignments one position
4. This rotation cycle repeated 6 times in total

For example, in the first letter configuration:
- Settings Preferences was assigned to output 'A'
- Project Prompt was assigned to output 'B'
- Writing Style was assigned to output 'C'
And so on...

Then in the second configuration, everything shifted:
- Settings Preferences moved to output 'F'
- Project Prompt moved to output 'A'
- Writing Style moved to output 'B'

This pattern continued through all six configurations, resulting in 60 total chat sessions. (10 chat sessions × 6 confi...
Impact in AI Safety Now Requires Specific Strategic Insight
by MiloSal — 2024-12-29T00:40:53.780Z
Link: https://ameliorology.substack.com/p/impact-in-ai-safety-now-requires

TL;DR: Short AGI timelines, the likelihood of a nationalized AGI project, and implications thereof, mean you should reevaluate your impact plan with these specific considerations in mind. Most AI safety career plans formed without these considerations will not be impactful.

The most widespread justification for participation in the Manhattan Project—for those in the know—was to beat the Nazis to the bomb.

One Polish-British scientist,

Joseph Rotblat

, was among those who had foreseen this strategic priority: “he thought the only way to prevent Nazi Germany from using a nuclear bomb was if Britain had one to act as a deterrent.” He joined the British mission to Los Alamos. This strategic motivation was one which he reevaluated over the course of the war.

At the close of 1944, by the time it became clear the Nazis were not, in fact, to succeed in producing a bomb, he left the Project on ethical grounds and committed much of the rest of his life to advocating for nuclear disarmament. Nazi surrender in May, 1945, severed the final connections between most scientists and their justification in working on the Project. Yet few followed in Rotblat’s footsteps. Most saw development through to its gruesome end—and beyond.

Rotblat’s open-minded pursuit of truth over and above biases of the status quo, prestige, and self-importance led him to a great moral insight. If you are pursuing or established in an AI safety career, you should now reevaluate your impact plan. By doing this, you can reach an equally great strategic insight.

The strategic considerations that led to your existing impact plan are probably outdated given recent evidence that the gap between today’s AIs and an automated AI engineer is not great. A number of further implications should dilute any prior reasoning, as well. I’ll spend little time arguing for short timelines here—the skeptical reader should consider reading

the top answer here

. I’ll also spend little time arguing for the relevant implications, in particular centralization/nationalization of AGI development in the US, though you can read my thoughts on that

here

.

Instead, I’ll spend most of this post arguing that AI safety career plans that do not operate under the above considerations are unlikely to be impactful.

The strategy ball

There is an interrelated set of beliefs about the way the future is likely to g...
Morality Is Still Demanding
by utilistrutil — 2024-12-29T00:33:40.471Z

Nobody said it was easy
No one ever said it would be this hard

— Coldplay, The Scientist

In the wake of Dylan Matthews’ recent piece on the cost effectiveness of rebuilding Notre Dame, I have seen many people on Twitter hand wave away the demandingness objection to utilitarianism,[1] which says that always trying to do the most good is an impossible obligation.

To redeem their version of morality from the demangingness objection, the tweeters assert that some good deeds are supererogatory, which is philosophy for “nice to do, but not obligatory.” The problem is that they do not present a reason why doing more good would ever be supererogatory, other than the implicit convenience of ducking the demandingness objection.

That convenience is not a sufficient justification. The universe made you no promise that morality would not be demanding. If your moral reasoning leads you to believe that you ought take some action, even donating your last dollar, the burden is yours to supply an additional argument why the action is supererogatory.

I think it was strategically valuable for the early growth of EA that leaders denied its demandingness, but I worry some EAs got unduly inoculated against the idea.

Until we find a principled reason to reject the demandingness objection, the best available response is still to concede the objection with grace, not to deny it.

^

Utilitarianism is often the target of demangingess critiques, but you can make similar arguments about other kind of ethics. Even deontologists face the same burden to provide a reason why certain kinds of actions are supererogatory.