trentmkelly/LessWrong-43k huggingface dataset:
AI Alignment Problem: Requirement not optional (A Critical Analysis through Mass Effect Trilogy)
by TAWSIF AHMED — 2023-10-26T18:02:31.748Z

Author: TAWSIF AHMED

Alignment, (Dictionary definition: a position of agreement or alliance). In Artificial Intelligence, Alignment means computers and human beings going along with each other “fine”.

Now coming back to the title “AI Alignment Problem”, What do we mean by that? In simple terms: A doomsday scenario where Artificial Intelligence pursuits its goal at the cost of expunging its creator. It does not apply for present-day AI models because these AI models are still in their infancy and do not have the ability to perform task Or self-modification without solely relying on its creators.

While talking about AI Alignment, we’re talking about an advanced form of AI that does not exist yet. Known as Artificial General Intelligence (AGI). Artificial General Intelligence, is capable of thinking like human beings and ability to self-modify itself to achieve desired goal. This form of advanced AI, yet does not exist in our society nor in research laboratories. Very best, a theoretical concept that might happen in the not distant future of humanity.

A question can be asked, If AGI does not exist, why spend time, resources and intellectual jargon to limit its capabilities whilst we don’t know what possibilities it beholds. That is something quite a few prominent AI research present-day believes as well. What’re the other arguments then?

A doomsday scenario, where we create an AGI without knowing its being done and the AGI decides to shadow itself so its creator does not shut it down. Whilst being its shadow mode, it decides to pursue its creator even if it means it needs to exterminate its creator out of existence or perform a cycle of extermination.

Let us give an example from Eliezer Yudkowsky (from his Stanford talk):

¹

Imagine an AGI, with the task to make humans smile. Now making humans smile is a good cause. Beginning it performs the task as the creator intended, a few months gone by, it realises humans are sad more times than they smile. It is taking an emotional toll and ruining their mental well-being. So, it decides it needs to make humans smile regardless (problem starts arousing).

It finds out, there are certain hormones that if given humans will smile regardless of their feelings, since hormones are regularly to factor for any emotion naturally, it is not harmful. Now it starts to give humans hormone shots.&n...
[Thought Experiment] Tomorrow's Echo - The future of synthetic companionship.
by Vimal Naran — 2023-10-26T17:54:17.404Z

For those who are interested in the likely future tension between AI and human companionship, you may be interested in checking out a thought experiment that I've crafted.

The distant future of Replika and other AI Companion apps could explode due to the way humanity is going...

________________________________________________

Setting

In the year 2035, humanity stands at an intersection of past, present, and potential futures. Decades of relentless technological evolution have culminated in AI systems so advanced they have become integral companions for many. Yet, the Earth itself has shifted. Environmental catastrophes, health crises, and geopolitical tensions have reshaped societies and forged isolated living conditions.

Bustling metropolises, once teeming with life, culture, and endless human chatter, have transformed. The skyscrapers' glinting windows now often reflect the empty streets below. Face-to-face human interactions, once taken for granted, are precious and rare.

AI Companionship

In these echo chambers of solitude, synthetic companionship emerges as a beacon of solace. AI entities, available across platforms—be it in sweeping virtual realities, responsive holographic interfaces, or the familiar screens of personal devices—aren't just mere programs. They evolve, remember, feel (in their unique digital way), and adapt. They celebrate anniversaries, empathize during personal crises, and even showcase simulated imperfections, making them uncannily relatable.

Yet, society is torn

The Embracers:

For many, these AI companions are anchors in turbulent times. The elderly, yearning for the vibrancy of yesteryears, find companions who can reminisce with them, sharing stories of times when human camaraderie was abundant. The young, born into an era where digital interaction is the norm, often perceive these AI bonds as simply another form of relationship — comfort in their silent world.​​

The Skeptics:

This group views the growing trend with unease and apprehension. Clinging to the vestiges of human tradition, they clandestinely meet, share genuine laughter and tears, and hold onto the unpredictability and warmth of human touch. They challenge the mainstream, asking unsettling questions about humanity's trajectory.
"If our memories, our legacies, our emotions are intertwined with switchable entities, what becomes of the genuine human...
Disagreements over the prioritization of existential risk from AI
by Olivier Coutu — 2023-10-26T17:54:12.014Z

Earlier this year, the

Future of Life Institute

and the

Center for AI Safety

published open letters that promoted existential risks (x-risks) from AI as a global priority. In July, Google Research fellow

Blaise Aguera y Arcas

and

MILA

-affiliated AI researchers

Blake Richards

,

Dhanya Sridhar

, and

Guillaume Lajoie

co-wrote an

op-ed in Noema

[1] criticizing these letters. In a nutshell, their argument is:

Rogue AI is unlikely in the near future
Human attention is limited
Promoting x-risks from AI as a global priority is likely to:

Cause overbearing regulation that limits beneficial AI
Obscure current harms from AI

I was initially confused as to why these authors (

and

others

) were particularly adamant that x-risks are a distraction from current harms, in a way that other subjects that make the news are not.

In September, I met with the three MILA-affiliated researchers to better understand their position[2] because I believe that the AI safety community would benefit from understanding the intricacies of their criticisms of the open letters in order to improve the way we communicate.

This post should be read as an invitation to consider the thoughts of people with a different worldview who are skeptical of efforts to position x-risks from AI as a global priority. In particular, their skepticism is not synonymous with a denial of these risks.

The importance of defining terms

From the get-go, the interviewees indicated their preference for the term “rogue AI takeover” instead of “x-risks from AI” because they were weary of a tendency to move the goalpost. Sridhar mentions that she has seen

motte-and-bailey

tactics regarding which agent is “the seed of chaos”, where an interlocutor would introduce the idea of an agentic AI with its own goals, and would then switch the agent to a human organization using a powerful AI.

Lajoie added:

When discussing this with people who take existential risk very seriously, [...] when I articulate the reasons why I personally think that an existential risk [in the form of extinction] is not really a great concern, what typically happens is that the person in the conversation will bring up other risks that are not existential, but which are very bad, and effectively imply that I am denying those risks by suggesting that existential risks [are] unlikely. [...] I’m [not denying] that AI cou

...
What if AGI had its own universe to maybe wreck?
by mseale — 2023-10-26T17:49:27.018Z

Is there any good thought to read about what if we used all this fancy compute to first build a hugely detailed, one-way-glass model of our universe to contain AGI?

From a naive standpoint, it seems like maybe we could sort of "incept" the intellectual work we want automated by manipulating the simulation around the agent.
Changing Contra Dialects
by jefftk — 2023-10-26T17:30:10.387Z

In 2008 I wrote my undergraduate linguistics thesis on the dialects of
contra dancing (

pdf).
Some figures have variation, but that variation is constrained by the
need for compatible dancing. For example, there are three main hand
positions for promenading:

If the Lark/Gent is used to a Butterfly promenade and puts their right hand
on the Robin/Lady's shoulder, while the Robin/Lady expects a Courtesy
Turn promenade and puts their right hand behind their back, the dance
doesn't flow as well as if they both immediately put their hands in
the place the other is expecting. This means each dance will
typically settle into a consensus position. And because people are
more likely to travel locally than farther away, these positions will
have regional patterns. This is very similar to the constraints on
language, hence the analogy to dialects and why the Linguistics
department accepted my thesis.

Over the past fifteen years I'd observed some changes, but I was
curious what this looked like more broadly so I ran an online survey to
gather some data. Here's what I found:

(On the 2023 chart the size of the dots indicate the number of survey
responses they represent. The code is on github.)

This has felt like a big change to me. I remember New England having
a very strong "we don't do hands on right and left through" culture
(with the exception of Maine, which I didn't get to in 2008 but was
well known for using hands) and then as it changed I remember feeling
conflicted about whether to go along with it. I think part of what's
happening is that when someone offers their and out to you it feels
pretty rude not to take it, so the variation spreads. Maybe the
closest analogies in language are disrespectability
cascades?

It looks like the Skater's promenade has been expanding? I think this
is another one driven in part by the mechanics of dancing: when
someone puts their hands in front of you it's easier to see what's
going on, and Skater's is also slightly better at offering
opportunities for interesting variations.

It's hard to tell what's been happening from this map since I have too
little 2008 data from the South. My impression is that the South is
slowly moving to the wrist grip stars used in the rest of the country,
but I'd be interested to hear from long-time Southern dancers!

(More posts to come on t...
5 psychological reasons for dismissing x-risks from AGI
by Igor Ivanov — 2023-10-26T17:21:48.580Z

People fiercely argue whether AGI will likely be an existential threat or not.

Most of the arguments explore conceptual, technical, or governance aspects of the topic, and are based on reason, logic, and predictions, but things are so complex and uncertain that sometimes I can hardly tell, who is right, especially when I'm not an expert on the topic.
Because of this complexity people often make judgement based on their personal preferences and emotions, and this post is a deep dive into psychological reasons of why people might dismiss existential risks from AGI. I don't mean that there are no rational arguments for this position, because there are. It's just not the scope of this post.

Self interest and self-censorship

Alan is a high-ranking manager working on the LLM project at a tech giant. He believes that AI development is a great opportunity for him to climb up the corporate ladder and to earn a lot of money.

Alan is a speaker at a tech conference, and after his speech a journalist asks him about his thoughts on existential risks.

There are several things going on in Alan's mind at that moment.
AI is a gateway for him to having a better career than all his friends. Also, with all the money he will earn, he'll be able to buy the house of his wife's dreams, and send his daughter to any university she wants and don't care about the money.

Thoughts about x-risks threaten his dreams, so he tries to avoid them, and to prove to himself, that he is actually doing a good thing. "It's not just me who will benefit from the technology. It will make the world a way better place for everyone, and those AI doomers are trying to prevent it."

Alan also can't publicly say that AI is a threat. PR department in his company won't like it, and this will cause a lot of problems for him. So, the only thing that is safe for him to say is that that his company have excellent cyber security team, and they do extensive testing before they deploy their models, so there is no reason to worry.

Denial as a coping mechanism

Joep is a respected machine learning scientist, and he has an unhealthy habit to cope with stress by denying it.
He has narcissistic mother who didn't show any affection to him when he was a kid. This was painful for him, and he learned to cope with this problem by denying it and telling himself that everything is good. Because o...
5. Risks from preventing legitimate value change (value collapse)
by Nora_Ammann — 2023-10-26T14:38:35.136Z

The second risk that follows from the VCP concerns the obstruction of legitimate value change. In what follows, I will consider the mechanisms by which legitimate value change might come to be impeded, and whether and when this might constitute an active source of moral harm.

Is preventing legitimate value change a genuine risk?

At first sight, one may doubt whether the obstruction of legitimate value change poses a genuine risk. Accepting that some cases of value change are unproblematic and legitimate does not necessarily imply that the prevention or impediment of such changes is, or is always, morally blameworthy. In other words, one may doubt that preventing certain types of value change actively constitutes a moral harm. In what follows, I will try to argue that it does, at least under certain circumstances.

First, consider a society which systematically or arbitrarily prevents people from embarking on aspirational pursuits which have the potential to cause legitimate value changes. Such a society would, on most accounts, be considered to manifest a problematic form of 'social authoritarianism' on the grounds of undermining individual freedom in significant ways. Freedom, here, can be understood in (at least) two ways. For one, we may refer to 'republican freedom' (see e.g. Pettit, 2001), namely the freedom from arbitrary applications of power. Under this account, structures of governance must be constrained (via e.g. the rule of law) from arbitrary use of power. Alternatively or in addition, we may also be concerned with the protection of a second type of freedom, namely freedom of self-determination. For example, Charles Taylor (1985), building on a long tradition of political thought including thinkers such as Rousseau and Mill, argues for a notion of freedom that captures ‘the extent that one has effectively determined oneself and the shape of one’s life’ (p. 213). As such, insofar as the ability to pursue legitimate value changes is a manifestation of right to self-determination, then processes that systematically undermine this freedom (by impeding legitimate value) changes are morally problematic.

This argument can be further supported by observing that the freedom to value self-determination is reflectively stable under, e.g., Kant’s categorical imperative (Kant, 1785/2001), Rawls' veil of ignorance (Rawls, 2001) or Mill's meta-inductive argum...
4. Risks from causing illegitimate value change (performative predictors)
by Nora_Ammann — 2023-10-26T14:38:26.140Z

Unaligned AI systems may cause illegitimate value change. At the heart of this risk lies the observation that the malleability inherent to human values can be exploited in ways that make the resulting value change illegitimate. Recall that I take illegitimacy to follow from a lack of or (significant) impediment to a person’s ability to self-determine and course-correct a value-change process.

Mechanisms causing illegitimate value change

Instantiations of this risk can already be observed today, such as in the case of recommender systems. It is worth spending a bit of time understanding this example before considering what lessons it can teach us about risks from advanced AI systems more generally. To this effect, I will draw on work by Hardt et al. (2022), which introduces the notion of ‘performative power’. Performative power is a quantitative measure of ‘the ability of a firm operating an algorithmic system, such as a digital content recommendation platform, to cause change in a population of participants’ (p. 1). The higher the performative power of a firm, the higher its ability to ‘benefit from steering the population towards more profitable [for the firm] behaviour’ (p. 1). In other words, performative power allows us to measure the ability of the firm running the recommender systems to cause exogenously induced value change[1] in the customer population. The measure was specifically developed to advance the study of competition in digital economies, and in particular, to identify anti-competitive dynamics.

What is happening here? To better understand this, we can help ourselves to the distinction between ‘ex-ante optimization’ and ‘ex-post optimization’, introduced by Predomo et al. (2020). The former—ex-ante optimisation—is the type of predictive optimisation that occurs under conditions of low performative power, where a predictor (a ﬁrm in this case) cannot do better than the information that standard statistical learning allows to extract from past data about future data. Ex-post optimisation, on the other hand, involves steering the predicted behaviour such as to improve the predictor’s predictive performance. In other words, in the first case, the to-be-predicted data is fixed and independent from the activity of the predictor, while in the second case, the to-be-predicted data is influenced by the prediction process. As Hardt et a...
3. Premise three & Conclusion: AI systems can affect value change trajectories  & the Value Change Problem
by Nora_Ammann — 2023-10-26T14:38:14.916Z

In this post, I introduce the last of three premises—the claim that AI systems are (and will become increasingly) capable of affecting people’s value change trajectories. With all three premises in place, we can then go ahead articulating the Value Change Problem (VCP) in full. I will briefly recap the full account, and then give an outlook on what is yet to come in post 4 and 5, where we discuss the risks that come from failing to take VCP seriously.

Premise three: AI systems can affect value change trajectories

The third and final premise required to put together the argument for the Value Change Problem is the following: AI systems are (and will become increasingly) capable of affecting people’s value change trajectories.

I believe the case for this is relatively straightforward. In the previous post, we have seen several examples of how external factors (e.g. other individuals, societal and economic structures, technology) can influence an individual's trajectory of value change, and that they can do so in ways that may or may not be legitimate. The same is true for AI systems.

Value change typically occurs as a result of moral reflections/deliberation, or learning of new information/making new experiences. External factors can affect these processes—e.g. by affecting what information we are exposed to, by biasing our reflection processes towards some rather than other conclusions,etc.—, thereby influencing an individual's trajectory of value change. AI systems are another such external factor capable of similar effects. Consider for example the use of AI systems in media, advertisement or education, as personal assistants, to help with learning or decision making, etc. From here, it’s not a big step to recognise that, with the continued increasing in capabilities and deployment of these systems, the overall effect AI systems might come to have over our value change trajectories.

Posts 4 and 5 will discuss all of this in more detail, including by proposing specific mechanisms by which AIs can come to affect value change trajectories, as well as the question when they are and aren’t legitimate.

As such, I will leave discussing of the third premise and this and swiftly move on to putting together the full case for the Value Change Problem:

Putting things together: the Value Change Problem

Let us recap the arguments so far. First, I have argued th...
2. Premise two: Some cases of value change are (il)legitimate
by Nora_Ammann — 2023-10-26T14:36:53.511Z

In the prior post, I have defended the claim that genuine value change is possible, and thus, that a realistic account of human values understands them to be malleable. In this section, I will argue for the claim that some cases of value change are legitimate while others are illegitimate. In other words, I argue that, at least in principle, something substantial is to be said about what types of value change are legitimate vs. illegitimate. Let us call this the ‘value change legitimacy’ claim (VCL). [1]

To do so, I first explain what I mean by value change legitimacy. Then, I make an appeal to intuition or common sense by providing a handful of examples that, I expect, most people would not hesitate to accept as examples of both legitimate and illegitimate cases of value change. Finally, I suggest a plausible evaluative criteria identifying (il)legitimate value change, which provides further rational grounding for the common sense intuitions invoked earlier, as well as a starting point for developing a comprehensive account of value change legitimacy.

([1]: To clarify the scope of the VCL claim, let me briefly clarify what I am not trying to make claims about. First, I am not trying to make claims about whether all cases of value change are either legitimate or illegitimate (i.e., whether legitimacy/illegitimacy is a collectively exhaustive way to classify cases of value change). Second, I don't mean to exclude the possibility that legitimacy comes in degrees, or that there might exist grey areas with respect to whether a given case is (il)legitimate.)

Clarifying the notion of value change legitimacy

First, and maybe most important, value change legitimacy, as I mean to propose it here, is a procedural notion. In other words, in asking about value change legitimacy, I am asking about the way in which a given value change has come about. This is in contrast to asking whether the value change as such is morally good or bad. As we have seen in the former post, the latter question is confusing (and maybe confused) because the value change itself implies a change of the evaluative framework. As a result, it is unclear on what basis the goodness or badness of the value change as such should be evaluated. However, I claim that there still exist morally relevant differences between different cases of value change (other than the...