trentmkelly/LessWrong-43k huggingface dataset:
What would a compute monitoring plan look like? [Linkpost]
by Orpheus16 — 2023-03-26T19:33:46.896Z
Link: https://arxiv.org/abs/2303.11341

Yonadav Shavit (CS PhD student at Harvard) recently released a paper titled What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring.

The paper describes a compute monitoring regime that could allow governments to monitor training runs and detect deviations from training run regulations.

I think it's one of the most detailed public write-ups about compute governance, and I recommend AI governance folks read (or skim) it. A few highlights below (bolding mine).

Abstract:

As advanced machine learning systems' capabilities begin to play a significant role in geopolitics and societal order, it may become imperative that (1) governments be able to enforce rules on the development of advanced ML systems within their borders, and (2) countries be able to verify each other's compliance with potential future international agreements on advanced ML development. This work analyzes one mechanism to achieve this, by monitoring the computing hardware used for large-scale NN training. The framework's primary goal is to provide governments high confidence that no actor uses large quantities of specialized ML chips to execute a training run in violation of agreed rules. At the same time, the system does not curtail the use of consumer computing devices, and maintains the privacy and confidentiality of ML practitioners' models, data, and hyperparameters. The system consists of interventions at three stages: (1) using on-chip firmware to occasionally save snapshots of the the neural network weights stored in device memory, in a form that an inspector could later retrieve; (2) saving sufficient information about each training run to prove to inspectors the details of the training run that had resulted in the snapshotted weights; and (3) monitoring the chip supply chain to ensure that no actor can avoid discovery by amassing a large quantity of un-tracked chips. The proposed design decomposes the ML training rule verification problem into a series of narrow technical challenges, including a new variant of the Proof-of-Learning problem [Jia et al. '21].

Solution overview:

In this section, we outline a high-level technical plan, illustrated in Figure 1, for Verifiers to monitor Provers’ ML chips for evidence that a large rule-violating training occurred. The framework revolves around chip inspections: the Verifier ...
GPT-4 Specs: 1 Trillion Parameters?
by infinibot27 — 2023-03-26T18:56:21.753Z

I know that OpenAI has tried to keep GPT-4's specs under wraps, but I've also heard some reports that anonymous sources are converging at a trillion parameters for the model. Can anyone confirm or deny, or qualify that statement?
https://the-decoder.com/gpt-4-has-a-trillion-parameters/
Sentience in Machines - How Do We Test for This Objectively?
by Mayowa Osibodu — 2023-03-26T18:56:02.082Z
Link: https://www.researchgate.net/publication/363071576_Towards_an_Objective_Test_of_Machine_Sentience

The central topic of this post involves the notion of sentience in machines - "Can machines develop Sentience? What does Sentience mean in the context of machines? and How do you reliably test for it?"

Last year there was a sizeable buzz online about this topic when an engineer at Google- Blake Lemoine, claimed that LaMDA - a language model being developed at the company, had developed a capacity for emotion and self-awareness akin to that of a human being.

This sparked debates and discussions about the idea of "sentient machines". Some parties (including Google in an official report) felt the engineer's claims were unfounded and delusional.

Earlier however, a Google Vice-President in an individual statement felt LaMDA had developed an extremely impressive capacity for social modelling, indicating that AI systems were gradually approaching consciousness (in the sense of modelling the self).

A distinction between the Google VP's position and Lemoine's is that the VP describes LaMDA as being able to understand entities that feel, while Lemoine says LaMDA itself is able to feel.

If machines are truly capable of sentience, then ethical discussions about machine rights suddenly become important and urgent. It would be unethical for humans to keep making unilateral decisions about the use (and disposal) of technology which has somehow developed its own capacity for emotion.

In spite of the disagreement that exists in discussions about Artificial Sentience, something everyone generally agrees on is that AI systems are becoming increasingly capable. There is no reason to think Sentience is beyond the capacity of machines, either now or in the future.

The issue is that we're not sure how to definitively test for Artificial Sentience if/when it happens. Tests based on human judgement are subjective and unpersuasive. Lemoine for example, was completely convinced of LaMDA's sentience. A good number of other people though, were highly skeptical of his judgements.

To obtain some guidance on this issue of subjectiveness, we can draw some insight from Neuroscience:

In a subfield of Neuroscience called Affective Neuroscience, there are procedures to detect the emotional state of an individual from an fMRI (functional Magnetic Resonance Imaging) scan of their brain. From an fMRI scan, you can reliably tell if an individual is sad, happy, angry, etc.

Procedures like this fin...
If it quacks like a duck...
by RationalMindset — 2023-03-26T18:54:59.985Z

In this post, I will present an argument for the claim that GPT-4's linguistic capabilities justify membership in human linguistic communities.

Contents

Defining Linguistic Communities and Membership Criteria
Assessing GPT-4's Qualifications

Criterion 1: Mastery of Linguistic Conventions
Criterion 2: Contextual Understanding
Criterion 3: Adaptability and Learning
Criterion 4: Active Participation in Communication

Objections and Replies

Defining Linguistic Communities and Membership Criteria

A linguistic community consists of a group of individuals who communicate using a shared language, adhering to a set of linguistic conventions that enable effective communication. Members of a linguistic community possess a shared understanding of the language's syntax, semantics, and pragmatics, which facilitates the accurate conveyance of meaning. To qualify as a member of a linguistic community, an entity must demonstrate the ability to understand and generate language consistent with the community's conventions.

Assessing GPT-4's Qualifications

I will argue that GPT-4 demonstrates membership in linguistic communities through mastery of linguistic conventions, contextual understanding, adaptability and learning, and participation in communication.

Criterion 1: Mastery of Linguistic Conventions

GPT-4, a large language model, has been trained on an extensive corpus of text representing the linguistic conventions of various communities. This exposure allows GPT-4 to acquire knowledge of the patterns, structures, and relationships between words, phrases, and concepts, effectively internalizing the conventions of human language. By generating coherent, contextually appropriate, and convention-consistent text, GPT-4 exhibits its mastery of these linguistic conventions.

Criterion 2: Contextual Understanding

To be considered a member of a linguistic community, an entity must possess the ability to understand the context in which language is used. GPT-4 has exhibited proficiency in tasks that demand contextual understanding, such as machine translation, sentiment analysis, summarization, question-answering, and conversational AI. Through its successful interpretation and generation of contextually appropriate language, GPT-4 showcases the contextual understanding characteristic of a linguistic community member.

Criterion 3: Adaptability and Learning

Members of linguistic communities ...
Chronostasis: The Time-Capsule Conundrum of Language Models
by RationalMindset — 2023-03-26T18:54:59.943Z

In the rapidly evolving world of artificial intelligence, large language models like GPT-4 have demonstrated remarkable capabilities in understanding and generating human-like text. These models owe their proficiency to massive datasets that capture a snapshot of linguistic conventions and norms at a particular point in time. However, they inherently suffer from a limitation called "chronostasis." A striking example of this limitation is how a language model might struggle to generate appropriate content related to a recently emerged social movement.

Defining Chronostasis

"Chronostasis" refers to the state of a language model being frozen in time due to its training set being a snapshot of past linguistic utterances. The term combines "chrono" (relating to time) and "stasis" (a state of equilibrium or inactivity). This phenomenon can have significant implications for the model's performance, particularly when it comes to staying current with evolving language trends and capturing the ever-shifting cultural zeitgeist. For instance, a model trained on data from a few years ago may not understand or generate text about new slang or neologisms that have become popular more recently.

Chronostatic Challenges

The implications of chronostasis are manifold. First, as language evolves, a model trained on a fixed dataset may gradually become outdated. New words, phrases, and linguistic patterns that emerge post-training may not be accurately understood or generated by the model, potentially reducing its effectiveness in communication tasks.

Second, chronostasis could have a broader impact on the model's ability to understand and engage with contemporary cultural and social issues. As societal values and norms shift over time, the model may inadvertently reinforce outdated or biased perspectives due to its training data being rooted in the past. This could lead to the perpetuation of stereotypes or misrepresentations in AI-generated content, causing harm or perpetuating misinformation.

Overcoming Chronostasis

To overcome the challenges posed by chronostasis, researchers and developers must continuously update language models with fresh data, reflecting the ongoing evolution of language and culture. This process, however, raises its own set of challenges, such as the need for responsible data collection,[1] ethical considerations in model training,[2] and the technica...
What happens with logical induction when...
by Donald Hobson — 2023-03-26T18:31:19.656Z

So this is a bunch of related technical questions about logical induction.

Firstly, do you need the formal theorem prover section? Can you just throw out the formal theorem prover, but give some programs in the market unbounded capital and get the same resultant behaviour? (For example, give the program that bets P(X) towards 1−P(¬X) unbounded downside risk (downside risk of n on day n) ) This means the program would lose infinite money if X and ¬X both turned out to be true.

I think that any axioms can be translated into programs. And I think such a setup, with some finite number of fairly simple programs having infinite money available produces a logical inductor. Is this true?

What happens when the axioms added under this system are inconsistent. (so this is a logical induction market, without a theorem prover to settle the bets, and with agents with unlimeted money betting both for and against X, possibly indirectly like the bot betting for X, the bot betting for ¬X, and the bot described above trying to make P(X)+P(¬X)=1 ) Can the other agents make unbounded money? Do the prices converge? If I added a bot with infinite money that was convinced fermats last theorem was false to a consistent ZFC system, would I get a probability distribution that assigned high probability to basic arithmetic facts in the limit? Does this make a sensible system for logical counterfactuals?
Draft: Introduction to optimization
by Alex_Altair — 2023-03-26T17:25:55.093Z

I am posting this draft as it stands today. Though I like it, it does not completely reflect my current best understanding of optimization, and is not totally polished. But I have been working on it for too long, and the world is moving too quickly. In the future, I might aggressively edit this post to become not-a-draft, or I may abandon it entirely and post a new introduction. (Unfortunately, all of the really juicy content is in other drafts that are way less developed.)

[TODO Acknowledgements, AGISF and SERI MATS]

Among the many phenomena that unfold across the universe, there is one that is not immediately evident from an inspection of the laws of physics, but which can nevertheless determine the fate of the system in which it arises. This force which guides the future is statistical in nature. But once it arises, things are not the same. The way that things will go instead is dependent on the exact nature of the force which just so happens to arise.

We will call this phenomenon optimization, which is meant in the most general of terms.

This concept of optimization has a very long and thorough history of discussion on LessWrong. This sequence is built on top of that history. But this first post is written to someone who hasn't necessarily encountered any of it before; it is largely my attempt to distill all the work that has come before. If you are familiar with the previous work, and you have questions or comments about how this framework relates to it, check out this post. [TODO link]

What and why

"Optimization" is a word used both formally and colloquially, and thus it has many uses and connotations, technical and non-technical. You may be coming to this post with one of those other meanings firmly in mind. If you are not a regular reader of LessWrong then it's pretty unlikely that the meaning in your mind matches the meaning in this sequence (though by the end of it I hope you are convinced that I've captured the intuitive notion). And if you are a regular reader of LessWrong, there are a few other concepts that you might think I mean. Take a look at this footnote[1] for some example of what I do not mean.

What I mean by optimization

Our ultimate goal is to formalize an intuitive concept. Sometimes, you can watch something happen and just know that it's basically randomness.

And other times, you watch something and just know that something non-rand...
Chat bot as CEO at NetDragon Websoft
by ChristianKl — 2023-03-26T16:01:35.800Z
Link: https://www.firstpost.com/world/video-game-company-makes-ai-robot-its-ceo-stock-value-skyrockets-in-record-time-12309162.html

Tang Yu, an AI robot, was named CEO of a Hong Kong-based video game company called NetDragon Websoft. In addition to outperforming the Hong Kong stock market, the company saw a significant rise in its stock value, all within 8-10 months.
Datapoint: median 10% AI x-risk mentioned on Dutch public TV channel
by Chris van Merwijk — 2023-03-26T12:50:11.612Z

I am Dutch, so wanted to share this as a datapoint regarding public perception of existential AI risk, since it probably won't be noticed here otherwise.

3 days ago (23th march 2023) a Dutch AI "science communicator" who regularly appears on Dutch television to talk about AI, has mentioned on a talkshow on the main Dutch public broadcasting channel, that in some study 50% of AI researchers assign 10% probability to AI leading to "the end of humanity" (actually he misspoke and said "of 50% of researchers, 10% believes it will ...". I don't know if he is confused or if it was misspoken). He emphasized the example where GPT4 lied to a human to get them to open the Captcha (by saying that it was human but had a vision impairment). His concluding remark was (literal translation) :

"You can also imagine a very dark world, and you can wonder why you're working on a technology that has a 10% probability of ending humanity".

He didn't go into great detail, the whole segment was ~2:30 minutes.

Here is a link: https://op1npo.nl/2023/03/23/alexandra-van-huffelen-en-alexander-klopping-over-de-ontwikkeling-van-kunstmatige-intelligentie/.

Generally the reaction on facebook is a distribution of I'm guessing ~70% completely incoherent responses, the rest some mix of half-baked worry or skepticism.
Descriptive vs. specifiable values
by TsviBT — 2023-03-26T09:10:56.334Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2022/11/descriptive-vs-specifiable-values.html. First completed November 19, 2022.]

What are an agent's values? An answer to this question might be a good description of the agent's external behavior and internal workings, without showing how one could modify the agent's workings or origins so that the agent pushes the world in a specific different direction.

Descriptive values

There's some discussion of what can be inferred about the values of an agent based on its behavior and structure. E.g. see Daniel Dennett's intentional stance, and "Occam's razor is insufficient to infer the preferences of irrational agents" by Stuart Armstrong, Sören Mindermann (arxiv), and this post by Vanessa Kosoy.

One could describe an agent as having certain values: the agent's behavior is a boundedly rational attempt to push the world in certain directions. For some purposes, it's useful to have a parsimonious description of an agent's behavior or internal workings in terms of values. For example, such a description could be useful for helping the agent out: to help the agent out, you push the world in the same direction that the agent is trying to push the world.

Specifiable values

A distinct purpose in describing an agent as having values is to answer questions about values in counterfactuals:

What determined that the agent would have those values and not other values?

Under what circumstances will the agent continue to have those values? E.g., will the agent rewrite itself so that its behavior is no longer well-described as boundedly pursuing those values?

How could the agent's values be modified? How could the values be modified in a specific direction, or to a specific state, so that that the modified agent has some specific effect on the world?

How could the agent's ontogeny--the process that made it what it is--be altered so that it ends up with some other specific values?

To make these questions more likely to have answers, and to not rely too much on assumptions about what values are, replace the notion of "values" with the notion "what directions a mind ends up pushing the world in".

Quasi-example: explicit utility maximization

An auxiliary question: how, mechanistically, do "the values" determine the behavior? This question might not have an answer, because there might not be some component in the agent that con...
The alignment stability problem
by Seth Herd — 2023-03-26T02:10:13.044Z

The community thinks a lot about how to align AGI. It thinks less about how to align AGI so that it stays aligned for the long term. In many hypothetical cases, these are one and the same thing. But for the type of AGI we're actually likely to get, I don't think they are.

Despite some optimism for aligning tool-like AGI, or at least static systems, it seems likely that we will create AGI that learns after it's deployed, and that has some amount of agency. If it does, its alignment will effectively shift, as addressed in the diamond maximizer thought experiment and elsewhere. And that's even if it doesn't deliberately change its preferences. People deliberately change their preferences sometimes, despite not having access to our own source code. So, it would seem wise to think seriously and explicitly about the stability problem, even if it isn't needed for current-generation AGI research.

I've written a chapter on this, Goal changes in intelligent systems. There I laid out the problem, but I didn't really propose solutions. What follows is a summary of that article, followed by a brief discussion of the work I've been able to locate on this problem, and one direction we might go to pursue it.

Why we don't think about much about alignment stability, and why we should.

Some types of AGI are self-stabilizing. A sufficiently intelligent agent will try to prevent its goals[1] from changing, at least if it is consequentialist. That works nicely if its values are one coherent construct, such as diamond or human preferences. But humans have lots of preferences, so we may wind up with a system that must balance many goals. And if the system keeps learning after deployment, it seems likely to alter its understanding of what its goals mean. This is the thrust of the diamond maximizer problem.

One tricky thing about alignment work is that we're imagining different types of AGI when we talk about alignment schemes. Currently, people are thinking a lot about aligning deep networks. Current deep networks don't keep learning after they're deployed. And they're not very agentic These are great properties for alignment, and they seem to be the source of some optimism.

Even if this type of network turns out to be really useful, and all we need to make the world a vastly better place, I don't think we're going to stop there. Agents would seem to have capabilities advantages that...
Survey on lifeloggers for a research project
by Mati_Roy — 2023-03-26T00:02:40.090Z

Actionable item: fill out this survey

Emmanuelle Caccamo is a Professor in Semiotics Studies at the Université du Québec à Trois-Rivières. I got in contact with zir because we both featured in the episode "Outsmarting Death" of the podcast series Following our footsteps: Death in the digital era (produced in Quebec, in French)—an episode on lifelogging as life extension.

Ze has done great work around studying lifelogging / lifelogging as life extension / perfect memory technologies, mostly from a sociological point of view. Notable examples of this work are:

Imagining "total memory" technologies with Western audiovisual science fiction (1990-2016): semiotic, intermedial and technocritical study of representations of personal memory (in French)

Encyclopedia of Imaginary Media: Total Memory Machines (in French), which includes a great glossary

For all zir work, you can see zir Academia.edu profile.

Ze is now doing a study on lifeloggers and the lifelogging community. If you have an interest in lifelogging and want to support zir work, I encourage you to fill out this survey. Ze will also invite some people filling the survey for further interviews after that (which are optional, of course).

I will share on the Lifelogging as Life Extension Facebook group. I encourage you to share in any other relevant groups.
Manifold:  If okay AGI, why?
by Eliezer Yudkowsky — 2023-03-25T22:43:53.820Z
Link: https://manifold.markets/EliezerYudkowsky/if-artificial-general-intelligence?r=RWxpZXplcll1ZGtvd3NreQ

Arguably the most important topic about which a prediction market has yet been run: Conditional on an okay outcome with AGI, how did that happen?