trentmkelly/LessWrong-43k huggingface dataset:
News : Biden-⁠Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI 
by Jonathan Claybrough — 2023-07-21T18:00:57.016Z
Link: https://www.whitehouse.gov/briefing-room/statements-releases/2023/07/21/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai/

Here are the extracts naming the companies and specific commitments :

As part of this commitment, President Biden is convening seven leading AI companies at the White House today – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to announce that the Biden-Harris Administration has secured voluntary commitments from these companies to help move toward safe, secure, and transparent development of AI technology.

Today, these seven leading AI companies are committing to:

Ensuring Products are Safe Before Introducing Them to the Public

The companies commit to internal and external security testing of their AI systems before their release. This testing, which will be carried out in part by independent experts, guards against some of the most significant sources of AI risks, such as biosecurity and cybersecurity, as well as its broader societal effects.
The companies commit to sharing information across the industry and with governments, civil society, and academia on managing AI risks. This includes best practices for safety, information on attempts to circumvent safeguards, and technical collaboration.

Building Systems that Put Security First

The companies commit to investing in cybersecurity and insider threat safeguards to protect proprietary and unreleased model weights. These model weights are the most essential part of an AI system, and the companies agree that it is vital that the model weights be released only when intended and when security risks are considered.
The companies commit to facilitating third-party discovery and reporting of vulnerabilities in their AI systems. Some issues may persist even after an AI system is released and a robust reporting mechanism enables them to be found and fixed quickly.

Earning the Public’s Trust

The companies commit to developing robust technical mechanisms to ensure that users know when content is AI generated, such as a watermarking system. This action enables creativity with AI to flourish but reduces the dangers of fraud and deception.
The companies commit to publicly reporting their AI systems’ capabilities, limitations, and areas of appropriate and inappropriate use. This report will cover both security risks and societal risks, such as the effects on fairness and bias.
The companies commit to prioritizing research on the societal risks that AI sys

...
The UAP Disclosure Act of 2023 and its implications
by andeslodes — 2023-07-21T17:21:35.623Z
Link: https://www.congress.gov/amendment/118th-congress/senate-amendment/797/text

TL;DR: A recently proposed (7/13/2023) bipartisan amendment to the NDAA for 2024, with senate majority leader Chuck Schumer as its first sponsor and with the aim of declassifying documents related to Unidentified Anomalous Phenomena (UAP). It explicitly defines UAPs as pertaining to objects exhibiting "performance characteristics and properties not previously known to be achievable based upon commonly accepted physical principles" such as, among other things, "instantaneous acceleration absent apparent inertia", "transmedium travel" and "positive lift contrary to known aerodynamic principles". It takes care to separate these from "temporarily non-attributed objects" with prosaic explanations. The terms "Non-human intelligence" and "Technologies of unknown origin" are explicitly defined and encountered multiple times throughout the text. The amendment also non-dismissively mentions "reverse engineering of technologies of unknown origins" and "examination of biological evidence of living or deceased non-human intelligence". It declares that "Legislation is necessary because credible evidence and testimony indicates that Federal Government unidentified anomalous phenomena records exist that have not been declassified".

It is surprising to me that the community has not noticed / discussed the Unidentified Anomalous Phenomena Disclosure Act of 2023, given how much discussion was generated by UFO Betting: Put Up or Shut Up (enough for Eliezer to indicate willingness to bet 150k yesterday) and Intelligence Officials Say U.S. Has Retrieved Craft of Non-Human Origin. I think that anyone that is dismissing the topic out of hand should have a good look at this document. While I do not claim that it provides direct proof of phenomena necessitating an "ontological crisis", I believe that it can be a source of major updates. I also strongly believe that this is the point where a serious discussion must be had about this topic by the community, given the accumulating evidence of *something* going on, together with the potential ramifications of an "ontologically shocking" outcome. The situation at this point displays parallels with the early stages of the COVID-19 pandemic, in that the topic is not seriously covered by mainstream media yet, despite the available evidence pointing towards some (at least) "interesting" outcomes.

The Document

The Unident...
To use computers well, learn their rules
by dkl9 — 2023-07-21T17:00:21.676Z
Link: https://dkl9.net/essays/computer_rules.html

To use computers effectively, you don't need to learn a lot about particular programs and features. Learn the near-universal rules of computing and common interface designs. The details are readily inferred.

The rules also apply to things beyond what you might think of as "computers". Smartphones are computers.

Here are some of those rules to demonstrate the technique, and to help you learn it.

How computers use numbers

Computers have limited memory, so they represent a number with (usually) a tiny amount of memory. Often, that amount is 16, 32, or 64 bits. In case that's meaningless to you: that means each number can only be one of a limited set of values. 66 thousand, 4.3 billion, 18 quintillion options, respectively, for those sizes.

If what you do introduces huge or precise numbers, at some point along the path of size or precision, the program doesn't handle the numbers quite as well, or, in some cases, breaks.

If you try to make the number too big, it's handled with less precision than you expect. For a clear example of this: try putting 10^20 + 999 in a calculator. Some calculators are forced to round it to 10^20, no 999.

If you try to make the number too precise, the little details will be cut off and won't help.

Quadratic scaling

Some things in computing, such as images or link-networks, have an inherent scale proportional to the square of the obvious variable. An image has around many pixels as the square of its width. A link-network has around half as many potential links as the square of the items it can link. The square of number grows much faster than the original number. Doubling the original quadruples the square.

The more detailed study of matters like these is computational complexity.

The size of data

Typical units are based around the byte (representing one character, at least in English plaintext), and extend it to the kilobyte/KB, megabyte/MB, gigabyte/GB, and terabyte/TB. Those last terms, respectively, mean 1 000, 1 000 000, 1 000 000 000, and 1 000 000 000 000 bytes, each one a thousand times the last. Sometimes (for good reason) the factor each time is instead the nearby 1 024: 1 024, 1 048 576, 1 073 741 824, 1 099 511 627 776.

The size of a file (or file-equivalent dataset) gives a hint as to how long it'll take to process. On machines of the 2010s, for straightforward operations (transferring, searching, reformatting, etc):

a kilobyte is tiny,

...
BCIs and the ecosystem of modular minds
by beren — 2023-07-21T15:58:27.081Z

Crossposted from my personal blog.

Epistemic status: Much more speculative than previous posts but points towards an aspect of the future that is becoming clearer which I think is underappreciated at present. If you are interested in any of these thoughts please reach out.

For many years, the primary AI risk model was one of rapid take-off (FOOM) of a single AI entering a recursive self-improvement loop and becoming utterly dominant over humanity. There were lots of debates about whether this 'fast-takeoff' model was correct or whether instead we would enter a slow-takeoff regime. In my opinion, the evidence is pretty definitive that at the moment we are entering a slow-takeoff regime[1], and arguably have been in it for the last few years (historically takeoff might be dated to the release of GPT-3).

The last few years have undoubtedly been years of scaling monolithic very large models. The primary mechanism of improvement has been increasing the size of a monolithic general model. We have discovered that a single large model can outperform many small, specialized models on a wide variety of tasks. This trend is especially strong for language models. We also see a similar trend in image models and other modalities where large transformer or diffusion architectures work extremely well and scaling them up in both parameter size and data leads to large and predictable gains. However, soon this scaling era will necessarily come to an end temporarily. This is necessary because the size of training runs and models is rapidly exceeding what companies can realistically spend on compute (and what NVIDIA can produce). GPT-4 training cost at least 100m. It is likely that GPT-5, or a successor run in the next few years will cost >1B. At this scale, only megacap tech companies can afford another OOM and beyond that there is only powerful nation-states, which seem to be years away. Other modalities such as visual and audio have several more OOMs of scaling to go yet but if the demand is there they can also be expended in a few years. More broadly, scaling up model training is now a firmly understood process and has moved from a science to engineering and there now exist battle-tested libraries (both internal to companies and somewhat open-source) which allow for large scale training runs to be primarily bottlenecked by hardware and not by sorting out the software and par...
Priorities for the UK Foundation Models Taskforce
by Andrea_Miotti — 2023-07-21T15:23:34.029Z
Link: https://www.conjecture.dev/uk-taskforce-priorities/

The UK government recently established the Foundation Models Taskforce, focused on AI safety and backed by £100M in funding. Founder, investor and AI expert Ian Hogarth leads the new organization.

The establishment of the Taskforce shows the UK’s intention to be a leading player in the greatest governance challenge of our times: keeping humanity in control of a future with increasingly powerful AIs. This is no small feat, and will require very ambitious policies that anticipate the rapid developments in the AI field, rather than just reacting to them.

Here are some recommendations on what the Taskforce should do. The recommendations fall into three categories: Communication and Education about AI risk, International Coordination, and Regulation and Monitoring.

Communication and Education about AI Risk

The Taskforce is uniquely positioned to educate and communicate about AI development and risks. Here is how it could do it:

Private education

The Taskforce should organize private education sessions for UK Members of Parliament, Lords, and high-ranking civil servants, in the form of presentations, workshops, and closed-door Q&As with Taskforce experts. These would help bridge the information gap between policymakers and the fast-moving AI field.

A new platform: ai.gov.uk

The Taskforce should take a proactive role in disseminating knowledge about AI progress, the state of the AI field, and the Taskforce’s own actions:

The Taskforce should publish bi-weekly or monthly Bulletins and Reports on AI on an official government website. The Taskforce can start doing this right away by publishing its bi-weekly or monthly bulletins and reports on the state of AI progress and AI risk on the UK government’s research and statistics portal.

The Taskforce should set up ai.gov.uk, an online platform modeled after the UK's COVID-19 dashboard. The platform’s main page should be a dashboard showing key information about AI progress and Taskforce progress in achieving its goals, that gets updated regularly. ai.gov.uk should have a progress bar trending towards 100% for all of the Task Force’s key objectives.

ai.gov.uk should also include a “Safety Plans of AI Companies” monthly report, with key insights visualized on the dashboard.

The Taskforce should send an official questionnaire to each frontier AI company to compile this report. This questionnaire should contain questions about ...
Training Process Transparency through Gradient Interpretability: Early experiments on toy language models
by robertzk — 2023-07-21T14:52:09.311Z

The work presented in this post was conducted during the SERI MATS 3.1 program. Thank you to Evan Hubinger for providing feedback on the outlined experiments.

Note: This post was drafted prior to the announcement of

Developmental Interpretability

, which offers a rigorous foundation for some of the ideas surrounding model explanations in light of the full training process. In any case, we believe the provided toy examples of gradient capture and analysis will be useful for validating future hypotheses in this space.

Introduction

Most attempts at mechanistic interpretability (mechint) focus on taking a completed trained model and performing a static analysis of specific aspects of its behavior and internals. This approach has yielded numerous fruits through well-known results such as

grokking

, the

IOI circuit

,

docstring completions

, and many others. However, mechint proceeds essentially in the dark without incorporating any information on the causal formation of features and mechanisms within the model. In particular, early training behavior is different from later training behavior, with

earlier training being more simplistic

.

Focusing on language models, we note that models exhibit “

consistent developmental stages

,” at first behaving similarly to n-gram models and later

exhibiting linguistic patterns

. By taking into account both these transitions and the ultimate source of the development of mechanisms (the content and sequencing of the training data), the task of mechint can become easier or at least provide more holistic explanations of model behaviors. This viewpoint is further elaborated by NYU researcher

Naomi Saphra

in a post where she urges for applying interpretability along the entire training process.

An additional reason this kind of approach could be important relates to the possibility of obfuscation and backdoors in models. In the more dangerous failure modes such as

deceptive alignment

or

gradient hacking

the final model may be in a structure in which a dangerous behavior is not amenable even to full

white-box

mechint. For example, it is possible to

plant backdoors in models

in such a way that no efficient distinguisher (e.g. a mechint technique) can discern the presence of the backdoor. If this happens within the SGD process itself, the only way to identify the existence of the defe...
Can AI Alignment please create a Reddit-like platform that would make it much easier for alignment researchers to find and help each other?
by  — 2023-07-21T14:03:10.105Z

Coming here for the first time I immediately noticed how unstructured the platform is. Imagine AI Alignment creating a brand new platform - or recreating this one if you prefer - that is structured exactly like Reddit. I mean exactly like Reddit. That way the many thousands of open-source developers out there working to solve the alignment problem would know exactly where to go to find the open questions in each category of development, and to also ask for help where they are stuck. Guys, this is a no-brainer. The way you have it now you make it so much more difficult for people to find questions and answers on the part of AI alignment that they are focused on.
Case for Foundation Models beyond English
by Varshul Gupta — 2023-07-21T13:59:20.520Z
Link: https://dubverseblack.substack.com/p/case-for-foundation-models-beyond

We live in a world enthralled by technology, so precisely and skilfully woven into the very fabric of our lives that it's impossible to untangle. Each thread of code, each strand of data, is meticulously encoded with language that is then translated into technology that builds the structures of our digital lives.

And at this very moment, the most dominant language in this realm (shoutout to the Marvel fans)—the lingua franca of our evolving digital society, if you will—is English.

Subscribe to get a blog every week on our learnings in Generative AI.

Andrej Karpathy wrote, "The hottest new programming language is English." If the language a common man across the globe needs to learn to take part in this language revolution is English, then the key to these advancements are no longer a proficiency in Python or Rust. Rather, the golden ticket becomes fluency in English.

Attention English is all you need.

This is a critical issue that requires inspection, especially if we align ourselves with the goal of an inclusive digital world. We should strive to create AI models that cater to more languages across the globe - the echoes of the Himalayas, the chatter of the Nile Delta and the whispers across the Pacific.

Think about it like this - have you ever found yourself lost in the charm of a song without understanding its actual words? I have been listening to

this

(maybe Japanese) song on loop for the last four days. I barely understand a word but I absolutely love it.

See, language goes beyond borders and comprehension. It's a pause in the pattern, an emotional connection. And if our AI models could encompass linguistic diversity - if they could 'hear' that song and 'understand' it - they would undoubtedly be more inclusive.

Matters of Tokenisation and Cost

"But GPT3.5 Turbo and GPT4 can't handle Indian languages well," you might argue. Well, they do a fine job, I'll admit, but at a cost. Surprisingly, processing Indian languages can be 5x pricier than English! This disparity springs from the concept of tokenisation–how words are deconstructed into tokens, not necessarily subwords or phonetics, but the frequency at which byte sequences come together.

number of tokens consumed in different languages

Then arises the concept of fertility. Originating from machine translation, fertility means the potential of a source token to birth multiple target t...
Meta is hiring for LLM red teaming position
by Michael Tontchev — 2023-07-21T13:57:27.239Z
Link: https://us.meta.talentnet.community/jobs/fdcbb283-42a8-4d1b-bcbf-d59f67cd9376

Apply at the link.

We're expanding our red team, whose job it is to try breaking our LLMs to make them exhibit unexpected and unsafe behaviors. Note that one of the requirements for this specific position is a PhD in linguistics. If you have prior red teaming experience (professional or personal), even better :) If you do apply at the link, please send me a message on here as well.

If you do not meet the PhD requirement but you think you're a good candidate for the red team (e.g. have discovered new jailbreaks or adversarial techniques, are curious and have a hacker mindset, have done red teaming before, etc), you can still message me on here so we can keep you in mind if other red teaming positions open up in the future without the PhD qualification.
[Linkpost] Interpreting Multimodal Video Transformers Using Brain Recordings
by Bogdan Ionut Cirstea — 2023-07-21T11:26:39.497Z

This is a linkpost for https://openreview.net/forum?id=p-vL3rmYoqh.

TL;DR: We show that fine-tuning on the vision-language task does not improve the alignment in brain regions that are thought to support the integration of multimodal information over their pre-trained counterparts.

Abstract: Integrating information from multiple modalities is arguably one of the essential prerequisites for grounding artificial intelligence systems with an understanding of the real world. Recent advances in video transformers that jointly learn from vision, text, and sound over time have made some progress toward this goal, but the degree to which these models integrate information from the input modalities still remains unclear. In this work, we present a promising approach for probing a multimodal video transformer model by leveraging neuroscientific evidence of multimodal information processing in the brain. We use the brain recordings of subjects watching a popular TV show to interpret the integration of multiple modalities in a video transformer, before and after it is trained to perform a question-answering task that requires vision and language information. For the early and middle layers, we show that fine-tuning on the vision-language task does not improve the alignment in brain regions that are thought to support the integration of multimodal information over their pre-trained counterparts. We further show that the top layers of the fine-tuned model align substantially less with the brain representations, and yield better task performances than other layers, which indicates that the task may require additional information from the one available in the brain recordings.
Decoding intermediate activations in llama-2-7b
by Nina Panickssery — 2023-07-21T05:35:03.430Z

Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort

Existing research, such as the post interpreting GPT: the logit lens and related paper Eliciting Latent Predictions from Transformers with the Tuned Lens, has shown that it is possible to decode intermediate states of transformer activations into interpretable tokens. I applied a similar technique to llama-2-7b, a 7-billion-parameter decoder-only transformer language model - you can find my code here (file with just relevant reusable code).

Unlike in the Eliciting Latent Predictions paper, I decoded not only the output of a full transformer layer but also the intermediate outputs post-MLP and post-attention-mechanism before and after merging into the residual stream.

This diagram shows the points at which the intermediate activations are decoded for a particular block. To decode them, I pass them through the same layer norm and final unembedding layer that is normally applied after the final transformer block.

Similarly to previous research, I found that the decoded block outputs at most layers, except a few early ones, were interpretable. I also found that the other intermediate outputs were interpretable and provided some intuition on what different layers were responsible for.

Furthermore, the intermediate decoded outputs gave useful information on where it could be most effective to apply a steering vector activation change. In a previous experiment I did, I found that merging activations from different inputs at a specific point in a transformer can result in effective concept mixing. However, such concept mixing does not work as effectively at each layer. By inspecting the decoded layer activations, one can more reliably predict where adding a steering vector will be most effective.

Analyzed example

If we look at the intermediate activations for the final token during a forward pass of 'The capital of Germany is,' we can see the following interesting features:

The first block of a transformer approximates a bigram predictor - it simply predicts the token that most often comes after 'is' in the corpus - in this case 'nt' because of the common bigram isn't.
By layer 11/31, we can first see information related to cities ('Metropol' and 'град' both mean 'city' in different languages).
By layer 14/31, we start to see the relevant context in English.
Lay

...