trentmkelly/LessWrong-43k huggingface dataset:
Self-Play By Analogy
by Amica Terra — 2024-03-24T20:06:20.174Z

This post is adapted from a longer, more wide-ranging post on Substack where I attempt to collect some of my thoughts about AI as a relative outsider to the field. The section I have decided to share here, though, I believe to be novel.

Success in self-play by game AIs like AlphaZero has led to some interest in its possibility of loosening (or even doing away with) the data bottleneck that threatens to strangle scaling as a path forward for LLMs. By making analogies to the human phenomena of dialects and group polarization, I provide a starting point for further, more analytically-framed arguments against self-play as a viable future for increasing the intelligence or linguistic capacity of LLMs.

The Viability of the Analogy

My argument rests, crucially, on an analogy between an AI using self-play to create its own training data and a human (or group of humans) interacting with each or the world to learn and adapt. I believe this is a defensible analogy.

Self-play is quite easy to translate into human cognitive experience. What defines self-play is that the same neural network which creates the new data is also the one which learns by it. The fact that this is possible might be philosophically interesting, but the actual experience is pedestrian.

When a chess player computes his next moves — “I do that, then she does that, I take there, she takes back, I move there, check, check, take, take… no, not good” — the chess player is doing a sort of self-play. Even more direct of an analogy is the fact that chess players often do play themselves in chess, and can learn from that experience. The fact that this is fairly directly analogous to AI self-play seems to me obvious.

What may be less obvious but also seems true to me is that self-play is functionally equivalent as well to a siloed group of functionally equivalent human individuals acting amongst one another. The important part is that they are siloed, that they only have the background conditions of cognition and the game itself to go off of. Given enough time and communication, knowledge becomes communal, and we can consider the group as a single intelligence. Or, if that doesn't seem right, we can pluck one person from the group and expect them to have similar behavior as anyone else in the group.

What is important for this analogy is that it is not surprising that AlphaZero was able to learn better chess than an...
Can quantised autoencoders find and interpret circuits in language models?
by charlieoneill — 2024-03-24T20:05:50.125Z

Executive Summary

I try vector-quantised autoencoders (VQ-VAEs) as an alternative compression scheme of transformer activations (as opposed to something like a sparse autoencoder).

Whilst people have danced around this idea before, discrete quantisation has only ever been tried in the actual transformer architecture itself, rather than on cached activations.

Specifically, I train a VQ-VAE on a data of 1000 cached model activations on the indirect objection identification (IOI) task (500 "positive" examples where the model is required to do IOI, and 500 "negative" examples where it isn't).

For each forward pass through the model (i.e. for each example), this produces a sequence of n_layers discrete integer codes, supposedly capturing the semantics of that progression of the residual stream.

I show that you can use something as simple as a categorical decision tree on these sequences of codes to extract very simple circuits from the model, and then examine the splits in the decision tree as a form of automatically interpreting this circuit.

I confirm that the "circuit" we found is quite close to the IOI circuit originally found in GPT-2 small, particularly since the whole process was unsupervised! Despite my current setup being very simplistic, I think that decision trees over these discrete codes can be a powerful tool for automated circuit discovery and interpretation.

I extend the decision-tree circuit-finder/interpreter to the high-dimensional continuous residual streams.
Finally, I briefly discuss how to make this whole process a bit more robust and how it might scale to more complex tasks and automated workflows.

All code for the VQ-VAE can be found here. All code for sampling can be found here.

Introduction

Mechanistic interpretability has recently made significant breakthroughs in automatically identifying circuits in real-world language models like GPT-2. However, no one has yet been able to automatically interpret these circuits once found; instead, each circuit requires significant manual inspection before we can begin to guess what is going on.

This post explores the use of compression in the form of autoencoders in order to make the residual streams of transformers acting on certain tasks much more conducive to automated analysis. Whilst previous groups have explored similar techniques for uncovering isolated features stored in specific layers (fo...
Mandolin Harp Sensor Placement
by jefftk — 2024-03-24T18:40:05.412Z

One of my goals in adding

electronic
"harp strings" to my mandolin is that I don't want to change
anything about my normal mandolin technique when I'm not using them.
I've been playing for decades, and like how I normally play. This
means I can't place these "teeth" anywhere my hands normally pass
through. With a bit of testing, I think these are the available
areas:

I don't normally pluck the strings that close to the fretboard, but
after some experimenting I realized that I do sometimes when I want a
different sound, so I need to keep that whole area open.

Since I think I want to be able to pluck a tooth on the downbeat and
then strum on the upbeat, the area above the neck sounds like a better
choice than below. I made a cardboard template so I could get a sense
of whether this got in my way when I was playing:

It felt good, so I used my jigsaw to transfer the outline to the piece
of 22ga (1/32") steel I'd attached my first two teeth to:

I cut a bunch more sections off the aluminum offset angle I used
before, and laid them out in a potential layout. FYI this isn't the
layout I ended up choosing:

These are all still pretty rough, so I filed off the burrs and rounded
the corners. Then I epoxied on the piezos and went to bed:

Today I cut out pieces of sorbothane and glued the teeth to the tops:

I decided to go with an offset layout, and will probably map it as Wicki–Hayden
(with the up stroke a half step below the down stroke). I glued the
teeth to the backing:

I taped the backing to my mandolin, and played with my normal
technique for a while to ensure it felt ok. No issues!

Next comes picking out a microcontroller, sorting out the circuits if
I can't just go in direct, and a ton of wiring. [1] Then software to
interpret the signals and generate MIDI, and integrating it into my
existing system. Then learning how to play it!

[1] I haven't figured out my wire routing yet. Possibly I'll use a
drill press to make a bunch of tiny holes, one next to each tooth, and
do all the routing on the back? Though deburring the tiny holes on a
sheet I've already glued the teeth to sounds annoying.

Comment via: facebook, mastodon
AI Alignment and the Classical Humanist Tradition
by PeteJ — 2024-03-24T13:37:28.578Z

Hi guys, I’d like to share a proposal regarding AI alignment. The proposal is that training AI in the curriculum of Classical Virtue ethics could be a promising approach to alignment. A) Because general virtues with many exemplifications can help us teach the AI what we would really want it to do, even when we can't micromanage it, and B) because this pedagogy seems to be a good fit for AI's style of learning more generally.

Background i) The Pedagogy of Classical Art

In the Classical Humanist Tradition the pedagogy of learning depends on whether one studies a subject, or practices an art. Art is understood as craft or skill, for example the art of speaking well and persuading (rhetoric), and the art of living (virtue ethics). Training in the arts is done by

Studying general principles of the art
Seeing the art practiced, either by seeing it done by a master, or by reading or otherwise studying exemplars
Practicing the art (imitation, etc.)

cf. John Sellars, «The Art of Living», and Walker, «The Genuine Teachers of This Art»

Notably, 2) seems to be close to how AI learns – by reading information, and then using it as a base for its own imitation and application.

Background ii) Classical Humanist Ethics as Art

The Classical Humanist tradition of Virtue ethics operates in the wake of Socrates, and includes Plato, Aristotle, the Stoics, the Epicureans, and others. Later some Catholics took this up as well. They all practiced virtue as an art rather than as (just) a science.

The Stoic tradition is especially practical. As described by John Sellars, it has general and theoretical texts on the virtues, especially on the «Cardinal» virtues of justice, temperance, courage, and wisdom. And they also have moral biographies to read to see how exemplars have lived up to or failed to live up to these virtues. In this way one can learn and adopt the virtues by imitating the exemplars. And one can also get a lot of experience second-hand, by reading about their lives.

Massimo Pigliucci has created a beginner’s curriculum in the last chapter of his «Quest for Character». Donald Robertson has written on the same topic in his «How to Think Like a Roman Emperor». They are both writing in the tradition of Pierre Hadot: Philosophy as a Way of Life.

Classical Humanist Virtue Ethics and AI

The combination of general concepts of virtues (for example jus...
UNGA Resolution on AI: 5 Key Takeaways Looking to Future Policy
by Heramb — 2024-03-24T12:23:51.698Z
Link: https://forum.effectivealtruism.org/posts/A7aC2QeyyEtb2u2WL/unga-resolution-on-ai-5-key-takeaways-looking-to-future

The United Nations General Assembly passed its first resolution on AI on 21st March 2024 and passed with the support of all 193 member states. The text is available here to read.

For the purposes of this summary, I focused on key aspects of the UNGA Resolution with regard to future-facing safety measures from advanced AI systems and looked over points such as reduction in the global digital divide, inclusivity and disinformation risks.

5 key takeaways:

1. Calls for Adequate Governance Mechanisms:

The resolution recognized the rapid acceleration in the AI landscape along with some encouragement for regulatory innovation, agile responses and the need for continued international dialogue.
Perhaps my favourite line in the document was, “International approaches (have to) keep pace with the evolution of artificial intelligence systems and their uses.”

2.It had a Little Bit in it for Everyone:

The resolution:

Called on businesses to adhere to the UN Guiding Principles on Business and Human Rights and implement the “Protect, Respect and Remedy” framework. It also alluded to the need for fair competition, particularly for SMEs in the space. At the same time, it called for appropriate safeguards to respect IP rights.
It touched on user rights, safeguarding privacy, raising awareness of the use of AI systems and the negative consequences of AI on the workforce.
For people concerned with future risks, there were risk assessments, evaluations, interoperable standards and the need for regulatory mechanisms to keep up with the evolution of AI systems.
For people concerned with ethics and human rights, the resolution also recognized that improper use of AI may lead to risks to human rights.

3.Encouraged a Rights-Based User-Centric Approach:

The resolution emphasized that human rights and international human rights law must be kept at the centre of discussions. This was a nice way to hint at AI systems that might not be aligned with human values.
It also called for feedback mechanisms for users to report any vulnerabilities or anomalies, as well as provenance mechanisms.

4.Investment and Global Cooperation is the Way Forward:

The resolution touched on the need for a shared international framework with interoperable technical tools, standards or practices while also respecting each country’s domestic regulatory needs.
The resolution highlights the need for "increased collaboration" among pub

...
Are (Motor)sports like F1 a good thing to calibrate estimates against?
by CstineSublime — 2024-03-24T09:07:43.951Z

Today a trend broke in Formula One. Max Verstappen didn't win a Grand Prix. Of the last 35 Formula One Grand Prix, Max Verstappen has won all but 5. Last season he had something like 86% dominance.
For context I believe that I am overall pessimistic when asked to give a probability range about something "working out". And since sports tend to vary in results if using a sport like Formula One would be a good source of data to make and compare predictions against?
Everything from estimating the range a pole position time, or the difference between pole and the last qualifier, from to a fastest lap in a race or what lap a driver will pit for fresh tyres.

What is the best way of doing it?

Every 2 weeks when there is a race at the start of a weekend making a series of estimates using only armchair-general knowledge of the sport, and then test and compare them with the actual numbers by the end of the racing weekend?
Should I actually attempt to build simple models and heuristics and deliberate and give specific reasoning?
Considering F1 stats go back to 1950 should I make guesses about historical races and championships rather than do it in real time?

The last idea raises the point that the intention is not to get better at predicting Formula One racing, but to reduce pessimistic bias from my own predictions on a wide variety of topics. Which gives me doubts about this exercise.

I wonder if I am wrong to think that making real time predictions would be better. Modern Formula One has become highly predictably after almost 15 years of Red Bull, Mercedes, and Red Bull dominance. It still is a sport and therefore no prediction model will ever be perfectly accurate. I think this hits the right balance of being able to predict within a range, while not being totally perfectly predictable and thus a good source of data to use. However, I don't intend for it to be the only one.
I could be way-way off and invite thoughts and experiences from others who have tried to get better at calibrating. And suggestions of how they chose the data sets they compared against?
Nuclear Quantum Immortality Hacking 
by Nezek — 2024-03-23T22:08:50.506Z

I will very briefly explain the abridged version of "Quantum Immortality" but you might want to read more about it.

The short version of Quantum Immortality:

"Imagine you are the cat in the Schrödinger experiment, if the many worlds interpretation of quantum mechanics is correct (a big 'if'), you will only detect a world where you are alive. since you cant notice a world where you are dead, so from the point of view of you the cat, the probability of survival is always 100%"

Simple Quantum Immortality Hacking:

This notion begs to be exploited, consider the case of wiring a random quantum experiment that would bet on a stock 50% of the times and against it for the other 50% of the times while also murdering you in the case of losing the random quantum bet.
like Schrödinger's cat you would only detect the world where you were not murdered and therefore made a profit.
Repeat until you are rich right? NO

For Goodness sake !

This is the time to say I strongly advise AGAINST any attempt to self harm or quantum probabilistic self harm, there are many good reasons to think this whole ordeal will not work, you can read all about it.

But for fun, lets assume it does work

Many of us might still feel uneasy with the thought of being murdered in most worlds, leaving our beloved family and friends behind in the vast majority of worlds having a very odd device attached to our corpse.
easy... we can just have our Quantum Immortality Machine Murder them as well in case of a losing bet.
and now our confused and probably resentful beloved will "travel" with us to the worlds where the quantum bet is always winning.
for the love of everything that is holy DONT DO THIS, I want to assume I am talking to adults who know the difference between an interesting thought experiment and homicide.

And for Narcissists Introducing:
'Nuclear Quantum Immortality Hacking':

Great, we have made a machine that would transport us and everyone we love to a world we can engineer, sounds great if all of the assumptions we rely upon are correct (a big 'if'). but what if I am a narcissist, and dislike the idea of the rest of humanity escaping my grasp and judgment, I don't want to just disappear from most worlds, I want to impose my will on all humans.
Easy-peasy.. you just wire your Quantum Immortality Murder Machine to an arsenal of world ending nukes that would eradicate all life o...
My Detailed Notes & Commentary from Secular Solstice
by Jeffrey Heninger — 2024-03-23T18:48:51.894Z

Previously: General Thoughts on Secular Solstice.

This blog post is my scattered notes and ramblings about the individual components (talks and songs) of Secular Solstice in Berkeley. Talks have their title in bold, and I split the post into two columns, with the notes I took about the content of the talk on the left and my comments on the talk on the right. Songs have normal formatting.

Bonfire

The Circle

This feels like a sort of whig history: a history that neglects most of the complexities and culture-dependence of the past in order to advance a teleological narrative. I do not think that whig histories are inherently wrong (although the term has negative connotations). Whig histories should be held to a very strict standard because they make claims about how most or all of human history functions.

The song describes morality in terms of an expanding circle of concern: kin → neighbor → humanity[1] → "feathers, fur, and silicon" → future.

Trying to line these up with historical societies or ideologies is ... difficult. Many societies do not have a concept of 'neighbor,'[2] and some do not understand ethics in terms of circles of moral concern.[3] A few moral systems are universalistic (i.e. they teach that people should have moral concern for all of humanity): Christianity,[4] liberal democracy,[5] and maybe Buddhism.[6] Actually practicing universalism is really hard: Most societies which preach universalism do not live up to its ideals.

Within one of these traditions, the whig version of history can make sense. Over the centuries, Christianity has dramatically expanded and Christian activists from Francis of Assisi to Martin Luther King have made it more true to the ideals of the New Testament. Similarly, liberal democracy has expanded dramatically, extended the right to vote for more people, and gotten better at defending many freedoms. (I don't know what's going on with Buddhism, but its failure to build/maintain a dominant position in India is evidence that universalist ideologies do not generally outcompete other ideologies.)

This song cannot be simply about the spread of an existing ideology like liberal democracy. It also looks beyond existing ideologies and wants to push its ethics to include animals, computers/software, and the long term future.[7] The whig history described by the song does not have good evidence when...
General Thoughts on Secular Solstice
by Jeffrey Heninger — 2024-03-23T18:48:43.940Z

I attended Secular Solstice in Berkeley last December.

My perspective is quite unusual: I live in a rationalist group house and work at an AI safety office, but I also am a Christian and attend church every week.[1]

I was originally not planning on going to Solstice, but a decent number of people (~5) told me in person that they would be particularly interested in my opinions of it. I realized that I was interested in learning what I would think of it too, so I went.

I took notes on my thoughts throughout the service.[2] This blog post is my broader thoughts on the experience. I also have blog posts for a fun little correction to one of the songs and my detailed notes & commentary.

Overarching Narrative

I do not agree with the overarching narrative presented at Solstice.

There is a narrative in my tradition about people becoming humble and turning to God. You can choose to be humble or you can be "compelled to be humble" by the difficult circumstances in life. I'm not super fond of this description because being humble and turning to God is always a choice. But there is some truth in it: many people do find themselves relying on God more and developing a deeper relationship with Him through the more difficult times in their lives.

The overarching narrative of Solstice felt like a transmogrified version of being compelled to be humble. The descent into darkness recognizes the problems of the human condition. Then, instead of turning to humility, it turns to a fulness of pride. We, humanity, through our own efforts, will solve all our problems, and become the grabby aliens we hope to be. There is some caution before the night, learning to accept things we cannot change, but this caution melts away before the imagined light of the Great Transhumanist Future.

AI X-Risk and AI Transhumanism

Existential Risk

A major cause for concern leading into the night was existential risk from AI: the chance that future artificial intelligence systems might kill everyone. This was talked about more than any other problem.

I expect that the organizers and speakers of Solstice are significantly more doomy than the audience.[3] The audience itself probably has selection effects that make it more doomy than AI researchers, or forecasters, or other groups of people who have thought about this possibility.

It is often the case that people's beliefs are more determined by wh...
How to make food/water testing cheaper/more scalable? [eg for purity/toxin testing] 
by Alex K. Chen (parrot) — 2024-03-23T05:28:17.273Z

Infrared spectroscopy, HPLC, Surface-enhanced Raman spectroscopy, and reverse ecology are examples. Figuring out the purity of one's foods/drugs (which is not done enough) can be a Pareto-efficient improvement on the attention economy of "pivotal actors", and can be done by environmental health specialists (whose skills are orthogonal to "pivotal actors"/those in our communities and are not appreciated enough - they were involved in the construction of the new Harvard SEAS building). Enhanced testing could be the first step towards improving the purity of one's food/water and help "cleaner thinking" (which alone has a lot of alignment potential).

Why is this important? B/c people are not mindful enough of the potential toxins/contaminants in their food (we are only beginning to learn the effects of microplastic accumulation in the body/brain), and given that some toxins bioaccumulate in the brain, the effects compound. AI gives rapid improvements to bioweapons near-term (which Kevin Esvelt is very concerned with), which makes contamination of the food/water supply by foreign agents particularly concerning.

Russia will continue to troll the world, and who knows what foreign biological agents that Russians may introduce to the food supply of "pivotal actors".

Mithridates, the Poison King, had deep knowledge of botany and employed many people to taste his food for him. An excellent biography (referenced by Peter Fedichev) is here: https://www.amazon.com/Poison-King-Legend-Mithradates-Deadliest/dp/0691150265

So one of the most important questions is.. what are some low-effort food spectrometers one can get? Given that short timeline/AI will deflate prices of some machines and occupational specialists, the cost should not be an issue, especially for those in high-value corporations that agents will try to infiltrate or poison (such as OpenAI).

Some links:

infrared spectroscopy: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8997473/

https://www.rapamycin.news/t/are-chromatography-mass-spec-assays-the-ideal-methods-for-determining-compound-purity-eg-for-rapamycin-or-semaglutide/12290