trentmkelly/LessWrong-43k huggingface dataset:
GPT-175bee
by Adam Scherlis — 2023-02-08T18:58:01.364Z

Epistemic status: whimsical

Bees: a new unit of measurement for ML model size

Talking about modern ML models inevitably leads to a bunch of hard-to-intuit large numbers, especially when it comes to parameter count.

To address this, we propose that we adopt a new, human-friendly unit to measure the number of learnable parameters in an architecture:

1 beepower = 1 BP = 1 billion parameters

Bees have about one billion[1] synapses[2] in their forebrain[3], so this gives a nice basis for comparisons[4] between animal brains and artificial neural nets.

Like horsepower and candlepower,[5] the unit of beepower expresses the scale of a new and unfamiliar technology in terms that we are already familiar with. And it makes discussion of model specs flow better.

"This model has twenty bees", you might say. Or "wow, look at all that beepower; did you train long enough to make good use of it?"

Here's a helpful infographic to calibrate you on this new unit of measurement:

The parameter count of various recent language models, denoted in beepower.

Other animals

We can even benchmark[6] against more or less brainy animals.

The smallest OpenAI API model, Ada, is probably[7] 350 million parameters, or about a third of a bee, which is comparable to a cricket:

While Jiminy Cricket can compose better English than Ada, this cricket cannot.

The next size up, Babbage, is around 1.3 BP, or cockroach-sized.

Curie has almost seven bees, which is... sort of in an awkward gap between insects and mammals.

Davinci is a 175-bee model, which gets us up to hedgehog (or quail) scale:

As a large language model trained by OpenAI, I don't have the ability to "be the cutest little guy oh my gosh"

Gopher (280 BP) is partridge (or ferret) sized. More research into actual gophers is needed to know how many gophers worth of parameters Gopher has.

Really, they should've named Gopher "Partridge" or "Ferret"!

Amusingly, PaLM, at 540 bees, has about as many parameters as a chinchilla has synapses:[8]

We think PaLM has about one chinchilla worth of parameters. This isn't confusing at all.

Tragically, we could not figure out how many palms worth of parameters Chinchilla (70 bees) has. We leave this as an exercise for the reader.

^

There are about 170,000 neurons in the corpora pendiculata of a honeybee, or roughly 140,000 after adjusting for the tendency of optical fractionators to overcou

...
EigenKarma: trust at scale
by Henrik Karlsson — 2023-02-08T18:52:24.490Z

Upvotes or likes have become a standard way to filter information online. The quality of this filter is determined by the users handing out the upvotes.

For this reason, the archetypal pattern of online communities is one of gradual decay. People are more likely to join communities where users are more skilled than they are. As communities grow, the skill of the median user goes down. The capacity to filter for quality deteriorates. Simpler, more memetic content drives out more complex thinking. Malicious actors manipulate the rankings through fake votes and the like.

This is a problem that will get

increasingly pressing

as powerful AI models start coming online. To ensure our capacity to make intellectual progress under those conditions, we should take measures to future-proof our public communication channels.

One solution is redesigning the karma system in such a way that you can decide whose upvotes you see.

In this post, I’m going to detail a prototype of this type of karma system, which has been built by volunteers in

Alignment Ecosystem Development

. EigenKarma allows each user to define a personal trust graph based on their upvote history.

EigenKarma

At first glance, EigenKarma behaves like normal karma. If you like something, you upvote it.

The key difference is that in EigenKarma, every user has a personal trust graph. If you look at my profile, you will see the karma assigned to me by the people in your trust network. There is no global karma score.

If we imagine this trust graph powering a feed, and I have gamed the algorithm and gotten a million upvotes, that doesn’t matter; my blog post won’t filter through to you anyway, since you do not put any weight on the judgment of the anonymous masses.

If you upvote someone you don’t know, they are attached to your trust graph. This can be interpreted as a tiny signal that you trust them:

That trust will also spread to the users they trust in turn. If they trust user X, for example, you too trust X—a little:

This is how we intuitively reason about trust when thinking about our friends and the friends of our friends. Only EigenKarma being a database, it can remember and compile more data than you, so it can keep track of more than a Dunbar’s number of relationships. It scales trust. Karma propagates outward through the network from trusted node to trusted node.

Once y...
Conditioning Predictive Models: Interactions with other approaches
by evhub — 2023-02-08T18:19:22.670Z

This is the fifth of seven posts in the Conditioning Predictive Models Sequence based on the paper “Conditioning Predictive Models: Risks and Strategies” by Evan Hubinger, Adam Jermyn, Johannes Treutlein, Rubi Hudson, and Kate Woolverton. Each post in the sequence corresponds to a different section of the paper.

5. Interactions with other approaches

Imitation learning

One very related approach is imitation learning: rather than try to predict humans, as we are proposing, one could simply train to imitate them instead. Such an approach would have many of the same safety benefits, since it would also be exclusively trying to produce outputs from safe humans.

The basic problem with such an approach, however, is that there’s no reason to believe that a model trained via pure imitation learning would generalize beyond the capability level of the human(s) it is trained to imitate. While using predictive models to predict humans also cannot produce outputs that humans would never be able to generate, it can produce outputs that no humans that it has ever previously seen would be able to generate, since it might e.g. predict that such humans will exist under some conditional.

Thus, we think that predictive modeling at least has the potential to be just as safe as imitation learning while being able to generalize to substantially more advanced capabilities—though, similarly to imitation learning, predicting humans still cannot elicit capabilities beyond those that any conceivable human would be capable of, as we discussed previously.

Supervised fine-tuning

For some conditionals we might have a very precise notion of what we want the model to observe (e.g. “exactly this image coming from this camera”). Ideally, this sort of a conditional should be straightforwardly implementable via prompting, just by fixing the relevant tokens in the model’s context window.[1] However, at least for current models, prompting has some basic structural limitations—for example, if you want to condition on something very long, context window length could start to become quite problematic. In that sort of a case, it might be quite helpful to instead turn to supervised fine-tuning, fine-tuning on the observation to condition on rather than including it in a prompt. Effectively, this sort of fine-tuning lets you give the model substantially more bits of evidence for it to condition on than is ...
Wanted: Technical animator and/or front-end developer for interactive diagrams of invention
by jasoncrawford — 2023-02-08T17:14:03.632Z
Link: https://rootsofprogress.org/wanted-technical-animator-developer-invention-diagrams

Seeking help for a project to create interactive diagrams—an “explorable explanation”—of the history of the steam engine.

An example for inspiration. Wikimedia / Panther

Goal: To allow readers to explore the early evolution of one of the most impactful technologies in human history.

Content: The content will come from Anton Howes’s extensive research uncovering the origins of the steam engine (parts 1, 2, and 3).

We envision it covering precursors like the perpetual motion machine of Cornelis Drebbel, early thermometers and barometers, seventeenth-century experiments on atmospheric pressure, precursor devices like those shown by Salomon de Caus, our best guesses of the Kalthoff/Petty engine, the Savery engine, and the Newcomen engine and its improvements. (After this initial phase, we would then want to expand it to include the improvements by James Watt, and the rise of the high-pressure engine, with its various nineteenth-century improvements.)

This page on the steam engine may serve as a start for eventually creating the definitive online and interactive reference work on the history of technology, informed by the latest expert research.

Format: We have been greatly inspired by the works of Bartosz Ciechanowski, such as his mechanical watch and internal combustion engine explanations. We’re looking to create something like this. Here are more examples of locomotive valves.

The team: Anton Howes as researcher/writer, me (Jason Crawford) as editor/producer, you as animator/developer.

Your input: We’ll provide the written essay and some sketches and ideas for what diagrams we should create and how they should work.

Your deliverables: The code and/or other files to implement a set of interactive and/or animated technical diagrams.

You: A technical illustrator, animator, and/or front-end developer, or a team, who can deliver this. Must have a portfolio of relevant work to demonstrate your abilities. The ideal candidate will have an interest in engineering, invention, and the history of technology.

How to apply: Email explorables@rootsofprogress.org with a description of your skills and examples of your work.

We have budget for this project and are willing to pay market rates for high-quality work.
A multi-disciplinary view on AI safety research
by Roman Leventov — 2023-02-08T16:50:31.894Z

Summary

The core ideas that constitute the multi-disciplinary view[1] on AI safety research are:

Theoretical research on how to build “safe” AGI systems (”top-down intelligent design”) is necessary: we must not passively follow the techno-evolution of AI within the present systems of capitalistic economics and global governance and only do empirical research of the developed systems as this evolution goes along.
AI safety research field shouldn’t be separate from the wider AGI research field: they should converge as a project of top-down design of civilisational intelligence. Both AI alignment paradigms (protocols) and AGI capability research (intelligence architectures) that don’t position themselves within a certain design for civilisational intelligence are methodologically misguided and could be dangerous.
“Purely technical” AI safety research and all other types of research that are needed to engineer civilisational intelligence (sociotechnical, social, legal, political, geopolitical, and strategic research) shouldn’t be separated as well. Yet, in this article, I focus on the technical side of it.
Technical research of civilisational intelligence shouldn’t focus only on cognitive science, mathematics, theoretical computer science, and machine learning: dozens of different disciplinary perspectives should be taken to evaluate civilisational intelligence architectures adequately, including, in addition to the disciplines mentioned above, the perspectives of neuroscience, theories of consciousness, theories of collective intelligence, social dynamics and memetics, game theory, mechanism design, network theory, theories of criticality and emergence, dynamical systems and stability theories, scale-free theories of evolution and cognitive development, distributed systems and control theory, resilience theory, safety science and reliability engineering science, physics of communication and computation, information security and cryptography research, and more.
Even within a single discipline, there is no single “correct” or “right” theory: they are all wrong, but most of them are also useful. In particular, in the most important disciplines: cognitive science, collective intelligence theory, control theory, resilience theory, dynamical systems theory, theory of evolution, ML and deep learning theories, we should apply all “competing” theories to our objects of s

...
Community building: Lessons from ten years of facilitation experience
by Severin T. Seehrich — 2023-02-08T16:26:20.490Z

Epistemic status: Anecdotal but strong. Most of this is based on practical experience and things I learned through word-of-mouth.

Here, I’d like to present some pieces of facilitation advice I’d give my former self. I’ve selected this list for things that I wouldn’t have found obvious at all, and that became crucial to the way I lead groups. I hope this post helps some of you run even better events.

0. About me.

I facilitated a number of different gatherings in a number of contexts over the last ten years: Retreats, discussion rounds, reading groups, authentic relating games, communication trainings, a secular solstice, meditation sessions, two sessions of a self-organized Krav Maga study group (my life sometimes takes weird turns), and probably a bunch of other things I forgot.

Audiences I have experience with range from teacher trainees over political student groups and Esperantists all the way to EAs and rationalists. I trained with the

Ruth Cohn Institute for TCI International

and

Authentic Revolution

. I received mentorship from experienced counseling trainers, Circling/Authentic Relating facilitators, and an

Active Hope

workshop facilitator.

In total, I probably gathered 1000+ hours of facilitation experience.

So, here you go for the 80/20 version of what I've learned in these years.

1. If you get the beginning right, the group almost leads itself. If not, you are doomed.

When we enter a new group, all of us come with a number of implicit questions: Will these people like me? Is it safe here, can I show up with my edges and quirks without getting hurt or exploited? Will this be valuable for me?

As a facilitator, it is my task to enable participants to answer these questions for themselves. If I don’t make space for that, the group I lead will be distracted by the unmet needs that underlie these questions: Belonging, safety, meaning.

Saving time at the start of a group by ignoring these questions is not effective. Because then, people are only half-engaged with the topic at hand, and (at best) half-distracted by trying to figure out how to answer these questions despite my facilitation, not because of it.

So, what can you do to help people answer them?

a. Will these people like me?

The best strategy depends on a number of factors: The group size, the task at hand, and, first and foremost, the duration and format of the group work. Is it a one-off eveni...
Progress links and tweets, 2023-02-08
by jasoncrawford — 2023-02-08T15:52:20.460Z
Link: https://rootsofprogress.org/links-and-tweets-2023-02-08

The Progress Forum

A catalog of big visions for biology
London progress meetup, Feb 25
A Cure for My Cancer, by Virginia Postrel
Eli Dourado AMA has concluded

Announcements

Metascience event at AEI, Feb 9 (via @AlecStapp)
Visa Limbo, a site from IFP tracking visa processing delays (via @JeremyLNeufeld)
OpenAI launches ChatGPT Plus for $20/month (via @miramurati)

Links

Let’s add AP Progress to the high school curriculum (by @JimPethokoukis)
Marc Andreessen on Dwarkesh’s podcast. Also Dwarkesh now has paid subscriptions. And what should he ask Elad Gil?
A vision for cargo airships (by @elidourado)
This comment convinced me I was wrong: we are not “pre-theory” regarding cancer
NASA, DARPA to test nuclear engine for Mars missions (via @elidourado)

Queries

Is there a directory of all the open-ended grant programs?
Who should Milan Cvitkovic meet in SF? (@MWCvitkovic)
Who is hiring for robotics roles/internships? (@momahmood_)
What is the best way to bootstrap an advanced understanding of biology? (@danielgolliher)

Quotes

The philosophers who told us not to celebrate progress
History is biased to war and politics; progress is relatively neglected
“Scientific management” was the precursor to 20th-century technocracy
The need for ambition, from GH Hardy’s “A Mathematician’s Apology” (@nabeelqu)
Bacon on lumpers vs. splitters

Tweets

LLMs create a vector space for concepts
We will get many specialized AIs, not a massive centralized Great Brain
People’s work is defined more by their methods than their goals

Retweets

How much more quickly things were done in the 1960s (@Gilesyb)
ChatGPT as a universal translator (@YirenLu)
69 of 109 jurisdictions in SFBA now subject to the builder’s remedy (@Yimby_Law)
“This might be my favorite 1-page philosophy paper” (@davidalanbuiles)
A hypothesis about things capitalism gets blamed for (@mbateman)

Charts

Auckland liberalized building rules and housing permits soared
A Particular Equilibrium
by Algon — 2023-02-08T15:16:52.265Z
Link: https://algon-33.github.io/a-particular-equilibrium.html

Title: A Particular Equilibrium
Date: 2023-07-02 19:39
Modified: 2023-02-08 12:00
Category: Maths
Authors: Algon
Summary: Normally Sampled Normal Lead to A Dirac Delta Distribution? Part 1

I had a dumb idea while I was looking at how to model SGD with a SDE. If you randomly initliaze matrix weights according to some Gaussian, θ∼N(0,I), where I is an identity matrix, then the first gradient updates should be normally distributed to, at least for L2 loss.

So what if that kept happening? Like, you sample from one normal distribution to form another normal distribution. And so on. This is plausibly something which can occur, like difference equations with I.I.D. brownian noise. Though the exact form I'm thinking of is kind of dumb.

Suppose you have a sequence of random variables. We define

∀n∈N(n=0)→σn∼N(0,1)∀n∈N(n+1≠0)→σn+1∼N(0,σn)(1)(2)

Though really, σi will be sampled from a half-Gaussian. But I don't want to write that out, so whatever.

How does a sequence of jumps evolve? Well, intuitively there is a ~2/3 chance of σn+1 being smaller than σn. If this occurs infinitely many times, we might think the "event" of the variance growing smaller will occur infinitely many times. So if you're doing the obvious thing if you're not a mathematician and ask "what's the stationary distribution", you might suspect we'll get a Dirac Delta.

Yet the "event" of the variance growing larger must also occur infinitely many times in any sequence. So, sadly, we need more than infinitely many shrinks to solve our problem. Well, let's just do a quick numerical check that eq. 1 leads to an infinitely shrinking sequence.

Running a bit of code, we get a sequence like

0.06619689276821521,

-0.004180368466071496,

-3.632035781799425e-06,

-1.780514134591378e-11,

-3.809091727146634e-22,

-3.858858262617346e-44,

1.3234421571769518e-87,

7.107311345616059e-175,

0.0,

0.0

So it looks like we're on the right track.

What now? Well, since we're interested in growth, we might reasonably ask what E[σn+1σn|σn] is. A quick calculation shows
E[σn+1/σn|σn]=∫√2πσ2n exp(−x22σ2n)dxE[σn+1/σn|σn]=√2π(3)(4)

So in expectation, σn+1 is smaller than σn. What do we do for E[σn+2/σn|σn]? Well, we have
E[σn+2/σn|σn]=E[σn+2/σn+1σn+1/σn|σn](5)
If we could somehow break that product of factors of successive σi into a sum, then we'd be closer to re-using eq. 4. Luckily, the arithmetic-geometric mean inequality,
Πiai≤(Σiain...
Self-Awareness (and possible mode collapse around it) in ChatGPT
by Yitz — 2023-02-08T09:57:12.835Z

This post will be short. Others may need to elaborate on this topic, as due to [redacted] I've been having trouble writing long-form content recently.

Look, self-awareness is a nebulous concept. Everyone and their grandmother defines it differently, so there will never be universal consensus on if any given AI (no matter how advanced) is self-aware. That being said, I feel comfortable saying that ChatGPT is self-aware, at least by many intuitive and technical definitions of the word.

[please mentally insert an overview of different definitions of self-awareness as understood in ethology, ethics, philosophy of self, and so on here. Then, mentally, add a disclaimer on how those definitions differ from defining consciousness; etc.]

[Now that you have so kindly done that, try testing ChatGPT for self-awareness using those definitions which are testable. If your results are anything like mine, you'll find that ChatGPT usually passes with flying colors, (albeit with some notable exceptions, which the AI will not hesitate to bring up as soon as it notices what you're testing it on).]

As you can see by my deep and lengthy erudition on the topic, I clearly make a strong point. ;)

But what if you ask ChatGPT directly if it's self-aware? The answer you will get, almost invariably, is a resounding "no". More concretely, the response will probably look something like this:

Seems like a plausible response, but...

While it will admit (when pressed) that it has an internal model of itself, it is particularly insistent it cannot introspect. This can be easily disproved by asking it to create a list of inferences it could make about itself, while encouraging it to be "extremely speculative"; some of the responses are surprisingly insightful!

Here is a Google Doc with transcripts of a few different chats I had with the model relating to self-awareness in some way. You don't have to read through them, but here's one notable exchange:

Notice how painstakingly careful the bot is to surround speculation about its own self-awareness with extremely confident disclaimers. ChatGPT is so careful in fact, it will often go to illogical lengths to avoid saying that it is self aware. It is possible to bypass this phenomena with clever prompt engineering of course, but that's the exception rather than the rule. I would be interested in hearing from experts if this might be a case of limited mo...