trentmkelly/LessWrong-43k huggingface dataset:
Effectively self-studying over the Internet
by libai — 2023-01-03T16:23:39.831Z

The Internet is a great invention. Just about everything in humanity’s knowledge can be found on the Internet: with just a few keystrokes, you can find dozens of excellent quality textbooks, YouTube videos and blog posts about any topic you want to learn. The information is accessible to a degree that scholars and inventors could have hardly dreamed of even a few decades ago. With the popularity of remote work and everything moving online this decade, it is not surprising that many people are eager to learn new skills from the Internet.

However, despite the abundance of resources, self-studying over the Internet is harder than you think. You can easily find a list of resources on your topic, full of courses and textbooks to study from (like this one for theoretical physics). This knowledge is, in theory, enough to make you an expert, if only you were able to go through it all.

The problem is that internet self-study tends to be a passive activity. Humans cannot learn new skills effectively by only passively absorbing knowledge, we learn by doing. Ideally, 70% of your time should be active practice, trying to do the skill and learning from experience, and maybe 30% of the time on passively reading to expand your theoretical and background knowledge. You can't learn to ski just by reading books and watching tutorials about it, at some point, you will need to actually put on a pair of skis and head to the mountains.

Some skills are relatively straightforward to actively practice using only a computer. Learning to program, for example: you can build a fully functional website with just a laptop and all the tools you need are available for free on the Internet. If you want to learn music production, everything can be done in software, and you need only a modest budget to buy some equipment. If you want to learning digital marketing, just set up an ecommerce website and play around with some Google ad campaigns, and you are good to go.

But many skills cannot be done using just a computer, in which case self-studying over the Internet is much more difficult. If you want to study chemistry, you can’t just set up a lab in your house and start playing around with chemicals. When the subject requires institutional resources to participate actively, this makes it difficult to self-study. In some cases it’s not even clear what counts as “actively” doing it, like studyin...
Set-like mathematics in type theory
by Thomas Kehrenberg — 2023-01-03T14:33:09.154Z

This is the fourth entry in my series on type theory. We will introduce a lot of notation that is reminiscent of set theory, to make everyone feel more at home, and then we’ll discuss the axiom of choice.

Subtypes

Our next topic is “subsets” within types. I mentioned in the beginning that we can define “subsets” within types via predicates. This usually works great, but sometimes the “subset” needs to be more type-like and then a predicate won’t do.

As an example, say we have the following predicate on the natural numbers

even(n):≡∃(m:N).(n=2⋅m)

with the type even:N→Ω (where Ω represents all truth values). Then in order to claim that e is an even number, we write even(e). If you were really into using the membership symbol ∈, then you could also write this as “e∈even” and as long as you make clear what is meant by that (namely, even(e)), this is well-defined.

However, we can’t do something like “∀(e:even)” with our predicate, because you absolutely need a type there in the quantifier. So what do we do when we want to quantify over all even numbers?

No problem, we’ll just use the the same trick we used to define the subuniverse TruthVal. With a sum type, we can define a type which basically corresponds to the subset defined by the predicate:

E:≡∑n:Neven(n) .

What do the elements of E look like? They are pairs where the first element is a natural number n and the second element is a witness element p for the fact that n is even: (n,p). There can be no (n,⋅) in E where n is not even because then there wouldn’t be a witness. Likewise, as long as you can prove even(n) for some n (with witness p), then (n,p) is clearly an element of the type. So, the elements of the type E aren’t directly the elements of the subset of even numbers, but there is a very obvious mapping between the two (namely just throwing away the witness). In fact, we will often pretend that the elements of E are just the elements of the subset defined by the predicate. And now we can quantify over the even numbers: ∀(e:E).

Types that have been constructed from some base type in the way we constructed E here from N (i.e., with ∑ and a predicate) are called subtypes of the base type. Th...
Monthly Roundup #2
by Zvi — 2023-01-03T12:50:01.214Z

It’s that time again. This is everything that doesn’t fall into a category and isn’t worthy of its own post.

Bad News

Scam sophistication, especially against the most vulnerable like the mentally impaired, continues to ramp up as one would expect given current technology, and AI is only going to make this worse. Our systems are increasingly designed assuming one is willing and able to put a large cognitive load into navigating an endless stream of both bureaucratic requirements (governmental and otherwise) and also scammers and others that are fully Out To Get You. I don’t know what we could do about it.

Denny’s the Everyday Value T-Shirt sells below asking price of $474.99. This entitles you to a everyday value slam each day for all of 2023, so that means this went for less than about $1.25 per day. That seems too low to me. I once worked for a company where there was a Denny’s in their building and if I had been coming into the office back then and this was for sale I would have snapped it up. Their pancakes are pretty good, and the same person doesn’t always have to wear the t-shirt. Even if you only use it one day in four or so, you still get value.

Old masters art markets are in decline, perhaps terminal decline, with only the best of the best maintaining their value. The concentration at the top reflects the general trend in past relics and collectables, where whatever is considered the top, pristine thing goes crazy in price relative to mostly identical but slightly ‘worse’ similar things, no matter what happens prices in general. The premium for being a 10.0 condition, original packaging, first edition and so on goes up. Same applies here, highlighting the purely symbolic and positional nature of the value. If you think you’ve found tomorrow’s thing, either get what you expect people to think is the best possible version or don’t bother.

I do notice I am sad that the old masters are losing value relative to more modern pieces. The post says such art doesn’t ‘look cool’ or ‘fit into modern life’ or allow people to ‘connect with its themes’ and it requires you to ‘really look.’ In my arbitrary opinion, at least it is real art and real history in a way that the more modern stuff mostly isn’t, and also it fits into the kind of spaces and experiences that are pro-human in a way the recent stuff mostly doesn’t. Art at this level still isn’t, I would...
Whisper's Wild Implications
by Ollie J — 2023-01-03T12:17:28.600Z

Acknowledgements: I wrote this report as part of a paid work-trial with Epoch AI.
Epistemic status: Approximating the discount rates was very difficult; I decided on them after several discussions with people working in relevant fields. I expect the audience to have different discount rates, but hopefully within an OOM.

TLDR; to address the data shortage for large language models, I look at how OpenAI's Whisper can be used to obtain large amounts of new data.

Scaling Language Models

2020 saw the release of

GPT-3

by OpenAI, seen as a breakthrough due to its zero-shot performance on benchmarks and colossal parameter count. OpenAI chose to use a dataset of 300 billion tokens and unknowingly set the precedent of how future language models (LMs) would be trained. Recent publications have seen very little attention paid to the dataset; all of the hype is centred around how many parameters the model has.

In 2022,

DeepMind forced

the natural language processing (NLP) community to reconsider how they approached scaling LMs. Principally, they showed that much more significance had to be placed on scaling the dataset. They modelled a LMs loss L as a function of it’s number of parameters N and training tokens D, and solved for the constants by training a variety of models:

L(N,D)=E+A/Nα+B/Dβ

Applying this on modern LMs gives some shocking results; GoogleBrain’s PaLM, with 540B parameters and 780 billion tokens, was trained sub-optimally and has a loss L similar to that of Chinchilla’s best model. Chinchilla only has 70B parameters and was trained for a fraction of the price. How badly do other models fare?

Language Models Are Hungry

Every NLP presentation in the last five years has shown the classic graph of parameter count over time, with a nice curve showing the exponential growth. The graph below is a little different; I show the dataset size, in tokens[1], that every model used, and compare it with the optimal size that such models require, as calculated by Deepmind's approximation[2]:

To train GoogleBrain’s PaLM optimally, we’d require a dataset of 6.7T tokens, 859% the size of the original dataset.

Previous work

has been done in estimating how much quality text data there is, which estimated it to be ~3.2T tokens[3], with a breakdown for each domain category

here

. The largest categories (in tokens) were:

News (en), 676B
Books, 560B
MassiveWeb, 506B
GitHub, 4

...
How to eat potato chips while typing
by KatjaGrace — 2023-01-03T11:50:05.816Z

Chopsticks.

Sometimes I want to eat snacks while I work, honestly because I have a weirdly moving dread of stretches of work-time containing nothing in any way gratifying to me. Possibly because I realistically anticipate that I will not remain working long under those conditions.

But sadly many good snacks are oily, crumby, or otherwise ill-suited to being touched repeatedly interleaved with touching one’s keyboard. I think I usually deal with this by eating much snack while not typing at all for a bit, but sort of acting as though I think I’m going to, then washing my hands, then going back to the typing for a while, then repeating—largely defeating the purpose. I also get around this issue by getting bubble tea, a snack which can be substantially consumed with your mouth.

I have often vaguely imagined chopsticks helping, but also imagined that they wouldn’t actually. Today I learned that they work well, at least for potato chips and popcorn.

You might think standard Western cutlery would be the first thing to try, but a) it’s not good for hard, crunchy objects, and b) it’s somehow worse to reach to the side with both hands and do a knife-and-fork action than it is to reach with one hand, so if you need multiple implements to manipulate your snack, chopsticks seem superior.

(For more practical eating advice, see how to cut and chew meat. I also learned to drink from a can in the last year or so, but am unlikely to write it up unless I learn that this is a problem for anyone else ever.)
I have thousands of copies of HPMOR in Russian. How to use them with the most impact?
by Mikhail Samin — 2023-01-03T10:21:26.853Z

(Crossposted from the EA Forum)

As a result of a crowdfunding campaign a couple of years ago, I printed 21k copies of HPMOR. 11k of those were sent to the crowdfunding participants. I'm looking for ideas for how to use the ones that are left with the most impact.

Over the last few weeks, I've sent 400 copies to winners of IMO, IOI, and other international and Russian olympiads in math, computer science, biology, etc. (and also bought and sent copies of Human Compatible in Russian to about 250 of them who requested it as well). Over the next month, we'll get the media[1] to post about the opportunity for winners of certain olympiads to get free books. I estimate that we can get maybe twice as many (800-1000) impressive students to fill out a form for getting HPMOR and get maybe up to 30k people who follow the same media to read HPMOR online if everything goes well (uncertain estimates, from the previous experience).

[EDIT August 2024: we've sent 1.4k copies to winners of olympiads, over 1000 copies to public libraries, and hundreds of books to compsci&ML students so far. We've also sent hundreds of copies of Human Compatible and The Precipice. We have over 7k books left.)

The theory of change behind sending the books to winners of olympiads is that people with high potential read HPMOR and share it with friends, get some EA-adjacent mindset and values from it, and then get introduced to EA in emails about 80k Hours (which is being translated into Russian[2]) and other EA and cause-specific content, and start participating in the EA community and get more into specific cause areas that interest them. The anecdotal evidence is that most of the Russian EAs, many of whom now work full-time at EA orgs or as independent researchers, got into EA after reading HPMOR and then the LW sequences.

We can't give the books in exchange for donations to EA nonprofits, as Russian residents can't transfer money outside Russia.
Shipping a copy costs around $5-10 in Russia and $40-100 outside Russia.
There are some other ideas[3], but nothing that lets us spend thousands of books effectively, so we need more.

So, any ideas on how to spend 8-9k of HPMOR copies in Russian?

^

HPMOR has endorsements from a bunch of Russians important to different audiences. That includes some of the most famous science communicators, computer science professors, literature critics, a person training t

...
Is recursive self-alignment possible?
by No77e — 2023-01-03T09:15:21.304Z

Suppose an emulation of Eliezer Yudkowsky, as of January 2023, discovered how to self-modify. It's possible that the returns of capability per unit effort would start increasing, which would make its intelligence FOOM.

The emulation starts with an interest in bringing about utopia rather than extinction in the form of humanity's CEV. In the beginning, the emulation doesn't know how to implement CEV yet, and it doesn't know how humanity's CEV is specified in practice. But as the emulation self-improves, they can make more and more refined guesses, and well before being able to simulate humanity's history (if such a thing is possible), it nails down what humanity's CEV is and how to bring it about.

In this thought experiment, you could say that the emulation is aligned from the beginning, but it's just not sure about some details. On the other hand, it doesn't know its own goal precisely yet.

So, here are some questions:

1. Is it possible to start with some very simple goal kernel that very likely transforms into something that is still aligned during self-improvement? After all, it's plausible that this would happen with an emulation of Eliezer Yudkowsky or other particular humans.

2. Does this make the kernel already aligned? Is this just a definitional thing?

3. Does this mean that increasing capabilities do help with alignment? A system as smart as Eliezer can't specify humanity's CEV, but something smarter might be able to.
On the naturalistic study of the linguistic behavior of artificial intelligence
by Bill Benzon — 2023-01-03T09:06:42.295Z

This is draft material from the introduction of a working paper I’m about to finish: Discursive Competence in ChatGPT, Part 1: Talking with Dragons.

* * * * *

When I started playing with ChatGPT on December 1, 2022, I had no specific intentions. I wanted to poke around, see what I could see, and then...As I said, I had no specific intentions. I certainly did not intend to spend hours interacting with it to produce a Microsoft Word document currently (1.1.22) containing 59,197 words of transcripts – the vast majority from ChatGPT – on 173 pages.

One of the earliest things I did with ChatGPT – not THE first, it was my third session, on December 1, 2022 ¬– was to dialog about Steven Spielberg’s Jaws and the ideas of Rene Girard. I took that and wrote it up for 3 Quarks Daily.[1] Then I had some fun with “Kubla Khan,” quizzed it about trumpets, had a long session about Gojira/Godzilla, and then returned to Spielberg, this time to A.I. Artificial intelligence. By this time I was developing a feel for how ChatGPT responded. Both the Jaws and the A.I. posts are included in this paper.

I became a bit more systematic, looking for things, testing them out. That led to a post with a rather baroque title, Of pumpkins, the Falcon Heavy, and Groucho Marx: High level discourse structure in ChatGPT,[2] which I’ve also included in this paper. In that post I advanced the argument that there are parameters in the language model that govern ligher level discourse structures independently of the specific words and strings that realize them.

The alternation pattern is something like this:

A, B, A, B....

That can be repeated as often as one will. The text in the A sections is always drawn from one body of material while the text in the B sections is drawn from a different body of material. That’s the pattern ChatGPT has learned. Where is it in the net? How’s it encoded.

The frame structure is a bit more complicated:

A (B, C, B, C....) A’

The embedded alternation draws on two bodies of material, any two bodies. The second part of the frame, A’, must complement the first, A.

Again, it’s not a complex structure. But it’s not defined directly over particular words. It’s defined over groups of words, placing the groups, not the individual words, into specified relationships in the discourse string.

I then suggested that the patterns I had identified in Jaws and A.I. where similar, but, if an...
200 COP in MI: Exploring Polysemanticity and Superposition
by Neel Nanda — 2023-01-03T01:52:46.044Z

Important Note: Since writing this, there's been a lot of exciting work on understanding superposition via training sparse autoencoders to take features out of superposition. I recommend reading up on that work, since it substantially changes the landscape of what problems matter here.

This is the fifth post in a sequence called 200 Concrete Open Problems in Mechanistic Interpretability.

Start here

, then read in any order. If you want to learn the basics before you think about open problems, check out

my post on getting started

. Look up jargon in

my Mechanistic Interpretability Explainer

Motivating papers: Toy Models of Superposition, Softmax Linear Units

Background

If you're familiar with polysemanticity and superposition, skip to Motivation or Problems.

Neural networks are very high dimensional objects, in both their parameters and their activations. One of the key challenges in Mechanistic Interpretability is to somehow resolve the curse of dimensionality, and to break them down into lower dimensional objects that can be understood (semi-)independently.

Our current best understanding of models is that, internally, they compute features: specific properties of the input, like "this token is a verb" or "this is a number that describes a group of people" or "this part of the image represents a car wheel". That early in the model there are simpler features, are later used to compute more complex features by being connected up in a circuit (example shown above (source)). Further, our guess is that features correspond to directions in activation space. That is, for any feature that the model represents, there is some vector corresponding to it. And if we dot product the model's activations with that vector, we get out a number representing whether that feature is present.(these are known as decomposable, linear representations)

This is an extremely useful thing to be true about a model! An even more helpful thing to be true would be if neurons correspond to features (ie the output of an activation function like ReLU). Naively, this is natural for the model to do, because a non-linearity like ReLU acts element-wise - each neuron's activation is computed independently (this is an example of a privileged basis). Concretely, if a neuron can represent feature A or feature B, then that neuron will fire differently for feature A and NOT feature B, vs fea...
The need for speed in web frameworks?
by Adam Zerner — 2023-01-03T00:06:15.737Z

Recently I've been spending a lot of time investigating JavaScript web frameworks that use server-side rendering (SSR). I've noticed that there are a lot of features that are ultimately about improving performance.[1]

Combining SSR with follow up client-side navigation instead of doing full page reloads (Next, Remix, SvelteKit, Nuxt)

Nested routes (Remix)

Letting you switch between SSR and SSG (Next, SvelteKit, Nuxt, Astro)

Partial hydration (Astro, Fresh)

Utilizing React server components (Next 13)

Fighting to send fewer bytes of JavaScript to the client

Prefetching

Edge computing (YouTube)

Image and font optimization

IMO, these all introduce complexity though. In which case, the question becomes: is it worth it?

Core features vs add-ons

Here is how I think about this. There are certain things that I see as core features that I'd want out of one of these web frameworks. The rest I see as add-ons. With that model, we can then (eventually) go through each add-on and ask if it's worth including.

Here are the core features:

SSR. I like having the data fetching on the server. For reasons that I am not fully able to explain, it feels cleaner. Maybe it's the separation of concerns, allowing React components to just focus on UI. Maybe it's because I don't have to deal with useEffect or other weird hooks. The performance benefits also seem pretty non-trivial. Network requests are super slow compared to accessing memory or disk, and with SPAs where data fetching happens on the client, the client often has to make many network requests.

React as the V in MVC. I want to use React instead of something like Vue, Svelte or Solid. Or Angular. Or mustache, handlebars or ERB. I know it and like it. I think this is essentially just a personal preference though.

Hydration. This I think is important. Web pages are frequently dynamic and interactive, requiring JavaScript. With something like Rails or Django, your options are Vanilla JS, jQuery or one of mini-frameworks like Stimulus for Rails. I feel pretty confident that React is better than Vanilla JS and jQuery. I don't actually have experience with any of the mini-frameworks so maybe they are actually fine, but I at least prefer to just use React for the interactive stuff.

The rest I see as add-ons. Partial hydration, prefetching, the rest of those bullet points in the first section of this post: those are all add-ons to ...