trentmkelly/LessWrong-43k huggingface dataset:
Internet Literacy Atrophy
by Elizabeth — 2021-12-26T12:30:01.540Z

It’s the holidays, which means it’s also “teach technology to your elderly relatives” season. Most of my elderly relatives are pretty smart, and were technically advanced in their day. Some were engineers or coders back when that was rare. When I was a kid they were often early adopters of tech. Nonetheless, they are now noticeably worse at technology than my friends’ 3 year old. That kid figured out how to take selfie videos on my phone after watching me do it once, and I wasn’t even deliberately demonstrating.

Meanwhile, my aunt (who was the first girl in her high school to be allowed into technical classes) got confused when attempting to use an HBOMax account I’d mostly already configured for her (I think she got confused by the new profile taste poll but I wasn’t there so I’ll never be sure). She pays a huge fee to use Go Go Grandparent instead of getting a smartphone and using Uber directly. I got excited when an uncle seemed to understand YouTube, until it was revealed that he didn’t know about channels and viewed the subscribe button as a probable trap. And of course, there was my time teaching my PhD statistician father how to use Google Sheets, which required learning a bunch of prerequisite skills he’d never needed before and I wouldn’t have had the patience to teach if it hadn’t benefited me directly.

[A friend at a party claimed Apple did a poll on this and found the subscribe button to be a common area of confusion for boomers, to the point they were thinking of changing the “subscribe” button to “follow”. And honestly, given how coy substack is around what exactly I’m subscribing to and how much it costs, this isn’t unreasonable.]

The problem isn’t that my relatives were never competent with technology, because some of them very much were at one point. I don’t think it’s a general loss of intelligence either, because they’re still very smart in other ways. Also they all seem to have kept up with shopping websites just fine. But actions I view as atomic clearly aren’t for them.

Meanwhile, I’m aging out of being the cool young demographic marketers crave. New apps appeal to me less and less often. Sometimes something does look fun, like video editing, but the learning curve is so steep and I don’t need to make an Eye of The Tiger style training montage of my friends’ baby learning to buckle his car seat that badly, so I pass it by and focus on the...
Merry Christmas
by lsusr — 2021-12-26T07:03:30.261Z

This year I was robbed at gunpoint for the first time. Most importantly, I wasn't shot. I don't think my assailants ran a careful calculation about whether it was in their interest to shoot me or know. I think they just preferred not to execute me unless they had to. True psychopaths are rare. I am grateful for the compassion those two desperate young men extended to me.

A reader of this blog sent me a lovely email complimenting me for being "adversarial". It made my day. When people write me nice comments and send me nice emails it's rare for me to read them fewer than three times.

There's a YouTube channel called Binkov's Battlegrounds which I sometimes watch when I'm doing research for this blog. The creator ends every video the same way.

[R]emember, Binkov may talk about hypothetical wars, but only real peace can bring us all together.

The same goes for this blog. I may talk about hypothetical wars…and real wars…and war crimes…and so on. But only real compassion can bring us all together.

This Christmas, I'd like to express love and gratitude.

For the readers who write nice comments and send me nice emails. I love you all.

For the readers who write mean comments and send me mean emails. I love you too.

For the readers who upvote and downvote my posts.

For the readers who have better things to do than create a Less Wrong user account.

For Daniel Kokotajlo and Steven Byrnes with whom I have had illuminative conversations.

For the Lightcone team.

For the authors who write books and articles I enjoy. There are too many of you to name.

For every person motivated to do good, regardless of its utility.

For the mad scientists doing gain-of-function research.

For my friends.

For my enemies.

For the factory workers and salespeople of Shenzhen who make the stuff I use and whose existence reminds me how cushy my life is.

For the proletariats of Vietnam who remind me how cushy life in China is.

For peasants of history who remind me how cushy life in modern Vietnam is.

For billions of strangers I will never meet.

For whatever comes after humanity, if it be sentient.

For incomprehensible horrors of incredible power.

For kittens.

For everyone who has ever harmed me. I forgive you.

For everyone who ever will harm me. I forgive you too.

For the two desperate young men who took my wallet. I hope you bought something nice with the $35 cash in it. I'm glad we could coordinat

...
Why don't governments seem to mind that companies are explicitly trying to make AGIs?
by ozziegooen — 2021-12-26T01:58:20.467Z
Link: https://forum.effectivealtruism.org/posts/wdk3LCg6iFxknCYG4/why-don-t-governments-seem-to-mind-that-companies-are

Epistemic Status: Quickly written, uncertain. I'm fairly sure there's very little in terms of the public or government concerned about AGI claims, but I'm sure there's a lot I'm missing. I'm not at all an expert on government or policy and AI.

This was originally posted to Facebook here, where it had some discussion. Many thanks to Rob Bensinger, Lady Jade Beacham, and others who engaged in the discussion there.

Multiple tech companies now are openly claiming to be working on developing AGI (Artificial General Intelligence).

As written in a lot of work on AGI (See Superintelligence, as an example), if any firm does establish sufficient dominance in AGI, they might have some really powerful capabilities.

Write bots that could convince (some) people to do almost anything
Hack into government weapons systems
Dominate vital parts of the economy
Find ways to interrupt other efforts to make AGI

And yet, from what I can tell, almost no one seems to really mind? Governments, in particular, seem really chill with it. Companies working on AGI get treated similarly to other exciting AI companies.

If some company were to make a claim like,

"We're building advanced capabilities that can hack and modify any computer on the planet"

or,

"We're building a private nuclear arsenal",

I'd expect that to draw attention.

But with AGI, crickets.

I assume that governments dismiss corporate claims of AGI development as overconfident marketing-speak or something.

You might think,

"But concerns about AGI are really remote and niche. State actors wouldn't have come across them."

That argument probably applied 10 years ago. But at this point, the conversation has spread a whole lot. Superintelligence was released in 2014 and was an NYT bestseller. There are hundreds of books out now about concerns about increasing AI capabilities. Elon Musk and Bill Gates both talked about it publicly. This should be one of the easiest social issues at this point for someone technically savvy to find.

The risks and dangers (of a large power-grab, not of alignment failures, though those too) are really straightforward and have been public for a long time.

Responses

In the comments to my post, a few points were made, some of which I was roughly expecting. Points include:

Companies saying they are making AGI are ridiculously overconfident
Governments are dramatically incompetent
AGI will roll out gradually and not give

...
My Overview of the AI Alignment Landscape: Threat Models
by Neel Nanda — 2021-12-25T23:07:10.846Z

This is the second post in a sequence mapping out the AI Alignment research landscape. The sequence will likely never be completed, but you can read a draft

here.

Disclaimer: I recently started as an interpretability researcher at Anthropic, but I wrote this post before starting, and it entirely represents my personal views not those of my employer

Intended audience: People who understand why you might think that AI Alignment is important, but want to understand what AI researchers actually do and why.

Pedagogy note: I link to many papers and blog posts to read more about each area. I think technical writing is often harder to digest without a big picture in mind, so where possible I link to Alignment Newsletter summaries for a piece. There are a lot of links, so I recommend reading the summaries for anything interesting, but being selective about which full-length works you read.

Terminology note: There is a lot of disagreement about what “intelligence”, “human-level”, “transformative” or AGI even means. For simplicity, I will use AGI as a catch-all term for ‘the kind of powerful AI that we care about’. If you find this unsatisfyingly vague,

OpenPhil’s definition of Transformative AI

is my favourite precise definition.

Introduction

A common approach when setting research agendas in AI Alignment is to be specific, and focus on a threat model. That is, to extrapolate from current work in AI and our theoretical understanding of what to expect, to come up with specific stories for how AGI could cause an existential catastrophe. And then to identify specific problems in current or future AI systems that make these failure modes more likely to happen, and try to solve them now.

It is obviously really hard to reason about the future in a specific way without being wildly off! But I am pretty excited about approaches like this. I think it's easy for research (or anything, really) to be meandering, undirected and not very useful, especially for vague and ungrounded problems such as AI Alignment, which is essentially trying to fix problems in a technology that doesn’t exist yet. And having a specific story to guide what you do can be a valuable source of direction, even if ultimately you know it will be flawed in many ways.

Nate Soares

makes the case for having a specific but flawed story in general well.

Note that I think there is very much a spectrum between this category ...
What is a probabilistic physical theory?
by Ege Erdil — 2021-12-25T16:30:27.331Z

This is a question I asked on Physics Stack Exchange a while back, and I thought it would be interesting to hear people's thoughts on it here. You can find the original question here.

What do we mean when we say that we have a probabilistic theory of some phenomenon?

Of course, we know from experience that probabilistic theories "work", in the sense that they can (somehow) be used to make predictions about the world, they can be considered to be refuted under appropriate circumstances and they generally appear to be subject to the same kinds of principles that govern other kinds of explanations of the world. The Ising model predicts the ferromagnetic phase transition, scattering amplitude computations of quantum field theories predict the rates of transition between different quantum states, and I can make impressively sharp predictions of the ensemble properties of a long sequence of coin tosses by using results such as the central limit theorem. Regardless, there seem to be a foundational problem at the center of the whole enterprise of probabilistic theorizing - the construction of what is sometimes called "an interpretation of the probability calculus" in the philosophical literature, which to me seems to be an insurmountable problem.

A probabilistic theory comes equipped with an event space and a probability measure attached to it, both of which are fixed by the theory in some manner. However, the probability measure occupies a strictly epiphenomenal position relative to what actually happens. Deterministic theories have the feature that they forbid some class of events from happening - for instance, the second law of thermodynamics forbids the flow of heat from a cold object to a hot object in an isolated system. The probabilistic component in a theory has no such character, even in principle. Even if we observed an event of zero probability, formally this would not be enough to reject the theory; since a set of zero probability measure need not be empty. (This raises the question of, for instance, whether a pure quantum state in some energy eigenstate could ever be measured to be outside of that eigenstate - is this merely an event of probability 0, or is it in fact forbidden?)

The legitimacy of using probabilistic theories then rests on the implicit assumption that events of zero (or sufficiently small) probability are in some sense negligible. However, ...
Belief-conditional things - things that only exist when you believe in them
by Jan — 2021-12-25T10:49:38.203Z
Link: https://universalprior.substack.com/p/belief-conditional-things-things

TL;DR: A little winter story about making beliefs come true. Epistemic status: this is >90% written to amuse. Not 100% though.

🎵 Do you believe... 🎵

It is the season, and I just finished watching the 2003 classic "Elf" with Will Ferrell as "Buddy", a human raised among elves at the North Pole who travels to New York to reunite with his biological father and to save Christmas. The movie got me thinking.

There is an interesting trope in the movie that's used as a story device: the magical powers of Santa are predicated on humans believing in Santa. Thus, Santa falls in the same category as some magical creatures, the value of fiat money and, arguably, god (as explained in the Book of Jezuboad and discussed here). To spread some magical fuzzy winter feelings, I want to share some thoughts about what I call "belief-conditional things". I think I'm on to something here.

Expect me to upgrade my CGD game even further in the new year! This looks a lot like Santa.

🎵 ... in love after love 🎵

What is the relationship between belief and reality? Some like to think in terms of "territory and map": the territory is everything out there. The map is a lossy mental representation of the territory. We know that the map is not the territory; sometimes, we are confused about how things work, sometimes we treat complex systems as black boxes, and some things are not represented in our map.

On the left there’s a map, on the right there’s… Middle earth?

The entire scientific endeavor might be summarized as reducing discrepancies between map and territory. Traditionally, you would update the map according to what you see in the environment.

Broke: Seeing that there is no dragon in your garage and giving up on the belief.

But if your metric is symmetric (and why would you call it a metric otherwise?) then it's equally effective to change the territory.

Woke: Believing in the dragon so hard that reality changes to match your belief.

Indeed, I think this is what motivates a substantial part of moral philosophy (and, for that matter, larping): Instead of accepting the world as it is, we conjure our ideas about how things ought to be into reality. Thus, it's not unheard of that reality is affected by our beliefs. This opens the door for...

Gödel's trick

When thinking about existence and belief, it is natural to think of epistemic logic - the subfield of the philosophy of knowledge that formali...
Tough Choices and Disappointment
by maralorn — 2021-12-24T21:59:00.970Z

I wrote a 4-minute read blog post about the idea that a tough choice is always preceded by a disappointment. I assume this is pretty straight forward and not very surprising for readers used to Bayesian thinking, but I have never seen it spelled out in this way, so I thought I could try it.

Epistemic Status: I think this is so obvious that it’s most certainly true in most situations. Don’t know if it’s helpful, though.

https://blog.maralorn.de/post/tough-choices

Also, I am trying to get better at writing (in English) so I am grateful for feedback.
Converging toward a Million Worlds
by Joe Kwon — 2021-12-24T21:33:52.732Z

Epistemic status: unfleshed thought that’s been brewing in my mind recently, but feels like it might be important. At the very least, I'm hoping to get pointers to writings and conversations that might've already transpired on this topic.

Inspired by recent trends in social media, upcoming tech in VR/AR, and reading some older posts about complexity of human values as well as the distinction between the conscious and unconscious mind.

A lot of technologies deployed today, including social media, have a core incentive to maximize attention. This can be achieved by doing “good” things e.g. showing interesting and relevant content to users. But I feel wary about this trend in increasing capabilities for capturing attention — specifically with regard to the possibility that we might warpspeed into a world in which people are increasingly stripped of agency and farmed according to values that they are never given a chance to sufficiently evaluate.

This tugs at something that is inconspicuous (to most), yet crucial to the human experience: if our subconscious and conscious utilities differ, to what extent is this phenomenon of attention-capture asymmetrically targeting and exploiting subconscious "feel-goods"/values? To clarify, I think the Experience Machine thought experiment is a fair analogy at the extreme. Or more concretely, maybe I'm joyful and having lots of laughs from viewing extremely optimized TikTok content for 16 hours a day. And yet, on days when I feel like I have especially high executive control and awareness, I realize that this is a bad behavior to engage in because I desire certain higher-level goals and a type of fulfillment that is being hindered by being sucked into entertaining content every single day.

In a non-social media context, mobile app video games have a huge userbase nowadays, especially with younger kids. I've played a few, and anecdotally I can say that they're incredibly well optimized to take advantage of subconscious and impulsive behaviors to maximize earnings -- through various in-app purchases, essentially veiled-gambling-for-kids "gacha" mechanisms, etc. In middle school and high school, I was obsessed with a lot of these types of games, and in hindsight I realize that at a time when I hadn't yet considered my values and goals, or experienced and learned more broadly about what life is like, these video games ena...
Understanding the tensor product formulation in Transformer Circuits
by Tom Lieberum — 2021-12-24T18:05:53.697Z

I was trying to understand the tensor product formulation in transformer circuits and I had basically forgotten all I ever knew about tensor products, if I ever knew anything. This very brief post is aimed at me from Wednesday 22nd when I didn't understand why that formulation of attention was true. It basically just gives a bit more background and includes a few more steps. I hope it will be helpful to someone else, too.

Tensor product

For understanding this, it is necessary to understand tensor products. Given two finite-dimensional vector spaces V,W we can construct the tensor product space V⊗W as the span[1] of all matrices v⊗w, where v∈V,w∈W, with the property (v⊗w)ij=viwj [2]. We can equivalently define it as a vector space with basis elements eVi⊗eWj, where we used the basis elements of V and W respectively.

But not only can we define tensor products between vectors but also between linear maps that map from one vector space to the other (i.e. matrices!):

Given two linear maps (matrices) A:V→X,B:W→Y we can define A⊗B:V⊗W→X⊗Y, where each map simply operates on its own vector space, not interacting with the other:

(A⊗B)(v⊗w)=A(v)⊗B(w)

For more information on the tensor product, I recommend this intuitive explanation and the Wikipedia entry.

How does this connect to the attention-only transformer?

In the "attention-only" formulation of the transformer we can write the "residual" of a fixed head as AXWVWO, with the values weight matrix WV, the attention matrix A, the output weight matrix WO, and the current embeddings at each position X

Let E be the embedding dimension, L the total context length and D the dimension of the values, then we have that

X is an L×E matrix,

A is a L×L matrix,

WV is a E×D, and

WO is a D×E matrix

Let's identify the participating vector spaces:

A maps from the "position" space back to the "position" space, which we will call P (and which is isomorphic to RL). Similarly, we have the "embedding" space E≅RE and the "value" space V≅RD.

It might become clear now that we can identify X with an element from P⊗E, i.e. that we can write X=Xij(ePi⊗eEj).

From that lense, we can see that right-multiplying X with WV is equivalent to multiplying with Id⊗WV, which maps an element from P⊗E to an element from P⊗V, by applying WV to the E-part of the tensor [3]:

(Id⊗WV)(X)=(Id⊗WV)∑ijXijePi⊗eEj=∑ijXijePi⊗WV(eEj)=∑ijXijePi⊗∑kWjkeVk=∑ik∑j(XijWjk)ePi⊗eVk=∑ik...
How to select a long-term goal and align my mind towards it?
by Alexander — 2021-12-24T11:40:37.659Z

I have been struggling with goal selection lately. I didn't struggle so much earlier in my life because others clearly defined my goals. We live in a high-dimensional world, as said elsewhere on LessWrong. There are many worthy directions to head towards, and I'm struggling to choose. I find myself doing a little here, a little there, but mostly converging on optimising for career success, mainly because it's a convenient goal.

I've cultivated many of the low hanging fruits (relatively speaking), such as getting a job that pays the bills, saving and investing, sustaining a social life, getting a decent education and maintaining good health. However, going after the less easily accessible fruits requires a focused effort. Imagine the brain is made up of thousands of sub-minds, each with its models and preferences. Achieving big goals requires aligning these sub-minds to work together towards those higher fruits.

What I am getting at is that I have a stag hunt scenario inside my head. If my sub-minds keep going in different directions, their unaligned efforts are not likely to amount to much.

I don't have a motivation problem. I work almost daily, and it's just that the work I do is unfocused and thus unlikely to amount to much. I think I'm lacking a dream. I like the saying, "If you can dream it, then you can achieve it."

Many studies show that setting goals increases performance and productivity. According to Wikipedia:

Difficult specific goals lead to significantly higher performance than easy goals, no goals, or even the setting of an abstract goal such as urging people to do their best.
Holding ability constant, and given that there is goal commitment, the higher the goal the higher the performance.
Variables such as praise, feedback, or the participation of people in decision-making about the goal only influence behaviour to the extent that they lead to the setting of and subsequent commitment to a specific difficult goal.

There are so many worthy goals directions to go:

Existential risk reduction
Helping people who are alive today
Fixing software bugs
Maximising my net worth
Learning new knowledge
Creating new knowledge
Maximising my social status
Maximising my pleasure
Etcetera.

I read "The Precipice" by Toby Ord some time ago, and it was a profoundly impactful book. Theoretically, I'm sold, and existential risk reduction is unambiguously the ultimate and most crucial g...