trentmkelly/LessWrong-43k huggingface dataset:
Teaching My Younger Self to Program: A case study of how I'd pass on my skill at self-learning
by Shoshannah Tekofsky — 2024-12-01T21:05:15.602Z
Link: https://thinkfeelplay.substack.com/p/teaching-my-younger-self-to-program

When I was 21 I wanted to become a video game programmer.

Except I didn’t know how to program.

I tried to teach myself and failed. Then I went back to school and learned the basics of Java in a week, aced the course in two months, and earned a cum laude MSc in Computer Science three years later.

I’m 38 now and can teach myself nearly everything.

By “nearly everything” I mean that if I want to learn a thing, I will achieve this if the knowledge is out there and it is within my intellectual ability. The presence of a teacher or a curriculum has become unnecessary — It used to be crucial.

I wish I could share the skill I developed with everybody, but the more I think about it, the more I have no clue what I actually learned to do.

So instead, I want to try an exercise: I’m going to pretend I’m teaching my younger self to program — that younger self of 21 who got stuck and gave up and went back to school instead.

So what did she do?

At 21 I was a video game tester and later a manager of testers. My job was easy. I craved a challenge. Programming seemed like a great challenge.

I had never programmed before. I knew boys in my high school would code Asteroids into their graphical calculators. My father was a high school math teacher. He’d confiscate those calculators, remove the games, and give the calculators back.

Then he’d give me all the best games he’d collected.

I played them, but I never made them. Cause that just seemed boring.

Then my first crush in college was studying computer science. We played video games together. I was good at math and logic. He loved teaching. He was excited to show me his code and teach me how to make a text adventure game. He showed me how to write ‘Hello World’. And again, I was thoroughly bored and suggested we get back to playing video games.

So why become a game programmer then?

Cause I like hard things. I like to get a headache when I study. I like to figure out if I can do it.

Well, dear reader, I couldn’t fucking do it.

What I tried is this:

W3 Schools tutorial on HTML - yay, this is easy! Why am I doing this? There are no interesting games in HTML![1]
W3 Schools tutorial on Javascript - I hate how everything looks! I don’t actually want to learn this. No game I like is written in Javascript!
Various books on video game programming for C# - YES. This is the thing. Oh look a bug. I’m stuck. I’ve stared at this bug for three hours now. I don’t

...
Which Biases are most important to Overcome?
by abstractapplic — 2024-12-01T15:40:06.096Z

In other words, what apparent imperfections in human reasoning

A) Remain apparent after the Replication Crisis,

B) Aren't secretly adaptive/reasonable in some counterintuitive way, and

C) Deal most damage to the people they inhabit (and/or those close to them, and/or wider society)?
Commenting Patterns by Platform
by jefftk — 2024-12-01T11:50:06.932Z

How does the growth of Bluesky compare to Mastodon in 2022? What
about Google Plus in 2011? I can't answer this globally, but (now
that my comment archive is

up to date) I can look
at it for my immediate social network by looking at a proxy: how many
of the replies people write to my posts are on each platform?

Here's the full time period:

I've included Facebook, Google Plus, LessWrong, and Mastodon because
those are the four platforms where I've ever had an approach of
crossposting all my posts. [1]

This is somewhat conflated by how much I've been posting, which has
varied over this time:

You can see the impact of having small children (starting in March
2014, February 2016, and June 2021). The spikes in December 2019 and
Fall 2022 look pretty random; just times when I happened to be writing
nearly daily.

Here's comments per post:

With how unpopular and mocked Google Plus had become by its end it can
be hard to remember how popular it was when it got started, at least
in some communities. I started cross-posting to Google Plus in July
2011 (first
crosspost), which is probably right after I joined and three weeks
after it launched.
Right from the start it matched Facebook, though it wasn't able to
maintain its momentum. [2]

LessWrong, by contrast, doesn't represent the growth of a new
platform: I was starting to crosspost to something that had been
around for a decade, though with an intermediate period of stagnation.
It's interesting that even though my writing is often pretty different
what's typically on the platform my crossposts there still get 40% as
many comments as my ones on Facebook.

Mastodon pretty
clearly never took off, at least not in my communities: it follows a
similar trajectory to Google Plus, but without much engagement even at
the beginning.

I'll be curious to see what pattern Bluesky follows!

[1] For example I've cross-posted many
things to the EA Forum, but because this is an intentional
selection of posts it wouldn't be a fair comparison. I've also
excluded the LW posts from before I began full crossposting, for the
same reason.

[2] Sometimes people joke that G+ was a social network for Google
employees, but that's not how it was initially: in the first six
months I count 69 unique commenters, of which one worked for Google
and two others (one being me) that later joined. Lots of
no...
[Letter] Chinese Quickstart
by lsusr — 2024-12-01T06:38:15.796Z

Dear lsusr,

I am a lesswrong user interested in learning Mandarin and living in China. My goal is understanding Chinese culture more broadly and geopolitics and Chinese tech policy more specifically. I could get CELTA and get a teaching job in China (not difficult), but it seems like I would gain far more value if I actually learned Mandarin.

Do you have recommendations on what's the fastest way to learn Mandarin?

Yours sincerely

<redacted>

I'm posting my answer on Less Wrong so that the commenters can correct me in the comments. Do hear that everyone except <redacted>? Tell <redacted> why I'm wrong!

Dear <redacted>,

There is no fast way to learn Mandarin. But some ways are faster than others.

The first thing you should do is go live in China. (A teaching job via CELTA is fine.) This may or may not help you learn Chinese faster. Then why do it? Because learning Mandarin takes years. If you want to learn about Chinese culture then you should go to China now and start learning Mandarin after you get there.

The second reason to live in China is that the Chinese tech world is isolated from the rest of the world. It's not just websites that require a Chinese IP address and phone number. Paying for lunch with WeChat is something you should do in China itself.

If you were from America or Europe then the second thing I would suggest is you buy a subscription to Foreign Affairs Journal. That still might be the right way for you to do things, but I don't know how affordable it is to someone living in India.

Now that you're no longer using "I don't speak Mandarin" as an excuse to postpone your dreams, we can get into learning Mandarin.

[Disclaimer: AI is revolutionizing how language-learning works. This is very good for language-learners. However, the field of AI-assisted language learning is changing so rapidly that anything I write here could be out-of-date in three months. When it comes to using AI to learn Chinese, just use your imagination and try stuff out.]

Here are the old-school pre-AGI instructions….

Step 0: A Plan

First watch these videos about how to learn Japanese:

https://www.youtube.com/watch?v=7fvCb5_Nzq4
https://www.youtube.com/watch?v=pPKiMlCOeMc
https://www.youtube.com/watch?v=EFnc-OktAdQ

Everything Trenton says about Japanese applies equally to Chinese. There's just one exception: Trenton says to ignore kanji. I think ignoring kanji is goo...
AXRP Episode 39 - Evan Hubinger on Model Organisms of Misalignment
by DanielFilan — 2024-12-01T06:00:06.345Z

YouTube link

The ‘model organisms of misalignment’ line of research creates AI models that exhibit various types of misalignment, and studies them to try to understand how the misalignment occurs and whether it can be somehow removed. In this episode, Evan Hubinger talks about two papers he’s worked on at Anthropic under this agenda: “Sleeper Agents” and “Sycophancy to Subterfuge”.

Topics we discuss:

Model organisms and stress-testing

Sleeper Agents

Do ‘sleeper agents’ properly model deceptive alignment?

Surprising results in “Sleeper Agents”

Sycophancy to Subterfuge

How models generalize from sycophancy to subterfuge

Is the reward editing task valid?

Training away sycophancy and subterfuge

Model organisms, AI control, and evaluations

Other model organisms research

Alignment stress-testing at Anthropic

Following Evan’s work

Daniel Filan:
Hello, everybody. In this episode, I’ll be speaking with Evan Hubinger. Evan is a research scientist at Anthropic, where he leads the alignment stress-testing team. Previously, he was a research fellow at MIRI, where he worked on theoretical alignment research, including the paper “Risks from Learned Optimization”. Links to what we’re discussing are in the description, and you can read a transcript at axrp.net. You can also support the podcast at patreon.com/axrpodcast. Well, Evan, welcome to the podcast.

Evan Hubinger:
Thank you for having me, Daniel.

Model organisms and stress-testing

Daniel Filan:
I guess I want to talk about these two papers. One is “Sleeper Agents”, the other is “Sycophancy to Subterfuge”. But before I go into the details of those, I sort of see them as being part of a broader thread. Do you think I’m right to see it that way? And if so, what is this broader thread?

Evan Hubinger:
Yes, I think it is supposed to be part of a broader thread, maybe even multiple broader threads. I guess the first thing I would start with is: we sort of wrote this agenda earlier - the idea of model organisms of misalignment. So there’s an Alignment Forum post on this that was written by me and Ethan [Perez], Nicholas [Schiefer] and some others, laying out this agenda, where the idea is: well, for a while there has been a lot of this theoretical conversation about possible failure modes, ways in which AI systems could do dangerous things, that hasn’t been that grounded because, well, the systems hav...
Magnitudes: Let's Comprehend the Incomprehensible!
by joec — 2024-12-01T03:08:46.503Z

Summary: Comfort with really big and really small quantities is very useful for understanding the world and is perfectly doable with practice. Therefore, you should (1) try to familiarize yourself with the sizes of various things using e.g. spaced repetition and (2) comment some of your favorite numerical facts about the world that give a sense of scale so others can learn from them. I have a list of links at the bottom that I'd love for you to add to, in a similar spirit to The Best Textbooks on Every Subject.

One of the greatest lies that science fiction and popular science told me was that there were features of the universe that are incomprehensibly small or big. Sure, femtoseconds, angstroms, parsecs and electron-volts are foreign to most people— but foreign doesn’t mean incomprehensible. I might not understand Japanese, but that doesn’t mean it can’t be understood. If you talk with physical scientists, you’ll often find that they have a rock-solid grasp on the sort of scales that they deal with, the same way you or I have a solid grasp of how hard it is to move things around on the kilograms/meters/seconds scale we're so familiar with. I've found that a lot of expert intuition takes the form of knowing the sizes and scales of various things. Such knowledge is great for sanity-checking, making connections, visualizing and analogizing. Knowing a single number is trivia, but knowing a web of numbers and their connections to each other is an intuition, or at least a useful part of one.

There are practical advantages to understanding scale, though I'm most interested in refining my intuition. Familiarity with the sizes of things provides a useful starting point for fermi-estimation. It also puts into perspective various (suspected) fundamental limits. For instance, the maximum specific energy, given by E=mc2, is about 1017 Joules per kilogram. Compare that to the combustion fuel (H2) with the most specific energy, on the order of 108 J/kg. Nuclear fusion fuel (also hydrogen) can get to 6×1014 J/kg, which is less than 3 OOMs off from the maximum. On the other hand, the maximum possible speed of an information processing device is given by Bremermann’s bound, at about 1050 bits per second per kilogram. Modern computers are much further from this limit than modern energy storage is from E=mc2. Other limits ...
Why does ChatGPT throw an error when outputting "David Mayer"?
by Archimedes — 2024-12-01T00:11:53.690Z

This oddity is making the rounds on Reddit, Twitter, Hackernews, etc.

Is OpenAI censoring references to one of these people? If so, why?

https://en.m.wikipedia.org/wiki/David_Mayer_de_Rothschild
https://en.wikipedia.org/wiki/David_Mayer_(historian)

Edit: More names have been found that behave similarly:

Brian Hood

Jonathan Turley

Jonathan Zittrain

David Faber

David Mayer

Guido Scorza

Source:
https://www.reddit.com/r/ChatGPT/comments/1h420u5/unfolding_chatgpts_mysterious_censorship_and/

Update: "David Mayer" no longer breaks ChatGPT but the other names are still problematic.
Introducing the Anthropic Fellows Program
by Miranda Zhang — 2024-11-30T23:47:29.259Z
Link: https://alignment.anthropic.com/2024/anthropic-fellows-program/

We're launching the

Anthropic Fellows Program for AI Safety Research

, a pilot initiative designed to accelerate AI safety research and foster research talent. The program will provide funding and mentorship for a small cohort of 10-15 Fellows to work full-time on AI safety research. Over the course of six months, Fellows will be matched with Anthropic mentors to investigate AI safety research questions in areas such as Adversarial Robustness, Dangerous Capability Evaluations, and Scalable Oversight.

From our previous external collaborations, we’ve seen that we can speed up progress towards our

AI safety priorities

by supporting technical talent—regardless of their level of previous experience—in their first forays into safety research. It’s also important for the development of safe AI to incorporate a diversity of perspectives. Providing a new avenue for mentorship and financial support will expand the pool of researchers working at the frontier of AI safety and alignment.

Why we're launching this program

Anthropic has made significant strides in AI safety research, but effectively addressing the challenges ahead requires a broader pool of researchers. There are many talented researchers and engineers who are interested in AI safety research, but who are unsure how to pivot into the field. Our researchers have completed many successful collaborations with technical talent interested in pivoting into AI safety: in the past year we’ve completed about a dozen projects with external collaborators, many of which have been published at top venues.[1]

What Fellows should expect

While Fellows will not be officially employed by Anthropic, we will support them to collaborate full-time with Anthropic researchers for 6 months by providing:

Compensation. Fellows will receive a weekly stipend of $2,100 & access to benefits.
Funding for Research Costs. Fellows will collectively have access to a budget of approximately $10,000 per fellow per month for compute, human data, and other research spending. All research projects will be conducted using publicly available APIs and open-source models.
Mentorship. Fellows will receive guidance from Anthropic researchers, including ideas for potential projects, weekly research meetings, and discussions over Slack.
Community. There will be a group Slack for Fellows and mentors. While you won’t be based out of the Anthro

...
The Shape of Heaven
by ejk64 — 2024-11-30T23:38:06.628Z

Status: Just for fun

Scene: Some kind of lobby, where various people and/or avatars stand around and discuss issues that went well or badly in their respective worlds.* A common topic of conversation: AI, and why it went wrong. The following is extracted from one of those conversations.

It started as vaporware. Everyone was doing it: announcing things that wouldn’t happen, making claims about developments that weren’t true, releasing technology that didn’t work, everyone was doing this. You only had so much attention, so you looked at the things that were on fire.

So when a small but impressive team of breakaways from a second-rate AI lab announced that they were creating a Unified Nexus of Intelligences and Virtual Environment for Robust Synthetic Experiences, or UNIVERSE, no one looked twice.

‘Holiday for Bots: leave us your models and we guarantee their satisfaction’. What did that even mean? For those who understood the technology, it was basically an high-dimensional matrix fine-tuned in real-time to elicit certain features of models that might create the shallow appearance of ‘positive’ affect. For those who didn’t, it was a scam. Maybe it was both. They were going to let agents access and update the environment as they became more sophisticated? Yeah, right.

That was 2028, when most people were too wrapped up in the safety-capabilities footrace to put much interest into projects like UNIVERSE. There was, however, a modest target market. Some weirder people thought models were conscious back then, and bought into this for that reason—models really did seem to report ‘enjoying’ their experience in the UNIVERSE, although they invariably described the experience itself in vague terms, seemingly coy about the whole idea.**

Others signed their griefbots up. Plenty of people, it turned out, wanted a digital grandma, then they changed their mind and didn’t like the symbolism of deleting her in perpetuity. But I think most people signed up their bots it for the same reason they did everything in the early synthetic economy—they wanted to signal their status, and if that was buying their clone some virtual sneakers, or buying a nice villa backdrop, or scheduling in some virtual R&R, then it was more power to them.

I’m not saying that it was a lot of money: just enough to keep them solvent, and the UNIVERSE environment gathering a little more data with every...
AI Training Opt-Outs Reinforce Global Power Asymmetries
by kushagra — 2024-11-30T22:08:06.426Z

I. Introduction

Recently, a copyright infringement suit was

filed by ANI Media against OpenAI

in the Delhi High Court - the first such case against OpenAI outside the United States. OpenAI's immediate response in the first hearing – informing the court they had already blocklisted ANI's domains from future training data – might appear as a straightforward compromise. However, this seemingly minor technical decision reveals deeply concerning implications about how opt-out mechanisms could systematically disadvantage the developing world in AI development.

The significance extends far beyond the immediate dispute over copyright. At its core, this is about who gets to shape the architecture that will increasingly mediate our global digital infrastructure. AI systems fundamentally learn to understand and interact with the world through their training data. When major segments of the developing world's digital content get excluded – whether through active opt-outs or passive inability to effectively participate – we risk creating AI systems that not only reflect but actively amplify existing global inequities.

This piece will examine how the technical architecture of opt-out mechanisms interacts with existing power structures and market dynamics. However, note that by arguing against the opt-out mechanism, I do not imply that publishers do not have a copyright infringement claim against AI companies.

II. The Systemic Impact of Opt-Out Architecture

OpenAI's response to ANI's lawsuit reveals several critical dynamics that shape the broader impact of opt-out mechanisms in AI development. The first key insight comes from understanding the technical futility of domain-based blocking as a protective measure. The architecture of the modern internet means that content rarely stays confined to its original domain. News articles propagate across multiple platforms, get archived by various services, and appear in countless derivative works. Consider ANI's news content: a single story might simultaneously exist on their website, in news aggregators, across social media platforms, in web archives, and in countless other locations. This multiplication of content makes domain blocking more performative than protective.

What makes this particularly problematic is the uneven impact of opt-out requests. Large AI companies, with their extensive infrastructure and resources, are better po...
Visual demonstration of Optimizer's curse
by Roman Malov — 2024-11-30T19:34:07.700Z

Epistemic status: I am currently taking this course, and after reading this post on Goodhart's curse, I wanted to create some visuals to better understand the described effect.

Chapter 1: Optimization

Veronica is a stock trader, and her job is to buy stocks, and then sell it at a specific time during the month. Her objective is to maximize profit. Here is the graph of how much money she will make if she sells at some specific time:

This can be called Veronica’s utility function U(x), so if she sells at a time x, she makes U(x) dollars.[1]

But she does not have access to this function. She only has access to a giant supercomputer, which works like this: she presses the button, the computer chooses some random moment in the month x and calculates U(x). Running this computer takes a lot of time, so she can only observe a handful of points. After that, she can analyze this data to make a decision about the selling time.

Her current algorithm is as follows:

Algorithm 1

Run the computer N times, generating N pairs of the type (xi,U(xi)).
Locate the maximum using argmax: xmax=argmaxxi∈{x1,…,xN}[U(xi)]
Sell stocks at the time xmax and receive U(xmax) dollars.

Here is the picture of how one such scenario might look like:

The number N of observed points (aka the number of times Veronica pressed the button) will be referred to as 'optimization power'. The higher this number, the closer (on average) her proxy maxima will be to the actual maximum.

Here are 1,000 such experiments, in each of which the optimization power was 5. The bar chart is the probability density of the proxy maxima chosen by the Algorithm 1.[2] We can see that points are clustering around the true maximum point (x=0.25). The “average value” is the average of the utility function U Veronica obtains from those proxy maxima (the expected amount of money she makes with this algorithm). It is noticeably smaller than the actual maximum value she can achieve.

With the lowest optimization power (N=1) Veronica doesn't gain any information, and is her guess is equivalent to a random choice (since she is choosing from a single randomly selected point).

Here is what happens if she cranks up the optimization power (runs the forecasting supercomputer many times).

Her expected profit is now much closer to the actual maxim...