trentmkelly/LessWrong-43k huggingface dataset:
Secular Solstice for children
by juliawise — 2022-12-22T14:33:46.168Z

Sometimes the Boston Secular Solstice has been hosted at my (Julia's) house because my husband runs the music and physical setup. While the adults are doing a long solstice downstairs, I’ve sometimes run a version for our kids upstairs. This year I did one heavily based on a text Dave Denkenberger wrote.

Dave’s text:

This Effective Altruism/Secular Solstice was adapted from Seattle and New York City events. The song (modified to make it easier to understand for kids) with the hand motions was a big hit with my 6 and 8 year olds.

Family EA Solstice

The solstice is about going from light to dark and back to light. So we will be talking about the good things we have accomplished, the problems we still need to work on, and how there is hope for the future.

We do this because we value community, and that if we want a brighter future, we’ll have to build it ourselves using empathy and reason. We hope to create community and share some of that vision with you tonight.

First, that simply by being here tonight - together, safe, warm, well-fed and at-peace - a million wishes from people in the past or in poor countries - are answered. Almost everyone in the past was a farmer barely surviving: no toilets, no school, not enough food, sickness, didn’t get to choose who to marry, didn’t get to vote. Tonight, we are living an impossible dream, and to every ancestor who lived and struggled and sacrificed for that dream, I say: Thank you.

Second, that our great-grandchildren, should we be so lucky, will tell stories about us. I want our great-grandchildren to tell stories of how we stood up to disease, animals being hurt, and the many ways the world could end.

Song: Six Days of the End of the World (adapted from Ray Arnold’s

X Days of X Risk

)

On the first day of the end of the world I could see:
A very bad disease

On the second day of the end of the world I could see:
Nuclear War!
A very bad disease

On the third day of the end of the world I could see:
Supervolcano
Nuclear War!
A very bad disease

On the fourth day of the end of the world I could see:
Asteroid or comet
Supervolcano
Nuclear War!
A very bad disease

On the fifth day of the end of the world I could see:
Robots taking over
Asteroid or comet
Supervolcano
Nuclear War!
A very bad disease

On the sixth day of the end of the world I could see:
Global warming
Robots taking over
Asteroid or comet
Supervolcano
Nuclear War!
A very bad disease...
Mental acceptance and reflection
by remember — 2022-12-22T14:32:15.589Z

Naive investigation of the world often works, especially when we have direct access, fast feedback loops, and can try many times. Sports, games, and even parts of science often have properties. When we investigate, say, microorganisms, we can directly experiment on what we are interested in and can rapidly iterate. On the other hand, broadly investigating ourselves can be very difficult. Biases, trauma, and all of the messy human stuff can interfere with directly accessing our motivations or reasons for decision-making. When we try to reflect on our actions, it is easy to become absorbed in our emotions and then unable to access the underlying reasons.

For example, I often have trouble initiating communication with people I don’t know. This hasn’t been crippling, but has prevented me from optimizing at times. Reflecting on this can make me feel guilty about how this has interfered with decent decision-making on my part. Reflecting is miserable because it brings up those feelings of guilt, and those very same feelings make it hard to consider what I should do to ameliorate these anxieties going forward or fix any mistakes. Yet, dealing with trauma or biases in how we view ourselves is also very difficult. It is hard not to feel guilt!

A potential way of bypassing this is by shifting our perspective of ourselves to that of a somewhat easier problem: science. We are all part of physics. We are not special. If we can see ourselves as a physical process, we can apply our epistemology to ourselves to better understand and shape the actions we take. Accepting that we are part of reality, just as much as a falling rock, we can investigate ourselves like any other object in science. By applying the scientific method to ourselves, we can observe phenomena and extract regularities in our behavior. By simply looking at causal relationships, we can step back and make ourselves more detached, but detached in a way that allows us to still consider our actions.

So instead of wrestling with these feelings of guilt, I can examine the causes of my anxieties. I’m fine talking with people once we start talking, so this anxiety is just isolated to the step of first contact. Once I isolate the cause, I can devise strategies of slowly acclimating myself, starting with, say, interacting with new people anonymously online. This approach of viewing myself as just another objec...
Against Diversification
by Jack Malde — 2022-12-22T13:29:38.765Z
Link: https://ethicaleconomist.substack.com/p/against-diversification

(Cross-posted from my substack The Ethical Economist: a blog covering Economics, Ethics and Effective Altruism.)

Don’t give to just one charity, there are so many good charities to give to! Also, what if the charity turns out to be ineffective? Then you just wasted all your money and did no good. Don’t worry, there’s a simple solution. Just give to multiple charities and spread risk!

Thinking along these lines is natural. Whether it’s risk aversion, or just an inherent desire to support multiple charities or causes, most of us diversify our philanthropic giving. If your goal is to do the most good however, you should fight this urge with all you’ve got.

Diversification does make some sense, some of the time. If you’re going to fill a charity’s budget with your giving then any further giving should probably go elsewhere.

But most of us are small donors. Our giving usually won't fill a budget, or hit diminishing returns. It certainly won’t hit diminishing returns at the level of an entire cause area, unless perhaps you’re a billionaire philanthropist, in which case well done you.

When you’re deciding where to give you likely have some idea of what the best option is. Maybe you want to help animals and are quite uncertain about how best to do so, but you lean towards thinking that giving to The Humane League (THL) to support their corporate campaigns is slightly better on the margin than giving to Faunalytics to support their research, even though you think there’s a possibility either option is ineffective. In this case, you should give your full philanthropic budget to THL. Fight that urge to give to both charities to cover your back if you make the wrong choice.

Giving to both charities reduces the risk of you doing no good. But, because you subjectively think that THL is slightly better than Faunalytics, it also reduces the amount of good you will actually do in expectation. If you think THL is the best, then why give to anything else? Giving to both means trading away expected good done to get more certainty that you yourself will have done some good. It’s putting your own satisfaction ahead of the expected good of the world. Don’t be that person.

At this point you might push back and say that I haven’t convincingly shown that there’s anything wrong with being risk averse in this way. That is, risk averse with respect to the amount of good a particular individual...
Notes on Meta's Diplomacy-Playing AI
by Erich_Grunewald — 2022-12-22T11:34:27.384Z
Link: https://www.erichgrunewald.com/posts/notes-on-metas-diplomacy-playing-ai/

Disclaimer: I'm not an expert at machine learning, AI safety or Diplomacy, so there may be errors here, though hopefully no major ones. For previous discussion of CICERO on here, see the comments in this post, this rundown and this commentary.

Summary

CICERO is a new AI developed by Meta AI that achieves good performance at the board game Diplomacy. Diplomacy involves tactical and strategic reasoning as well as natural language communication: players must negotiate, cooperate and occasionally deceive in order to win.

CICERO comprises (1) a strategic model deciding which moves to make on the board and (2) a dialogue model communicating with the other players.

CICERO is honest in the sense that the dialogue model, when it communicates, always tries to communicate the strategy model's actual intent; however, it can omit information and change its mind in the middle of a conversation, meaning it can behave deceptively or treacherously.

Some who are concerned with risks from advanced AI think the CICERO research project is unusually bad or risky.

It has at least three potentially-concerning aspects:

It may present an advancement in AIs' strategic and/or tactical capabilities.

It may present an advancement in AIs' deception and/or persuasion capabilities.

It may be illustrative of cultural issues in AI labs like Meta's.

My low-confidence take is that (1) and (2) are false because CICERO doesn't seem to contain any new insights that markedly advance either of these areas of study. Those capabilities are mostly the product of using reinforcement learning to master a game where tactics, strategy, deception and persuasion are useful, and I think there's nothing surprising or technologically novel about this.

I think, with low confidence, that (3) may be true, but perhaps no more true than of any other AI project of that scale.

Neural networks using reinforcement learning are always (?) trained in simulated worlds. Chess presents a very simple world; Diplomacy, with its negotiation phase, is a substantially more complex world. Scaling up AIs to transformative and/or general heights using the reinforcement learning paradigm may require more complex and/or detailed simulations.

Simulation could be a bottleneck in creating AGI because (1) an accurate enough simulation may already give you the answers you want, (2) an accurate and/or complex enough simulation ma

...
Take 13: RLHF bad, conditioning good.
by Charlie Steiner — 2022-12-22T10:44:06.359Z

As a writing exercise, I'm writing an AI Alignment Hot Take Advent Calendar - one new hot take, written every day some days for 25 days. I have now procrastinated enough that I probably have enough hot takes.

Edit - I should have cited Buck's recent post somewhere.

Hyperbolic title, sorry. But seriously, conditioning is better than RLHF for current language models. For agents navigating the real world, both have issues and it's not clear-cut where progress will come from.

By "conditioning", I mean the decision transformer trick to do conditional inference: get human ratings of sequences of tokens, and then make a dataset where you append the ratings to the front of the sequences. A model trained on this dataset for next-token prediction will have to learn the distribution of text conditional on the rating - so if you prompt it with a high rating and then the start of an answer, it will try to continue the answer in a way humans would rate highly.

This can be very similar to RLHF - especially if you augment the training data by building a model of human ratings, and train a model to do conditional inference by finetuning a model trained normally. But in the right perspective, the resulting AIs are trying to do quite different things.

RLHF is sorta training the AI to be an agent. Not an agent that navigates the real world, but an agent that navigates the state-space of text. It learns to prefer certain trajectories of the text, and takes actions (outputs words) to steer the text onto favored trajectories. Conditioning, on the other hand, is trying to faithfully learn the distribution of possible human responses - it's getting trained to be a simulator that can predict many different sorts of agents.

The difference is stark in their reactions to variance. RLHF wants to eliminate variance that might make a material difference in the trajectory (when the KL penalty is small relative to the Bayesian-updating KL penalty), while conditioning on rating still tries to produce something that looks like the training distribution.

This makes conditioning way better whenever you care about the diversity of options produced by a language model - e.g. if you're trying to get the AI to generate something specific yet hard to specify, and you want to be able to sift through several continuations. Or if you're building a product that works like souped-up autocorrect, and want to auto...
Applied Linear Algebra Lecture Series
by johnswentworth — 2022-12-22T06:57:26.643Z

Over the past couple months, I gave weekly lectures on applied linear algebra. The lectures cover a grab-bag of topics which I've needed to know for my own work, but which typically either aren't covered in courses or are covered only briefly in advanced courses which use them (like e.g. quantum). The series is now complete, and recordings of all the lectures are available here.

Be warned: all of the lectures were given with zero review and minimal prep. There are errors. There are poor explanations and too few examples. There are places where I only vaguely gesture at an idea and then say to google it if and when you need it. The flip side is that you will see only things I know off the top of my head - and therefore things which I've found useful enough often enough to remember.

Outline of Topics

Lecture 1

Prototypical use cases of linear algebra

First-order approximation of systems of equations for solving or stability analysis
Second-order approximation of a scalar function in many dimensions for optimization or characterizing of peak/bowl shape
First-order approximation of a dynamical system near a steady state
Principal components of a covariance matrix

Lecture 2

Working with efficient representations of large matrices

Tricks for jacobian and hessian matrices
Prototypical API for implicit matrix representations: scipy's LinearOperator

Lecture 3

Suppose we look at a matrix (e.g. using pyplot.matshow()). What patterns are we most likely to see, and what can we do with them?

Recognizing sparse & low-rank structure
Interpreting sparse & low-rank structure
Leveraging sparse & low-rank structure

Lecture 4

Matrix calculus, with a focus on stability of eigendecomposition

Basics: tensor notation
Differentiating eigendecomposition
Instability of eigenvectors of (approximately) repeated eigenvalues

Lecture 5

Leveraging symmetry

Suppose my system is invariant under some permutation (e.g. a PDE with wraparound boundary, or exchangeable variables in a covariance matrix). How can I leverage that to more efficiently find an eigendecomposition (and invert the matrix etc)?
What Fourier transforms have to do with symmetry, and how to compute them quickly

How to represent rotations/orthogonal matrices

Lecture 6

Wedge products: those "dx dy dz" things in integrals

How to do coordinate transformations with things like "dx dy", even when embedded in a higher-dimensional space

Map between funct

...
Naive Set Theory, Halmos
by David Udell — 2022-12-22T02:34:38.509Z

Epistemic status: Unorganized thoughts of mine generated after reading this book -- not a concerted attempt to convey any single particular mathematical insight.

Set theory is the mathematical study of collections. These collections -- sets -- can be either the collection of nothing (the empty set), or collections... of other collections. Countries, for example, are sets of people, making the United Nations a set of sets of people.

Somewhat surprisingly, just about all the rest of mathematics can be framed as the study of sets (of sets of sets of sets... ). Knowing some set theory lets you translate the myriad objects and claims that you encounter elsewhere in the various areas of math into more set theory, putting math on one common intellectual foundation.

Contents and Notes

1. The Axiom of Extension

2. The Axiom of Specification

All the basic principles of set theory, except only the axiom of extension, are designed to make new sets out of old ones (p. 4).

Reading this book, I picked up the idea that sets with arbitrary properties aren't simply assumed to exist in math. As Halmos puts it,

It is impossible, especially in mathematics, to get something for nothing. To specify a set, it is not enough to pronounce some magic words (which may form a sentence such as "x∉x"); it is necessary also to have at hand a set to whose elements the magic words apply (p. 7).

Another related "correctly orienting towards mathematics" maxim along these lines is: "assume nothing, prove nothing." You don't do math without any premises, because nothing interesting follows from no premises at all. Nor do you freely assume arbitrary premises and see what follows deductively from them. Sets in set theory are not generally assumed to exist -- with the sole exception of the axiom of infinity, our existential starting point. Sets in set theory are instead constructed in steps, from the axiom of infinity, using the existential conditionals of the remaining axioms of set theory.

3. Unordered Pairs

4. Unions and Intersections

5. Complements and Powers

6. Ordered Pairs

The ordered pair of a and b, with the first coordinate a and second coordinate b, is the set (a,b) defined by

(a,b)={{a},{a,b}}

(p. 23).

Sets do not natively track the ordering of their elements: {x,y,z}={z,y,x}. We can introduce the notion of ordering of elements, though, in the above fashion.

Yo...
Not Getting Hacked
by jefftk — 2022-12-21T21:40:05.254Z

A lot of people get hacked or phished, and being completely secure is
somewhere between difficult and impossible, especially if someone
trying hard to hack you in particular. But there are several simple
things you can do that decrease your risk a lot:

Use a password manager that fills in fields automatically.

Configure 2FA with security keys.

Read "sign in with provider" (oAuth) screens carefully.

Encrypt your laptop.

Be cautious with inbound communication.

Why these things? You can make computers do stuff random other people
normally can't: transfer your money, post to social media as you, read
your private documents, control your employer's systems, etc. If an
attacker can trick the right systems into thinking the are you, they
can take your money, exploit your secrets, or use you to get to
others.

A strategy of putting in a lot of mental effort to always make the
right security decision in the moment doesn't work well for humans: we
make mistakes, especially when we're doing routine things or are
distracted, tired, hungry, etc. Instead:

Use tools so you can't mess up even if you're not paying
attention.

Understand the remaining cases where you do need to take
special care.

Password managers, laptop encryption, and security keys are
examples of the former, while being careful with 2FA codes, oAuth
screens, and inbound communication are examples of the latter.

In the most common attacks someone gets a copy of what the computer
checks to make sure you are you (your "credentials"). For example, if
your email is protected only with a password then if they can get your
password they can log into your email.

The most common way for someone to learn your password is using the
same one on multiple sites. Say you use "hunter2" everywhere and you
create an account on "Joe's Shoes and Wallpaper" to buy sneakers.
They later get hacked, their passwords get cracked, and now the
attacker knows you used "hunter2". They try this on your email
account, learning your address from Joe's records, and it works. This
is an especially worrying scenario because you probably have a lot of
other accounts that will send password resets to your email, and now
the attacker can control those as well.

Remembering passwords for hundreds of sites isn't practical, so you
want something that lets you generate a random p...
Metaphor.systems
by the gears to ascension — 2022-12-21T21:31:17.373Z
Link: https://metaphor.systems/

https://metaphor.systems has been absolutely incredible for finding new things on the internet. It hands-down beats every other site I've tried, including some of the ones you'd think of when I say something like that. I'm going to put some examples in the comments over the next day or so asynchronously, please add your own as well.

please vote up or down the examples you find most useful, and provide an example or two of your own if you come up with anything particularly good. there's also a discord server where people can share prompts, though those prompts are usually not as effective as mine, and I haven't been sharing mine there much.

You can also use metaphor to do a similarity search for any url. The results can sometimes be a bit samey, but often it's good for finding more services like a given service.

my single favorite site I've found on metaphor so far is microsolidarity.cc
How much is DQC (Dynamic Quantum Clustering) currently looked into in AI Capabilities Research? 
by macmillan — 2022-12-21T20:46:55.448Z

Dynamic Quantum Clustering is a method for analyzing data quickly and with a smaller amount of compute, but producing unexpected results. (https://quantuminsights.io/wp-content/uploads/2019/03/Dyanamic-Quantum-Clustering.pdf)
Freeman Dyson mentioned this as one of the more promising areas of AI Research in a talk he gave at Dublin University.
Think wider about the root causes of progress
by jasoncrawford — 2022-12-21T20:05:46.986Z
Link: https://rootsofprogress.org/think-wider

Too much discussion of the Industrial Revolution is myopic, focused narrowly on a few highlights such as steam and coal. The IR was part of broader trends that are wider in scope and longer in time than its traditional definition encompasses. To understand anything, it is crucial to get the correct scope for the phenomenon in question.

Here are some ways in which we have to widen our focus in order to see the big picture.

Wider than coal and steam

Some explanations of the IR focus on steam engines, and especially on the coal that fueled them. Economist Robert Allen, for instance, has one of the best-researched and most convincing arguments for cheap coal as a requirement for the development of steam power.

A weak version of this claim, such as “coal was a crucial factor in the IR,” is certainly true. But sometimes a much stronger claim is made, to the effect that the IR couldn’t have happened without abundant, accessible coal, and that this is the main factor explaining why it happened in Europe and especially in Britain. Or more broadly, that all of material progress is driven by fossil fuels (with the implication that once we are, sooner or later, forced to transition away from fossil fuels—whether by geology, economics, or politics—growth will inevitably slow).

But much of the early IR wasn’t dependent on steam power:

Textile machinery, such as Arkwright’s spinning machines, were originally powered by water.
Henry Maudslay’s earliest machine tools were made to manufacture locks; another key application of machine tools was guns with interchangeable parts. This was not motivated by steam power.
Improvements in factory organization, such as the arrangement of Wedgwood’s pottery manufacturing operation, were management techniques, independent of power sources.
The reaper was pulled by horses—even after the development of steam power, because steam tractors were too heavy for use in fields. Even stationary agricultural machines such as for threshing or winnowing were often muscle-powered.

You could argue that all of these inventions would have reached a plateau and would not have had as much economic impact without eventually being hooked up to steam or gas engines. But the fact remains that they were initially powered by water or muscle, and they were economically useful in those first incarnations, often achieving 10x or more gains in productivity. They didn’t need c...