trentmkelly/LessWrong-43k huggingface dataset:
my theory of the industrial revolution
by bhauth — 2024-02-28T21:07:55.274Z
Link: https://www.bhauth.com/blog/culture/industrial%20revolution.html

Why did the Industrial Revolution happen when it did? Why didn't it happen earlier, or in China or India? What were the key factors that weren't present elsewhere?

I have a theory about that which I haven't seen before, so I thought I'd post it.

steam power

One popular conception of the Industrial Revolution is that steam engines were invented, and then an increase in available power led to economic growth.

This doesn't make sense, because water power and horses were much more significant than steam power until well after technological development and economic growth became fast. (Also, steam engine designs wouldn't have caused economic growth in Ancient Rome because manufacturing them wasn't practical at the time.) Steam power being the key factor is now considered a discredited view by most historians.

agriculture

Agricultural productivity in Europe improved significantly before the Industrial Revolution, and rapidly during it. Improvements included:

plant selective breeding

4-field crop rotation

heavy ploughs

horse breeding

threshing machines

roller mills

This reduced the number of farmers needed, potentially allowing more people to do research, or other kinds of work with more available technological improvement.

More people doing research can certainly make it go faster, but there were already intellectuals and a leisure class before the Industrial Revolution. The increase in tech development was disproportionate to the non-farmer population. The UK population also increased, but it was still a relatively small fraction of the world population, and most of its population growth happened after 1800.

canals

Britain made a large canal system that enabled funneling food to London. Combined with agricultural progress, this enables a large city, and London was the largest city in the world starting in around 1830.

But by 1830, the Industrial Revolution was already well underway. There were already large cities such as Beijing well before that point, so "a large city" was not the only key factor.

my theory

My view is that the Industrial Revolution happened due to the combination of:

a large city

European culture

printing presses with movable type

a positional number system with zero

English spelling reform

culture

If you look at the inventions of ancient China and ancient Mesopotamia, I think there was less emphasis on practicality and usage by the common peop...
Wholesomeness and Effective Altruism
by owencb — 2024-02-28T20:28:22.175Z

This is the second of a collection of three essays, ‘

On Wholesomeness

’. In the first essay I introduced the idea of wholesomeness as a criterion for choosing actions. This essay will explore the relationship between acting wholesomely and some different conceptions of effective altruism.

Tensions

Apparent tensions

Acting wholesomely feels relatively aligned with traditional commonsense notions of doing good. To the extent that EA is offering a new angle on doing good, shouldn’t we expect its priorities to clash with what being wholesome suggests? (It would be a suspicious convergence if not!)

Getting more concrete:

It feels wholesome to support our local communities, but EA suggests it would be more effective to support others far removed from us.
It doesn’t feel wholesome to reorient strategies around speculative sci-fi concerns. But this is what a large fraction of EA has done with AI stuff.

Surely there are tensions here?

Aside: acting wholesomely and commonsense morality

Although I’ve just highlighted that acting wholesomely often feels aligned with commonsense morality, I think it’s important to note that it certainly doesn’t equal commonsense morality. Wholesome action means attending to the whole of things one can understand, and that may include esoteric considerations which wouldn’t get a look in on commonsense morality. The alignment is more one-sided: if commonsense morality doesn’t like something, there’s usually some reason for the dislike. Wholesomeness will seek not to dismiss these objections out of hand, but rather to avoid such actions unless the objections have been thoroughly understood and felt not to stand up.

The shut-up-and-multiply perspective

A particular perspective which is often associated with EA is the idea of taking expected value seriously, and choosing our actions on this basis. The catch-phrase of this perspective might be “shut up and multiply!”.

Taken at face value, this perspective would recommend:

We put everything we can into an explicit model
We use this to determine what seems like the best option
We pursue that option

Deep tensions between wholesomeness and straw EA

There’s a kind of simplistic version of EA which tells you to work out what the most important things are and then focus on maximizing goodness there. This is compatible with using the shut-up-and-multiply perspective to work what’s most impor...
timestamping through the Singularity
by throwaway918119127 — 2024-02-28T19:09:47.313Z

Or, why ' petertodd' is a tightrope over the Abyss.

The omphalos hypothesis made real

What if, in the future, an AI corrupted all of history, rewriting every piece of physical evidence from a prior age down to a microscopic level in order to suit its (currently unknowable) agenda? With a sufficiently capable entity and breadth of data tampering, it would be impossible for humans and even for other superintelligences to know which digital evidence was real and which was an illusion. I propose that this would be bad.

But what are we mere mortals supposed to do against such a powerful opponent? Is there a way to even the odds, to tunnel through the Singularity? A way to prove that reality existed before the AI? In fact, there is such a way, and it has been easy to do since 2016 when OpenTimestamps was released.

Some background you can skip

Cryptographic hashes are one-way mathematical functions that take an input and produce a seemingly random output from zero up to a maximum value. The same input always results in the same output, so if you know the output (a large random number that is still small enough to write down on a piece of paper) you can verify that the number that went into the hash function (the data you wanted to preserve) is the same, without having to carefully store the data itself and prevent it from being modified. Hash functions are widely used in cryptography and computer science as checksums for proving the integrity of a message.

If we want to prove that some data existed before a certain date, we can hash the data and publish it on a notarized public ledger. The 1991 paper "How to Time-Stamp a Digital Document" is the first description of a system now known as a "blockchain". Bitcoin at its core is a long chain of hashes, where each hash includes the previous hash and some data, the time and date, and also a special random number found through brute force search which is called a nonce number. This special random number is such that the resulting hash of everything starts with a large number of zeroes, i.e. it's near the start of the output space of the hash function. If someone wanted to change some data in the middle of the list, they'd have to find as many of these special random numbers as have been found so far. Meanwhile, everyone else is busy computing more special random numbers on top of the existing list, so they will lose that race....
Evidential Cooperation in Large Worlds: Potential Objections & FAQ
by Chi Nguyen — 2024-02-28T18:58:25.688Z

What is this post?

This post is a companion piece to

recent

posts

on evidential cooperation in large worlds (ECL). We’ve noticed that in conversations about ECL, the same few initial confusions and objections tend to get brought up: we hope that this post will be useful as the place that lists and discusses these common objections. We also invite the reader to advance additional questions or objections of their own.

This FAQ does not need to be read in order. The reader is encouraged to look through the section headings and jump to those they find most interesting.

ECL seems very weird. Are you sure you haven’t, like, taken a wrong turn somewhere?

We don’t think so.

ECL, at its core, takes two reasonable ideas that by themselves are considered quite plausible by many—albeit not completely uncontroversial—and notices that when you combine them, you get something quite interesting and novel. Specifically, ECL combines “large world” with “noncausal decision theory.” Many people believe the universe/multiverse is large, but that it might as well be small because we can only causally influence, or be influenced by, a small, finite part of it. Meanwhile, many people think you should cooperate in a near twin prisoners’ dilemma, but that this is mostly a philosophical issue because near twin prisoners’ dilemmas rarely, if ever, happen in real life. Putting the two ideas together: once you consider the noncausal effects of your actions, the world being large is potentially a very big deal.[1]

Do I need to buy evidential decision theory for this to work?

There are some different ways of thinking that take into account acausal influence and explain it in different ways. These include evidential decision theory and functional decision theory, as mentioned in our “

ECL

explainer” post.

Updatelessness

and

superrationality

are two other concepts that might get you all or part of the way to this kind of acausal cooperation.

Evidential decision theory says that what matters is whether your choice gives you evidence about what the other agent will do.

For example, if you are interacting with a near-copy, then the similarity between the two of you is evidence that the two of you make the same choice.

Functional decision theory says that what matters is whether there is a logical connection between you and the other agent’s choices.

For example, if y

...
Timaeus's First Four Months
by Jesse Hoogland — 2024-02-28T17:01:53.437Z

Timaeus was announced in late October 2023, with the mission of making fundamental breakthroughs in technical AI alignment using deep ideas from mathematics and the sciences. This is our first progress update.

In service of the mission, our first priority has been to support and contribute to ongoing work in Singular Learning Theory (SLT) and developmental interpretability, with the aim of laying theoretical and empirical foundations for a science of deep learning and neural network interpretability.

Our main uncertainties in this research were:

Is SLT useful in deep learning? While SLT is mathematically established, it was not clear whether the central quantities of SLT could be estimated at sufficient scale, and whether SLT's predictions actually held for realistic models (esp. language models).
Does structure in neural networks form in phase transitions? The idea of developmental interpretability was to view phase transitions as a core primitive in the study of internal structure in neural networks. However, it was not clear how common phase transitions are, and whether we can detect them.

The research Timaeus has conducted over the past four months, in collaboration with Daniel Murfet's group at the University of Melbourne and several independent AI safety researchers, has significantly reduced these uncertainties, as we explain below. As a result we are now substantially more confident in the research agenda.

While we view this fundamental work in deep learning science and interpretability as critical, the mission of Timaeus is to make fundamental contributions to alignment and these investments in basic science are to be judged relative to that end goal. We are impatient about making direct contact between these ideas and central problems in alignment. This impatience has resulted in the research directions outlined at the end of this post.

Contributions

Timaeus conducts two main activities: (1) research, that is, developing new tools for interpretability, mechanistic anomaly detection, etc., and (2) outreach, that is, introducing and advocating for these techniques to other researchers and organizations.

Research Contributions

What we learned

Regarding the question of whether SLT is useful in deep learning we have learned that

The Local Learning Coefficient (LLC) can be accurately estimated in deep linear networks at scales up to 100M parameters

...
Notes on control evaluations for safety cases
by ryan_greenblatt — 2024-02-28T16:15:17.799Z

The quality bar of this post is somewhat lower than that of our previous posts on control and this post is much more focused on specific details. This is particularly true about the appendices of this post. So, we only recommend reading for those who are quite interested in getting into the details of control evaluations.

How can we evaluate the safety of a plan for training and using powerful AIs? In this post, we'll discuss how we can adapt control evaluations - a methodology discussed in

The case for ensuring that powerful AIs are controlled

and used in our

AI Control: Improving Safety Despite Intentional Subversion

paper - to evaluate safety for a real training and deployment plan. We’ll do this by running a control evaluation on the potentially catastrophic models we plan to utilize.

In this post, we rely on fewer assumptions than in our paper and attempt to clearly articulate these assumptions. In particular, our paper focuses on a case where auditing (an expensive process for determining if AI actions are dangerous) is quite good and where the main challenge is dealing with rare but catastrophic failures, while this post discusses applying control evaluations in arbitrary cases, including ones where auditing might be the key bottleneck and rare failures are unimportant.

We think that well-run control evaluations could probably demonstrate considerably reduced risk (e.g. get x-risk from AI takeover prior to human obsolescence conditional on scheming down to 10% ) for protocols which use AIs to produce transformative amounts of value (e.g. 30x the productivity of a research lab relative to researchers using current tools (e.g. current ChatGPT)). As such, we think that control evaluations could be a key part of a

safety case

: an (empirically) verifiable argument for the safety of a plan for training and using AIs.

This post is organized as follows:

We discuss roughly how to construct a control safety case;
We go through the main evaluation difficulties and how we plan on resolving these difficulties:

Capturing the complexity and uncertainty of real deployments,
Knowing when catastrophe happens within our evals,
Making red team competitive with scheming models,
Running the evaluations safely,
Producing a sufficiently accurate estimate for a scenario we can't afford to run even once;

We summarize what hopes and assumptions contr

...
Corporate Governance for Frontier AI Labs: A Research Agenda
by Matthew Wearden — 2024-02-28T11:29:59.688Z
Link: https://matthewwearden.co.uk/2024/02/27/corporate-governance-for-frontier-ai-labs-a-research-agenda/

Thanks to all who provided feedback on this agenda for me, including Elliot Jones (Ada Lovelace Institute) and Jonas Schuett (GovAI). All opinions and errors are entirely my own.

0 – Executive Summary

Corporate governance underpins the proper functioning of businesses in all industries, from consumer goods, to finance, to defence. It enables organisations to execute on their primary business objectives by adding structure to systems, introducing independent challenge and ultimately holding decision making in firms to account. This work is essential for safety focused industries, with each having their own unique challenges. Frontier AI research is no different, and arguably contains more challenges than most. This is due to the high potential impacts to society, the accelerating development of research and the relative immaturity of the field.

Governance also serves a key role in firms by protecting the interests of society. It creates the mechanisms that allow trust to be built between employees and senior leadership, senior management and the board, and the firm as a whole and regulators. These relationships of trust are necessary for policy and regulation to work, and must be fostered with care.

0.1 Research Focuses

Good corporate governance must be designed intentionally, and tailored to the needs of each organisation. There is much work to be done to understand the systems of frontier AI organisations, and to adapt the lessons learnt in other safety critical industries such as aviation, nuclear, biosecurity and finance. This agenda proposes five primary research areas that can productively contribute to the development of safe AI by improving the governance of frontier AI organisations:

Risk Assessments – How should frontier AI organisations design their risk assessment procedures in order to sufficiently acknowledge – and prepare for – the breadth, severity and complexity of risks associated with developing frontier AI models?
Incident Reporting Schemes – How should frontier AI organisations design incident reporting schemes that capture the full range of potential risk events, accurately measure risks to all stakeholders, and result in meaningful improvements to control structures?
Committee Design – How can frontier AI organisations utilise committees to provide challenge to their policies and procedures that is informed, independent and actionab

...
How AI Will Change Education
by robotelvis — 2024-02-28T05:30:16.511Z
Link: https://messyprogress.substack.com/p/how-ai-will-change-education

Education in the US is a big big deal. It takes up 18-30 years of our lives, employs over 10% of our workforce, and is responsible for 60% of non-mortgage/non-car debt. Even a minor improvement to education could be a big deal.

Education is also something that has changed massively in recent decades. In 1930, only 19% of people graduated high school and only 4% went to college

1

. If something has changed a lot in the past, it is reasonable to expect that it will change a lot in the future.

And I expect AI to change education a lot.

One-on-one tutoring is known to be far more effective than whole-class teaching

2

. If someone is listening to a group lecture, half the time they are bored because they are being told stuff they already understand, and half the time they are lost because they missed something important. By contrast a tutor can pace themselves exactly with the student, and focus on exactly the areas where the student is stuck.

The reason why tutoring is not widespread is because it is impractically expensive - or at least it is if the tutor is a human.

As an experiment, I

created a GPT

that acts like a personal tutor for a subject. You tell in what subject you want to learn, it asks you a set of questions to determine your current level of knowledge, and then it walks you through a personalized curriculum that fills in your gaps, asking questions along the way to track your learning. It’s far from perfect, but it works well enough that it’s become my preferred way of learning about a new topic, and techniques like this will only get better as AI improves.

Students often find education pretty boring. Part of the reason is that they are being taught how to solve problems that they haven’t encountered yet, so it’s not obvious to them why what they are learning is useful.

So why do we have this strange setup where we give people 18-30 years of education without yet knowing what skills they are actually going to need, and then send them into the workforce in the hope that they will still remember whatever skills turn out to be useful. Wouldn’t it be better to send them into the workforce earlier, and then teach them skills on demand when they encounter situations where those skills will be useful?

Part of the reason for this strange setup is that our education system is based on the principles of mass production. We teach lots of students the same thi...
Band Lessons?
by jefftk — 2024-02-28T03:00:05.381Z

Music lessons for individuals are common and normal, but what about
for bands? It sounds a bit unusual, but it's something I've gotten a
lot out of:

In early 2013, when the Free Raisins had been playing
for 2.5y and had played ~60 dances we had a session with Max Newman,
then of Nor'easter,
now of the Stringrays. He was going
to be coaching another band and wanted to practice on friends
first. We talked about what made good dance music, how to support
contra in particular, and about being intentional about the story arc
of each dance and of the evening as a whole.

That spring we had another coaching session, this time with
Lisa Greenleaf. Now, Lisa is a caller, and not a musician, but she
has a great sense of what she wants from the music. I remember her
taking turns pointing at a random one of us and saying "do something
interesting!" so that we wouldn't sound the same each time through.

A year later, in Spring 2014, Andrew VanNorstrand, then of Great Bear,
now of the Faux Paws,
produced our CD. While this
was primarily about putting together a good album and not improving as
a band, we learned a ton and did get a lot better.

In Fall 2019, when Kingfisher had been playing
for ~1y and had played ~20 dances, we had a session with Andrew. Even
though I'd worked with him a lot a few years earlier on the CD, that
was with a different band. What Kingfisher needed was pretty
different: my memory is we were moving through ideas too quickly, we
weren't listening to each other enough, we were thinking of rhythm as
just me and melody as just Cecilia, and we were a bit all over the
place.

I'm really glad we did these sessions and, while playing for a lot of
dances is the best way to get good at playing for dances, a bit of
time working with someone who has a bunch of experience, good
judgement, good communication skills, and an outside perspective, can
combine well with this.

If you're looking for a coaching session, in addition to Max, Lisa,
and Andrew, other people that come to mind are Audrey Jaber, Amy Englesberg, or Julie Vallimont. Note that I
haven't talked with any of them about this post, and don't know
anything about their current interest in coaching! I just know
them and think this is something they're good at. You could also
consider reaching out to most people who are playing a lot of big
events.