trentmkelly/LessWrong-43k huggingface dataset:
More Growth, Melancholy, and MindCraft @3QD [revised and updated]
by Bill Benzon — 2024-12-05T19:36:02.289Z

This is cross-posted from New Savanna.

I’ve got a new article at 3 Quarks Daily: Melancholy and Growth: Toward a Mindcraft for an Emerging World.

I’m of two minds about it: On the one hand, I think it’s one of my best non-technical pieces in a decade, maybe more. I enjoyed doing it. I learned a lot. But it was tricky. The ending was not as good as it needs to be.

“Why not?” you may ask. I ran out of time. I had to upload it to meet the deadline. “Why did that happen?” you ask. I didn’t know where I was going. I started out with some great material – the charts of my blog posting frequency, the counts of tag numbers. But as the material developed, it started changing up on me. In a good way, mind you. But it was becoming a new piece, one based on the material I started with, but going in a different direction. I no longer had a sense out where this thing was going. Sure, by the time I had to upload I’d managed to reorient my thinking, but I didn’t have enough to reconfigure the last half of the article.

Still, I like it. It’s some of the best work I’ve done in a few years.

* * * * *

In the rest of this post: 1) I lay out the core argument in prose, then in 2) I dissect that argument into roughly seven assertions, leading to 3) some final comments based on the Young Lady's Illustrated Primer in Neal Stephenson’s The Diamond Age.

The core argument

My argument starts with the following chart, which depicts my blogging activity since I first established New Savanna in April 2010. The chart is based on the number of posts I make per month.

The dominant feature of the chart is that it is very spiky; there are times when I make many posts per month and other times where I make very few. Furthermore, those ups and downs seem to be fairly regular, with the periods of low positing coming in the winter months. Now, since I lived the experience of producing those posts, and I felt (moderately) depressed during the slumps, that suggests that I might be prey to seasonal affective disorder (SAD), which appears to be organic in nature.

That hypothesis, however, is contradicted by my blogging in the last few years. In the article I present a chart that takes a closer look at the period. There we see two stretches where I’m down in the summer and up in the winter. That’s opposite of the pattern typical for SAD.

Something else seems to be going on. What? I argue that it’...
Expevolu, a laissez-faire approach to country creation
by Fernando — 2024-12-05T19:29:24.011Z
Link: https://expevolu.substack.com/p/expevolu

I write this post to present expevolu[1], a system to enable people to establish new independent countries peacefully, through the legal acquisition of territorial rights via trade.

This is the first post in a three part series introducing the idea.

This post, part I, is dedicated to explaining the basics of the system. Part II will deal mostly with a single important practical problem and its proposed solution. The third post, part III, will first discuss incentives of political agents (such as voters and politicians) in regards to the institution of the system and then provide a brief conclusion to the whole series.

The core of the system is in this first post and you can get a practical overview of it in the first sections. So, although the post is a bit lengthy, you'll be able to determine if it's something that would interest you right on the beginning, with only a small amount of effort.

Additionally, I've put some "checkpoint" marks on spots in the text that I thought would make for good places to pause; in case you want to break the reading in smaller pieces.

Part I

Table of contents:

0) Tl;dr: The expevolu system in a nutshell[2]
1) Intro to Trading for Independent Land
2) The Expevolu System (Simplified Version)
Citizens' Bundle, CB
Transition: From Practice to Theory
3) Rights Consolidation in Expevolu
4) Summary
5) Why Dormant CBs
6) Foreign Founders
7) Citizens Outrage and Direct Ownership
8) CBs and Citizenships
9) CB-Citizenship Binding and the Threat of Re-anexation
10) Equality of CBs
11) Purposes of the System
The name expevolu
12) State Owned Territories and Areas with no Defined Owners
13) Independent Zones
14) Protection of Citizens
15) Problems with Adopting Countries Interfering with IZs
16) Conclusion of Part I

0) Tl;dr: The expevolu system in a nutshell

-Expevolu is a policy that grants citizens control over their territorial rights and facilitates the creation of new countries.

-It would need to be willingly adopted by the governments of existing countries to come into being. The land for new countries would be drawn from the territories of these adopting countries.

-Governments of countries that implement expevolu would grant each of their citizens a ‘Citizens’ Bundle’, CB for short, which is a title representing citizens’ territorial rights over their countries.

-To determine CBs, the area of the whole country would be divided by the number of citizens, resulti...
Are SAE features from the Base Model still meaningful to LLaVA?
by Shan23Chen — 2024-12-05T19:24:34.727Z

Shan Chen, Jack Gallifant, Kuleen Sasse, Danielle Bitterman[1]
Please read this as a work in progress where we are colleagues sharing this in a lab (https://www.bittermanlab.org) meeting to help/motivate potential parallel research.

TL;DR:

Recent work has evaluated the generalizability of Sparse Autoencoder (SAE) features; this study examines their effectiveness in multimodal settings.
We evaluate feature extraction using a CIFAR-100-inspired explainable classification task, analyzing the impact of pooling strategies, binarization, and layer selection on performance.
SAE features generalize effectively across multimodal domains and recover nearly 100% of the ViT performance (this LLaVA used).
Feature extraction, particularly leveraging middle-layer features with binarized activations and larger feature sets, enables robust classification even in low-data scenarios, demonstrating the potential for simple models in resource-constrained environments.

Introduction

The pursuit of universal and interpretable features has long captivated researchers in AI, with Sparse Autoencoders (SAEs) emerging as a promising tool for extracting meaningful representations. Universality, in this context, refers to the ability of features to transcend domains, languages, modalities, model architectures, sizes, and training strategies. Recent advances have shed light on key properties of these representations, including their dataset-dependent nature, their relationship with the granularity of training data, and their transferability across tasks. Notably, studies have demonstrated the intriguing ability of features to transfer from base models to fine-tuned models, such as Kissane et al. (2024) and Kutsyk et al. (2024), and have even hinted at their generalization across layers (Ghilardi et al. 2024). However, one critical question remains underexplored: can features trained in unimodal contexts (e.g., text-only or image-only models) effectively generalize to multimodal systems?

In this work, we focus on bridging this "modality gap" by investigating the applicability of SAE-derived features in multimodal settings. Specifically, we explore LLaVA (Liu et al. 2024), a popular multimodal model that integrates vision and language tasks. Leveraging the CIFAR-100 dataset, which provides a challenging fine-grained classification task, we assess the transferability and interpretabilit...
OpenAI o1 + ChatGPT Pro release
by anaguma — 2024-12-05T19:13:21.843Z
Link: https://openai.com/index/introducing-chatgpt-pro/

As AI becomes more advanced, it will solve increasingly complex and critical problems. It also takes significantly more compute to power these capabilities.

Today, we’re adding ChatGPT Pro, a $200 monthly plan that enables scaled access to the best of OpenAI’s models and tools. This plan includes unlimited access to our smartest model, OpenAI o1, as well as to o1-mini, GPT-4o, and Advanced Voice. It also includes o1 pro mode, a version of o1 that uses more compute to think harder and provide even better answers to the hardest problems. In the future, we expect to add more powerful, compute-intensive productivity features to this plan.

ChatGPT Pro provides a way for researchers, engineers, and other individuals who use research-grade intelligence daily to accelerate their productivity and be at the cutting edge of advancements in AI.
Smart people should do biology
by  — 2024-12-05T19:11:20.671Z

In school, it often felt like an unspoken rule: the “smart kids” did physics and chemistry, while biology was relegated to those willing to memorize disconnected facts. A high school teacher once told me biology was his least favorite science because “there are just too many things to remember.” He argued that physics offered laws, and chemistry gave the periodic table—but biology? It was a mess of facts with no unifying principles. While I see his point, he was wrong. Biology is not about memorization. Biology is about exploration, grappling with complexity, and engaging in some of the most exciting intellectual challenges of our time.

The Challenge of Biology: Thinking in Systems

Biology’s complexity isn’t a weakness; it’s a strength. Unlike physics, where idealized models simplify reality, or chemistry, where periodic trends provide predictability, biology forces us to confront the messy, dynamic systems of life itself. Even seemingly simple tasks—like growing E. coli in a lab—require navigating intricate dependencies. A friend of mine, who quit a PhD in CRISPR research, described it perfectly: “If I left for a weekend, all my cells would die.” Biology doesn’t let you assume a “spherical cow.” It demands that you grapple with the full richness of life from the start.

Take the example of yeast aging research, as described by Laura Deming in her week-long exploration of whether it’s possible to make yeast immortal. Her approach exemplifies the intellectual thrill of thinking deeply about biology. She started with curiosity, asking herself what it might feel like to “be a yeast cell.” Then, armed with data from BioNumbers and a deep understanding of yeast physiology, she built a mental model of the cell’s inner workings. She questioned how aging affects everything from cell size to ATP concentration, and how sporulation—a process some yeast use to regenerate—might extend lifespan. Her exploration wasn’t just about finding answers; it was about wrestling with the unknown, articulating plausible hypotheses, and imagining novel experiments.

This kind of thinking—creative, systems-level, and deeply exploratory—is where biology shines. It’s not about memorizing facts but about asking better questions. What causes yeast to age? Can we reverse it? What does it even mean to be immortal? Biology challenges us to think not just deeply but differently.

Why Biology’s Complex

...
Announcement: AI for Math Fund
by sarahconstantin — 2024-12-05T18:33:13.556Z
Link: https://renaissancephilanthropy.org/news-and-insights/renaissance-philanthropy-and-xtx-markets-launch-new-9-million-ai-for-math-fund/

Renaissance Philanthropy and XTX Markets today announced the launch of the AI for Math Fund. The fund will commit $9.2 million to support the development of new AI tools, which will serve as long-term building blocks to advance mathematics.

An increasing number of researchers, including some of the world’s leading mathematicians, are embracing AI to push the boundaries of mathematical discovery and learning. The AI for Math Fund will support projects that expand the use of leading AI technology by mathematicians globally.

Alex Gerko, Founder and co-CEO, XTX Markets said, “The fund will support this critical intersection between AI and math. Working in partnership with Renaissance Philanthropy, we want to give mathematicians the tools they need to advance the field. As AI continues to transform other sciences, we believe that mathematics will be next.”

Renaissance Philanthropy and XTX Markets are inviting proposals for innovative projects led by researchers, non-profits, companies, mathematicians, software engineers and computer scientists that are unlikely to occur under business-as-usual conditions. Click here to submit an application.

Proposals should be aligned with one of the following categories:

Production grade software tools: AI for auto-formalization, proof generation, synthesis of verifiable code, and more
Datasets: Open-source collections of theorems, proofs, and math problems
Field building: Textbooks, courses, and resources to grow the AI-for-math community
Breakthrough ideas: High-risk, high-reward approaches to AI-driven math research

XTX Markets is the founding donor of the AI for Math Fund.

“We are excited to partner with XTX Markets on this important initiative,” said Tom Kalil, CEO of Renaissance Philanthropy. “The convergence of AI and math has the potential to advance fundamental mathematics, the reasoning capability of AI systems, and the synthesis of verifiable code.”

Following a rigorous assessment of the proposals, individual grants of up to $1 million will be awarded for projects lasting up to 24 months.

Terence Tao, UCLA, Fields Medalist and AI for Math Fund advisor said, “The next generation of AI models and tools have the potential to enhance collaboration among mathematicians that was previously impossible. I am delighted to work with Renaissance Philanthropy and XTX Markets to realize this potential through the AI for Math Fund.”

About Rena...
Detection of Asymptomatically Spreading Pathogens
by jefftk — 2024-12-05T18:20:02.473Z

Cross-posted from my NAO
Notebook.

This is an edited transcript of a talk I just gave at CBD
S&T, a chem-bio defence conference. I needed to submit the slides
several months in advance, so I tried out a new-to-me approach where the
slides are visual support only and I finalized the text of the talk later
on. This does mean there are places where it would be great to have
additional slides to illustrate some concepts

Additionally, this was the first time I gave a talk where I wrote out
everything I wanted to say in advance. I think this made for a much less
engaging talk, and for future ones I'm planning to go back to speaking from
bullets.

I'm Jeff Kaufman, and I co-lead the Nucleic Acid Observatory at
SecureBio, where we work to develop biosurveillance methods capable of
detecting biological threats released by governments or non-state
actors. Today I'm going to be talking about the computational
methods we use in our pathogen-agnostic early warning system.

Before I get started, however, this is all the work of a team:

I'm honored to work with this group, and Will, Lenni, and Simon are here today.

As I go through this talk, please leave questions in the app. If I've
managed my timing correctly, always a risk, we should have a good bit of time for them at the end.

So: what are we up against?

A typical pandemic starts with an infected individual.

They start having symptoms and become infectious.

Someone else gets infected.

Then a few more.

Those people start having symptoms and become infectious.

More people get infected.

There is now a cluster of unusual cases.

Perhaps at this point doctors
notice something unusual is happening and raise the alarm.

But what if there's a long pre-symptomatic period? Imagine something
like a faster-spreading HIV, maybe airborne.

You'd still start with one infected individual.

Then another.

Still without the symptoms that would tell us something was wrong.

No one notices.

Then the first person starts showing symptoms

But that's unlikely to
be enough for someone to notice something is wrong: there are
individuals who report unusual symptoms all the time.

Another person starts showing the same unsual symptoms.

But they're
far enough away that no one puts the pieces together yet.

By the time you have symptom clusters...
Model Integrity: MAI on Value Alignment
by Jonas Hallgren — 2024-12-05T17:11:31.707Z
Link: https://meaningalignment.substack.com/p/model-integrity

EVERYONE, CALM DOWN!

Meaning Alignment Institute just dropped their first post in basically a year and it seems like they've been up to some cool stuff.

Their perspective on value alignment really grabbed my attention because it reframes our usual technical alignment conversations around rules and reward functions into something more fundamental - what makes humans actually reliably good and cooperative?

I really like their frame of a moral graph and locally maximally good values to follow as another way of imagining alignment, it is a lot more similar to that which happened during cultural evolution as explored in for example The Secret of Our Success. It kind of seems like they're taking evolutionary psychology and morality research and group selection and applying the results to how to align models and I'm all for it.

It could be especially relevant for thorny problems like multi-agent coordination - just as humans with shared values can cooperate effectively even without explicit rules, AI systems might achieve more robust coordination through genuine internalization of values rather than pure game theory or rule-following.
This is part of my take nowadays, we need more work on things that work in grayer, multi-agent scenarios as we're likely going into a multi-polar future with some degree of slower takeoff.
Social Science in its epistemological context
by Arturo Macias — 2024-12-05T16:12:29.034Z
Link: https://www.theseedsofscience.pub/p/prediction-and-control-in-natural

Seeds of Science has kindly published my essay “Prediction and control in natural and social systems”. This short essay : i) address and contextualize the modest predictive success of Social Science and ii) describes what kind of scientific intervention on the human society can be successful.

Its main arguments are the following:

Exact Sciences and Artificial Systems: Physics and Chemistry are considered exact because they operate on artificial systems developed through scientific processes, unlike complex natural systems. The precision seen in Celestial Mechanics (the only natural system on which we can consider Science to be exact) is replicated in other physical systems like lenses, batteries, and microchips, but they are human-made. We mostly have exact predictive science on systems designed to be predicted. Even chemists' first task was to create pure substances, which are essential for applying the rules of stoichiometry and chemical thermodynamics.

Complex Natural Systems: Beyond exact sciences, other natural sciences study complex systems, both animate and inanimate. Complex systems can be categorized into modular systems, which allow controlled experiments (vg. Medicine random trials), and non-modular systems, where direct experimentation is not feasible (the system is too big, no easy control points are found).

Non-Modular Complex Systems: These systems, such as the atmosphere, ecosystems, and populations, are studied by Climatology, Ecology, Population Genetics, and Evolutionary Biology, and are not easily amenable to controlled experiments on due to their scale and complexity (that imply that it is difficult to alter a single element alone in the system).

Social Sciences: When compared with natural sciences studying complex non-modular systems, social sciences' predictive success and techniques align with those of similar natural sciences, challenging the notion that social science is less rigorous.

Institutional Design and Progress: increasing the "hardness" of any science involves moving from observation to control, and that is why the Mechanism Design research agenda is superior to the policy design paradigm. Unlike the High Modernity project of vertical rule by a technocratic class, institutional design is (and has been since Solon and the American founding fathers) an inclusive project where the intellectual elite proposes the rules fo...
Higher and lower pleasures
by Chris_Leong — 2024-12-05T13:13:46.526Z

I used to think that talk about more sophisticated forms of art providing "higher forms of pleasure" was mere pretentious, but meditation has shifted my view here by making me more conscious of how of things affect my experience.

Art can do two things. It can provide immediate pleasure. This is all that "disposable " entertainment provides. Alternatively, it can shape the way you can make sense of the world.

One way it can do this is by providing you with a greater sense of purpose, that allows you to push through obstacles with less suffering. As an example, let's suppose you watch an inspirational story about someone who grinds at work (such as the Pursuit of Happiness). Perhaps before you watch it, when you're at work, every few minutes you think, "I hate my job, life is suffering, someone please shoot me". Perhaps after that your work becomes meaningful and you no longer are pulled down by such thoughts.
Another example: there is a scene in American Beauty where Rick Fitts calls a scene with a plastic bag floating "the most beautiful thing in the world". We can imagine that this teaches someone to appreciate beauty in the everyday, beauty they would have previously overlooked.

Over a longer period of time, you'd expect to increase your utility more by watching something that positively transforms the way that you experience the world than something that just provides immediate pleasure.

Now, I'm not saying that none of this talk of higher and lower pleasures is pretentiousness, but I think there's something there. And it's easy to underestimate the value of these "higher pleasures" as the effect of these is less obvious than that immediate enjoyment.
(This post is in response to the Astral Codex Ten post Friendly And Hostile Analogies For Taste).