trentmkelly/LessWrong-43k huggingface dataset:
Poker is a bad game for teaching epistemics. Figgie is a better one.
by rossry — 2024-07-08T06:05:20.459Z
Link: https://blog.rossry.net/figgie/

Editor's note: Somewhat after I posted this on my own blog, Max Chiswick cornered me at LessOnline / Manifest and gave me a whole new perspective on this topic. I now believe that there is a way to use poker to sharpen epistemics that works dramatically better than anything I had been considering. I hope to write it up—together with Max—when I have time. Anyway, I'm still happy to keep this post around as a record of my first thoughts on the matter, and because it's better than nothing in the time before Max and I get around to writing up our joint second thoughts.

As an epilogue to this story, Max and I are now running a beta test for a course on making AIs to play poker and other games. The course will a synthesis of our respective theories of pedagogy re: games, and you can read more here or in the comments. The beta will run July 15-August 15, in-person in SF, and will be free but with limited signups has gone to waitlist-only. We're hoping to run additional iterations in-person in NYC and remote-first starting in September, so please sign up to the mailing list if either of those are of interest.

Some trading firms are driven by good decisions made by humans. (Some aren't, but we can set those aside. This post is about the ones that are.) Humans don't make better-than-average-quality decisions by default, so the better class of intellectually-driven quantitative trading firm realizes that they are in the business of training humans to make better decisions. (The second-best class of firm contents themselves with merely selecting talent.) Some firms, famously, use poker to teach traders about decision making under uncertainty.

First, the case for poker-as-educational-tool: You have to make decisions. (Goodbye, Candy Land.) You have to make them under uncertainty. (Goodbye, chess.) If you want to win against smart competition, you have to reverse-engineer the state of your competitors' uncertainty from their decisions, in order to make better decisions yourself. (Goodbye, blackjack.)

It's the last of these that is the rarest among games. In Camel Up—which is a great game for sharpening certain skills—you place bets and make trades on the outcome of a Candy Land-style camel race. Whether you should take one coin for sure or risk one to win five if the red camel holds the lead for another round... Turn after turn, you have to make these calculations and decis...
Controlled Creative Destruction
by Martin Sustrik — 2024-07-08T04:36:52.274Z

This is a crosspost from https://250bpm.substack.com/p/controlled-creative-destruction.

Creative destruction is a process which lets the inefficient fail and the efficient survive. It's the economists' way of saying of "survival of the fittest".

In free markets, creative destruction reigns supreme. Firms build competing products. Consumers buy the best products. The firms that cannot deliver go bankrupt.

But it not just about money. Open source programmers work for prestige or joy, users choose the software to use based on what works best for them. Popular bloggers gain followers, others fade into obscurity.

Joseph Schumpeter, a Marxist economist who coined the term in 1940's, looked at creative destruction with a mix of awe and fear. Awe at its efficiency, fear that the destructive element may prevail in the long term.

The modern, colloquial usage is more in line with Daron Acemoglu, who believes that creative destruction is vital to economy and elites blocking creative destruction is a major force leading to economic stagnation.

So far, so good.

But sometimes it is the case that destruction of an institution would be extremely harmful. Think state. Nobody wants to live in a failed state and so the state is allowed to persist with no competition and no elimination of the underperforming. As time goes it accumulates dysfunction.

Once the situation becomes intolerable, revolution may occur. People rise up in arms and destroy the state. That's bad. No only is the disruption of the state often accompanied by bloodshed. Also, there's no telling how will the new institutions look like, whether they will be better or worse than the old ones. In history, there's no lack of revolutions that resulted in improvement, but also those that changed nothing or descended into chaos.

One possible way to deal with it is the libertarian one. The state is too big to fail, so let’s make it as lean as possible. Outsource as much stuff as possible to private enterprises, to NGOs, to voluntary associations. All of those are allowed to fail. Creative destruction naturally follows.

Yet, this approach only delays the inevitable. Even a minimal state will eventually accumulate dysfunction and face a potential revolution.

An alternative approach is to use what could be called "controlled destruction": The system is allowed to fail, but in a controlled, piecemeal manner, rather that collapsing all ...
On saying "Thank you" instead of "I'm Sorry"
by Michael Cohn — 2024-07-08T03:13:50.663Z

Back in 2016 or so, I ran into an idea going around the self-help / trauma-informed-therapy / cognitive-behavioral internet: Learn to say “thank you” instead of “I’m sorry”. It’s turned out to be one of the most transformative pieces of advice I’ve ever taken. I’d like to share what it’s done for me, with just enough context to help others think about adopting it.

The idea

Whenever you want to apologize to someone who has done something for you, consider thanking them instead.

Examples

I trip and fall, and you help me up. I could apologize for inconveniencing you or I could thank you for helping me.
I refer to the fat guy statue in a Chinese restaurant as Buddha, and you politely inform me that it's actually Budai / Hotei. I could apologize for being stupid or I could thank you for making me smarter.
I'm having an absolute garbage day and in the middle of an intellectual discussion with you I start crying. You stop talking, listen to me sympathetically, maybe give me a hug. I could apologize for being a mess or I could thank you for being kind.

In all these cases I’ve found that I end up feeling better about myself and more positive towards the other person if I thank them for helping me instead.

Is this just a generic post about growth mindset / cognitive-behavioral therapy / positivity bias?

It's got elements of all those things but I think there are some much more specific shifts that it creates in me and in the person I'm thanking. See below for more.

But first, counterexamples

I do still apologize if I’ve objectively harmed someone or failed to fulfill a duty or a promise. Like:

I trip and fall, spilling coffee on you.
I tell you the guy is the Buddha, you believe me and repeat it around a group of Chinese people, and they think you're an idiot.
I’m having a terrible day and in the middle of an intellectual discussion with you I call you an idiot.

That’s what apologies are for. But I’ve learned that a lot of my apologies were just for, like, existing, and that’s where I’ve found it awesome to express gratitude instead.

Why “thank you” is awesome

Ways saying “thank you” affects me

It frames things in terms of a positive emotion, gratitude[1], instead of a negative emotion, regret.
It puts us on the same side. When I apologize, I feel like there’s me, the hapless mess, and the other person, who is competent and picking up the slack for me. When I tha

...
How can I get over my fear of becoming an emulated consciousness?
by James Dowdell — 2024-07-07T22:02:43.520Z

Summary

In the last few months I've been suffering from bouts of abject terror related to the thought of experiencing reality as a thinking being only - one with no senses of any kind, as well as no agency - presumably as a result of unintended consequences from asking to have my mind uploaded and emulated in a computer. I don't have anybody to talk to about this, who can recommend practical mitigation, and hope to request help from the LessWrong community for recommended reading.

Lead-Up / Premise

This is related and relevant, I promise: in theory I love the idea of dirigibles and airships. I hate that we spend so much energy (literally) and focus on fixed wing aircraft and drones, when we could just be drifting along gracefully through the air. Airship Ventures' "Eureka" airship (while it was flying) was one of my favorite things about Silicon Valley.

My wife knew this, and for my 30th birthday, arranged for us to go on a surprise hot air balloon tour of Napa Valley. I was so excited getting into the basket, I was grinning ear to ear! But the moment the basket lifted off the ground - and somehow it was only then that it hit me - I remembered I had a pretty serious fear of heights, and ended up collapsed on the floor of the basket, shaking pretty much throughout the whole flight.

Jumping topic but not point, I have been extremely keen, for decades, under a mechanistic "I am my brain" assumption, to destructively scan my brain and upload my mind into a computer, where it would not be subject to biological death, and could be backed up and made quite durable. This desire was a very natural evolution of my experience with NES and SNES video game emulation in the 1990s, and the idea is well treated by both pop sci-fi as well as more academic treatments on sites such as this one. We don't have the ability to do something like this yet, and I grow concerned as I approach 40 years old, that we may not have it solved in my lifetime. But I think I'm "reasonably hopeful" that between progress at the Allen Institute, with brain scanning; and the work by Robert McIntyre at Nectome and the Brain Preservation Foundation more generally; that if I live a typical lifespan (with about 35 years of life left), I will have the option to undergo physician-assisted suicide with something like Robert's aldehyde-stabilized cryopreservation ...
An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2
by Neel Nanda — 2024-07-07T17:39:35.064Z

This post represents my personal hot takes, not the opinions of my team or employer. This is a massively updated version of a similar list I made two years ago

There’s a lot of mechanistic interpretability papers, and more come out all the time. This can be pretty intimidating if you’re new to the field! To try helping out, here's a reading list of my favourite mech interp papers: papers which I think are important to be aware of, often worth skimming, and something worth reading deeply (time permitting). I’ve annotated these with my key takeaways, what I like about each paper, which bits to deeply engage with vs skim, etc. I wrote

a similar post

2 years ago, but a lot has changed since then, thus v2!

Note that this is not trying to be a comprehensive literature review - this is my answer to “if you have limited time and want to get up to speed on the field as fast as you can, what should you do”. I’m deliberately not following academic norms like necessarily citing the first paper introducing something, or all papers doing some work, and am massively biased towards recent work that is more relevant to the cutting edge. I also shamelessly recommend a bunch of my own work here, and probably haven't always clearly indicated which papers I was involved in, sorry!

How to read this post: I've bolded the most important papers to read, which I recommend prioritising. All of the papers are annotated with my interpretation and key takeaways, and tbh I think reading that may be comparable good to skimming the paper. And there's far too many papers to read all of them deeply unless you want to make that a significant priority. I recommend reading all my summaries, noting the papers and areas that excite you, and then trying to dive deeply into those.

Foundational Work

A Mathematical Framework for Transformer Circuits

(Nelson Elhage et al, Anthropic) - absolute classic, foundational ideas for how to think about transformers. See my

youtube tutorial

(I hear this is best watched after reading the paper, and adds additional clarity).

Deeply engage with:

All the ideas in the overview section, especially:

Understanding the residual stream and why it’s fundamental.
The notion of interpreting paths between interpretable bits (eg input tokens and output logits) where the path is a composition of matrices and how this is different from interpreting eve

...
Joint mandatory donation as a way to increase the number of donations
by Crazy philosopher — 2024-07-07T10:56:57.222Z

Let's say each person in your community has 1 resource unit. They are 10 peoples in community including you.

Each member of the community can either consume 1 resource himself or give it to a fund that distributes resources to those in need in third world countries. Each resource consumed by one of you brings him 1 unit of happiness, but each resource given to charity brings 5 units of happiness to some third world resident (he is poorer and needs more). For each of you, the happiness of others is 10 times less valuable than your own happiness.

R=ressourcesFutility=Rconsumedinyourcomm/10+donatedR∗5/10+Rconsumedbyyou

Thus, it is preferable for each of you that everyone else donate their resources, but you consume your resources. Also, everyone prefers a joint donation to a joint one consumption. In the absence of coordination, if each of you consumed its resource, and you will all receive 1.9 units of utility, which is much less than 5, therefore, you need coordination.

Thus, each of you is interested in others donating money, and he is not, but at the same time everyone prefers joint donation to joint consumption. In the absence of coordination, each of you will consume your resource, and you will all receive 1.9 units of utility, which is much less than 5. Therefore, you need coordination.

So how can you coordinate your community?

Option 1: you can gather a few friends who are interested in effective charity, tell them the idea of this article, and then declare that if at least one of them refuses to donate to charity, then you will not either. If each of your friends also agrees to promise that if someone does not donate to charity, then they will not either, then each of you have not option [everyone exept me donate to chatiry], so you will choose between [everyone donates to charity] and [no one donates to charity]. If your friends are altruistic enough, then everyone will choose to donate, and you rich good equilibria.

Of course, the more people there are and the less altruistic they are, the less chance it will work. If you decide to arrange such a donation with 10 people, but 1 of them will be selfish, or he simply will not have money, then he will refuse, and you will not receive anything. In general, this option is good for a few friends, which is not bad, but it will not be able to scale.

Option 2: make donations of some percentage of income a prerequisite...
Beyond Biomarkers: Understanding Multiscale Causality
by Matěj Nekoranec — 2024-07-07T09:56:22.554Z

Photo by Jigar Panchal on Unsplash

In exercise science, we typically derive causality in a bottom-up manner. When we evaluate performance, we assess factors such as cardiovascular capacity, metabolic efficiency, or muscular contractile capacity. However, I’ve always grappled with a chicken-and-egg dilemma in exercise physiology. This dilemma highlights the challenge of understanding sequences of events where mutual dependencies exist — each outcome depends on a preceding event, and vice versa.

Consider a simple example: biomechanical testing of an NBA basketball player might reveal that certain parameters (x, y, z) predispose them to excel at that competition level. However, we can also argue that these parameters likely developed in response to the competitive demands of the game. As players advance to higher leagues, they face greater technical demands, which drive their development and the evolution of their biomechanical parameters.

This creates a paradoxical situation. If structure gives rise to behaviour, but structure simultaneously evolves in response to our behaviour, where does causality lie? What comes first, the chicken or the egg? Does causality arise from the bottom-up physiological blueprint or the top-down constraints of a specific ecological niche?

Top-down or bottom-up?

To understand biological causation, I came across an interesting stream of research from the well-known Oxford physiologist Denis Noble, the founder of modern electrophysiology of the heart. He discusses the concept of biological relativity, where no level of the biological hierarchy holds privileged causation (Noble et al., 2019). In simple terms, lower levels are responsible for dynamics, while higher levels constrain the lower levels by setting boundary conditions.

Example derived from Noble et al. (2019). Global cell properties, such as electric potential, regulate molecular-level properties, such as ion channel proteins, which in turn influence changes in cell properties.

Noble explains that differential equations at lower biological levels have an infinite set of solutions until constrained by higher-level boundaries. This means that multiscale nested biological systems operate as a two-way street, with higher emergent biological levels imposing boundary conditions on lower levels, thus serving as top-down controllers. While low-level descriptions define the system’s dyn...
Goodhart's Law and Emotions
by Zero Contradictions — 2024-07-07T08:32:27.134Z
Link: https://expandingrationality.substack.com/p/goodharts-law-and-emotions

Goodhart's Law is an important principle about using a measure to drive action, and there are many examples of Goodhart's law and its importance in human affairs. This essay focuses on how Goodhart's Law applies to human desire in the modern environment.

Emotions do not directly measure the adaptiveness of an action because they are a crude mechanism. They are heuristic, ad hoc and stimulus-dependent. Although emotions are not a direct measure of adaptiveness, they evolved to motivate adaptive behavior, and thus motivation is essentially a proxy for what is adaptive in the current situation. In modern civilization, we are gaming our emotions, thus making them a terrible measure of adaptiveness.

Imagine if the engineer viewed increasing the odometer number as the purpose of the car, and the movement of the car on the road as just a way to increase that number. He would view putting the car up on blocks as progress. That is what modern man is doing with respect to emotions and adaptiveness. He is spinning his emotional wheels and going nowhere.
Reflections on Less Online
by Error — 2024-07-07T03:49:44.534Z

Meta: This post turned out longer, slower, and less well-written than I hoped. I don’t see any similar posts in a quick search, though, so
I'm posting it anyway. I’ve tried to front-load
feedback that might be useful to the organizers, and put more personal
stuff towards the end. For context, I attended LessOnline and the Manifest-branded Summer
Camp, but not Manifest itself, and my main prior experience with events like
this is fandom conventions such as (local to me) Dragoncon.

As I left the Lighthaven dorm to find breakfast, five people at a table
in the courtyard invited me to join a game of Zendo. This was the first
notable thing to happen to me at LessOnline. It was also the thing that
convinced me that yes, the trip across the country to attend would be
Worth It.

I have never played Zendo before, and don’t expect to play it again
anytime soon. That the game was specifically Zendo is not important. The
important part is that five people in the same place knew what Zendo is
and found that kind of game worth playing.

There’s an attitude that I associate with normies, aptly summarized by
Tycho Brahe (the writer, not the astronomer) as: “Many people respond to
new information, especially densely coded information, as something
between an insult and a chop to the trachea.”

There’s a different attitude, one that I associate with security
mindset, aptly summarized by John Gordon as: “Alice will happily
attempt, with someone she doesn't trust, whom she cannot hear clearly,
and who is probably someone else, to fiddle her tax returns and to
organise a coup d'etat, while at the same time minimising the cost of
the phone call. A coding theorist is someone who doesn't think Alice is
crazy.”

A lot of things happened over the course of my trip, but what
made it worth it wasn’t any particular event. It was spending a week
around the sort of people that play Zendo, take dense coding in stride,
and think Alice is a necessary kind of crazy.

Lighthaven

First and most critical to minimizing P(doom), look at the adorable doggie!

His name is Leo. As best I could tell from asking others, he’s not
attached to the site, he hails from one of the adjacent properties and
just likes the people. He belongs to a Lighthaven maintenance person with excellent taste in dogs. I was going to nominate him as the LessOnline
mascot, but must admit that Agendra might be more appropriate.

Ahem. So.

L...
LK-99 in retrospect
by bhauth — 2024-07-07T02:06:27.660Z
Link: https://www.bhauth.com/blog/flawed%20ideas/lk99.html

About a year ago, there was a lot of public interest in a supposed room-temperature superconductor called LK-99. What I publicly said at the time was, basically:

We should remember the possibility that apparent levitation is from ferromagnetism or paramagnetism. Iron filings can stand up on a magnet, and pyrolytic graphite can float over a strong magnet.

If we consider some known high-temperature superconductors:

YBCO has flat sheets of copper oxide, and superconductivity happens along those planes. The copper in that has high positive charge density, comparable to aluminum atoms in alumina, which gives strong bonding to the oxygen.

H3S (paper) has unusually strong bonds between the sulfur and hydrogen, which only form because the atoms are pressed into each other with enough pressure to substantially compress liquid water.

Superconductivity comes from flow of Cooper pairs, and the electron-phonon interaction must be stronger than random thermal movement. LK-99 doesn't seem to have any reason to have exceptionally strong such interactions. (Yes, I'm simplifying, you have to consider phonon bandgaps, but the point is at least directionally correct.)

The focus on "room-temperature" superconductivity is a bit silly. Even with systems using liquid nitrogen cooling, the superconducting wires are much more expensive than the cooling. What's really needed for superconductors to be practical is cheaper superconducting wires, not higher-temperature ones.

At the time, I found the unusual amount of public interest a bit bemusing. There have been various claims of near-room-temp superconductivity, but none of them attracted as much public attention as LK-99. A few months earlier, Ranga Dias published a paper claiming room-temperature superconductivity; he's now up to 5 retractions.

What was different about LK-99?

That was supposedly superconducting at ambient pressure, which makes it more practical, but also means less specialized equipment is needed to replicate it - or claim to replicate it.

LK-99 had a video that appealed to people.

There were also a few social conditions that I think were important:

It had been a while since that last major excitement about fake science news. After some big story that turns out to be wrong, people are more skeptical of science stories in every field for a while, and then things gradually go back to a baseline. (That

...