trentmkelly/LessWrong-43k huggingface dataset:
Socialism in large organizations
by Adam Zerner — 2023-07-30T07:25:57.736Z

I'm a programmer who's into startups. For my first startup, a site that provided student super in depth student reviews of colleges, I remember asking what people thought. I'd get all of these really encouraging responses. "Oh, that's so cool! I wish that existed when I was applying! That's gonna be so helpful to prospective students!"

Then for my second startup, I had similar experiences. I built an app that helps people study poker and received lots of great feedback. But for both startups, when it actually came time to sign up: crickets. When it actually came time to fork over some money: crickets.

The lesson? Talk is cheap. Actions speak louder than words. It's all about the Benjamins. That sort of stuff.

Now I work as a programmer in a large organization. Things are very different.

My team builds a tool that is used by many other teams in the organization. We don't, in my opinion, do a very good job of, before building a feature, checking to see if it truly addresses a pain point, and if so, how large that pain point is.

As an entrepreneur, if you do a poor job of this, you fail. You don't make money. You don't eat. Survival of the fittest. But in a large organization? It doesn't really matter. You still get your paycheck at the end of the day. There's no feedback mechanism. Well, I guess there is some feedback mechanism, but it's not nearly as strong as the feedback mechanism of having users voluntarily open up their own wallets and handing you money for the thing you're providing to them.

It reminds me of socialism. In socialism, from what I understand, there is a centrally planned economy. Some pointy-haired bosses decide that this group of people will produce this widget, and that group of people will produce that widget. If you do a clearly horrible job of producing the widget, you'll get fired. If you do a clearly incredible job, and respect the chain of command, you'll get promoted. But almost always, you'll just walk away with your paycheck. It doesn't matter too much how good of a job you do.

Well, sometimes it does. Sometimes there's more elasticity with who gets fired and who gets promoted. But even in those scenarios, it's highly based on KPIs that are legible.

I've been watching The Wire recently, and the show revolves around this a lot. Teachers having their students memorize things for the test. Police officers prioritizing quantity over quality ...
How to make real-money prediction markets on arbitrary topics (Outdated)
by yutaka — 2023-07-30T02:11:47.050Z

Intro

On current real-money event market platforms, traders are limited to viewing and betting on markets made by platform operators. This means the questions tend to be confined to popular subjects like politics and sporting outcomes, and the sites can't be used for financing estimates on practical or limited-interest questions, like alignment expectations or CEO performance.

OpenPredict is a toolkit for managing prediction markets which regular people can use for this purpose. We're calling it a "toolkit" instead of a "website" because OpenPredict's developers do not act as either arbitrators or market makers; instead, traders set criteria and manage resolution themselves. Likewise, all of OpenPredict's components, including its web client, are entirely open-source and store important metadata about markets on Solana and IPFS. This means that while you are free to use the public instance at openpredict.org it is an unprivileged, replaceable frontend that you may (and are in fact encouraged to) forgo in favor of a self-hosted client.

The OpenPredict network uses cryptocurrency to settle accounts, but even if you don't have a crypto wallet, creating a new prediction market on OpenPredict takes less than five minutes. As a tutorial, I'll be making a market about the outcome of this $150,000 bet between EliezerYudkowsky and the UFO guy. You can view and trade outcomes on the resulting market here.

Market Creation

Step One: Sign up

On OpenPredict, your identity is synonymous with your cryptocurrency wallet. If you have one, you can login with that. If you don't have one, all you need is an email address and the site will provision one for you. Simply enter it into the signup box in the top right and follow the shown instructions:

After typing in my email

After following the link and entering the code

Once you've done this, you'll see two wallet addresses generated automatically on the right. The one marked "SOL" is your user ID and what will initially be displayed when you comment, make markets, or trade. When you fill your wallet with some money, you'll be able to register a username that will be shown instead of the wallet address.

Step Two: Fill your Solana Wallet

You need one of two things to trade on OpenPredict: SOL or USDC. SOL is the network currency of Solana and is what's used to pay transaction fees - generally under a tenth of a cent per transaction. USDC is a...
Does decidability of a theory imply completeness of the theory?
by Noosphere89 — 2023-07-29T23:53:08.166Z

I know right now that completeness implies decidability of a theory, but the question is essentially the converse of this:

Given an arbitrary formal theory that is somehow decidable by a Turing Machines, in the sense that for any sentence in the theory, you can decide it's truth or falsity by some means, can you show that it will inevitably be a complete theory?

If it isn't universally the case that decidability of a theory implies the completeness of a theory, are there any conditions that are required to have decidability of a theory be equivalent to completeness of a theory?
If I showed the EQ-SQ theory's findings to be due to measurement bias, would anyone change their minds about it?
by tailcalled — 2023-07-29T19:38:13.285Z

Background on EQ-SQ

The EQ-SQ theory is a theory by Simon Baron-Cohen which states that there's a biological sex difference in tendency to empathize with people (Empathizing) versus try to understand deterministic systems (Systematizing), and that autism represents the extreme Systematizing end of this spectrum.

Measurement bias

Since writing this subthread where I found ambiguous and partial evidence that the EQ-SQ theory's findings are due to measurement bias, I've been considering collecting direct evidence with SBC's EQ and SQ-R scales, and write up a detailed post investigating measurement bias.

Measurement bias, if it exists (as it very much seems to do on e.g. the EQ scale), would most likely involve observing that the sex difference on the scales is limited to one subset of the items, the autistic-allistic difference is limited to another subset of the items, and maybe also the main thing the scale measures exists on a yet third subset of the items.

That is, it would involve observing that the scales mix different things together, and that autism and sex have distinct relationships with the things that get mixed together. As an analogy, imagine showing that autism is extreme masculinity by creating a "maleness test" which mixes together physical strength and poor social skills. Yes, this would have a large sex difference and a large autistic-allistic difference, but that'd transparently be measurement bias; the reason for the large sex difference (it measures physical strength) would be distinct from the reason for the large autistic-allistic difference (it measures social skills).

My concern

My concern is that people didn't start believing in the EQ-SQ theory based on statistical correlations found with Simon Baron-Cohen's scales. They presumably started believing in it based on fuzzy intuitions arrived at through social experience.

So I could imagine that if I did show SBC's findings to be due to measurement error, people would sort of dismiss it as poorly designed scales and still continue believing in the theory. Like basically maybe SBC's studies provide a faux-objective criterion they can use to throw at feminists, but if the criterion doesn't validate sex differences then EQ-SQ enthusiasts would discard it and pick a new one.

Thoughts?
Self-driving car bets
by paulfchristiano — 2023-07-29T18:10:01.112Z

This month I lost a bunch of bets.

Back in early 2016 I bet at even odds that self-driving ride sharing would be available in 10 US cities by July 2023. Then I made similar bets a dozen times because everyone disagreed with me.

The first deployment to potentially meet our bar was Phoenix in 2022. I think Waymo is close to offering public rides in SF, and there are a few more cities being tested, but it looks like it will be at least a couple of years before we get 10 cities even if everything goes well.

Waymo’s current coverage of Phoenix (here)

Back in 2016 it looked plausible to me that the technology would be ready in 7 years. People I talked to in tech, in academia, and in the self-driving car industry were very skeptical. After talking with them it felt to me like they were overconfident. So I was happy to bet at even odds as a test of the general principle that 7 years is a long time and people are unjustifiably confident in extrapolating from current limitations.

In April of 2016 I gave a 60% probability to 10 cities. The main point of making the bets was to stake out my position and maximize volume, I was obviously not trying to extract profit given that I was giving myself very little edge. In mid 2017 I said my probability was 50-60%, and by 2018 I was under 50%.

If 34-year-old Paul was looking at the same evidence that 26-year-old Paul had in 2016 I think I would have given it a 30-40% chance instead of a 60% chance. I had only 10-20 hours of information about the field, and while it’s true that 7 years is a long time it’s also true that things take longer than you’d think, 10 cities is a lot, and expert consensus really does reflect a lot of information about barriers that aren’t easy to articulate clearly. 30% still would have made me more optimistic than a large majority of people I talked to, and so I still would have lost plenty of bets, but I would have made fewer bets and gotten better odds.

But I think 10% would have been about as unreasonable a prediction as 60%. The technology and regulation are mature enough to make deployment possible, so exactly when we get to 10 cities looks very contingent. If the technology was better then deployment would be significantly faster, and I think we should all have wide error bars about 7 years of tech progress. And the pandemic seems to have been a major setback for ride hailing. I’m not saying I got ...
The Parable of the Dagger - The Animation
by Writer — 2023-07-29T14:03:12.023Z
Link: https://youtu.be/sLVrQTd-OHI

This video is an animation of The Parable of the Dagger, by Eliezer Yudkowsky. It's the first article of A Human's Guide To Words.

The narration is almost word for word. The only discrepancies between the narration and the original text are some deletions of words such as "replied" or "said", which are redundant in an animation. But one remains intact:

"How?!" cried the jester in horror, as he was dragged away. "It's logically impossible!"

The rationale for leaving that unchanged is that I'd like to modify the original articles as little as possible, so I felt iffy about deleting so many words, even if the visuals convey a similar meaning. Moreover, having "cried the jester in horror" makes clear to the viewer that the video is adapted from something you can read somewhere, not an original story written for Rational Animations. That said, I'm not sure it was the right decision in the end, although it probably doesn't matter much.

There is one actual modification to the parable, although it doesn't change its meaning. It is a small non-narrated addition at the end. The video would be way too sad without it.

We've also included a brief call to action on screen: "Read this story, and many more at readthesequences.com", plus some more links and details in the description and pinned comment. For now, I'll direct people to that site instead of directly here.
Are Guitars Obsolete?
by jefftk — 2023-07-29T13:20:01.482Z

I got an email from a reader:

I was wondering, like you said that jams
make less sense in the world of easy refrigeration, do you think
guitars make less sense when amazing MIDI keyboards are available now?

It's a good question! In general, if an instrument becomes more
capable then, yes, it's going to make more sense for people to play it
relative to its closest competitors. The range of sounds and textures
it's possible to play on a keyboard is a big advance over what was
possible on a piano, and they're also far more portable, need less
maintenance, are more flexible with volume, and are cheaper. And you
can face other people while you play!

On the other hand, guitar as an instrument has also gotten better.
We've figured out how to make good-sounding acoustic guitars much more
cheaply over time, low-end acoustics and electrics are much more
playable than they used to be, and the range of what you can do with
pedals has expanded. For example, an octave pedal adds a lot to a dance band where
the lowest instrument is a guitar, enough that I'd guess 80% of the
top guitar-lowest contra dance bands use one.

Another reason we see less substitution than you might expect is that
the guitar and piano have very different strengths, and the piano's
increased versatility doesn't affect that tradeoff very much. The
piano is much more
automated: you have much less direct control over the production
of the sound. On the guitar you touch the strings, and very small
differences change the feeling of your music. On the piano you give
that up, but in exchange you can play many more notes at once over a
larger range.

If we could see a chart of how new players, comparing guitar vs piano,
my guess is we'd see a long slow decline of piano stretching back to
~1910, driven by recorded music substituting for playing at home.
Then we'd see guitar rising starting in the 40s as electric
amplification became mainstream and made the guitar able to hold it's
own in bands. Because musicians are very conservative with instrument choice, it
would take decades for the guitar to build up to where it 'should' be
for it's strengths. Then, as good portable electronic keyboards became
to be available in the late 80s I'd expect to see piano start to
recover from its slump, though not pass guitar? If someone has data
on this I'd love to see it—quickly searching ...
NAMSI: A promising approach to alignment
by  — 2023-07-29T07:03:51.930Z

Media-driven fears about AI causing major havoc that includes human extinction have as their foundation the fear that we will not solve alignment before we reach AGI. What hasn't been sufficiently appreciated is that alignment is most fundamentally about morality.

This is where narrow AI systems trained to understand morality hold great promise. We humans may not have the intelligence to sufficiently solve alignment but by creating narrow AI systems that understand and advance morality we can solve it sooner.

Since our greatest alignment fears are about when we reach artificial super-intelligence, (ASI) perhaps narrow morality-focused ASIs should take the lead on that work. Narrow AI systems already approach top level legal and medical expertise. And because progress in those two domains is so rapid, we can expect major advances in the next few years.

We can develop a top level narrow super-intelligent AI that advances the morality at the heart of alignment. Such a system might be dubbed Narrow Artificial Moral Super-intelligence, or NAMSI.

Some developers like Stability AI understand the advantage of developing narrow AI rather than working on more ambitious, but less attainable, AGI. In fact Stability's business model is about selling narrow AI to countries and corporations.

A question we face as a global society is to what might we best apply AI? Considering the absolute necessity of solving alignment and appreciating that morality is our central challenge here, developing NAMSI may prove our most promising application as we near AGI.

But why go for narrow artificial moral super-intelligence rather than simply artificial moral intelligence? Because this is within our grasp. While morality has great complexities that challenge humans, our success with narrow legal and medical AI tells us something. We have reason to be confident that if we train AI systems to better understand the workings of morality, we can expect that they will achieve a level of expertise in this domain that far exceeds our own. This expertise could then guide them in more effectively solving alignment than what seems currently possible through human intelligence.
Understanding and Aligning a Human-like Inductive Bias with Cognitive Science: a Review of Related Literature 
by Claire Short — 2023-07-29T06:10:38.353Z

Produced as part of the SERI ML Alignment Theory Scholars Program with support from Cavendish Labs, Many thanks go to the following for reading and commenting: Dennis Akar, Walter Laurito, Soroush Pour, Kaarel Hänni
Goals of post:

Awareness of current inductive bias research related to human-like decisions
Understand the known mechanisms behind inductive bias development
Compare similarities of humans and why this could be important
What’s the bigger picture importance and potential alignment research connection

Aim to answer these questions:

What does human cognition-inspired inductive bias look like in a model?
Is this divide in model and human bias simply a result of the different types of data that humans take in vs the model?
What are the mechanics behind certain inductive biases in humans?
Is it possible that these findings could translate to a generalization of other human-like decisions in AI?
What elements help create the conditions necessary to shift a model’s inductive bias to match that of a human?

(Future post: 2. Potential ways to create further convergence, experimental ideas)

What is inductive bias

Models

In machine learning, when referring to the inductive bias of a particular architecture and training process, we are pointing to the distribution of models it produces. For example, if a CNN is trained to classify a training set of red apples vs green pears, is it more likely to become a red-vs-green-object-classifier, or a round-vs-pointy-object-classifier (or something entirely different). The term ‘bias’ in this case refers to any basis for choosing one generalization over another one, differing from strictly following the observed training instances.

Inductive bias can be thought of as the set of rules governing what kind of algorithm is likely to end up being implemented in the model architecture, including patterns in how it will generalize to test data and out-of-distribution. This allows the model to generalize.

Occam's razor

is a classic example of inductive bias, which says the simplest consistent explanation is generally the most likely to be the correct one.

There are some known factors that contribute to the creation of a model’s inductive bias, such as: choice in model architecture, training data, feature selection/preprocessing, and optimization algorithms such as Stochastic Gradient Descent (SGD) can all introduce ...
Why You Should Never Update Your Beliefs
by Arjun Panickssery — 2023-07-29T00:27:01.899Z
Link: https://arjunpanickssery.substack.com/p/why-you-should-never-update-your

Epistemic status: Invincible

Since Cavalry scouts are often in direct contact with the enemy, their job can be considered one of the most dangerous jobs the Army has to offer.

—

something called “Operation Military Kids

”

There’s some irony that Julia Galef’s rationalist self-help book The Scout Mindset compares favorably the scout, who hunts for new and reliable evidence, to the soldier, who fights off threats. But scouts have one of the most dangerous military occupations. To quote a random website,

“cavalry scouts and recon units tread uncharted ground when it comes to conflict zones. They are usually at the tip of any advance and, therefore, meet the brunt of whatever resistance is lying in wait for them.”

Uncharted epistemic territory is dangerous because it’s awash with incorrect arguments which might convince you of their false conclusions. Many of these arguments are designed to be persuasive regardless of their accuracy. Scott Alexander describes

succumbing to an “epistemic learned helplessness”

after his failure to refute crackpots whose arguments are too carefully crafted to refute in any reasonable length of time:

What finally broke me out wasn’t so much the lucidity of the consensus view so much as starting to sample different crackpots. Some were almost as bright and rhetorically gifted as Velikovsky, all presented insurmountable evidence for their theories, and all had mutually exclusive ideas. After all, Noah’s Flood couldn’t have been a cultural memory both of the fall of Atlantis and of a change in the Earth’s orbit, let alone of a lost Ice Age civilization or of megatsunamis from a meteor strike. So given that at least some of those arguments are wrong and all seemed practically proven, I am obviously just gullible in the field of ancient history. Given a total lack of independent intellectual steering power and no desire to spend thirty years building an independent knowledge base of Near Eastern history, I choose to just accept the ideas of the prestigious people with professorships in Archaeology, rather than those of the universally reviled crackpots who write books about

Venus being a comet

.

You could consider this a form of epistemic learned helplessness, where I know any attempt to evaluate the arguments is just going to be a bad idea so I don’t even try. If you have a good argument that the Early Bronze Age worked completely differently f

...
Thoughts about the Mechanistic Interpretability Challenge #2 (EIS VII #2)
by RGRGRG — 2023-07-28T20:44:36.868Z

1) Introduction

In February,

Stephen Casper

posted

two Mechanistic Interpretability challenges

. The first of these challenges asks participants to uncover a secret labeling function from a trained CNN and was

solved

by

Stefan Heimersheim

and

Marius Hobbhahn

.

The second of these challenges, which will be the focus of this post, asks participants to uncover a different secret labeling function from a trained transformer and was

solved*

by the same individuals. Stephen marked this second problem as “solved*” (with an asterisk) since “

[this solution] did not find pseudocode for the labeling function, but instead made a strong case that it would not be tractable to find this. In this case, the network seemed to learn to label points by interpolating from nearby ones rather than developing an interesting, coherent internal algorithm.

”

However, I believe that there is “an interesting, coherent internal algorithm” inside of the network. Additionally, there is at least one interesting mechanic in the attention mechanism, even though the

the authors of the solution didn’t expect there to be any

.

Specifically, I argue that the network appears to learn patterns in its extended embeddings (an Anthropic-style circuit of 5 matrices, defined later). Extended embeddings of the input’s first term (X) dictate high-level patterns for each row. We see increasing/decreasing patterns in the extended embeddings for the X dimension and Y (second) dimension. Additionally, we observe interpretable clusters of MLPin neurons (using an Anthropic-style circuit of 3 matrices).

In this post, I present my current understanding of the internal workings of this model. I begin by presenting the mechanism in detail. I next demonstrate how four specific MLPin neurons support this mechanism and argue how these make up larger interpretable clusters of MLPin neurons. I present several interpretable patterns in the attention mechanism. Next, I argue that the model is learning interpretable extended embeddings of input terms X and Y. I conclude by discussing potential short-comings of my and other such interpretability analyses.

My code has been uploaded on Github

. Work was conducted on a Dell (Windows) XPS17 laptop with an NVIDIA GeForce RTX4080 with 12 GB GDDR6 GPU. Bugs, erro...