trentmkelly/LessWrong-43k huggingface dataset:
Pronouns are Annoying
by ymeskhout — 2024-09-18T13:30:04.620Z
Link: https://www.ymeskhout.com/p/pronouns-are-annoying

This post isn’t totally about the culture war topic du jour. Not at first.

As with any other topic that soaks up angst like an ultra-absorbent sponge, I wonder how many have lost track of how we arrived here. Why are pronouns? Pronouns have always been meant to serve as a shortcut substitute reference for other nouns, and the efficiency they provide is starkly demonstrated through their boycott:

Abdulrahmanmustafa went to the store because Abdulrahmanmustafa wanted to buy groceries for Abdulrahmanmustafa’s dinner. When Abdulrahmanmustafa arrived, Abdulrahmanmustafa realized that Abdulrahmanmustafa had forgotten Abdulrahmanmustafa’s wallet, so Abdulrahmanmustafa had to return to Abdulrahmanmustafa’s house to get Abdulrahmanmustafa’s wallet.

So that’s definitely a mouthful, and using he/his in place of Abdulrahmanmustafa helps lubricate. Again, pronouns are nothing more than a shortcut referent. Zoom out a bit and consider all the other communication shortcuts we regularly use. We could say National Aeronautics and Space Administration, or we can take the first letter of each word and just concatenate it into NASA instead. We could append ‘dollars’ after a number, or we could just use $ instead.

The tradeoff with all of these shortcuts is precision. Depending on the context, NASA, for example, might also refer to the National Association of Students of Architecture in India, or some mountain in Sweden. Dollar signs typically refer to American dollars, but they’re also used to denote several other currency denominations. The same risk applies to pronouns. It’s not a problem when we’re dealing with only one subject, but notice what happens when we introduce another dude to the pile:

John told Mark that he should administer the medication immediately because he was in critical condition, but he refused.

Wait, who is in critical condition? Which one refused? Who’s supposed to be administering the meds? And administer to whom? Impossible to answer without additional context.

One way to deal with ambiguous referents is to just increase the number of possible referents. Abbreviations could have a higher level of fidelity if they took the first two letters of every word instead of just one, then no one would risk confusing NaAeSpAd with NaAsStAr. For full fidelity, abbreviations should use every letter of every word but then…obviously there’s an inherent tension between effi...
Is "superhuman" AI forecasting BS? Some experiments on the "539" bot from the Centre for AI Safety
by titotal — 2024-09-18T13:07:40.754Z
Link: https://open.substack.com/pub/titotal/p/is-superhuman-ai-forecasting-bs-some?r=1e0is3&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Disclaimer: I am a computational physicist’s and this investigation is outside of my immediate area of expertise. Feel free to peruse the experiments and take everything I say with appropriate levels of skepticism.

Introduction:

The centre for AI safety is a prominent AI safety research group doing technical AI research as well as regulatory activism. It’s headed by Dan Hendrycks, who has a PHD in computer science from Berkeley and some notable contributions to AI research.

Last week CAIS released a blog post, entitled “superhuman automated forecasting”, announcing a forecasting bot developed by a team including Hendrycks, along with a technical report and a website “five thirty nine”, where users can try out the bot for themselves. The blog post makes several grandiose claims, claiming to rebut Nate silvers claims that superhuman forecasting is 15-20 years away, and that:

Our bot performs better than experienced human forecasters and performs roughly the same as (and sometimes even better than) crowds of experienced forecasters; since crowds are for the most part superhuman, so is FiveThirtyNine.

He paired this with a twitter post, declaring:

We've created a demo of an AI that can predict the future at a superhuman level (on par with groups of human forecasters working together). Consequently I think AI forecasters will soon automate most prediction markets.

The claim is this: Via a chain of prompting, GPT4-o can be harnessed for superhuman prediction. Step 1 is to ask GPT to figure out the most relevant search terms for a forecasting questions, then those are fed into a web search to yield a number of relevant news articles, to extract the information within. The contents of these news articles are then appended to a specially designed prompt which is fed back to GPT-4o. The prompt instructs it to boil down the articles into a list of arguments “for” and “against” the proposition and rate the strength of each, to analyse the results and give an initial numerical estimate, and then do one last sanity check and analysis before yielding a final percentage estimate.

How do they know it works? Well, they claim to have run the bot on several metacalculus questions and achieved accuracy greater than both the crowd average and a test using the prompt of a competing model. Importantly, this was a retrodiction: they tried to run questions from last year, while restrict...
Knowledge's practicability
by Ted Nguyễn — 2024-09-18T02:31:59.018Z
Link: https://tednguyen.substack.com/p/knowledges-practicability

Knowledge can be useless. Much of what we learned isn't used.

The most famous example of useless knowledge must be schools.

But before discussing why school isn't practical, let's first talk about when knowledge is practical. I use this simple formula to decide if something has high application:

The practicality of some knowledge = Its usefulness (quality) X Total chances I can apply it (quantity)

Simply put, if a piece of knowledge is useful, or it isn't very useful but can be used a lot, or it is both useful and can be used a lot, then it's worth learning.

Let's first talk about the usefulness of knowledge.

Because some knowledge is more valuable than others, we compare and pick those that benefit us the most.

For example, suppose you have two ways to solve an issue. If the second way is better, then if you already have a lot of knowledge about it (when/how to apply the second way, how to handle its side effects, etc.), it is better than you already know about the first way. In another situation, if the first way is more efficient, then having the knowledge about it will be better. How useful a piece of knowledge is depends on the situation you apply it to.

It would be best if you understand both solutions well, but since we have limited time and resources, the best we can do is pick what we think will be the most useful and learn in advance.

Knowledge's usefulness is a good factor that helps us pick what to learn. An equally useful factor is how much we can apply a piece of knowledge.

Let's have an example to illustrate that.

One skill that is useful but isn't used a lot is interview skills. Suppose "good interviewing skills" have 1000 units of usefulness, but you only use it 10 times in your whole life; then its practicality can be calculated as:

Good interview skills practicality = 1,000 x 10 = 10,000

In contrast, "good dishwashing skills" (by having better gloves, better cleaners, knowing which one should be cleaned first, etc.) may only have 1 unit of usefulness, but because you need to wash dishes so many times in your whole life (supposedly 10,000 times), you have 10,000 chances to use these dishwashing tricks.

Good dishwashing skills practicality = 1 x 10,000 = 10,000

Although these two skills have many differences, they give you the same value.

Next, let's talk about how we can increase the usefulness of knowledge or how we can find more chances to apply knowledg...
Skills from a year of Purposeful Rationality Practice
by Raemon — 2024-09-18T02:05:58.726Z

A year ago, I started trying to deliberate practice skills that would "help people figure out the answers to confusing, important questions." I experimented with Thinking Physics questions, GPQA questions, Puzzle Games , Strategy Games, and a stupid twitchy reflex game I had struggled to beat for 8 years[1]. Then I went back to my day job and tried figuring stuff out there too.

The most important skill I was trying to learn was Metastrategic Brainstorming – the skill of looking at a confusing, hopeless situation, and nonetheless brainstorming useful ways to get traction or avoid wasted motion.

Normally, when you want to get good at something, it's great to stand on the shoulders of giants and copy all the existing techniques. But this is challenging if you're trying to solve important, confusing problems because there probably isn't (much) established wisdom on how to solve it. You may need to discover techniques that haven't been invented yet, or synthesize multiple approaches that haven't previously been combined. At the very least, you may need to find an existing technique buried in the internet somewhere, which hasn't been linked to your problem with easy-to-search keywords, without anyone to help you.

In the process of doing this, I found a few skills that came up over and over again.

I didn't invent the following skills, but I feel like I "won" them in some sense via a painstaking "throw myself into the deep end" method. I feel slightly wary of publishing them in a list here, because I think it was useful to me to have to figure out for myself that they were the right tool for the job. And they seem like kinda useful "entry level" techniques, that you're more likely to successfully discover for yourself.

But, I think this is hard enough, and forcing people to discover everything for themselves seems unlikely to be worth it.

The skills that seemed most general, in both practice and on my day job, are:

Taking breaks/naps
Working Memory facility
Patience
Knowing what confusion/deconfusion feels like
Actually Fucking Backchain
Asking "what is my goal?"
Having multiple plans

There were other skills I already was tracking, like Noticing, or Focusing. There were also somewhat more classic "How to Solve It" style tools for breaking down problems. There are also a host of skills I need when translating this all into my day-job, like "setting reminders for myself" and ...
Superposition through Active Learning Lens
by akankshanc — 2024-09-17T17:32:56.583Z

Introduction:

'How can we control whether superposition and polysemanticity occur?’ was one of the open questions mentioned by the authors of ‘Toy Models of Superposition’ and this work was an attempt to do so with the help of Active Learning.
The idea behind this work was to examine whether it is possible to decode Superposition using Active Learning methods. While it seems that Superposition is an attempt to arrange more features in a smaller space to utilize the limited resources better, it might be worth inspecting if Superposition is dependent on any other factors. I have compared the presence of Superposition in ResNet18’s baseline and Active Learning model trained on CIFAR10 and TinyImageNet dataset. The presence of Superposition in the above models is inspected across multiple criteria, including t-SNE visualizations, cosine similarity histograms, Silhouette Scores, and Davies-Bouldin Indexes. Contrary to my expectations, the active learning model did not significantly outperform the baseline in terms of feature separation and overall accuracy. This suggests that non-informative sample selection and potential overfitting to uncertain samples may have hindered the active learning model's ability to generalize better suggesting more sophisticated approaches might be needed to decode superposition and potentially reduce it.

Figure 1: Hypothesis

I wanted to test whether the hypothesis in Figure 1 is plausible, it felt intuitive to imagine the superposition being reduced if the model has been able to ‘focus’ more on difficult features, allowing it to give less attention to the redundant features.
Thus reducing the overall required feature representation and helping in observing lesser superposition.

Superposition:

For most, when one hears the term, Superposition it is immediately clubbed with quantum mechanics, the very intriguing quantum entanglement, and always reminds us of the ‘thought experiment’ by Erwin Schrödinger in 1935. But the circuits thread introduced this word Superposition in a new light, a new meaning for anyone interested in the field of mechanistic interpretability. Just as Schrödinger’s cat can be both alive and dead at the same time in his thought experiment, it was observed how a neuron can activate for both a car wheel and a dog snoot. The authors of Zoom In: An Introduction to Circuits observed that neurons had conn...
Survey - Psychological Impact of Long-Term AI Engagement
by Manuela García — 2024-09-17T17:31:38.397Z

As part of the AI Safety, Ethics and Society Course, I’m conducting a survey to better understand the psychological and emotional effects of long-term engagement with AI technologies, particularly within the AI safety community. This is an invitation for you to take part in this anonymous questionnaire, which explores how engagement with AI could influence emotions, stress levels, and mental health.
Who should participate?

• Anyone involved in AI development, research, or policy

• Members of the AI safety community, including advocates and researchers

• Individuals concerned about the societal and existential implications of AI
For participants interested, the report and analysis of this questionnaire will be shared once it’s released.

Link to the Form

Your contribution is deeply valued; this is how we can generate a greater understanding of the psychological challenges faced by individuals in the AI community, and in turn, more effectively address the stress and anxiety caused by this issue, building the resiliency needed to navigate these challenges assertively and empathetically.
Finally, I’m committed to discussing any emotional challenges related to AI in more detail, therefore feel free to reach out at

manugarciaat@gmail.com

.
Thank you in advance for your time.
Survey - Psychological Impact of Long-Term AI Engagement
by Manuela García — 2024-09-17T17:31:38.383Z

Hello everyone,
I’m conducting a survey to better understand the psychological and emotional effects of long-term engagement with AI technologies, particularly within the AI safety community. This is an invitation for you to take part in this anonymous questionnaire, which explores how engagement with AI could influence emotions, stress levels, and mental health.
Who should participate?

• Anyone involved in AI development, research, or policy

• Members of the AI safety community, including advocates and researchers

• Individuals concerned about the societal and existential implications of AI
For participants interested, the report and analysis of this questionnaire will be shared once it’s released.

Link to the Form

Your contribution is deeply valued; this is how we can generate a greater understanding of the psychological challenges faced by individuals in the AI community, and in turn, more effectively address the stress and anxiety caused by this issue, building the resiliency needed to navigate these challenges assertively and empathetically.
Finally, I’m committed to discussing any emotional challenges related to AI in more detail, therefore feel free to reach out at

manugarciaat@gmail.com

.
Thank you in advance for your time.
What does it mean for an event or observation to have probability 0 or 1 in Bayesian terms?
by Noosphere89 — 2024-09-17T17:28:52.731Z

Okay, this one is a simple probability question/puzzle:

What does it actually mean for a probability 0 or 1 event to actually occur, or for those who like subjective credences more, what does it mean to have a probability 0 or 1 observation in Bayesian terms?

Part of my motivation here is to address the limiting cases of beliefs, where the probabilities are as extreme as they can get, and to see what results from taking the probability to the extremes.
How harmful is music, really?
by dkl9 — 2024-09-17T14:53:25.426Z
Link: https://dkl9.net/essays/music_experiment.html

For a while, I thought music was harmful, due largely to pervasive and arbitrary earworms. More recently, I started to find that earworms are ephemeral and lawful. A contrarian belief held like the former for years gets stuck as part of my identity, but maybe I should find the truth.

"Music is harmful" is hard to measure and verify. "Listening to music is harmful" is both easier to measure and more readily useful, for you can make a randomised controlled trial out of it.

Methods

Given that I deliberately listen to music only on rare occasion, it's easy, in my case, to let a column of random booleans in a spreadsheet dictate whether I listen to music each day. Sometimes I forgot to listen to music when the spreadsheet said I should, and sometimes I heard a lot of incidental music on days when the spreadsheet said I should abstain. To account for both cases, I kept a record of whether I actually did listen to music each day. Whether I actually listened to music is the explanatory variable, which ended up 50% correlated (phi coefficient) with whether the random boolean generator said I should.

The response variables are my mood — -1 to 1 — and the song stuck in my head — one of four categories:

no song (N)
a song played back deliberately (D)
a song I heard recently (R)
any other song (O)

Both response variables were queried by surprise, 0 to 23 times per day (median 6), constrained by convenience.

Analysis

I ran the experiment over 51 days. In all analysis here, I exclude three long intervals (11 days, 5 days, 4 days) of consecutive musical abstention due to outside constraints, leaving 31 days to examine.

Given these measurements, we can find the effects of listening to music by comparing the averages from days with music to those from days without music. It seems plausible that the effects of music lag or persist past the day of listening. Perhaps the better averages to compare would come from

music days, plus days just after music days, versus
all other days

What kind of harm do I expect to see from listening music?

It could worsen my mood.
It could make earworms play for more of the time, i.e. increase the ratio of D+R+O to N.
It could make more of my earworms accidental, i.e. increase the ratio of R+O to N+D.
It could make whatever particular music I listen to show up more often as accidental earworms, i.e. increase the ratio of R to O.

Results

What does my data say about all ...
Monthly Roundup #22: September 2024
by Zvi — 2024-09-17T12:20:08.297Z

It’s that time again for all the sufficiently interesting news that isn’t otherwise fit to print, also known as the Monthly Roundup.

Bad News

Beware the failure mode in strategy and decisions that implicitly assumes competence, or wishes away difficulties, and remember to reverse all advice you hear.

Stefan Schubert (quoting Tyler Cowen on raising people’s ambitions often being very high value): I think lowering others’ aspirations can also be high-return. I know of people who would have had a better life by now if someone could have persuaded them to pursue more realistic plans.

Rob Miles: There’s a specific failure mode which I don’t have a name for, which is similar to “be too ambitious” but is closer to “have an unrealistic plan”. The illustrative example I use is:

Suppose by some strange circumstance you have to represent your country at olympic gymnastics next week. One approach is to look at last year’s gold, and try to do that routine. This will fail. You’ll do better by finding one or two things you can actually do, and doing them well

There’s a common failure of rationality which looks like “Figure out what strategy an ideal reasoner would use, then employ that strategy”.

It’s often valuable to think about the optimal policy, but you must understand the difference between knowing the path, and walking the path

I do think that more often ‘raise people’s ambitions’ is the right move, but you need to carry both cards around with you for different people in different situations.

Theory that Starlink, by giving people good internet access, ruined Burning Man. Seems highly plausible. One person reported that they managed to leave the internet behind anyway, so they still got the Burning Man experience.

Tyler Cowen essentially despairs of reducing regulations or the number of bureaucrats, because it’s all embedded in a complex web of regulations and institutions and our businesses rely upon all that to be able to function. Otherwise business would be paralyzed. There are some exceptions, you can perhaps wholesale axe entire departments like education. He suggests we focus on limiting regulations on new economic areas. He doesn’t mention AI, but presumably that’s a lot of what’s motivating his views there.

I agree that ‘one does not simply’ cut existing regulations in many cases, and that ‘fire everyone and then it will all work out’ is not ...
I finally got ChatGPT to sound like me
by lsusr — 2024-09-17T09:39:59.415Z

I publish lots of words on the Internet. These words are fed into LLMs such as ChatGPT. I have published enough training data that the world's most powerful AIs have a simulation of me inside of them. I have basically uploaded a copy of myself into the cloud. The AI simulation of me gets smarter every time OpenAI and its competitors upgrade their AI.

Cool, huh? Who doesn't want a simulation of themselves that thinks way faster and is networked deep into the world's most powerful superintelligences?

Ever since GPT-3 came out, I've been trying to talk to the doppelgänger of me hidden inside. There are a few problems. First of all, GPT needs to know who I am with high fidelity. At first it just generated a stereotypical LessWrong blogger. That wasn't interesting to me. I didn't want a clone of a stereotype of my demographic group. I want a clone of me, with all my quirks and weirdness. For this to work, GPT would have to know who I am i.e. where I exist in its vector space representation of the world.

Today, ChatGPT-4o explained to my satisfaction what makes me different from other writers on this website.

What makes lsusr's writing interesting is the subtlety with which they engage complex issues. Many rationalist bloggers can become quite verbose or dogmatic in their pursuit of certain truths. Lsusr, by contrast, exhibits restraint and humility in the face of uncertainty. They’re willing to question common assumptions within the rationalist sphere and sometimes explore paths that others might find unconventional, often leading to unique insights.

⋮

In essence, lsusr strikes a balance between rigorous analysis and a kind of minimalist clarity that leaves room for the reader to reflect and draw their own conclusions, rather than being led to a definitive answer. This makes the blog a place of exploration rather than indoctrination, offering readers the tools and ideas to enhance their own thinking rather than a packaged belief system.

Now that ChatGPT knows what my writing is like, my next step in summoning my digital clone was to get it to write a blog post in my style. I tried simply asking ChatGPT variants of "write a blog post in the style of lsusr", but that doesn't work. I'm rigorous, unconventional and anti-dogmatic (that's not my subjective self-judgment; the AI said it itself), whereas ChatGPT has a tendency to regress to the mean writer in its training c...