trentmkelly/LessWrong-43k huggingface dataset:
Every niche event should also be a meetup
by DMMF — 2024-11-19T20:47:50.053Z
Link: https://danfrank.ca/every-niche-event-should-also-be-a-meetup/

I was just at a Tyler Cowen lecture in NYC — one of my heroes and biggest intellectual influences — and watching everyone file out after the talk, I kept thinking about this really great comment someone had made on the event announcement: “Do these 92Y talks typically offer space to build community and talk with other attendees, or are they pretty stale pay > attend > leave?”

This hits on something important. The type of person who wants to attend a Tyler Cowen lecture is quite filtered — there is a very good chance that I would enjoy being friends with most people who love Tyler enough to show up. But we all just… left without saying hi.

I’m not sure if we’re truly in a “loneliness crisis,” as the internet suggests, but it’s clear that many people today are actively seeking deeper connections and a sense of community. Niche events already gather people with shared passions—people who care enough to show up in person, which is no small thing. With such a clear alignment of interests, it feels like a huge missed opportunity not to create spaces for attendees to actually connect and get to know each other.

Since moving to NYC this fall, I’ve been thinking about this a lot. I have a lot of niche interests and have been attending lots of niche events, hoping to meet like minded people and build my community here. Despite these events attracting people who are quite similar to me and many potential friends, there’s still very little opportunity to actually connect. I am taking initiative, being hyper-social, but I’m sorry, it’s just hard.

But I’ve seen what’s possible. I was recently at a Disco Biscuits concert and at the front of the floor (~1,500 people in attendance), nearly every person knew each other. Like 30+ people, all giving high fives, hugging, and catching up. This was actually quite annoying during the show, but it was really wonderful to see. The reason for this is that the Disco Biscuits have hosted many festivals and other multi-day concert runs where people travel to attend, which naturally leads to people meeting and connecting and the merging of friend groups. The end result is that, at a random concert in NYC, there are huge groups of people who are part of an integrated, shared community.

During the summer, the band King Gizzard had a large North American tour where before each show, fans organized formal meetups for everyone to attend...
U.S.-China Economic and Security Review Commission pushes Manhattan Project-style AI initiative
by worse — 2024-11-19T18:42:43.296Z

https://x.com/hamandcheese/status/1858897287268725080

"The annual report of the US-China Economic and Security Review Commission is now live. 🚨

Its top recommendation is for Congress and the DoD to fund a Manhattan Project-like program to race to AGI."
Intrinsic Power-Seeking: AI Might Seek Power for Power’s Sake
by TurnTrout — 2024-11-19T18:36:20.721Z
Link: https://turntrout.com/dangers-of-intrinsic-power-seeking

I think AI agents (trained end-to-end) might intrinsically prefer power-seeking, in addition to whatever instrumental drives they gain.

The logical structure of the argument

Premises

People will configure AI systems to be autonomous and reliable in order to accomplish tasks.
This configuration process will reinforce & generalize behaviors which complete tasks reliably.
Many tasks involve power-seeking.
The AI will complete these tasks by seeking power.
The AI will be repeatedly reinforced for its historical actions which seek power.
There is a decent chance the reinforced circuits (“subshards”) prioritize gaining power for the AI’s own sake, not just for the user’s benefit.

Conclusion: There is a decent chance the AI seeks power for itself, when possible.

Read the full post at turntrout.com/intrinsic-power-seeking

Find out when I post more content: newsletter & RSS

Note that I don't generally read or reply to comments on LessWrong. To contact me, email alex@turntrout.com.
Evolution's selection target depends on your weighting
by tailcalled — 2024-11-19T18:24:53.117Z

I think it's common on LessWrong to think of evolution's selection target as inclusive genetic fitness - that evolution tries to create organisms which make as many organisms with similar DNA to themselves as possible. But what exactly does this select for? Do humans have high inclusive genetic fitness?

One way to think of it is that all organisms alive today are "winners"/selected-for by that competition, but that seems unreasonable to me, since some individual organisms clearly have genetic disorders or similar which make them unfit according to this criterion.

There's some sort of consensus that we can assign individual organisms to "species", and then we could count it by the number of members of that species. Supposedly, the most numerous species is Pelagibacter communis, with 10^28 individuals, vastly outnumbering humanity. Maybe we could say that this is the selection target of evolution?

Of course as would be expected, pelagibacter is a very minimalist species, being single-celled and having very few genes. This minimalism also makes it hard to notice, to the point where according to Wikipedia, it was first discovered in 1990. (I wonder if there's another species that's smaller, more common, and even harder to notice...) This raises the question of pure numerousity is the correct way of thinking of it.

If we instead weight by biomass, most life is in the form of plants, and I think more specifically trees. This makes perfect sense to me - trees evolve from a direct competition for height, which is one of the traits most directly related to mass. And in a way, biomass is more sensible to weight by than numerousity, since it is less dependent on the way you slice a species into individual organisms.

But trees are pretty static. Maybe the problem is that since mass has inertia, this weighting implicitly discourages more dynamic species, like humans? An alternative is to weight by energy flow, but in that case, algae and grasses end up accounting for most of it. Sensible, because if you go up the trophic levels, you rapidly lose energy. That said, energy flow does have the dissatisfying (to me) element that it is "shared" between organisms that predate upon each other. I wonder if one could use something like entropy production to get a conceptually similar metric that's more attributable to a single organism.

I don't know of any weightings or metrics where hu...
AISN #44: The Trump Circle on AI Safety 
Plus, Chinese researchers used Llama to create a military tool for the PLA, a Google AI system discovered a zero-day cybersecurity vulnerability, and Complex Systems 
by Corin Katzke — 2024-11-19T16:36:40.501Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-44-the-trump

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.

Subscribe here to receive future versions.

The Trump Circle on AI Safety

The incoming Trump administration is likely to significantly alter the US government’s approach to AI safety. For example, Trump is likely to immediately repeal Biden’s Executive Order on AI.

However, some of Trump’s circle appear to take AI safety seriously. The most prominent AI safety advocate close to Trump is Elon Musk, who earlier this year supported SB 1047. However, he is not alone. Below, we’ve gathered some promising perspectives from other members of Trump’s circle and incoming administration.

Trump and Musk at UFC 309. Photo Source.

Robert F. Kennedy Jr, Trump’s pick for Secretary of Health and Human Services, said in an interview with Forbes: “countries all over the world that are now developing very very high performance AI that is very frightening. We need to be talking to all of them and making sure that we develop treaties that protect humanity.”
Tulsi Gabbard, Trump’s pick to serve as Director of National Intelligence, said in an interview with Joe Rogan: "very real dangers of [AI] being weaponized… just like with nuclear arms race, there are no winners."
Representative Mike Waltz, Trump’s pick for National Security Advisor, said during an Atlantic Council seminar: “before we have full breakout [with AI] … get some regulatory structure around it… These breakout moments … could prove to be very dangerous.”
Vivek Ramaswamy, cohead of the new Department of Government Efficiency, said in an interview with KCCI: "if you're developing an AI algorithm today that has a negative impact on other people, you should bear the liability for it."

Others with potential influence in a Trump administration include Tucker Carlson, who in another Joe Rogan interview said: “Machines are helpmates, like we created them to help us, to make our lives better, not to take orders from them … If we agree that the outcome is bad and specifically it’s bad for people … then we should strangle it in its crib right now,” and Ivanka Trump, who tweeted about the risks of AGI.

Chinese Researchers Used Llama to Create a Military Tool for the PLA

Chinese military researchers have repurposed Meta's open-we...
Visualizing small Attention-only Transformers
by WCargo — 2024-11-19T09:37:42.213Z

Visualizing small Attention-only Transformers

Work done during an internship at MILES, Paris Dauphine University, under the supervision of Yann Chevaleyre. You can find the git page for the post here.

Research has indicated that in large Transformers, facts are primarily stored in the MLP layer rather than the attention layer. However, it's worth exploring whether the attention layer also plays a role in memorizing some part of the data. Can an attention layer memorize information, and if so, how?

In this blog-post, we define the memorization task as predicting the correct next token for a pair of input tokens. Our goal was to determine if the Transformer exhibits any structure that supports memorization for this task. For instance, it is conceivable that attention heads could distribute the workload, with each head remembering specific associations.

Ultimately, we were unable to identify any clear structure within the Transformer that would allow us to propose a definitive algorithm for how memorization occurs. The internal workings are too complex, and the basis of the attention heads does not reveal any structure. While this could be due to the inherent lack of structure in the data itself, it might also suggest that the optimal configuration of the Transformer is inherently "messy". Therefore, we aim to present this complexity and how it can be broken down into smaller, more understandable components. Even if the entire system remains opaque, some sub-steps can be understood relatively well.

The Transformer architecture

We will use a standard architecture for the Transformer, but in a very simplified form. The dictionary size is N, and the sequence length is S. Each token is embedded in a d-dimensional vector space and added to its positional embedding. These vectors are then passed through a layer normalization followed by an attention module with H heads, each of dimension d_h. The outputs of the heads are summed back into the residual stream and projected to logits through an unembedding matrix.[1]

Here is the equation for a one-layer Transformer predicting only the last token:

T(t1:S)=WU(H∑h=1Ah(t1:S)+e(tS)+posS)Ah(t1:S)=WOVh∑s=1Sahs(t1:S)LN(e(ts)+poss)ah(t1:S)=softmax(LN(e(tS)+posS)TWhQKLN(e(ts)+poss),s=1:S)

Simplifications and Setup

Now to visualize this Transformer, we want to use some assumptions which will give nicer results without changing too much the ...
Americans are fat and sick—and it’s their fault…right?
by Declan Molony — 2024-11-19T06:41:36.648Z

People eat more calories than they expend. They gain weight. If nobody is forcing them to eat to excess, then they only have themselves to blame, right?

I harbored this belief for a long time until I read the book

The Obesity Epidemic

by Dr. Robyn Toomath.[1] The premise is that too much emphasis on personal responsibility is (1) negating the changing macroeconomic factors that ballooned the population’s weight, and (2) the government’s responsibility (and failure) to regulate the food industry. She makes the case that the food industry acts to addict us in the same way Big Tobacco did in the 20th century. And one of the clearest ways the food industry does this is in how we shop for groceries.

Supermarkets—drug stores that happen to sell vegetables

Dr. Toomath explains that the development of the global industrial diet is, to a large degree, the fault of supermarkets.

The free market is supposed to empower the consumer, but we hand this power over to multinational companies who in turn create demand (having figured out what sells). Yes, we get to choose this brand of chips over another, but larger forces determine the cheapness of chips and influence our preferences for these over rice or vegetables.

Price is just one way in which our behavior is manipulated. We’re also influenced by the location of certain products on the supermarket shelves.[2] People are bored in checkout lines. Stores know this. It’s not a coincidence that impulse-buy candy is located there.[3]

Supermarkets are implementing the concept authors Richard Thaler & Cass Sunstein describe in their book Nudge.

Nudge: Any choice [made by a decision maker] that alters people’s behavior in a predictable way without forbidding any options.

Where personal responsibility meets subtle manipulation

Back in 2019, my friends and I were in a theater waiting for our movie to begin when an M&Ms ad played. I remember thinking, “what’s the point of ads like this? It’s not like I’ll suddenly be persuaded by this ad to actually get out of my seat and buy M&Ms at the concession stand.” Meanwhile, one of my friends left the theater and returned with newly purchased M&Ms. I was flabbergasted.

Me: “What made you get M&Ms?”

Him: “I don’t know, I just felt like it.”

Me: “So the ad didn’t persuade you?”

Him: “What ad?”

A less obvious nudge than direct advertising is the geographic loca...
Announcing the CLR Foundations Course and CLR S-Risk Seminars
by JamesFaville — 2024-11-19T01:18:10.085Z

Center on Long-Term Risk (CLR) is accepting applications to the next iteration of our introductory fellowship. We are now planning to run two related programs: the CLR Foundations Course and the CLR S-Risk Seminars.[1]

The CLR Foundations Course will introduce people to CLR’s research on s-risk reduction. Apply by Wednesday November 27th, 23:59 Pacific Time.

Apply to the CLR Foundations Course

The CLR S-Risk Seminars will dive deeper into different priority cause areas within s-risk reduction and is intended for people who have completed the CLR Foundations Course or have a similar level of context.

Register your interest for the CLR S-Risk Seminars

What topics does the CLR Foundations Course discuss?

The Foundations Course is designed to introduce participants to CLR’s research on how transformative AI (TAI) might be involved in the creation of large amounts of suffering (s-risks). Our priority areas for addressing these risks include work on multiagent AI safety, AI governance, epistemology, risks from malevolent or fanatical actors, and macrostrategy.

We recommend people consider Center for Reducing Suffering’s S-risk Introductory Fellowship if they are interested in learning about work on s-risks beyond these priority areas, and consider BlueDot’s AI Safety courses if they are interested in learning about existential risks from TAI.

Note that some prior familiarity with discourse surrounding TAI and AI safety will be helpful.

Who are these programs a good fit for?

We think the Foundations Course and S-Risk Seminars will be most useful for you if you are interested in reducing s-risk through contributions to CLR’s priority areas, and are seriously considering making this a priority for your career. They could also be useful if you are already working in an area that overlaps with CLR’s priorities (e.g. AI governance, AI alignment), and are interested in ways you can help reduce s-risks in the course of your current work.

We recommend those interested in our Summer Research Fellowship first take part in the Foundations Course and an S-Risk Seminar, though participation is not a prerequisite. This way you have a better sense of what work on our priority areas involves, and are better able to “hit the ground running” if accepted.

There might be more idiosyncratic reasons to apply and the criteria above are intended...
No Electricity in Manchuria 
by winstonBosan — 2024-11-19T01:11:58.661Z

TLDR: For a period of time in 2021, there were electric shortages throughout China. Especially in the North Eastern Provinces. China use coal extensively for both heating and power generation. Because of an unfortunate overlapping of policy changes (carbon neutral + market reform) and other factors, in order to keeping humans alive, factories were allowed limited work times and civilian uses were rationed. At its worst, there were on average 3~8 hours of downtime intermittently throughout the winter months. (Mostly Nov~Dec)*

Electricity is important. China is important. And I am going to make the naive assumption that happenings regarding China's Electrical grid system is at least somewhat important.

In addition, by looking at how a portion of the grid "failed", it would also shine on the effects of how some macro/foreign policy changes can have immediate and consequential effect on the life of the people.

I promise that by the end of this article[1]:

You will have a slight better understanding of Chinese geography.
What are some reasons behind the recent-ish (~2021) electrical shortage in China, especially what happened in the Three North-Eastern provinces. (historically referred to as Manchuria)

Yer a Power Plant Boss, Harry

Here what your life looks like if you ran a power plant in 2021:

You get instruction from your "boss" telling you how much energy you should produce in any given period.
Your profit margins are locked to roughly 0.03～0.2 RMB per kilowatt-hour at the best of times. Because you don't get to set prices - your margin is predetermined by your customer.
You are not getting as much subsidies as before, because the cool kids running solar and wind are getting them. And because they are cool, the government is letting them into this new "market" economy thingie.
You only have one customer, and that is the Grid (电网). Back in the old days, the plants and the Grid were basically the same company and had monopoly over all power in China; the good old days are gone, at least for you.
You have long term contracts with your coal suppliers.
Oh, did I tell you your boss and your customer is the same person?

So, what do you have to do when you are asked to produce power at a loss while running low on coal with limited supplies? You have a couple obvious choices:

Just stop making electricity.
Buy more coals and make negative monies (usually from the nea

...
Looking back on the Future of Humanity Institute - Asterisk
by jakeeaton — 2024-11-19T00:44:40.928Z

Asterisk's latest is a retrospective on FHI by Tom Ough.

Including (stealing from Tom here:)

☛ Its apparent origin: a candlelit nerdsnipe

☛ When Bostrom and Anders Sandberg lodged with Derek Parfit

☛ The 'whiteboard panopticon'

☛ FHI's profound influence on the development of AI

☛ The ingredients of Bostrom's 'elixir'
☛ How the relationship with the university went downhill

☛ The attempts to save FHI, inc. a mooted transfer to the Department of Physics

☛ The moment the university swung the axe

☛ FHI's 'wake' at a local pub, and the FHI-inspired work that has ensued

☛ Unmasked for the first time: the nerf gun bandit who terrorised Littlegate House
Don't Dismiss on Epistemics
by ggex — 2024-11-19T00:44:05.329Z

Disclaimer: this is my first post and so is extra moderated. For the intended reading experience, kindly skip the next 2 sentences. I am trying to convey: ideas based on broken epistemology can be useful and have working components, and you should not reject them automatically. Similarly, when ideas do work, it doesn't prove the supporting theory correct.

Have you heard the truth about chiropractics? If not, sit down.

They're quacks. Their entire field was created in nineteenth century America by a magnetic healer with no medical training.

I made the mistake then of sharing this with my parents at thanksgiving; their responses were predictable:[1]

"But chiropractics are covered by our insurance." - Well yes, but only because they sued the American Medical Association for anti-trust violations.

"Who is this American Medical Association? Surely the state of the science has advanced since this lawsuit took place."

-The AMA is exactly what it sounds like: a professional association of physicians in the United States of America. Also this lawsuit was settled in the 1990's; chiropractic was developed in the 1890's.

"Chiropractics are doctors, I'm sure they know more than you do"

- Ok first - weird ad hominem - of course chiropractors know more about chiropractic than I do, just as clerics know more scripture than you do; this isn't good evidence. Besides, my appeal to authority trumps yours. Yes, they successfully lobbied the American legal system for the title of doctor - arguably this degrades the meaning of the word. Do you take physicians or the American legal system to be the higher authority on matters of health? The physicians aren't convinced.

"I can't believe you would say such a thing. My chiropractor helps me." - And that's where I didn't have a good answer.

Because they were right of course. Going to a chiropractic had helped them. Arguably chiropractic "helps" most as a very clear signal that you are being cared for by your tribe. And perhaps the title of doctor and it's presentation alongside more rigorous practices (ie. physiotherapy or speech therapy) simply makes it a strong placebo. Or just maybe, after years of practice, they have a treatment that sometimes works. Nonetheless it does seem to make people feel better, even if the treatment itself is an offshoot of nineteenth century magnetic healing.[2][3]

Recently I went to a buddhist retr...