trentmkelly/LessWrong-43k huggingface dataset:
Developer experience for the motivation
by Adam Zerner — 2022-11-16T07:12:19.893Z

I watched this YouTube video a few months ago that really stuck with me. It was about how to organize your kitchen. The video went through 10 tips, but before getting in to those tips it talked about the why. Why is organizing your kitchen important in the first place?

Well, the standard and obvious answer is that it makes the process of cooking smoother and more efficient. No digging around for those tongs as your chicken threatens to burn. But here's another answer that is perhaps less obvious and more important: motivation.

That's right, motivation. The desire to even be in the kitchen in the first place. If you have to dig through a chaotic cabinet under the sink to find the right pan, in practice you're just gonna end up ordering takeout. And that's a problem that compounds upon itself. If you cooked instead of ordering takeout you'd be building up your skill, making things easier for next time. Instead, your failures and bad experiences make you averse to wanting to cook in the future.

I think something similar is true in programming.

One simple example of this is with Vite vs Webpack. I recently was working on a side project that used Webpack. `npm run start` would take a good ten seconds or so. And I restart my dev server a lot, imposing this annoying ten second cost on myself beacuse... well, I'm not sure. I'm paranoid about hot reloading not working? Because I suck?

Anyway, I ended up switching to Vite which is more like one second instead of ten (don't quote me though). Soooo much nicer. I notice myself smiling inside whenever I restart my dev server. It's like reaching for a knife resting neatly on a wall mounted magnetic strip instead of having to dig through a messy drawer.

Another example that comes to my mind is with flaky tests. When I was working on Premium Poker Tools, my end-to-end tests got out of control. I'd make a small code change, random tests would fail, it didn't make sense, I'd have to spend hours debugging. Ugh. It definitely made me feel unmotivated.

But as a sort of counterexample, for a side project called Calibration Training, I didn't have any tests. At first it was fine, but at some point I started feeling nervous that a code change here would introduce a bug there. It's a little bit like the feeling of being in one of those antique shops: you don't want to accidentally bump into something and then hear the sound of glass shatter...
Progress links and tweets, 2022-11-15
by jasoncrawford — 2022-11-16T03:21:46.159Z
Link: https://rootsofprogress.org/links-and-tweets-2022-11-15

Announcements

The Roots of Progress email newsletter is now on Substack. If you have a Substack, I’d love a recommendation—thank you!
The Science for Progress Initiative to catalyze research on the scientific process (announcement via @heidilwilliams_)
Biotech startup hiring RAs and Scientists to develop therapies against aging and age-related diseases (@ArtirKel)
Cameron Wiese (World’s Fair Co) is looking for an events/ops person “to help create a magical experience next month” (@camwiese)

Links

Are technologies inevitable? (by @mattsclancy)
Nuclear energy is misunderstood, rarely celebrated, and deeply underinvested in (by @juliadewahl)
Econ paper claiming that part of the explanation for the British Industrial Revolution is that British engineers were working on more central/fundamental technological problems
Sunny Bono says that “entrepreneurial capitalism” is the solution to extreme poverty, and he’s been reading Where Is My Flying Car? (via Marginal Revolution and @_TamaraWinter)

Queries

What are the most interesting megastructure ideas? (@michael_nielsen)
What is the right context & funding model for an inventor today? (@piammichel)
Looking for good examples of technology overlapping S curves (@_brianpotter)
What are some lesser-known enormous infrastructure, construction, or R&D projects in US history? (@_brianpotter)
If a media company wanted to inspire hope & self-efficacy what type of content would they produce? (@willobri)
What are all the attempts to create a pro nuclear coalition in the western world? (@PradyuPrasad)
What’s the best way to get old annual reports for public companies? (@_brianpotter)
What explains “maximalism”? (@kanjun)

Tweets

We are problem-solving animals
We live in the future
Some events that took place in the same time in history but don’t seem like they would have

Retweets

Cruise expands driverless service area to cover almost all of SF (@kvogt)
222nm light to kill pathogens is probably a decade or more away from widespread usage (@davidmanheim)
Why the First Amendment guarantees freedom of religion (@BretDevereaux)
Zeynep’s law: assume counterintuitive findings to be false, and second-order effects to be dwarfed by first-order ones (@zeynep)
Search books by concept instead of keyword, using AI (@dwarkesh_sp). And: =GPT3(), a way to run GPT-3 prompts in Google Sheets (@shubroski)
Every new Lego piece has to fit with every piece ever made,

...
EA & LW Forums Weekly Summary (7th Nov - 13th Nov 22')
by Zoe Williams — 2022-11-16T03:04:39.398Z

Supported by Rethink Priorities

This is part of a weekly series summarizing the top (40+ karma) posts on the EA and LW forums - you can see the full collection here. The first post includes some details on purpose and methodology.

If you'd like to receive these summaries via email, you can subscribe here.

Podcast version: prefer your summaries in podcast form? A big thanks to Coleman Snell for producing these! Subscribe on your favorite podcast app by searching for 'Effective Altruism Forum Podcast'.
Author's note: I'm currently travelling, so there won't be any summary post next week. It'll be back the week after with the summaries for those 2 weeks of content at a higher karma bar.

Top / Curated Readings

FTX

FTX filed for bankruptcy on 11th Nov. The FTX Future Fund, financed by FTX and its founder SBF (Sam Bankman-Fried), had been a major funder in EA since its launch in Feb this year.
What happened: Binance, a competitor, sold their stake in FTX’s primary coin - this caused it to drop in value, and led to the equivalent of a bank run ie. a large number of clients attempting to withdraw funds. FTX didn’t have enough money to pay them, and there have been claims this is due to SBF misusing customer funds to prop up his trading company Alameda Research.

This has meant thousands of people with funds in FTX have likely lost them. If you need someone to talk to, even just to vent, CEA’s community health team

is available

or you can access peer support set up by Rethink Wellbeing and the Mental Health Navigator

here

.

FTX FAQ

by Hamish Doodles provides a good overview of the situation as of Sunday 13th November.

Other key points from posts:

The EA community has distanced itself from FTX and is strongly condemning any misuse of customer funds that might have occurred. This includes statements by

Robert Wiblin

and

William MacAskill

,

the FTX fund team

(who have resigned), and many others.
Upcoming grantees of the FTX Future Fund who have yet to receive money are unlikely to. Money received from FTX after August 11th

may also be clawed back

(currently uncertain - there’s also some chance of earlier grants being clawed back but this is much less likely). If this puts you in financial distress, limited emergency funding is available via

Nonlinear

or the

AI Safety Microgrants round

. These budgets are comparatively small, so donation

...
The FTX Saga - Simplified
by Annapurna — 2022-11-16T02:42:55.739Z
Link: https://jorgevelez.substack.com/p/ftx

Disclaimer: This is an evolving situation. It might take months to get an accurate picture of what exactly happened. This post is my attempt to simplify what is currently the general theory of what happened at FTX as of November 16, 2022. The goal of this post is to explain to those not well versed in investment finance how it all (potentially) went wrong. If you have a sophisticated understanding of investment finance, this post is probably too elementary for you.

This is a story about lending.

But before I begin the story, I would like to explain how certain financial entities work today.
A bank, in simple terms, is an entity where a customer can deposit their money (and get paid interest), or can borrow money (and pay interest to the bank). The way a bank makes money is that it charges more interest to the borrower than what it pays to the depositor. A bank is allowed to lend your money out without your explicit permission. Because of this, governments heavily regulate banks. They are limited as to who they can lend to and how much. They are audited frequently and have robust external oversight.

A brokerage, in simple terms, is an entity where a customer can deposit their money and buy investment assets, in hopes to grow their money over time. Typically, a customer can open two types of accounts in a brokerage:

Cash account: This account can exclusively buy and sell assets
Margin account: This account can not only buy and sell assets, it can borrow against those assets. For example, you can deposit $10,000 USD and buy Apple stock. You can then use that Apple stock as collateral and borrow against it. Typically you could borrow up to a certain amount. For a stock like Apple, you could borrow up to 70% of the value of the stock, so in our example, you could borrow up to $7,000 USD, which you could use to buy whatever you want. However, there is a caveat: if Apple stock falls 50%, you would be faced with a margin call: Your account balance is $5,000 USD in Apple stock, but you borrowed $7,000. When this happens the brokerage will call you and ask you to deposit $5,000 USD in collateral so your account is covered. If you don’t have the money, the brokerage has the right to sell your remaining Apple stock to cover your bad debt. However in our example, there is a shortfall of $2,000 USD ($5,000 from the sale of Apple stock - $7,000 loan you t

...
Utilitarianism and the idea of a "rational agent" are fundamentally inconsistent with reality
by banev — 2022-11-16T00:19:14.649Z

The currently unsolvable problem with the ethical branch of consequentialism and its subtype utilitarianism, on the basis of which the whole concept of effective altruism and the theory of rational agents are built, is that since the world is an extremely complex tangle of interwoven systems, you cannot predict the consequences of any change even a few steps ahead. Not only because you can't adequately model them, but also because any measurement error is enough to make it impossible to predict even the 0/1 (true/false) status of these systems and their parts after very few steps. So to assume that you can estimate the maximal utility of any action by any criteria over any considerable period of time is an overconfidence that is characteristic of many of those who consider themselves rationalists.

This is not to say that one should do nothing. It is to say that it is necessary to act, and to plan action, on principles other than consequentialism or utilitarianism.

And to be a little more humble. Less wrong, you know.
Is the speed of training large models going to increase significantly in the near future due to Cerebras Andromeda?
by Amal  — 2022-11-15T22:50:22.968Z

Cerebras recently unveiled Andromeda - https://www.cerebras.net/andromeda/, an AI supercomputer that enables near linear scaling. Do I understand correctly that this might have a big impact on the large (language) models research, since it would significantly speed up the training? E.g. if current models take 30+ days long to train, we can just 10x the number of machines and have it done in three days? Also, it seems to be much simpler to use, thus decreasing the cost of development and the hassle with dstributed computing.

If so, I think its almost certain that large companies would do it, and this in turn would significantly speed up the research/training/algorithm development of large models such as GPT, GATO and similar? It seems like this type of development should affect the discussion about timelines, however I haven't seen it mentioned anywhere else before.
What is our current best infohazard policy for AGI (safety) research?
by Roman Leventov — 2022-11-15T22:33:34.768Z

I struggle to see what the infohazard policy for AGI safety research writing should be. For example, it has been pointed out multiple times that GPT-3 paper was an infohazard, because so many people realised this kind of model was possible, and developed its equivalents, making big progress towards AGI overall. On the other hand, imagine we would still know nothing about LLMs on the capability level of GPT-3 and higher. So many independent research with these models would have not been done. Safety teams at OpenAI and other leading orgs would do something, but the alignment community complains that the size of these teams is inadequate to the si,e of their capability teams, so we should sort of want AGI labs to publish? Or we also think that the number of AGI wannabies outside the leading AGI labs who want to build something impressive upon the insights from the leading AGI labs and don't concern about safety pretty much at all is still many times larger than the number of AGI safety researchers who use GPT-3 and other proto-AGI results in "differentially right way"?

Similar weighting of harm and benefit should be performed regarding the community involvement: it's known that publishing of GPT-3 paper was a wake-up call for many researchers, and, I believe, have directly led to founding of Conjecture. Similarly, publishing of the many impressive results earlier this year, primarily by Google and DeepMind, was a wake-up call for many other people concerned about the x-risk, including me. But it's harder to estimate the corresponding effect on the excitement of another groups of people, and estimate the numbers of people who decided to become involved and push towards AGI, as the result of these publications. (I believe John Carmack is an example of such a person.)

What are the best resources on this topic? Why they are not promoted as a policy on LessWrong?

Related to the question is the idea that the conceptual/inferential, social, organisational, and political division between the AGI capabilities R&D community and the alignment community is deeply problematic: what we should have is a single community with a singular goal of developing a transformative AI that won't lead to a catastrophe.
SBF x LoL
by Nicholas / Heather Kross — 2022-11-15T20:24:52.041Z

Epistemic status: community conversation starter.

Content warnings: mental illness, IQ, drug abuse, speculation about unproven events, the concept of permanent mental degradation, flashing lights / loud sounds on a linked video, a link to the LessWrong tag about (discussion of the meta-idea of) infohazards, League of Legends.

If any ideas have gaps / are poorly-developed, click the nearby links and read them.

Boring Context Part

I saw this link to this post and also saw this comment, and I wanted to respond.

Time Is Not Skill

"SBF is an idiot because he's bad at League of Legends" is the wildest argument I've seen in a while.

I want to first unpack this explicitly, as the article went into minor depth about it:

While this [Sam Bankman-Fried being bad at the game League of Legends] may initially seem like a frivolous observation, it is actually shocking that SBF was unable to rank higher than the Bronze or Silver league after years of regular play across hundreds if not thousands of individual games. It is reflective of an incredibly impaired level of cognition, much like someone who was unable to learn how to ride a bicycle after an entire month of practice or someone who never progresses past the level of an average elementary school student after five years’ of daily piano practice. To me, this is one of the strongest indications possible that SBF is, to be blunt, “not entirely there.”

I have over 2,000 hours in the FPS Team Fortress 2. Shouldn't I be a master at the game?

Not necessarily.

Many (most?) people who are addicted to a video game, are doing it to relax, not to think.

Relatedly, "dark flow" can happen, where we get a "flow state" where our abilities (at the moment) are matched exactly to the challenge (at the moment)... but "our abilities" are "I'm tired/understimulated", and "the challenge" is "play the game somewhat poorly". This is why "mindless" activities are usually more addictive than non-mindless ones; for most people, most of their gaming time is not high-energy, so high-challenge flow is partly achieved by just playing the game at all.

The analogy used in the article doesn't work, since it compares deliberate-practice hours (piano practice, riding a bike) with those dark-flow hours of League [1].

Many of my hours in Team Fortress 2 were spent idling, as I left the game running as background noise/imagery while I did something

...
Some research ideas in forecasting
by Jsevillamol — 2022-11-15T19:47:09.422Z

In the past, I have researched how we can effectively pool the predictions of many experts. For the most part, I am now focusing on directing Epoch and AI forecasting.

However, I have accumulated a log of research projects related to forecasting. I have the vague intention of working on them at some point, but this will likely be months or years. Meanwhile, I would be elated if someone else took my ideas and developed them.

And with the Million Predictions Hackathon by Metaculus looming, now seems a particularly good moment to write down some of these project ideas.

Compare different aggregation methods

Difficulty: easy

The ultimate arbiter of what aggregation works is what performs best in practice.

Redoing a comparison of forecast aggregation methods on Metaculus / INFER / etc questions would be helpful data for that purpose.

For example, here is a script I adapted from Simon M to compare some aggregation methods, and the results I obtained:

Method	Weighted	Brier	-log	Questions
Neyman aggregate (p=0.36)	Yes	0.106	0.340	899
Extremized mean of logodds (d=1.55)	Yes	0.111	0.350	899
Neyman aggregate (p=0.5)	Yes	0.111	0.351	899
Extremized mean of probabilities (d=1.60)	Yes	0.112	0.355	899
Metaculus prediction	Yes	0.111	0.361	774
Mean of logodds	Yes	0.116	0.370	899
Neyman aggregate (p=0.36)	No	0.120	0.377	899
Median	Yes	0.121	0.381	899
Extremized mean of logodds (d=1.50)	No	0.126	0.391	899
Mean of probabilities	Yes	0.122	0.392	899
Neyman aggregate (o=1.00)	No	0.126	0.393	899
Extremized mean of probabilities (d=1.60)	No	0.127	0.399	899
Mean of logodds	No	0.130	0.410	899
Median	No	0.134	0.418	899
Mean of probabilities	No	0.138	0.439	899
Baseline (p = 0.36)	N/A	0.230	0.652	899

It would be straightforward to extend this analysis with new questions that have been resolved since then, other datasets or new techniques.

Literature review of weight aggregation

Difficulty: easy

When aggregating forecasts, we usually resort to formulas like ∑iailogoi, where oi are the individual predictions (expressed in odds) and ai the weights assigned to each prediction.

Right now I have a lot of uncertainty about what are the best theoretical and empirical approaches to assigning weights to predictions. These could be based on factors like the date of the prediction, the track record of the forecaster or other factors.

The first step would be to conduct a literature review of schemes to weigh experts' predictions when aggregating and ...
Strategy of Inner Conflict
by Jonathan Moregård — 2022-11-15T19:38:02.781Z
Link: https://honestliving.substack.com/p/strategy-of-inner-conflict

Epistemic status: Non-validated conjecture served with a pinch of salt
Category: In-depth analysis of everyday things

Dall-e: “A person with an exploding head, depicted as a nuclear mushroom cloud, digital art“

Do you want to understand why it makes sense to threaten civilian lives with horrific nuclear weapons? Then I can recommend the Game Theory classic “The strategy of conflict” [pdf-link]. The book was written way back in 1981, by Thomas C. Schelling. The lessons are as relevant as ever, including ideas on salary negotiations, hostage situations and traffic.[1]

In this post, I’ll apply concepts from the book on inner conflicts, using the Internal Family Systems framework. We’ll use ideas and concepts from the book to explore things like procrastination, not going to bed on time, and the nature of dessert.

The text will end with some practical ideas for how to circumvent the need for draining inner conflicts. Enjoy!

Part 1: The Strategy of Inner Conflict

Context: Internal Family Systems (IFS)

“Internal family systems” (IFS) is a school of thought within psychotherapy. In internal family systems, you picture internal conflicts as a struggle between different parts[2] of yourself. This conflict can then be resolved using techniques from group counselling.

A smoker might have a short-term oriented part that wants to cope with stress by smoking cigarettes, and a long-term oriented part that wants to avoid dying of lung cancer. The back-and-forth most smokers do while quitting can be visualized as a battle raging between these parts,[3] with power changing hands over time.

And with battle comes strategy. Let’s go through some ideas from “The Strategy of Conflict” to see how we can apply them to inner conflict.

Idea 1: Schelling Points

“The Strategy of Conflict” introduces the concept of a “Schelling Point”.[4] Schelling points are natural/obvious places for "negotiations" to end up. To illustrate the point, let's start with a very special kind of negotiation: negotiation without communication.

Example - without communication

You are separated from your friend while shopping at Megamart. Your friend is mobile-free for obscure contrarian reasons. Communication is blocked, but “negotiation” is not; you’re able to “agree” on where to go by trying to pick an “obvious” meeting point. After some thinking, you both head to the checkout line, where you reunite....