trentmkelly/LessWrong-43k huggingface dataset:
Some Thoughts on Conditional Forecasts – Lessons from the 2020 Election
by Javier — 2023-06-05T11:58:10.606Z

Disclaimer: This post was written as part of my job at Open Phil, but it hasn’t been reviewed closely by anyone else at Open Phil, and the opinions and recommendations I put forward are mine only and don’t reflect Open Phil’s views.

In early 2020, Open Philanthropy commissioned forecasts from Hypermind on five socioeconomic indicators. These forecasts were conditional on the result of the 2020 election, i.e. questions were of the form "What will be the value $X on <date> if Trump <wins / doesn't win>?". The five indicators (all about the US) were: total COVID deaths, GDP, S&P 500, prime-age employment-population ratio, and rank in the World Happiness Report. The resolution dates were EOY 2022 and 2024. Forecasts were submitted between August and November 2020.

Now that the 2022 data is in, I summarize what I see as the key results in the tables and figures below.[1] I will only evaluate the forecasts using the mainline aggregation algorithm evaluated on the last day the forecasting window was open. This means I won’t talk about (i) individual differences in forecasting skill, (ii) how forecasts changed over time, or (iii) the merits of different aggregation methods.

Results

Accuracy

2/5 forecasts were better than chance, as measured by the log score (zero being chance level).
4/5 missed the mark by quite a lot, as measured by the CDF evaluated at the true value (which should be 0.5 on average).

Question	Log score[2]	CDF[3]	Interpretation
COVID Deaths	-inf[4]	1.000	Much worse than chance, wildly optimistic
World Happiness Report Rank	-1.054	0.052	Worse than chance, somewhat pessimistic
Employment to Population Ratio	-2.380	0.995	Worse than chance, pessimistic
S&P 500	2.881	0.418	Beats chance, no evidence of bias
GDP	1.440	0.933	Beats chance, somewhat pessimistic

Differences between conditions

The differences between the two conditions were small in general

Only 1/10 forecasts showed a mean difference between conditions of >10% of the relevant range,[5] and in 6/10 this difference was <5%.[6]
Forecasts were similarly uncertain in both conditions, as evidenced by the fact that their entropies differed by <10% in 8/10.

Forecasters expected worse and more uncertain outcomes under Trump, with a few small exceptions like employment and GDP being marginally better in 2022.

Question	Year	Entropy ratio[7]	Mean difference / range[8]
COVID Deaths	2022	1.015	0.088
2024	1.027	0.

...
5/23
by Celer — 2023-06-05T05:50:07.543Z

AI

Congress had a hearing on AI: Gary Marcus, Sam Altman, and Christina Montgomery.

My takeaways:

Congress sees what happened with social media regulation as a mistake driven by a failure to regulate adequately. There is bipartisan unhappiness on this point. They see AI as analogous in many ways, though the harms vary.

People are taking this seriously.

OpenAI’s thoughts on superintelligence.

Fun game / exercise in prompt engineering

Interesting piece on applying an environmental justice lens to algorithmic accountability. Interesting primarily because the authors are clearly social conservatives. AI is bad because it could “disrupt” existing social relations, without any serious interrogation of whether those disruptions have the possibility of being good. AI could also interfere with “attachment to particular territories”, which is not a particularly subtle critique of the global citizens that international media and easier translations produce. A general trend as new areas become salient is that people wind up with more inconsistent views, relative to the rest of their ideology. Whether it’s committed liberals suddenly concerned that licensing regulation could create a regulatory moat or conservatives horrified at the idea of companies getting to choose to avoid offending people and trying to make regulation to stop them, expect to see more ideological cross-pressures when innovation is high.

Relatedly, if you want one reason why the discussions around race and AI are so weird: white people were underrepresented among AI PhDs in 2020 and falling (AI PhD recipients, US population by age).

Excellent roundup of news on AI, if you’re into general news. I usually recommend against, but understanding what the median educated elite is reading can be useful.

Chapter from a philosophy book on how AI poses an existential risk. One of my big open questions is how to disentangle cohort effects and time effects for the spread of influential EAs. Some of it is clearly that people made plans in college and now they’re paying off. Some of it is clearly that EA concerns, particularly AI, are becoming more salient to the general public.

So, the EU is passing legislation on AI. Draft legislation as of May 5 is here. Here’s my favorite critical piece, published May 13. A very clear website is made by the Future of Life Institute, detailing strengths and weaknesses.

Personal

I will be in...
We Are Less Wrong than E. T. Jaynes on Loss Functions in Human Society
by Zack_M_Davis — 2023-06-05T05:34:59.440Z

These paragraphs from E. T. Jaynes's Probability Theory: The Logic of Science (in §13.12.2, "Loss functions in human society") are fascinating from the perspective of a regular reader of this website:

We note the sharp contrast between the roles of prior probabilities and loss functions in human relations. People with similar prior proabilities get along well together, because they have about the same general view of the world and philosophy of life. People with radically different prior probabilities cannot get along—this has been the root cause of all the religious wars and most of the political repressions throughout history.

Loss functions operate in just the opposite way. People with similar loss functions are after the same thing, and are in contention with each other. People with different loss functions get along well because each is willing to give something the other wants. Amicable trade or business transactions, advantageous to all, are possible only between parties with very different loss functions. We illustrated this by the example of insurance above.

(Jaynes writes in terms of loss functions for which lower values are better, whereas we more often speak of utility functions for which higher values are better, but the choice of convention doesn't matter—as long as you're extremely sure which one you're using.)

The passage is fascinating because the conclusion looks so self-evidently wrong from our perspective. Agents with the same goals are in contention with each other? Agents with different goals get along? What!?

The disagreement stems from a clash of implicit assumptions. On this website, our prototypical agent is the superintelligent paperclip maximizer, with a utility function about the universe—specifically, the number of paperclips in it—not about itself. It doesn't care who makes the paperclips. It probably doesn't even need to trade with anyone.

In contrast, although Probability Theory speaks of programming a robot to reason as a rhetorical device[1], this passage seems to suggest that Jaynes hadn't thought much about how ideal agents might differ from humans? Humans are built to be mostly selfish: we eat to satisfy our own hunger, not as part of some universe-spanning hunger-minimization scheme. Besides being favored by evolution, selfish goals do offer some conveniences of implementation: my own hunger can be computed as a much simp...
Monthly Shorts 8/21
by Celer — 2023-06-05T05:30:07.037Z

Arts

Highlights from the stock phrase to start a fairytale in other languages

Back in the days when it was still of help to wish for a thing —German

Cross seven countries, through the sea of Operencia, through the glass mountains, where the curly-tailed pig delves —Hungarian

I remember something that our father told me and that is this —Iraqw [typo?]

A long long time ago, when goats had feathers —Kazakh

Back when tigers used to smoke tobacco —Korean

Where the water was being strewn and the sand poured —Slovak

In a time already long past, when it was still of use to cast a spell —Esperanto

Once there was, and once there wasn't. In the long-distant days of yore, when haystacks winnowed sieves, when genies played jereed in the old bathhouse, [when] fleas were barbers, [when] camels were town criers, [and when] I softly rocked my baby grandmother to sleep in her creaking cradle, there was/lived, in an exotic land, far, far away, a/an —Turkish

SCP-5031 is a sweet story, about prisons and the Other. For context, SCP stands for Secure, Contain, Protect, and in-universe is an organization that protects humanity from anomalous entities. Many of these entities are hostile, and some of them are incompatible with humanity.

Vampire Flower Language is a romance written about actually remotely sensible people. The characters enter conflict over a deep moral question, and resolve it by growing as people and talking with each other. Strongly recommended: I have been reading it for a while, but this is the month it finished.

vellartrill’s someone who I lost touch with after following them for a while on tumblr, and since then they’ve gone off the deep end, but they do have a beautiful essay on transhumanism

eternity is our rightful inheritance, the stars our appointed clay, space our canvas and matter our ink. and they have been stolen away from us by the petty machinations of the cosmos.

how can we simply accept this, the greatest crime imaginable, committed against every being that has ever lived and ever will live?

Sciences

A Method for Measuring Investigative Journalism in Local Papers is sort of useless by itself, but is just begging to be combined with other datasets. You could do that! I have more projects than I have time for, but am happy to provide guidance to anyone interested but unsure how to start.

I’ve been travelling this month: t...
Ages Survey: Results
by jefftk — 2023-06-05T02:10:06.986Z

A few weeks ago after the

community
meeting with the Department of Children and Families I decided to
run a survey to learn more about the range of ages at which people
generally thought kids might be ready to do various activities without
supervision. Stay home for a few hours? Play at the park alone?
Take public transit? I put something together on

Google Forms and shared
it

on
Facebook and over email. Several friends shared it further, and
over the next two weeks it gathered 219 responses. In reading the
following please remember that this is not a representative sample of
anything: it's heavily oversampling people geographically and socially
similar to me!

The main part of the survey was a series of questions about scenarios
("play in an unfenced backyard", "cross a medium-traffic street",
"spend the night home alone") and asking "what age you'd guess the
typical child in your area could learn to do it without supervision".
Here are the average responses:

The survey also asked "the ages you'd guess for an especially mature
child (90th percentile) or an especially immature child (10th
percentile)", assuming "the child doesn't have any relevant
disabilities." Here's that chart again, with the addition of the
estimates for especially mature or immature children:

Taking medians is helpful for distilling each question down to a few
numbers, but the wide range of responses are interesting. Here are CDF
plots for each question. The way to read these is that at a given
age (x-axis) they tell you what fraction of parents (y-axis) think a
child this old is likely ready. You can think of the previous bar
chart as a collection of the the y=50% slices from each of the charts
below:

I'm also interested in how people's backgrounds relate to their
answers in these scenarios. First, let's look at what demographics we
saw. The median respondent was 40 when they filled out the survey:

Most respondents were female (83%), followed by male (13%), and
non-binary (5%).

Living in an urban area, most of the people I was able to survey also
lived in an urban area:

They overwhelmingly grew up in more suburban areas, though:

Most respondents were parents (88%) and the most common number of
children was 2 (45%):

The median parent's oldest child was 8 and most of the parents had an
oldest child under 18 (92%),...
Meta-conversation shouldn't be taboo
by Adam Zerner — 2023-06-05T00:19:41.015Z

I often find myself in a conversation that feels like it's falling into some sort of failure mode. Some examples:

A few years ago in some job as a programmer, I remember being in this meeting. We were trying to figure out the architecture of some feature that was important and we were behind on. It shouldn't have been that hard, but the people who were working on it just kept running into issues, so someone called a big meeting with maybe 12 people in it to just "get in a room together and figure it out". But even in that meeting we weren't figuring it out. I think a big reason why is because there were too many chefs in the kitchen.
At work a few months ago I recall a meeting where Bob (let's call him), continuously interrupted Alice. In such a way where Alice wasn't really ever able to express her thoughts. And I wanted to hear her thoughts. Bob had objections to all of her thoughts, hence the interruptions, but I didn't think they were strong enough objections to justify the interruptions.
A few years ago there was this conversation that has always stuck with me. I was visiting my mom with my girlfriend. The three of us were chatting. I'm not big on interrupting. I do it when it seems worth it, but try to avoid it. But they just constantly interrupted each other. Like, one of them would be about 80% finished saying what they were trying to say, and the other would interrupt and start talking. I couldn't get a word in.
This one is hard to describe, but... ok. When you're talking to someone, they talk, they finish what they're saying, they pause, and then you talk. Some people pause for, what seems to me, a particularly long period of time, and then continue talking. And during that pause I assume they're done and I start talking, and we end up talking over each other.
A few months ago I was at Enthea Teahouse, hanging out with some fellow rationalists. At first we were all sitting around the same table downstairs, but then as the group got larger, we split up, with half of us moving upstairs. Once we moved upstairs, two non-rationalists joined us. A girl and a guy who were boyfriend and girlfriend. For the next 2-3 hours, the girl dominated the conversation. She probably did about 80% of the talking in a group of maybe 7 of us. It was an energetic, fast-paced, and rushed type of talking. She'd also interrupt a lot. At one point she even mentioned that she gets

...
The ants and the grasshopper
by Richard_Ngo — 2023-06-04T22:00:04.577Z
Link: https://www.narrativeark.xyz/p/the-ants-and-grasshopperhtml

One winter a grasshopper, starving and frail, approaches a colony of ants drying out their grain in the sun, to ask for food.

“Did you not store up food during the summer?” the ants ask.

“No”, says the grasshopper. “I lost track of time, because I was singing and dancing all summer long.”

The ants, disgusted, turn away and go back to work.

One winter a grasshopper, starving and frail, approaches a colony of ants drying out their grain in the sun, to ask for food.

“Did you not store up food during the summer?” the ants ask.

“No”, says the grasshopper. “I lost track of time, because I was singing and dancing all summer long.”

The ants are sympathetic. “We wish we could help you”, they say, “but it sets up the wrong incentives. We need to conditionalize our philanthropy to avoid procrastination like yours leading to a shortfall of food.”

And they turn away and go back to their work, with a renewed sense of purpose.

...And they turn away and go back to their work, with a flicker of pride kindling in their minds, for being the types of creatures that are too clever to help others when it would lead to bad long-term outcomes.

...“Did you not store up food during the summer?” the ants ask.

“Of course I did”, the grasshopper says. “But it was all washed away by a flash flood, and now I have nothing.”

The ants express their sympathy, and feed the grasshopper abundantly. The grasshopper rejoices, and tells others of the kindness and generosity shown to it. The ants start to receive dozens of requests for food, then hundreds, each accompanied by a compelling and tragic story of accidental loss. The ants cannot feed them all; they now have to assign additional workers to guard their doors and food supplies, and rue the day they ever gave food to the grasshopper.

...The ants start to receive dozens of requests for food, then hundreds, each accompanied by a compelling and tragic story of accidental loss—and while many are fraudulent, enough are real that they are moved to act. In order to set incentives correctly, the ants decide to only give food to those who can prove that they lost their food supplies through no fault of their own, and set up a system for vetting claims.

This works well for a time—but as fraudsters grow more sophisticated, the ants’ bureaucratic requirements grow more onerous. In order to meet them, other creatures start to deposit their food in large group storehou...
implications of NN design for education
by bhauth — 2023-06-04T20:50:49.325Z

Even the most basic aspects of the design and training of artificial neural networks seem (to me) to have large implications for the education of humans. Just the fact that complex information can be represented as a point in a shared high-dimensional Euclidean latent space seems to overturn centuries of philosophy and psychology. People used to argue about the Sapir-Whorf hypothesis, but now there are massively multilingual language models that render that debate not just resolved but obsolete.

However, when I try to explain something like latent spaces to someone involved in education, it seems to come across merely as: <math people> invented <math thing> that's <interesting for some nerds>.

The people here are interested in both AI and approaches to learning, right? What's an implication that you think the design and training of neural networks has for education? How would you explain it to ordinary people?
Nature < Nurture for AIs
by scottviteri — 2023-06-04T20:38:07.519Z

This is a cross-link for https://scottviteri.github.io/post/nature-v-nurture-for-ais.

Let's imagine a hypothetical scenario where an AI is somehow trained in a way that is analogous to a human childhood in all of the relevant ways. Maybe it has a loving mother and family, maybe it has to learn to play well with peers, and maybe it has to learn to integrate into a larger social context. My expectation is that most members of the AI alignment community would expect this to create a fundamentally alien kind of intelligence. I will argue contrarily that nurture (the training data and training procedure) matters more than nature (transistors, gradient descent, and transformers) for a sufficiently capable AI. ﻿

This piece will have the following structure:

Counter-arguments to my point that nurture trumps nature for AI’s
A selection effect that prevents the main counter-argument from applying
Relation to the Bitter Lesson

Counter-arguments

Here are some arguments for why the human-trained AI might be alien (nature over nurture):

Empirically speaking, even large, well-fed base models are indeed pretty alien.
But maybe base models + RL are a better analogy for the way that the steering and learning subsystems combine anyway. In which case I expect that you could do some RL training to get a pretty convincing human. It is already the case that 30% of people can't tell if they are talking to a person or not over a two-minute conversation at

https://www.humanornot.ai/

(it's a fun game!).
There are just so many object-level differences between people and modern AI's that it is apriori unlikely to think they should be similar, absent some extra selection effect.

Speed difference
For instance, computers run way faster than human brains on a variety of tasks, and are continuing to speed up. Maybe this has to do with brains optimizing for low energy expenditure rather than speed (can someone confirm by reading

https://www.lesswrong.com/posts/xwBuoE9p8GE7RAuhd/brain-efficiency-much-more-than-you-wanted-to-know

)? Of course, what it means to say that computers are running faster than brains depends on the task -- it is certainly true for calculators and less true for writing a high quality code base or research proposal. But the set of tasks for which the computer is faster is expanding rapidly, and that set will probably continue to expand.
Different learning algorit

...
One implementation of regulatory GPU restrictions
by porby — 2023-06-04T20:34:37.090Z

Some observations:

ML-relevant hardware supply is bottlenecked at several points.
One company, NVIDIA, is currently responsible for most purchasable hardware.[1]
NVIDIA already implements driver licensing to force data center customers to buy into the more expensive product line.[2]
NVIDIA would likely not oppose even onerous regulation on the use of its ML hardware if it gives them more of a competitive moat.

Where can you stick a regulatory lever to prevent improper use of GPUs at scale?

Here's one option![3]

Align greed with oversight

Buff up driver licensing with some form of hardware authentication so that only appropriately signed drivers can run. (NVIDIA may do this already; it would make sense!)
Modify ML-focused products to brick themselves if they do not receive a periodic green light signal from a regulatory source with proper authentication.
Require chain of custody for ML-focused products. If an auditable chain of custody cannot be produced for a particular piece of hardware, stop sending a green light.
Use both targeted and randomized audits to confirm that hardware is being used for the stated purpose. Audits are not primarily automatic: regulators should have on-demand access to the physical hardware and verifiable detailed information on how the system is being used.
If audits are blocked or refused by a product owner, stop sending a green light.
While not a primary form of security, the hardware/driver could self-report on some kinds of usage patterns. It does not seem realistic to collect invasive data[4], but the combination of chain of custody and high-level samples of hardware utilization over time could help corroborate client claims or flag installations for audits.
I imagine NVIDIA would be happy to offer Certified ML-MOAT-2023-B Compliant hardware and drivers. Given the opportunity, they may even push regulators to implement progressively more difficult forms of verification to make it more expensive for smaller hardware providers to break into the market.[5]

Between the cost of the hardware involved and the chain of custody information, it is unlikely that many small businesses would suffer excessive burdens as a result of the regulation. A company with 16 H100's is far less of a risk than a company with 140,000 H100's; you can probably skip the audit unless you have some reason to suspect the ostensibly small installations are attempting to expl...