trentmkelly/LessWrong-43k huggingface dataset:
Paradigms and Theory Choice in AI: Adaptivity, Economy and Control
by particlemania — 2023-08-28T22:19:11.167Z

Epistemic status: Some general thoughts on philosophy of design, that have turned out to be useful for reasoning about AI paradigms for a few years for me.

Summary:

I briefly discuss the role of paradigms in Kuhnian philosophy of science, and the problem of theory choice.
I introduce the notion of design paradigms, when discussing design and engineering. For machine learning, this concept is similar to the concept of

training stories

.

I then introduce three criteria (or theoretical virtues) that design paradigms can be evaluated on, that play a role in shaping theory choice in design and engineering generally, and AI specifically.
I then apply these three criteria to some criticisms of the CAIS paradigm (as compared to AI agents), as a case study.

1. Paradigms in Science

In the philosophy of science, Thomas Kuhn introduced the notion of a ‘paradigm’, roughly referring to a class of key theories, tools and approaches, and methodological and theoretical assumptions that govern progress within some discipline of science. The function of a paradigm, as per

SEP

, is to “to supply puzzles for scientists to solve and to provide the tools for their solution”. Kuhn’s interest in the concept was to provide a commentary on revolutions and incommensurability.

According to Kuhn, scientific revolutions are characterized by disruptions in periods of ‘normal science’ (or within-paradigm progress), wherein new paradigms are introduced leading to a shift in the assumptions, methods and evaluative criteria that govern the discipline. Some examples include the shift from classical mechanics to relativistic mechanics, and the shift from miasma theory to germ theory of disease.

Sometimes paradigms can refer to an overlapping object of inquiry and continue to remain productive, and yet be incommensurable (i.e. lacking common measure) with respect to each other. When the theories provide no basis for evaluation on each other’s standards, and science lacks a common standard to compare them (for example because they have incompatible methodological assumptions, or operate in incompatible ontologies), it can make it hard to provide ‘objective’ justifications for choosing between paradigms. Kuhn called this the problem of Theory Choice, and raised the question of which criteria we do/should use in making subjective choices here.[1]

2. Paradigms in Design and Enginee

...
Humanities In A Post-Conscious AI World?
by Netcentrica — 2023-08-28T21:59:57.848Z

I have a simple question – Are you aware of any institution, public or private, that is engaging in serious research regarding humanities issues in a "post-conscious AI" world? Despite the now thousands of hours I’ve put into researching AI for the eight near-future, science fiction novellas on the subject I have written I have not found one.

Sure it’s entertaining to see social issues regarding AI or companions/androids/social robots explored in games like Detroit: Become Human but I’m curious to know if anyone is doing serious academic research towards the eventuality of conscious AI being present in society. While there is a wide range of opinions, a great many people do seem to believe that it is an inevitability, so where's the research?

I realize there are plenty of opinion pieces that have been written by a variety of individuals from the academic and private sectors but what I am asking about is something more like UC Berkeley’s Center for Human-Compatible AI or its Machine Intelligence Research Institute or even the Cambridge Centre for the Study of Existential Risk but which are instead focused on post-conscious AI social/humanities issues.

I use the term humanities because I write near future stories about AI Companions which cover a wide range of subjects but NOT things like hardware or software. I write stories about whether things like social values, art, spirituality and even human-to-AI metamorphosis are evolutionary inevitabilities.

For this post however by humanities issues I mean things like ethics and justice, in particular things like civil or constitutional law. I have no legal training but am simply interested in the fact that people discuss how we might measure or determine consciousness in AI (https://arxiv.org/abs/2308.08708) but no one seems very interested in researching the eventuality from a social perspective. What happens when it is determined that an AI is in fact conscious?

Should you even be allowed to try to create a conscious AI or should that be regulated and licensed?

If you create a conscious AI can you own it?

Would a conscious AI have the right to citizenship with its inherent rights and freedoms?

If you created one would you have “necessities of life” responsibilities similar to what parents have? https://laws-lois.justice.gc.ca/eng/acts/c-46/section-215.html

I know these questions will li...
Introducing the Center for AI Policy (& we're hiring!)
by Thomas Larsen — 2023-08-28T21:17:11.703Z
Link: https://www.aipolicy.us/blog/hiring

Summary

The Center for AI Policy

is a new organization developing and advocating for policy to mitigate catastrophic AI risks.
We are hiring for a

Government Affairs Director

and a

Communications Director

. We’re also open to other roles. The deadline to apply is October 30th, 2023.

About Us

The Center for AI Policy is a new DC-based organization developing and advocating for policy to mitigate catastrophic AI risks.

Our current focus is building capacity in the US government to safeguard AI development. Our proposed legislation would establish a federal authority to monitor hardware and license frontier AI development, ensuring we can identify and respond to risks. It would also create strict liability for severe harms caused by AI systems, increasing accountability and improving incentives to developers.

Our team includes Thomas Larsen (Executive Director), a former technical AI safety researcher; Jason Green-Lowe (Legislative Director), a lawyer and policy analyst; Jakub Kraus (Operations Director), who has a computer science background; and Olivia Jimenez (Chief of Staff), who has an AI policy and field building background. We’re advised by experts from other organizations and supported by several volunteers.

How the Center for AI Policy differs from other AI governance organizations

Many AI governance organizations are focused on doing research and building up infrastructure/credibility that can be used later. We’re focused on developing and advocating for significant, shippable policy now. We want to harness the current energy to pass meaningful legislation during this policy window, in addition to building a coalition for the future. While we engage in conversation with policymakers about a diverse range of AI risks, we are also upfront about our focus on catastrophic risk.

We’re hiring

We think we’re strong at developing policies that would significantly reduce catastrophic risk if passed. To get these policies passed, we need to scale our efforts and bring in more advocacy, policy, and DC experience.

That’s why we’re hiring a

Government Affairs Director

and a

Communications Director

. Our Government Affairs Director will design and execute our strategy for passing meaningful legislation. Our Communications Director will design and execute our strategy for promoting our ideas. For more informatio...
45% to 55% vs. 90% to 100%
by yhoiseth — 2023-08-28T19:15:18.524Z

In Never Split The Difference: Negotiating As If Your Life Depended On It, Chris Voss discusses cognitive biases:

Through decades of research with Tversky, Kahneman proved that humans all suffer from Cognitive Bias, that is unconscious—and irrational—brain processes that literally distort the way we see the world. Kahneman and Tversky discovered more than 150 of them.

There’s the Framing Effect, which demonstrates that people respond differently to the same choice depending on how it is framed (people place greater value on moving from 90 percent to 100 percent—high probability to certainty—than from 45 percent to 55 percent, even though they’re both ten percentage points) (p. 12).

Isn’t it rational to value 90% → 100% more than 45% → 55%?

Even going from 90% to 95% means you are wrong half as often — 120 instead of 110 — whereas going from 45% to 55% only removes about 20% of your errors.

Is my thinking and math correct? If not, how am I wrong?

Assuming I’m right, I would also really appreciate a better way to explain this.
Information warfare historically revolved around human conduits
by trevor — 2023-08-28T18:54:27.169Z

Epistemic status: This is a central gear in any model of propaganda, public opinion, information warfare, PR, hybrid warfare, and any other adversarial information environment. Due to fundamental mathematical laws, like power law distribution, not all targets in the battlespace are created equal. This is important for defensive purposes.

______________________________________________

Generally, when people think about propaganda, censorship, public opinion, PR, and social psychology, they tend to think of the human receivers/viewers/readers/listeners as each being the ultimate goal— either the message is accepted, ignored, or rejected, either consciously or subconsciously, and each person either gains your side a single a point or loses you a point.

This is actually a bad model of large-scale influence. People with better models here are usually more inclined to use phrases like "steering public opinion", because the actual goal is to create human conduits, who internalize the message and spread it to their friends in personal conversations.

Messages that spread in that way are not only personalized influence by-default, even if initiated by early 20th century technology like radio or propaganda posters, but messages are also perceived as much more trustworthy when heard directly from the mouth of a friend than from one-to-many communication like broadcasting or posters, which were unambiguously spread by at least one elite with the resources to facilitate that and who you are not allowed to meet, even if they seem like a person just like you (although many people develop positive feelings towards messages from a likable or relatable persona, such as bumbling/buffoon politicians or critical voices of justice). Propaganda posters totally fell out of style, because it was plain as day that they were there to influence you; radio and television survived, including as a tool of state power, because they did not stick out so badly.

Censorship, on the other hand, is the prevention of this dynamic from emerging in the first place, which is likely a factor explaining why censorship is so widely accepted or tolerated by elites in authoritarian countries.

The battlespace

It's important to note that not all human conduits are created equal. This dynamic ultimately results in intelligent minds not merely relaying the message, but also using their human intelligenc...
The Evidence for Question Decomposition is Weak
by niplav — 2023-08-28T15:46:31.529Z

Question decomposition appears to be a relatively common method for forecasting, see Allyn-Feuer & Sanders 2023, Silver 2016, Kaufman 2011 and Hanson 2011, but there have been conceptual arguments against this technique, see Yudkowsky 2017 and Gwern 2019, which both state that it reliably underestimates the probability of events.

What is the empirical evidence for decomposition?

Lawrence et al. 2006 summarize the state of the field:

Decomposition methods are designed to improve accuracy by splitting the judgmental task into a series of smaller and cognitively less demanding tasks, and
then combining the resulting judgements. Armstrong (2001) distinguishes between decomposition, where the breakdown of the task is multiplicative (e.g. sales forecast=market size forecast×market share forecast), and segmentation, where it is additive (e.g. sales forecast=Northern region forecast+Western region forecast+Central region forecast), but we will use the term for both approaches here. Surprisingly, there has been relatively little research over the last 25 years into the value of decomposition and the conditions under which it is likely to improve accuracy. In only a few cases has the accuracy of forecasts resulting from decomposition been tested against those of control groups making forecasts holistically. One exception is Edmundson (1990) who found that for a time series extrapolation task, obtaining separate estimates of the trend, seasonal and random components and then combining these to obtain forecasts led to greater accuracy than could be obtained from holistic forecasts.
Similarly, Webby, O’Connor and Edmundson (2005) showed that, when a time series was disturbed in some periods by several simultaneous special events, accuracy was greater when forecasters were required to make separate estimates for the effect of each event, rather than estimating the combined effects holistically. Armstrong and Collopy (1993) also constructed more accurate forecasts by structuring the selection and weighting of statistical forecasts around the judge’s knowledge of separate factors that influence the trends in time series (causal forces).
Many other proposals for decomposition methods have been based on an act of faith that breaking down judgmental tasks is bound to improve accuracy or upon the fact that decomposition yields an audit trail and hence a defensible rationale

...
The Anthropic Principle Tells Us That AGI Will Not Be Conscious
by nem — 2023-08-28T15:25:28.569Z

More specifically, the Anthropic Principle tells us that AGI/TAI are unlikely to be conscious in a world where 1) TAI is achieved, 2) Alignment fails, and 3) the 'prominence' of consciousness scales with either increasing levels of capability or with a greater number of conscious being/the time of their existence.
The argument is simple. If the future is filled with artificial intelligence of human origin, and if that AI is conscious, then any given observer should expect to be one of those AIs. This means that, on balance, one of the following is likely true:
1) The anthropic principle does not hold.
You and I, as observers, are simply an incredibly unlikely, but also inevitable exception. After all, in a world where pre-historic humans contemplated the Anthropic Principle, they would have concluded the unlikelihood of modern civilization.
Or perhaps the principle doesn't hold because it is simply inaccurate to model consciousness as a universal game of Plinko.
2) There are many AI consciousnesses alongside biological consciousnesses in spacetime.
This indicates perhaps that alignment efforts will succeed. However, this introduces another anthropic bind, this time in relation to humanity's current single planet, type 0 status.
3) There are not that many AI consciousnesses throughout spacetime.
This could support the conclusion that humanity will not create TAI.
In certain models, it could also indicate that any AI consciousness will be concentrated in a relatively small number of minds, and that for the purposes of the Anthropic Principle, quantity of minds is more important than some absolute 'level' of consciousness.
Most saliently to me, is the slight update towards the possibility that whatever minds will populate the universe for the majority of time will not be conscious in a way that is applicable to the Anthropic Principle.
This post is just a musing. I don't put much weight behind it. I am in fact most inclined to believe #1, that the Anthropic Principle is not a good model for predicting the future of a civilization from inside itself. However, I have never seen this particular anthopic musing, and I wanted to, well... muse.
Please chime in with any musings of of your own.
No More Freezer Pucks
by jefftk — 2023-08-28T15:20:02.929Z

A while ago I wrote about how I'd make

freezer pucks:

These work ok, but there are a couple issues:

A large batch requires a lot of muffin tins

Cleaning muffin tins is annoying

It can be hard to get them out of the tins

I've switched to a different system where I freeze a flat tray in a
thin enough layer that I can break it easily:

It takes a bit of practice getting the layer thickness right:
generally the higher the water content the thinner it needs to be if
you don't want an awkward block.

I put a layer in a freezer bag:

Before putting in additional layers I put down a scrap of parchment
paper or wax paper:

It's easier and faster than the puck method, though with higher risk
of accidentally ending up with a block and needing to defrost some.

Comment via: facebook, mastodon
The mind as a polyviscous fluid
by Bill Benzon — 2023-08-28T14:38:26.937Z

Cross-posted from New Savanna.

About a year ago I uploaded a post with a typically ungainly title, The structured physical system hypothesis (SPSH), Polyviscous connectivity [The brain as a physical system]. It’s that word, “polyviscous,” that’s got my present attention. Since then I’ve done a number of posts using that idea, whatever it is. This is another of those ideas.

So, viscosity. Honey is more viscous than, say, water. It flows more slowly, much more. What happens if you drop a lump of honey into a tumbler of water? It sinks to the bottom in a continuous lump and flattens out along the bottom. It will begin to diffuse into the water along the boundary, but I don’t know how long, if ever, it will take to mix completely. Now, put a stick down into the tumbler until it extends into the honey. Give is a stir or three, but no more. Now you’ll have gobs and threads of honey mixed in with water in a complex and somewhat irregular and ragged way. That’s a simple polyviscous fluid. It has regions of relatively high viscosity and other regions of relatively low viscosity. Now imagine a fluid with 5, 10, 27, 48, and more different levels of viscosity, from all but solid like cold tar through the wispiest whatever. Polyviscosity.

As the title of the year-old post indicates, I was thinking in terms of connectivity:

Thus I say that the cortical network as a whole exhibits polyviscous connectivity. What do I mean? Some connections are highly resistant to change, and thus have high viscosity. Others change quite readily, and have low viscosity.

OK. Now let’s shift our thinking just a bit and think of the mind as a polyviscous fluid. The mind, as the saying goes, is what the brain does. And that is very complex.

Imagine that you’re watching a movie, make it a Hong Kong martial arts movie. Your mind is entrained to the images on the screen. During a fight scene the level of mental viscosity is relatively how. The fight is over and the hero rests, contemplating the sunset, let’s say. The viscosity is somewhat higher.

Yet, while you’re entrained by the film, you’re not completely absorbed into. While the hero contemplates the sunset, you take a bit of popcorn. And maybe you were munching furiously during the fight. So, even as you were watching the film you slipped in some mental popcorn “frames” among the film frames. Very slippery, low viscosity.

When I wrote my book on...
Who can most reduce X-Risk?
by sudhanshu_kasewa — 2023-08-28T14:38:07.188Z

Just wanted to poll the world about who we think are most able to reduce X-Risks (even if they currently aren't doing so). Folks who are most able to but aren't seem like the most crucial ones to reach out to.

Please add only one person (or collection of people) per answer.

Please check the list to see if your submission isn't already mentioned. If so, please upvote them rather than add them a second time.
Dear Self; we need to talk about ambition
by Elizabeth — 2023-08-27T23:10:04.720Z

I keep seeing advice on ambition, aimed at people in college or early in their career, that would have been really bad for me at similar ages. Rather than contribute (more) to the list of people giving poorly universalized advice on ambition, I have written a letter to the one person I know my advice is right for: myself in the past.

The Letter

Dear Past Elizabeth,

Your life is, in some sense, a series of definitions of success.

First you’re in early school, and success is defined for you by a handful of adults. You go where they say, do the assignments they say, when they say, and doing well means meeting the goals they set for you. Even your hippie elementary school gives you very few choices about life. You get choices in your leisure activity, but that (as they have explained to you) is leisure and thus unimportant, and there’s no success or failure in it.

Then you get further in school, and the authorities give you some choice over the hoops you jump through. You can choose which book you write your report on or even what classes you take (within a predetermined set). This feels like freedom, but you’re in still a system someone else designed and set the win conditions for. You can fulfill a college distribution requirement with any history class at all- but you are going to take one, and the professor is the one determining if you succeeded at it.

More insidiously, you’ll like it. Creating your own definition of success feels scary; enacting it feels impossible. The fact that school lays out neat little hoops for you to jump through is a feature.

Work (you’ll be a programmer) is where things get screwy. Programming contains multiple definitions of success (manager, principal, freelancing, development, testing, bigtech, start-up, money-maxing, altruistic projects…), and multiple ways to go about them. If your goals lie outside of programming altogether (art, parenting, travel..), it’s relatively easy to work out a way to fund it via programming while still having the time to do what you want. Not trivial, but have you seen what people in other jobs go through? With programming it’s at least possible.

But you like hoops. You’re comfortable with hoops. So you’re going to waste years chasing down various definitions of success within programming, and by the time you give up will be too exhausted to continue in it at all. I think you (I) s...