trentmkelly/LessWrong-43k huggingface dataset:
Pointing to a Flower
by johnswentworth — 2020-05-18T18:54:53.711Z

Imagine I have a highly detailed low-level simulation (e.g. molecular dynamics) of a garden. The initial conditions include a flower, and I would like to write some code to “point” to that particular flower. At any given time, I should be able to use this code to do things like:

compute a bounding box around the flower
render a picture which shows just the flower, with the background removed
list all of the particles which are currently inside the flower

Meanwhile, it should be robust to things like:

most of the molecules in the flower turning over on a regular basis
the flower moving around in space and/or relative to other flowers
the flower growing, including blooming/wilting/other large morphological change
other flowers looking similar

That said, there’s a limit to what we can expect; our code can just return an error if e.g. the flower has died and rotted away and there is no distinguishable flower left. In short: we want this code to capture roughly the same notion of “this flower” that a human would.

We’ll allow an external user to draw a boundary around the flower in the initial conditions, just to define which object we’re talking about. But after that, our code should be able to robustly keep track of our particular flower.

How could we write that code, even in principle?

“Why Not Just… ”

There’s a lot of obvious hackish ways to answer the question - and obvious problems/counterexamples for each of them. I’ll list a few here, since the counterexamples make good test cases for our eventual answer, and illustrate just how involved the human concept of a flower is.

Flower = molecules inside the flower-boundary at time zero. Problem: most of the molecules comprising a flower turn over on a regular basis.
Flower = whatever’s inside the boundary which defined the flower at time zero. Counterexample: the flower might move.
Flower = things which look (in a rendered image) like whatever was inside the boundary at time zero. Counterexample: the flower might bloom/wilt/etc. Another counterexample: there may be other, similar-looking flowers.
Flower = instance of a recurring pattern in the data, defined by clustering. Counterexample: there may not be any other flowers. (More generally: we can recognize “weird” objects in the world which don’t resemble anything else we’ve ever seen.)
Flower = region of high density contiguous in space-time with our initial region. Counterexampl

...
Why Artists Study Anatomy
by Sisi Cheng — 2020-05-18T18:44:23.109Z

I’ve been interested in drawing ever since I was a little kid. I like drawing people. I practiced a lot by looking at a cool picture of a character I like, and copying it as closely as I can.

The better I got at copying, the better my work became. I got some great results this way. I even advanced to the level where I could take photos of real-life people, then painstakingly copying it to produce a portrait. I was pleased by and proud of many of my works. Many agreed that I was “such a good artist, wow!”

Here’s one piece I was particularly proud of; my sketch on the left, the reference on the right.

In middle school, I took on a new challenge: I wanted to draw commissions. People would give me a character design, sometimes with a few reference images, and I would draw their characters. This was really hard for some reason…

I had no images to copy directly from. My results were inconsistent. I spend a lot of my time tweaking things like moving around the eyes, redrawing the nose, etc. After a bunch of commissions, I got reasonably decent at drawing a character at the ¾ angle, the most common one for portraiture. I did hundreds of commissions, almost all of them at that angle. Soon, I felt uncomfortable drawing faces at any other angle.

Some examples of the stuff I drew. Some of them look kinda strangely similar, hmm...

I stopped doing art for a while because college was busy. When I started again, I took

an anatomy class

. I learned a lot of great things. But the biggest lesson I learned was that I’ve been approaching drawing all wrong. The biggest thing I had missed was that I had to understand the process that generates the image.

The art an artist draws is a projection of the 3D object onto a 2D surface. By practicing by only copying from 2D images without studying the 3D structures that generated them, I treated my reference images as being generated by some black box, and ignored the underlying gears and structures.

For example, the shape of an eye is determined by the 3D structures of the eyeball projected into 2D. I never learned what an eye looks like in 3D, so I didn’t know how to draw it from an angle different from the 2D references I’ve seen. It turns out, the eye isn’t a flat almond-shape like it seems in a picture. It’s a sphere inset into the skull, wrapped from the top and bott...
Extracting Value from Inadequate Equilibria
by  — 2020-05-18T17:25:09.781Z

[Much expanded from my comment here. Pure speculation, but I’m confident that the bones of this make sense, even if it ends up being unrealistic in practice. Cross-posted from Grand, Unified, Crazy.]

A lot of problems are coordination problems. An easy example that comes to mind is scientific publishing: everybody knows that some journal publishers are charging ridiculous prices relative to what they actually provide, but those journals have momentum. It’s too costly for any individual scientist or university to buck the trend; what we need is coordinated action.

Eliezer Yudkowsky talks about these problems in his sequence Inadequate Equilibria, and proposes off-hand the idea of a Kickstarter for Coordinated Action. While Kickstarter is a great metaphor for understanding the basic principle of “timed-collective-action-threshold-conditional-commitment”, I think it’s ultimately led the discussion of this idea down a less fruitful path because Kickstarter is focused on individuals, and most high-value coordination problems happen at the level of institutions.

Consider journal publishing again. Certainly a sufficient mass of individual scientists could coordinate to switch publishers all at once. But no matter what individual scientists agree to, this is not a complete or perfect solution:

Almost no individual scientists are paying directly for these subscriptions – their universities are, often via long-term bulk contracts.
University hiring decisions involve people in the HR and finance departments of a university who have no interest in a coordinated “stop publishing in predatory journals” action. They only care about the prestige and credentials of the people they hire. Publications in those journals would still be a strong signal for them.
Tenure decisions involve more peer scientists than hiring, but would suffer at least partly from the same issue as hiring.

What’s needed for an action like this isn’t a Kickstarter-style website for scientists to sign up on – it’s coordinated action between universities at an institutional level. Many of the other examples discussed in Inadequate Equilibria fit the same pattern: the problems with healthcare in the U.S. aren’t caused by insufficient coordination between individual doctors, they’re caused by institutional coordination...
The Mechanistic and Normative Structure of Agency
by Gordon Seidoh Worley — 2020-05-18T16:03:35.485Z
Link: https://philpapers.org/rec/WINTMA-3

Winning, Jason (2019). The Mechanistic and Normative Structure of Agency. Dissertation, University of California San Diego.

Abstract:

I develop an interdisciplinary framework for understanding the nature of agents and agency that is compatible with recent developments in the metaphysics of science and that also does justice to the mechanistic and normative characteristics of agents and agency as they are understood in moral philosophy, social psychology, neuroscience, robotics, and economics. The framework I develop is internal perspectivalist. That is to say, it counts agents as real in a perspective-dependent way, but not in a way that depends on an external perspective. Whether or not something counts as an agent depends on whether it is able to have a certain kind of perspective. My approach differs from many others by treating possession of a perspective as more basic than the possession of agency, representational content/vehicles, cognition, intentions, goals, concepts, or mental or psychological states; these latter capabilities require the former, not the other way around. I explain what it means for a system to be able to have a perspective at all, beginning with simple cases in biology, and show how self-contained normative perspectives about proper function and control can emerge from mechanisms with relatively simple dynamics. I then describe how increasingly complex control architectures can become organized that allow for more complex perspectives that approach agency. Next, I provide my own account of the kind of perspective that is necessary for agency itself, the goal being to provide a reference against which other accounts can be compared. Finally, I introduce a crucial distinction that is necessary for understanding human agency: that between inclinational and committal agency, and venture a hypothesis about how the normative perspective underlying committal agency might be mechanistically realized.

I have not had a chance to read this, and my time is rather constrained at the moment so it's unlikely I will, but I stumbled across this, and it piqued my interest. Better understanding of agency appears important to the success of many research programs in AI safety, and this abstract matches enough of the pattern of what LW/AF has figured out matters about agency that it seems well worth sharing.

Full text of the dissertation here.
Reward functions and updating assumptions can hide a multitude of sins
by Stuart_Armstrong — 2020-05-18T15:18:07.871Z

1 Introduction

This post started off as a response to a pair of comments by Rohin Shah, and by Charlie Steiner. But the insights are (I hope) sufficiently general that I made them into a full post.

EDIT: It seems I misunderstood Rohin's point. In any case, I'll keep this post up, because the things I wrote in response to the things I thought Rohin said, are useful things to have written out.

In my original post, I pointed out that since an agent's preferences are not simply facts about the universe, "updating" a prior requires bridging assumptions - something that tells the AI how to update human preferences, depending on what the human does.

It needs to know that:

If the human goes to a restaurant, orders Sichuan Hot Pot, eats it, comments on how much they enjoyed it, goes back to the restaurant again and orders Sichuan Hot Pot after recommending it to their friends... this is at evidence that they like Sichuan Hot Pot.

However, if that restaurant is run by the mob, and the human owes money to the prominent mobster that owns the restaurant, and that mobster occasionally cooks the Sichuan Hot Pot themselves... then this isn't really evidence of much. Same goes if the restaurant is owned by the human's boss, a prominent local politician with power over something the human wants, the human's probation officer, and so on.

Similarly, to 2., if the restaurant is owned by the human's child, or their lover, or a close friend - though for subtly different reasons, and to different extents.

Anyway, not to belabour the point, but the AI needs some bridging assumptions so that it can take context like that into account when figuring out human preferences.

Rohin Shah and Charlie Steiner pointed to some ways that CIRL (Cooperative Inverse Reinforcement Learning) might get round this problem.

Thanks for those responses! They're very useful. Still, I'll argue in this post that the problems still remain (and that the only of solving them is to solve the hard problems of what human preferences and meta preferences are, reproducing human theory of mind in AI format, without being able to use machine learning for the key assumptions).

2 Rigging learning or reward-maximisation?

Rohin Shah's comment starts out with:

The key point is that in a CIRL game, by construction there is a true (unknown) reward function, and thus an optimal policy must be viewable as being Bayesian about th

...
English Bread Regulations
by jefftk — 2020-05-18T13:20:02.672Z

Yesterday I stumbled on the

Assize of
Bread, which set the price of bread in England by the price of
wheat with rules like:

When a quarter of wheat is sold for 18d., then wastel bread of a
farthing white and well-baked shall weigh £4 10s. 8d.

A 'quarter'
here means eight bushels, or a quarter of a ton. This is a large
amount of wheat, which makes sense since it's the baker buying.
'Wastel' bread meant good-quality white bread, and a 'farthing' is a
quarter of a penny. In the British tradition of using confusingly
similar terms for weight and currency, '£4 10s 8d' is a weight.
With 20 pennies (d) to the shilling and 12 shillings
(s) to the pound (£), that's 4.53 Tower
pounds, which would be 3.49 lb
(international avoirdupois pound, as used in the US).

I had thought the standard bread would be something cheaper than
white bread, and this does look like the case later, but here
cheaper breads are all given with prices relative to white bread (Ross 1956):

When the Quarter of Wheat is sold for 12d., then the
Farthing Wastel Loaf shall weigh six pounds and sixteen shillings; the
Cocket Loaf of the same corn and the same fineness of sieving shall
weigh two shillings more than the Wastel. [The Cocket Loaf] of corn of
less price shall weigh five shillings more than the Wastel. And the
Simnel Loaf shall weigh two shillings less than the Wastel because it
has been baked twice. The Farthing whole-loaf of wheat shall weigh a
Cocket and a half. And the Treet Loaf shall weigh two Wastels. And the
Loaf de omni blado shall weigh two Cockets.

The rule gave different loaf weights for different wheat prices:

Each row is a wheat price (in shillings and pence per quarter ton) and
the columns are for different sources. For example, if a quarter ton
of wheat cost six shillings, most sources would give a £1 2s 8d
(£1.2) weight for a farthing (quarter penny) loaf.

We can also convert this into modern prices. The commodity price of
wheat today is $5/bushel ($40/quarter) and I'm going to assume the
bushel hasn't changed size much. The table says that if a quarter of
wheat costs six shillings, then a farthing gets you a 1.2 Tower pound
(15oz avoirdupois) loaf. Six shillings is 480 farthings, so
a $40 quarter gives a $0.08 15oz loaf, or $0.1/lb. Cheap white bread
in the grocery store today is about ten times the cost. I expect this
mostly ...
"Starwink" by Alicorn
by Zack_M_Davis — 2020-05-18T08:17:53.193Z
Link: http://alicorn.elcenia.com/stories/starwink.shtml

A novelette-length adaptation of Eliezer Yudkowsky's "That Alien Message", by Alicorn. (August 2019, ~8800 words)
Isn't Tesla stock highly undervalued?
by Liron — 2020-05-18T01:56:58.415Z

I'm asking this here because I'm by no means an expert on Tesla or stock picking. My only relevant skill is "order-of-magnitude perspective", a general ability to notice when a quantity seems to be out of whack by an order of magnitude.

Right now I'd guess there's a ballpark 50% chance that Tesla's market cap surpasses $1T by end of 2027, and 30% chance it surpasses $2T. Therefore my expected value for the stock price in 2027 is about $1T, or a 30%/yr return if purchased at today's $150B market cap.

Here's why I think Tesla is a clear pick to be one of the top 5 most valuable companies in the world within 10 years:

Their 5+ year lead in market-proven designs of electric cars, and of the factories that make those cars is, by itself, a really exciting thing to bet on
Their 5+ year lead in autonomous driving is, by itself, a really exciting thing to bet on
Their 5+ year lead in battery technology and manufacturing is, by itself, a really exciting thing to bet on. Even if they were to exit the car industry, they could earn $100B+/yr revenue making batteries for countless applications
Model 3 is the most popular electric vehicle in the world by far, and there's still plenty of room for quality improvement and price lowering.
Tesla is already profitable, yet are still many ways they can increase their gross margins, the most obvious of which is simply scaling up and optimizing everything further (they're the youngest major car company and have done this the least)

And here's why I think other investors are currently undervaluing it:

Most consumers still haven't considered buying a Tesla and are unaware of its key benefits such as safety, race-car-like acceleration, and rapidly increasing semi-autonomous driving capabilities. Even though Tesla stock is currently worth 2-3x more than the rest of the US car industry combined, it's still very much *not* a household name today (for neither consumers nor investors), compared to where it's about to be in 5 years or less.
Model 3 is still just an early version of what Tesla is capable of. Their Cybertruck, semi truck, and future sedans are going to be even bigger hits in the market.
Tesla is probably only a few months away from meeting the profitability requirements to be listed on the S&P, at which point a bunch of funds will automatically buy in and its

...
Replication Dynamics Bridge to RL in Thermodynamic Limit
by Past Account — 2020-05-18T01:02:53.417Z

Epistemological Status: I ran a basic simulation and this checks out. Specific details need refinement. Probably just a curiosity, but I think there's a chance there's a deeper connection here. I opted to post in the earlier stages incase I'm missing something big.

In biology, the quasispecies equation is used to model population structures of viruses. States are usually interpreted as genotypes for an individual. Commonly, these states represent proto-viral DNA/RNA strands. Transitions are mutations between these genotypes. Usually, the state genotype confers the replication rate and transitions are equivalent to mutations. The literal equation states that we can study the population dynamics π with the mutation matrix Q using,
πt+1=Qπt
Let's switch to the MDP setting. Assume the actions of the individual(s) have a deterministic effect on the environment transitions. We're going to interpret the rewards as a fitness score allowing the agent to continue propagating. First, let the transition matrix for the system be given as Tij. Second, transform each reward to the fitness rij→e1λrij. The Quasispecies formala relates the population of individuals at each state after one stage of replication after we set Qij=Tijrij.

The individuals aren't intelligent. Instead, the fitness controls the replication rate of transitions. If r=0 then the transition is neutral and the number of individuals collected on a state is neither amplified nor diminished. If r≪0 then the transition is extremely harmful and if r≫0 the transition is extremely helpful.

Notice that we can study the space of all possible transitions and conclude that,
πt(st)=∑st−1Qst−1,stπt−1(st−1)=∑st−2∑st−1Qst−1,stQst−2,st−1πt−2(st−2)=…=∑s0,…,st−1(Πtk=1Qsk−1,sk)π0(s0)
To make further progress, remember that actions have deterministic effects so it's okay to assume individuals are fully random in their exploration. This allows us to simplify the product into,
Πtk=1Qsk−1,sk=Πtk=1Tsk−1,sk^rsk−1,sk=t∏k=1e1λrsk−1,sk=e1λ∑krsk−1,sk
In words, we have decomposed the evolution of the population into a summation over all the paths an agent could take through the system. The twist, is that each path is weighted by an exponential term proportional to the reward that path receives from the environment.

Philosophically, this has the same spirit as the path integral approach used in physics. If we send λ→0 in the path integral, ...
Effective altruism and criticism toward activism: Answer to a paradox
by Arthur Milchior — 2020-05-17T21:15:09.354Z

Since a little while now, I have been exploring the notion of Effective Altruism - EA for short. My readings on the topic so far have been very interesting [1], and I would like to add my own idea that I deem important and have yet to read elsewhere. If ever this has been written down somewhere, I can at least attest to it being all well too hidden. Personally, I believe that it should be discussed in introductions to the EA topic.

Why is it bad form to critic others' actions

In the groups I have been in, criticizing the actions of another volunteer or activist is very unwelcome. Since the very notion of telling others how they should conduct their own actions is at the core of Effective Altruism, it would then follow that EA as a whole would be unwelcome in those groups. So I strived to reconcile the two. To do that, I first needed to understand why - precisely - telling someone else not to act is bad form. I know of two reasons:

Do not act

The first one is that criticisms about someone else's action often come from people who do seemingly nothing. It is way harder to act than not to, and criticizing is easy. Since a lot of those criticisms are non-constructive, effectively following them would mean not doing anything at all. And people not doing anything at all is not - generally - a better situation then them following through their actions.

It is worthy of note that the advice being given is not usually "Stop". But it can for instance come under the form of "You're harming our cause". Typically, it is said systematically to other activists who are said to "give bad press to the group". This may happen when they would get worked up, rise their voices, insult others, tag, break glasses, protest on strikes, and so on...

Your struggle is not important

The second one, more dire, is that a lot of those criticisms stems from struggle appropriation. I have often heard the story of racialized LGB or T friends acting on LGBT issues that they would get told constantly -by their family or other racialized peers- that they are merely wasting their time and should focus on what really matters: Racism. Sometimes, they even lack the imagination to perceive the activist as belonging to both groups, since being non-white does not follow the LGBT stereotypes.

Likewise, I have heard that social unions have been violently rejecting homosexual issues for a long time, as being

...