Talk:Utility monster

message: This has no citations, no indications of notability... It could be merged with either Nozick or one of the utilitarianism articles. I would probably favour expanding the section in Nozick's article and having it redirect to there (as with e.g. non-overlapping magisteria). Richard001 (talk) 11:41, 13 May 2008 (UTC)
thread_title: Worthy of an article?

message: "Nozick's criticism of utilitarianism relies on a cardinal rather than ordinal definition of utility.  Cardinal utility, which is quantifiable and comparable interpersonally as proposed originally by Jeremy Bentham, has been rejected by most utilitarians since John Stuart Mill.  Ordinal utilitarianism would hold the interpersonal, or in this case intermonstropersonal, comparison needed to justify "that we all be sacrificed in the monster's maw" as invalid."
Who says that? Please cite a quality source for that statement. And don't use wikipedia to express your own opinion in an article! 
188.23.64.27 (talk) 22:17, 3 May 2009 (UTC)
thread_title:  No original research and no opinions please! 

message: The notion that mere carelessness of programmers would lead to creation of a well grounded concept of real world paperclip (and a well defined count of 'real world paperclips'), resistant to any potential reinterpretation by a superhuman intelligence, has nothing to do with utility monster. It is also a fringe view coming from people that know very little about AI programming. 78.60.253.249 (talk) 12:05, 25 November 2012 (UTC)
replies: text: I don't particularly see it as a fringe view, given that it's discussed—in conjunction with Parfit—in academic publications. I guess I understand your objection to the scenario, but I think the hypothetical is clearly relevant: that a powerful intelligence without the flexibility of human moral reasoning might prioritize some outcome that most humans would consider horrifying. Do you really think the article is better off with this section excluded? I'd like to restore it, but I'll wait a little while for other comments. groupuscule (talk) 23:57, 26 November 2012 (UTC)
replies: text: Well, the primary issue is that it has nothing to do with "utility monster". Two folks cited are both soliciting donations for them preventing the destruction of the world due to the aforementioned carelessness, too. I think it should be treated the same as commercial linking irrelevant to the subject of the article in general. 78.60.253.249 (talk) 16:00, 28 November 2012 (UTC)
replies: text: What is relevance of "Sara Goldsmith, "The Perfection of the Paper Clip", Slate, 22 May 2012" ? 78.60.253.249 (talk) 03:06, 29 November 2012 (UTC)
replies: text: To help establish outside notability for the "paperclip monster" concept. I understand your concern about Yudkowski and Bostrom, but ultimately I think this is an ad hominem argument that could be leveled at many scholars. I don't see how it diminishes the relevance of the thought experiment. groupuscule (talk) 03:57, 29 November 2012 (UTC)
replies: text:  The relevance of it to the article is zero. One naturally looks why irrelevant, non notable (1 minor reference in Slate?) things are added to the article. Typically there is financial incentive. Indeed, there is: the only two people you are linking to that actually discuss it, are taking donations for this. That's clearly not neutral coverage. Normally, for a notable concept you could e.g. find a philosopher that is not soliciting donations (let alone soliciting donations via fear mongering), and link that. 78.60.253.249 (talk) 08:33, 30 November 2012 (UTC)
replies: text:  It looks like the paperclip maximizer has been removed which is probably for the best. I strongly disagree that the paperclip monster is a niche concept; it's the canonical example in CS ethics about the effects of building an unsupervised general AI which optimizes for some outcome which is orthogonal or secondary to the primary outcomes that humans want -- yes, we want our paperclip AI to build as many paperclips as it reasonably can, but not at the cost of accumulating all the processing power and wealth in the world and effectively enslaving and harvesting the human race and later eliminating life on earth to secure more resources for paperclip production. It's a similar style of argument to the utility monster but I don't actually think it's the same; the whole point is that the paperclip AI generates a massive amount of negative utility because it doesn't have any mechanism to calculate utility, only to calculate the rate of paperclip production. If the paperclip AI could calculate utility correctly then it should stop producing more paperclips when the utility to the population of creating new paperclips reaches equilibrium to the negative utility to the population of the actions the AI needs to take to produce more. It is less an argument about ethical frameworks and more an argument that (under any reasonable ethical framework), a sufficiently-advanced algorithm would still need to be supervised by humans. It's certainly not nonsense though; while we don't yet have general AI that's sufficiently-advanced enough to enslave the human race and end the world, we do certainly have an increasing number of unsupervised algorithms which create large amounts of disutility because they optimize for their function without consideration to their effects on society. The algorithms that Facebook used to determine which posts appeared on your 'wall' generated so much free publicity for sensational and false news articles that Facebook has started to hire manual fact-checkers to mark them. It turns out, it's easier to write interesting news articles that people want to click on, read, and share, if you can make up whatever you like (and whatever most appeals to your audience) instead of reporting the truth, and the Facebook algorithms optimize for popularity, not for truth. --24.108.52.222 (talk) 21:30, 29 September 2018 (UTC)
thread_title:  Why is the 'paperclip maximizer' even there? 

message: There is no source cited for the claim that a utility monster "receives as much or more utility from each additional unit of a resource he consumes as the first unit he consumes." Nor would this imply that all resources should be given to the utility monster, since it says nothing about how the utility the monster gets from each resource compares to the utility anyone else gets from the resource. The quote from Nozick establishes a completely different definition: that a utility monster gets more utility from each unit of resource than others would get from the same resource. As far as I've been able to find, Nozick never says anything about the monster not having diminishing marginal returns. If no one can find a source for the claim about constant or increasing marginal returns, it should be changed to the definition quoted by Nozick. 50.0.142.172 (talk) 18:12, 19 August 2013 (UTC)
replies: text: I have now removed the mention of marginal returns, and replaced it with Nozick's original definition. [Edit: I am the same person who brought up the objection originally.] 24.7.24.119 (talk) 22:55, 15 January 2014 (UTC)
thread_title:  Increasing marginal returns 

message: As I see it, the link adds no value to the article and should be removed. --Benjamin Schwarz (talk) 12:05, 4 May 2014 (UTC)
replies: text: I agree. I've removed the webcomic link from the article. —Mark Bao (talk) 02:31, 8 May 2014 (UTC)
thread_title:  Link to chaospet web comic 

message: On the article of Utility Monster, I would like to add the following corrections:
-Add a link to the Wikipedia "Utilitarianism" page when it is mentioned
-Citation of Frederick Kennard's quotes throughout the article, where citations are currently established as needed
-More background or history of the concept would be extremely beneficial, so readers can better grasp the subject and it's relevance to thought today because they understand from where the idea arose.
-This article needs more material that is not quoted or closely summarized.
In conclusion, I would add a history of the subject through explaining Nozick's approach to ethics, and refine what is already written to include less quotes and close paraphrasing.
Bibliography of corrections and additions: *there are not many reputable sources published on the topic, so I would dig deeper into the first two sources here (which are already referenced on the Wikipedia page, but minimally so), and add information from the last source about Nozick's approach.
"Utilitarianism". Wikipedia. 2016-10-14.
Kennard, Frederick (March 20, 2015). Thought Experiments: Popular Thought Experiments in Philosophy, Physics, Ethics, Computer Science & Mathematics (First ed.). AMF. p. 322. .
"Nozick, Robert | Internet Encyclopedia of Philosophy". www.iep.utm.edu. Retrieved 2016-10-18.
Petra Sen (talk) 23:25, 18 October 2016 (UTC)
thread_title:  Revisions and Additions to Bibliography and Content 

message: I thought you did a great job on adding to the article Utility Monster.  At the top of the page, it stated that it was an "orphan" and needed links, and you addressed that need right away by adding the link to the Wikipedia Utilitarianism page.  Additionally, you inputted two citations where requested for sources.  The sources look good and credible too according to Wiki guidelines. The only suggestion I have is the one line, "Any other people who not in the said group or not the individual are left with less happy units to be split among them" does not really flow off the tongue easily.  I had to reread it a few times to understand.  If it is not a direct quote, whichI believe it is paraphrased, maybe you could reword that just a bit to be easily understood.  I understand what it is saying, but I think the sentence could be structured a bit better to read.  Something about "people who not in the said group" maybe needs rewording.  I learned something today about Utility Monster because of you.  I had no idea.  I want lots of happy units!!!  I think you did a nice addition to the article.Tyler Chinappi (talk) 03:43, 10 November 2016 (UTC)
thread_title: Peer Review

message: This needs to be tidied up. Does anyone have any suggestions on how to merge the History, Relevance, and Social implications sections into fewer sections without eliminating the useful context that each of these provide? ... JimsMaher (talk) 01:02, 5 January 2017 (UTC)
thread_title:  History / Relevance / Social implications 

message: The article currently states that "It can be shown that all consequentialist systems based on maximizing a global function are subject to utility monsters". But the claim seems similar to the observation that any estimator with a low breakdown point can be thrown off by a single arbitrarily large outlier. I don't have access to the source. Does it take analogs of robust statistical measures into account? If it does, perhaps the article should say so. 46.66.178.159 (talk) 13:40, 20 September 2019 (UTC)
thread_title:  Every optimization function has a monster? 

message: nozick does not once say the word "cookie" in ASU, last I checked. nor does he say "happiness units", or "happiness hogs". by this I mean to say the whole page is discussing this thought experiment with very little reference to the original text. the main references are from secondary sources - only one reference is actually nozick himself, and it is a single quote from ASU. 
these references shouldn't be removed, but need to be clarified. kennard's discussion /is/ interesting - but that's kennard's work, not nozick's. the original text uses the thought experiment to prove the holes in utilitarian thought through inconsistencies regarding animal rights. this is crucial to interpreting nozick's work correctly. 
that's not to say, of course, the contemporary popularisation of the thought experiment (i.e. kennard and the likes) are also referring to animal rights. that is obviously not the case and the discussion has largely been divorced overtime. 
it is important here to iterate, I think - nozick did not spend long on the utility monster at all. he may have been the first to bring it up, but he did not develop it in great depth (outside his discussion of animal rights). that credit ought to be more clearly attributed to other scholars so that people can find relevant sources more easily (and better understand ASU as a text). 
I'll overhaul this page at a later date but will need to do some further reading outside of nozick's work to do a good job of that. any help in the meantime is appreciated. thx yall. Meikkon (talk) 17:39, 12 December 2023 (UTC)
thread_title:  on animal rights / adding this to watch list