Talk:Akaike information criterion

message: Hi all,
I recently made a small change to the AICc formula to add the simplified version of the AICc (i.e. not in terms of the AIC equation above, but the direct formula) so that readers could see both the AICc's relation to the AIC (as a "correction") and as the formula recommended by Burnham and Anderson for general use [edit number 618051554].
An unnamed user reverted the edit, citing "invalid simplification." Does anyone (the reverting editor from ip 2.25.180.99 included) have any reason to not want the edit I made to stand? It adds the second line to the equation below,
:\begin{align}AIC_c &= AIC + \frac{2k(k + 1)}{n - k - 1}\\
&= \frac{2kn}{n-k-1} - 2 \ln{(L)}\end{align}
The main point I have in favor of the change is that Burnham and Anderson point out in multiple locations that the AIC should be thought of as the asymptotic version of the AICc, rather than thinking of the AICc as a correction to the AIC. This second equation shows easily (by comparison with the AIC formula) how that asymptotic relationship holds, and it shows how to compute the AICc itself directly.
The point against it, as far as I can see, is just that there is now a second line of math (which I can imagine some people being opposed to...)
Any opinions? If no, I'll change my edit back in a few days/weeks.
Dgianotti (talk) 18:03, 31 July 2014 (UTC)
replies: text: The second equation is certainly valid.  I do not see why the proposed new formula is a simplification though; for me, the first (current) formula is simpler.  Regarding the asymptotic relationship, this is shown much more clearly by the first formula: as n→ ∞, the second term plainly goes to 0; so the formula becomes just AIC, as Burnham and Anderson state.  Hence I prefer the current formula.  86.152.236.37 (talk) 21:11, 13 September 2014 (UTC)
thread_title: AICc formula

message: I've written a short article on DIC, please look it over and edit. Bill Jefferys 22:55, 7 December 2005 (UTC)
replies: text: Great, thanks very much. I've edited a bit for style, no major changes though. Cheers, --MarkSweep (call me collect) 23:18, 7 December 2005 (UTC)
thread_title: Deviance Information Criterion

message: is AIC *derived* from anything or is it just a hack? BIC is at least derivable from some postulate. WHy would you ever use AIC over BIC or, better, cross validation?
There is a link on the page () which shows a proof that AIC can be derived from the same postulate as BIC and vice versa. Cross validation is good but computationally expensive compared to A/BIC - a problem for large scale optimisations. The actual discussion over BIC/AIC as a weapon of choice seems to be long, immensely technical/theoretical and not a little boring 128.240.229.7 12:37, 28 February 2007 (UTC)
replies: text: An external link http://www4.ncsu.edu/~shu3/Presentation/AIC.pdf  mentioned in the current article gives a coherent explanation of the goal of the AIC and the "corrected" AIC.  (See the section "Model Selection Criterion" on page 7)  Using general concepts, it defines a quantity that we seek to maximize.  The AIC and the "corrected" AIC formulae are presumably ways to compute something proportional to that quantity in special situations.  Tashiro~enwiki (talk) 17:38, 9 December 2017 (UTC)
replies: text: It took me years to understand what the text of this article is actually about, pity the poor reader. One major source of confusion is the omission of a crisp definition of what are the 'statistical models' that AIC applies to, to wit, it applies only to 1D random variates, which is a small subset of the uses of density models. For example, if we want to fit a concentration of drug in blood curve with a 'statistical model,' like a gamma distribution, GD(t|a,b), we would minimize the error of C(t) ≈ AUC GD(t|a,b), where C(t) are concentration 2D "blood samples" in time, and AUC, in this case, is area from time is {0,infinity} of concentration under the AUC GD(t|a,b) curve, as the AUC of GD(t|a,b) = 1. Note that AIC values can be obtained from concentration curves, but only indirectly. For example, maximum likelihood fitting can be performed, but one would apply maximum likelihood to a 1D list of residuals from {C(t),AUC GD(t|a,b)} and not to the 2D concentration data itself. One could call this process 'residual maximum likelihood.' Now the way this article presents AIC and 'statistical models' excludes discussion of all other density functions, which, like concentration density functions, are not probability density functions (PDF), and that is the majority of models out there. Using the definitions implied by, but not explained in this article, the gamma distribution usage above would not be called a statistical model, which is not a good idea; too confusing, especially given that its residuals can be 'statistical.' To be clear, a gamma distribution is a density function, and although density functions can be applied to random variates, in which case, and as far as I can tell only in that case, can we speak of density functions as being PDF. Models are never 'statistical' or 'not statistical' only data can be statistical in the sense of being random, and one should not consider a PDF, or a PMF for that matter, as being anything other than deterministic; they just are not random; they have no noise. Thus, I highly recommend not saying 'statistical models,' when you mean models applied to statistical data. That is a grammatical error called a misplaced modifier, for other examples see https://www.scribbr.com/language-rules/misplaced-modifier/. There is no such literal thing as a 'statistical model' as that says that the model is randomized. Statistical modelling is the modelling of statistics, it does not use 'statistical models,' but rather models of statistical processes. CarlWesolowski (talk) 06:38, 24 July 2022 (UTC)
thread_title: Justification/derivation

message: AIC was said to stand for "An Information Criterion" by Akaike, not "Akaike information Criterion" Yoderj 19:39, 16 February 2007 (UTC)
replies: text: This is similar to Peter Ryom developing the RV-index for the works of Vivaldi - instead of the official Répertoire Vivaldi, his index of course became known as the Ryom Verzeichnis... I am inclined to believe this is not unintentional on the part of the developer; it's probably (false?) modesty. Classical geographer 12:18, 2 April 2007 (UTC)

text: WAIC and AIC are not the same. I have now listed a paper for WAIC in the "Further reading" section. SolidPhase (talk) 07:31, 16 May 2016 (UTC)
thread_title: Origin of Name

message: I have sent an e-mail to this Mr. Gee who is cited as a possible reference, with the following text:
::Dear Mr. Gee, 
::For some time now your name is mentioned in the Wikipedia article on the Akaike Information Criterion (http://en.wikipedia.org/wiki/Akaike_information_criterion). You are cited as having developed a pseudo-R2 derived from the AIC. However, no exact reference is given. I'd be glad to hear from you whether you actually developed this, and where, if anywhere, you have published this measure.
::Thank you for your cooperation.
However, he has not answered. I will remove the reference. Classical geographer 12:18, 2 April 2007 (UTC)
thread_title: Travis Gee

message: Please write the pronunciation using the International phonetic alphabet, as specified in Wikipedia:Manual of style (pronunciation). -Pgan002 05:09, 10 May 2007 (UTC)
thread_title:  Denoting pronunciation 

message: Query - should this page not also link to Bayes' Factor?
I'm not an expert in model selection but in my field (molecular phylogenetics) model selection is an increasingly important problem in methods involves Bayesian inference (e.g. MyBayes, BEAST) and AIC is apparently 'not appropriate' for these models 
Any thoughts anyone? I've also posted this on the model selection page. Thanks.--Comrade jo (talk) 12:19, 19 December 2007 (UTC)
thread_title:  Bayes' Factor 

message: The RSS in the definition is not a likelihood function!  However, it turns out that the log likelihood looks similar to RSS.  —Preceding unsigned comment added by 203.185.215.144 (talk) 23:12, 7 January 2008 (UTC)
replies: text: I agree. What's written is actually the special case of AIC with least squares estimation with normally distributed errors. (As stated in Burnham, Anderson, "Model selection and inference". p48) Furthermore you can factor out ln(2pi)*n and an increase in K due to using least squares. These are both constant when available data is given, so they can be ignored. The AIC as Burnham and Anderson present it, is really a tool for ranking possible models, with the one with the lowest AIC being the best, the actual AIC value is of less importance. EverGreg (talk) 15:27, 29 April 2008 (UTC)
thread_title:  Confusion 

message: I think the link that appeared at the bottom "A tool for fitting distributions, times series and copulas using AIC with Excel by Vose Software" is not too relevant and only one of many tools that may incorporate AIC.  I am not certain enough to remove it myself.  Dirkjot (talk) 16:36, 17 November 2008 (UTC)
thread_title:  Link spam? 

message: What on earth is this section?  It should be properly explained, with real references, or permanently deleted!  I would like to see a book on model selection which describes AIC in detail, but also points out these supposed controversies!  True bugman (talk) 11:50, 7 September 2010 (UTC)
replies: text: This is a good question, but there is probably something that does need to be said about properties of AIC, not necessarily under "Controversy". For example, in this online dissertation, I found "AIC and other constant penalties notoriously include too many irrelevant predictors (Breiman and Freedman, 1983)" with the reference being: L. Breiman and D. Freedman. "How many variables should be entered in a regression equation?" Journal of the American Statistical Association, pages 131–136, 1983. There are similar results for using AIC to select a model order in time series analysis. But these results just reflect the penalty on large models that is inherent in AIC, and arises from the underlying derivation of AIC as something to optimise. Melcombe (talk) 16:04, 7 September 2010 (UTC)
replies: text: Time series data is only one of many data types where modelling and AIC are used together.  Something like this should be included in a special section dedicated to time series data.  True bugman (talk) 12:09, 8 September 2010 (UTC)
replies: text: The point was that the supposed problem with AIC is known to occur for both regression and time series, in exactly the same way, so it would be silly to have to say it twice in separete sections. Melcombe (talk) 16:51, 8 September 2010 (UTC)
replies: text: Regression, as an example, is not covered in this article.  Neither is time series.  But yes, AIC is not perfect, and yes this should probably be discussed.  But in a neutral way, this is by no means a controversy.  I believe the entire 'controversy' section should be deleted.  These are all recent changes from different IP addresses (110.32.136.51, 150.243.64.1, 99.188.106.28, 130.239.101.140) unsupported by citation, irrelevant for the article (the controversial topics discussed are not even in the article), and it is very poorly written (again there is no connection to the article).  What does "crossover design", "given to us a priori by pre-testing", and "Monte Carlo testing" even mean?  This section is written as an attack on the technique rather than a non-biased source of information.  It is not verifiable WP:V nor written with a neutral point of view WP:NPOV.  It must go.  True bugman (talk) 17:19, 8 September 2010 (UTC)
thread_title:  Controversy?! 

message: I removed the part on Takeuchi information criterion (based on matrix trace), because this seemed to give credit to Claeskens & Hjort.  There could be a new section on TIC, if someone wanted to write one; for now, I included a reference to the 1976 paper.  Note that Burnham & Anderson (2002) discuss TIC at length, and a section on TIC should cite their discussion.  TIC is rarely useful in practice; rather, it is an important intermediate step in the most-general derivation of AIC and AICc.   86.170.206.175 (talk) 16:24, 14 April 2011 (UTC)
thread_title:  Takeuchi information criterion 

message: I made a few minor edits in the BIC section to try to keep it a *little* more neutral, but it still reads with a very biased tone. I imagine a bunch of AIC proponents had a huge argument with BIC proponents and then decided to write that section as pro-AIC propaganda.  You can find just as many papers in the literature that unjustifiably argue that BIC is "better" than AIC, as you can find papers that unjustifiably argue AIC is "better" than BIC.  Furthermore, if AIC can be derived from the BIC formalism by just taking a different prior, then one might argue AIC is essentially contained within "generalized BIC", so how can BIC, in general, be "worse" than AIC if AIC can be derived through the BIC framework?  
The truth is that neither AIC nor BIC is inherently "better" or "worse" than the other until you define a specific application (and by AIC, I include AICc and minor variants, and by BIC I include variants also to be fair).  You can find applications where AIC fails miserably and BIC works wonderfully, and vice versa.  To argue that this or that method is better in practice, because of asymptotic results or because of a handful of research papers, is flawed since, for most applications, you never get close to the fantasy world of "asymptopia" where asymptotic results can actually be used for justification, and you can almost always find a handful of research papers that argue method A is better than method B when, in truth, method A is only better than method B for the specific application they were working on.  — Preceding unsigned comment added by 173.3.109.197 (talk) 17:44, 15 April 2012 (UTC)
thread_title:  Biased Tone of BIC Comparison Section 

message: The article states that AIC is applicable for nested and non-nested models, with a reference to Anderson (2008). However, looking up the source, there's no explicit indication that the AIC should be used for nested models. Instead, the indicated reference just states that the AIC can be valuable for non-nested models. Are there other sources that might be more explicit?  — Preceding unsigned comment added by Redsilk09 (talk • contribs) 10:11, 18 July 2012 (UTC)
thread_title:  AIC for nested models? 

message: The BIC section claims Akaike derived BIC independently and credits him as much as anyone else in discovering BIC.  However, I have always read in the history books that Akaike was very excited when he first saw (Schwartz's?) a BIC derivation, and that after seeing that it inspired him to develop his own Bayesian version of AIC.  I thought it was well-documented historically that this was the case, and that he was a very graceful man who didn't think of BIC as a competitor to him, but thought of it as just yet another very useful and interesting result.  His only disappointment, many accounts do claim, was that he didn't think of it himself earlier.  Isn't that the standard way that all the historical accounts read?
:Your version of events seems right to me.  Akaike found his Bayesian version of AIC after seeing Schwartz's BIC.  BIC and the Bayesian version of AIC turned out to be the same thing.  (Maybe you should edit the article?)  — Preceding unsigned comment added by 86.156.204.205 (talk) 14:10, 14 December 2012 (UTC)
thread_title:  BIC Section 

message: I removed the following sentence:
"This form is often convenient, because most model-fitting programs produce \chi^2 as a statistic for the fit."
The \chi^2 statistic produced with many model-fitting programs is in fact the RSS (e.g. Origin ). But the RSS cannot simply replace \chi^2 in these equations. Either the σi has to be known or the following formula should be used AIC = n ln(RSS/n) + 2k + C.  — Preceding unsigned comment added by 129.67.70.165 (talk) 14:34, 21 February 2013 (UTC)
thread_title:  Removed confusing sentence 

message: The example from U. Georgia is no longer found;  so I deleted it.  It was:
* Example Calculation (University of Georgia)
* ie, * http://coopunit.forestry.uga.edu/Coop_Wkshop/inference_effects/aic_reg.pdf Example Calculation (U.GA)
I added the best example I could find with a Google-search: [Akaike example filetype:pdf]  — Charles Edwin Shipp (talk) 13:31, 11 September 2013 (UTC)
thread_title:  Example? 

message: Substituted at 19:44, 1 May 2016 (UTC)
thread_title: Assessment comment

message: This section is created to discuss the recent edits by anonymous IPs. SolidPhase (talk) 10:28, 31 October 2016 (UTC)
thread_title: Recent edit war by anonymous IPs

message: In the article, it is written "If the model under consideration is a linear regression, k {\displaystyle k} k is the number of regressors, including the intercept".
This is wrong, isn't it? What about the error variance? Shouldn't the error variance also count as a parameter?  — Preceding unsigned comment added by 193.174.15.2 (talk) 09:30, 3 January 2017 (UTC)
thread_title:  Number of parameters? 

message: Hello fellow Wikipedians,
I have just modified one external link on Akaike information criterion. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20120315165255/http://www.sortie-nd.org/lme/Statistical%20Papers/Burnham_and_Anderson_2004_Multimodel_Inference.pdf to http://www.sortie-nd.org/lme/Statistical%20Papers/Burnham_and_Anderson_2004_Multimodel_Inference.pdf
*Added  tag to http://www.mun.ca/biology/quant/BurnhamBES11.pdf
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 00:45, 29 June 2017 (UTC)
thread_title:  External links modified 

message: , could you please explain why the sentence “Asymptotically, AIC selects the model that minimizes the mean squared error of (out-of-sample) prediction,” is wrong and/or misleading? By definition, an asymptotically efficient (information) criterion chooses the (candidate) model that minimises the mean squared error of prediction, and by Stone (1977) AIC is asymptotically efficient. --bender235 (talk) 23:56, 16 January 2018 (UTC)
replies: text: The statement is true in some cases, but it is not known to be true in general, e.g. for panel data. Stone (1977) considers regression only, and the article does say that "in the context of regression … AIC is asymptotically optimal for selecting the model with the least mean squared error". Even for univariate time series, the status of the statement is only partially known (Ing & Wei, Annals of Statistics, 2005). If you (or someone else) want to study this issue more, perhaps also consider the cited work of Akaike (1985).  BetterMath (talk) 00:50, 17 January 2018 (UTC)
thread_title:  AIC asymptotics, misleadingly phrased? 

message: A proposed revision is below. The first paragraph is the same as before; the second paragraph is expanded; the third paragraph is the same as before; the fourth paragraph is new (and it mentions Hirotugu Akaike, as recommended by User:Reidgreg).
:The Akaike information criterion (AIC) is an estimator of the relative quality of statistical models for a given set of data. Given a collection of models for the data, AIC estimates the quality of each model, relative to each of the other models. Thus, AIC provides a means for model selection.
:AIC is founded on information theory: it estimates the relative information lost when a given model is used to represent the process that generated the data. The less information is lost, the higher is the quality of the model. (In making an estimate of the information lost, AIC deals with the trade-off between the goodness of fit of the model and the simplicity of the model.)
:AIC does not provide a test of a model in the sense of testing a null hypothesis. It tells nothing about the absolute quality of a model, only the quality relative to other models. Thus, if all the candidate models fit poorly, AIC will not give any warning of that.
:AIC was formulated by the statistician Hirotugu Akaike. It now forms the basis of a paradigm for the foundations of statistics; as well, it is widely used for statistical inference. 
Perhaps the third paragraph should be moved to the Definition section.
SolidPhase (talk) 09:45, 19 July 2018 (UTC)
thread_title: Proposed revision

message: There is a long equation in the section "Replicating Student's t-test". The equation is so long that it is split over two lines. Should the equals sign go on the first line (at the end) or on the second line (at the beginning)?
I did not find a relevant policy or guideline on this. I prefer that the equals sign be on the first line, for two reasons. First, because it makes the first line clearer: when someone reads the first line, they know immediately that the next line is the other side of an equation, and that eases reading of the whole equation. Second, because it lessens the overall display width.
BetterMath (talk) 10:43, 27 July 2019 (UTC)
thread_title:  splitting long equation 

message: There is a misleading statement in the section on "Foundations of Statistics," where one editor claims that the Akaike Information Criterion (AIC) can form a foundation of statistics that is distinct from both frequentism and Bayesianism. Upon reviewing the cited reference (Burnham & Anderson, 2002 [p. 99]), it appears that the statement is not adequately supported. The relevant statement in the reference mentions that information criteria can be computed and interpreted without subjective judgment or the use of significance levels or Bayesian priors, but it does not suggest that information criteria constitute a separate statistical paradigm.
Additionally, after examining the book "Philosophy of Statistics," it is evident that no claim is made regarding AIC or information criteria forming a distinct paradigm within statistics. In the chapter specifically discussing AIC, BIC, and model selection, the authors treat AIC as a model selection rule that aids in statistical inference.
Based on these observations, it seems that the editor who wrote that particular part of the section may have aggregated and overextended the statements in the references. I recommend that the section be removed entirely, but gathering input from other editors would be valuable before making a final decision. Jourdy345 (talk) 15:04, 10 May 2023 (UTC)
thread_title:  Information criteria are hardly a statistical paradigm 

message: Online essay ordering services play a crucial role in alleviating the burden of academic workload for students. With the increasing demands of coursework and deadlines, these services offer a lifeline by providing timely and customized educational assistance. By outsourcing essay writing tasks to professional writers, students can focus on other essential aspects of their education, such as studying for exams, participating in extracurricular activities, or gaining practical experience through internships. The convenience of accessing these services online empowers students to strike a balance between their academic responsibilities and personal pursuits, ultimately enhancing their overall learning experience and academic performance. The order essay online process provides a practical solution for those seeking assistance with complex assignments or seeking to manage their academic workload more effectively. Lucas Herry (talk) 07:28, 8 November 2023 (UTC)
thread_title:  How Online Essay Ordering Services Help Students Manage Academic Workload