Wikipedia talk:Avoid using meta-templates/Archive 1

message: Stub templates aren't the only meta-templates in use.  See Template:Message box. -- AllyUnion (talk) 07:08, 4 Mar 2005 (UTC)
thread_title:  Template:Message box 

message: As the stub messages are being turned from readable semantically-marked-up code to less-readable stuff with DIVs flyign to and fro, I gotta as this.
As Jamesday pointed out, when a page's entry in the cache is empty, a database query is made to fetch that page. Then another one for each template it contains, and so forth. Why on earth aren't we caching the templates as well?
From what Jamesday said, this is the current process: Say page1 contains template, which contains meta-template. page2 also contains template. Then, if meta-template is changed, loading the two pages causes this to happen:
*page1(template(meta-template)) is dirty, needs to be refreshed. Fetch page1 from db.
**Fetch template from db.
***Fetch meta-template from db.
**Assemble template(meta-template).
*Assemble page1(template(meta-template)).
*Serve (and cache) page1(template(meta-template)).
*Fetch page2 from db.
**Fetch template from db.
***Fetch meta-template from db.
**Assemble template(meta-template).
*Assemble page2(template(meta-template)).
*Serve (and cache) page2(template(meta-template)).
This can't possibly be right, but it looks like this is what Jamesday is saying is happening. If so, no wonder meta-templates are such a performance hit. What's wrong with caching a version of the page with references to the templates, so they can be substituted into an already in-cache page without dirtying it? So we'd avoid dirtying zillions of pages and requiring them to be fetched all the way from the database again when, really, nothing's actually changed in them.
Am I way off base here? Someone set me straight. grendel|khan 02:07, 2005 Mar 4 (UTC)
:#It seems like we can change the way pages are cached instead of inconveniencing everyone by removing meta-templates.  True?
:#If "glorified stylesheets" are converted into css classes, the classes have to be as visible and easy to use as the templates and also editable. - Omegatron 23:41, Mar 21, 2005 (UTC)
:Eventually that can be done. Doesn't change what's happening at present, though. One issue we were discussing a few days ago was the cost of lots of round trips to the cache server for each page build and the desirability of avoiding those, particularly as we move to building sites at different data centers. That forced us for now to want to stick to sites very close to the current Florida location, because of the delays involved with lots of sequential cache requests at perhaps 200ms each from korea. Asking the database slave which could be located in Korea would actually have been better than using the different memcached cache for that. Not ideal and I hope that will also change.
:Where would you do the cache lookups? Options are:
:*Squids (which would increase the cost per page for the squids and is something we can't currently do anyway because what we have doesn't support it
:*Apaches and memcached. Could do. 30 or so of them are needed to do the 25% of work the Squids don't do. Is four times as many an improvement? Probably not, so wouldn't want to do it all the time, which leaves the squids caches being purged and them reloading from an apache as at present. I don't know whether the parser cache currently saves expanded or non-expanded templates - not expanding them and caching them individually might be an improvement. Certainly could cache the templates if they aren't currently cached and that would help with the page views for those who are logged in and for the rebuilds and database queries. Jamesday 15:24, 7 Apr 2005 (UTC)
thread_title:  Question about caching. 

message: * Template:31DayCalendarStartingOnMonday
* Template:31DayCalendarStartingOnTuesday
* Template:31DayCalendarStartingOnWednesday
* Template:31DayCalendarStartingOnThursday
* Template:31DayCalendarStartingOnFriday
* Template:31DayCalendarStartingOnSaturday
* Template:31DayCalendarStartingOnSunday
Used on the calender sources then used on the calendar themselves!  Triple include! -- AllyUnion (talk) 06:48, 30 Mar 2005 (UTC)
replies: text: For each template, a fetch from the database is required, right?  So if you have a template within a template, the server will need to attempt to retrieve the data two times before correctly displaying the information.  Here, we have a case of templates being applied three or four times, requiring the server to make several trips to retrieve the data.  Add it to a normal article, and you add more trip.  Which increases the server load. -- AllyUnion (talk) 08:47, 9 Apr 2005 (UTC)
thread_title:  Worst cases 

message: *  This is a personal opinion piece by one user, and should be acknowledged as such.
::I am suggesting that it goes into user space as User:Netoholic/Avoid using meta-templates, or into Meta as m:Avoid using meta-templates. Susvolans (pigs can fly) 12:31, 31 Mar 2005 (UTC)
thread_title: Requested move

message: Since we have now realized that this is a temporary treatment for a (lack of) hardware issue, how do we know at which point we no longer have to do it?   How temporary is it? - Omegatron 20:18, Apr 22, 2005 (UTC)
: Ah, but that's the genius of my "Emergency measures" (above). They are generally good ideas at all times, and we never have to stop taking them. 
: On the other hand, the emergency will never be "over". This is a not-for-profit enterprise, therefore chronically short of cash, no matter what optimists may say. We may temporarily get ahead of expenses, but the maw will quickly gobble up any excess and we'll be short again. Healthy profit is one way to tell when a not-for-profit enterprise has strayed from its primary purpose. — Xiong熊talk 18:07, 2005 Apr 24 (UTC)
::Mmm. The project page itself should probably include links near the top to a couple of diffs where Jamesday sets out the problems in detail. Metatemplates =  severe lack of caching, which can be problematic when you have the proportion of reader users vs editor users Wikipedia has (quite a lot) - David Gerard 23:21, 27 Apr 2005 (UTC)
replies: text: The information from Jamesday is fragmented into discussions in several different places, including Template talk:Sisterproject, his User page, and probably elsewhere. It would be nice to see his comments and discussions refactored into some sort of coherent whole. There are also a number of questions that have been asked here and elsewhere that I don't think have been fully answered, but they will need to be answered from one of the developers, or wait till Jamesday returns.  I also think that this discussion needs should be expanded to cover all the problems/ramifications of using Templates on the Wikipedia, and not be just about the single problem of Meta-Templates. BlankVerse ∅  06:32, 28 Apr 2005 (UTC)
thread_title:  How long do we do this? 

message: I would not really consider any performance questions serious until MediaWiki 1.5 is deployed on cluster. This will change whole view how we treat our systems and how those do respond. When we know what 1.5 is all about, and what should be do next, we will be able to decide on such issues. For now I'd like to be as conservative as possible. Domas Mituzas 14:57, 2005 May 3 (UTC)
:Conservative in which direction? ;-) - Omegatron 15:18, May 3, 2005 (UTC)
::In the direction of mucking about with a little as possible in the spirit of WP:POINT (IMNSHO, obviously). HTH HAND --Phil | Talk 17:04, May 3, 2005 (UTC)
:::WTFDTAM? - Omegatron 18:49, May 3, 2005 (UTC)
::::IMNSHO, naturally :-) HTH HAND --Phil | Talk 08:51, May 4, 2005 (UTC)
I don't understand Domas, you're saying we should wait until an upgrade to a more effecient version instead of engaging in pure speculation about perforamance from people who don't understand the current implemntation to begin with?  Nonsense!  By all means, lets treat this exclusively as a hardware problem where hardware is delivered by snails! -Nathan J. Yoder 18:03, 11 May 2005 (UTC)
thread_title:  let's wait until 1.5 

message: Do meta-templates still cause a significant resource hit in the current version of the Wiki software? Will they ever not be discouraged in the near or distant future? I'm working on a set of infobox templates and want to know if I should consider using them. android79 00:53, July 28, 2005 (UTC)
:I don't know for sure, but my guess is that the answers are yes (still a resource hit) and no (will still be discouraged in at least the near future).  If soneone more in the know than me doesn't respond within a few days, you might want to directly ask user:Jamesday. -- Rick Block (talk) 02:28, July 28, 2005 (UTC)
:: See the Mediawiki 1.5 technical review section below for updates on 1.5. Jamesday 15:10, 6 August 2005 (UTC)
thread_title:  Still a problem? 

message: (This topic inserted after the fact, for those trying to judge the veracity and timeliness of this guideline).
 
The issue of meta-template impact was referred to the m:developer committee as part of an arbitration request, whose input gives weight to guide (see below). The original impulse for guideline creation (performance impact at that time) occurred under 1.4 Mediawiki, and was reduced by 1.5 improvements.  The necessity for the guideline may further diminish with future improvements. -- LarryLACa 18:04, 14 November 2005 (UTC)
thread_title:  Developer committee review 

message: One change which may reduce overhead would be to provide a way to have templates which do not cause a cache flush.  This would be most useful for templates which do not change often.  It has been mentioned that Eloquence  recently proposed shared spaces for information such as the capital of a country.
* Does it seem like a good idea to allow transclusion to a Commons namespace?  For discussion purposes I'll call it Global:.
* Does transclusion currently support interwiki references?
* Would editing a transcluded entry in the Commons database trigger a cache flush?
* Editing a Global: entry, for example to change the capital of a country, should cause no changes until the next edit.  An editor should know that the country's main article should be tickled to update that obvious resource, and other less critical articles would be updated later.
(SEWilco)
* Is Commons in a separate data base server with a slower data path, which would slow the processing of an edit on en: servers?  Or would processing be faster due to DB overhead being on Commons servers?  (SEWilco 01:35, 15 August 2005 (UTC)
thread_title:  Low-Priority Template space 

message: ::Another thing that needs to be rewritten on the WP:AUM page is the "Server load" section. None of the stub templates have called the metastub templates since around the time that page was created, so the example of  Template:Canada-stub calling Template:MetaPicstub has not been a valid example for a very long time. BlankVerse 06:53, 15 November 2005 (UTC)
thread_title: Update guideline text: Server Load 

message: I'm glad to see changes being made to the page and being kept up-to-date.  
Is the scope creep section really appropriate, though?  The whole point of using templates is that they save work in creating new content.   So this is a benefit, not a "harmful effect".  The creation of lots of stub templates is a problem for WikiProject Stub sorting, not here. — Omegatron 18:13, 15 November 2005 (UTC)
thread_title:  Scope creep section