Talk:Extract, transform, load

message: Due to the technical nature of ETL GA compliance may not be possible. However, the content should not be sales literature for software companies including open source projects. I plan on sinking that content to the bottom into a list of ETL apps. 
* The diagram used here is not indicative of using multiple data mediums for I/O. Specifically Input of csv files, spreadsheets, databases, other data streams.
* I will take the lead on sanitizing this article
Mkevlar (talk 4:34, 1 April 2022 (UTC)
thread_title: General audience rework

message: IMO it makes sense to include Jaspersoft ETL in the list of Proprietary ETL frameworks  —Preceding unsigned comment added by 178.236.241.148 (talk) 12:55, 16 February 2011 (UTC)
thread_title: Jaspersoft ETL

message: I'm not an expert on this subject. But you guys who are experts should update this to include at least a reference to ELT (Extract, Load, Transform). You can find many articles about this and many huge companies using this approach for many-hundreds-terabytes datawarehouses.  —Preceding unsigned comment added by 201.48.130.65 (talk) 20:00, 28 December 2009 (UTC)
thread_title: Refer ELT

message: Specifically referring to the statement, "Increasingly, companies are buying ETL tools to help in the creation of ETL processes.[citation needed],"... Really? I'm not sure we need citations when statements are common sense. Companies are always looking for ways to increase the bottom line and IT departments to increase the "plug-and-play" supportability of components. Certainly this is not accomplished through creating (or allowing to be created) tons of proprietary, in-house scripts. So, obviously, companies that discover ETL tools will buy ETL tools to create ETL processes. I wouldn't expect to see "Wind is the movement of air [citation needed]" That's just silly. 71.67.189.53 (talk) 01:42, 24 October 2009 (UTC)
replies: text: I do think that the citation is needed to validate the term "Increasingly" used in the paragraph.  I believe there is still massive amount of manual creation of ETL without the use of ETL tools so some proof that this is diminishing and the use of COTS products are filling in that part of the marketshare would be useful. 157.203.43.103 (talk) 14:35, 11 March 2010 (UTC)

text: I'm with the guy above. As someone who has worked in the industry for 20+ years, there's an awful lot of "ETL Tool evaluation" goes on, but companies (IMO) still shy away from the up-front cost, and err towards sticking with their in-house scripted solutions. This might be changing, but we do need a citation. If it were my $2 I'd say it's not on the increase. Michaelfromtheuk (talk) 09:18, 17 July 2012 (UTC)
thread_title: Citation Needed??

message: I suggest that deleting the "External links to other tools" section is long overdue.  It has become a spam magnet and a link repository, and Wikipedia is not a link repository. —Veyklevar 05:24, 12 May 2006 (UTC)
thread_title: External links to other tools

message: Some of the "Challenges" section of this article has been copied verbatim from . Haakon 11:57, 22 May 2006 (UTC)
thread_title: Copyvio

message: This list of ETL tools is a little selection of the usable products. A recent software is'nt present in this list : Talend Open Studio. I have find severals english source of information of this product and its show the relevance of this software:
* http://linux.sys-con.com/read/283125.htm
* http://opensource.sys-con.com/read/278184.htm
* http://www.galileo.fr/eng/default.php?num=7
* http://www.decideo.com/Talend-pushes-online-the-public-beta-of-first-ever-etl-open-source-tool_a1318.html
* https://analyticscanvas.com  — Preceding unsigned comment added by 184.147.110.172 (talk) 15:38, 23 March 2021 (UTC)
thread_title:  Some ETL Tools 

message: Under the section that lists open source tools, shouldn't we mention tools like Cloudbase, Hive, and Pig (all based off Hadoop) as well?  —Preceding unsigned comment added by Saurabhnanda (talk • contribs) 04:51, 2 June 2009 (UTC)
thread_title:  Hadoop based tools 

message: Kettle seems to be a synonym for Pentaho Data Integration. Useful to mention this somewhere in the tool list, and maybe also have a disambiguation page for "Kettle". March 2010, ThP  —Preceding unsigned comment added by 88.101.4.80 (talk) 10:30, 19 March 2010 (UTC)
thread_title:  Kettle 

message: In addition to the criticism that has been posted as to the "recipe" nature of part of this section (do this first, then do that, etc), as a database consultant with almost 20 years experience, I actually disagree with the fundamental point the author is making here. He says the "bottleneck" is in the loading of the database. Firstly, the "speed" of a load is rarely the step that takes the longest time in any migration / data-cleansing type project. It's the *analysis* that must be performed to map data and data types(!) from disparate sources into a new "consolidated" target database. This involves many many man-hours and meetings and talking to subject-mater experts, etc. In other words, the "technical" aspect of actually loading the final data is in a sense the *easiest* part of the project. It's the figuring out of all the nasty problems that crop up *before* one has a clean mapping of old-->new data and old--> new data types that take up the most time and intellectual effort. Finally, with the ever increasing speeds of computers and ever increasing quantities of fast memory, technical "loading" of data (yes, even with constraints etc needing to be created) is simply not even in the top 5 problems on a person's mind when working on such a project (ETL).
In conclusion, there are many difficult intellectual problems associated with ETL. However the one mentioned by the author (DB load speed) is probably the least among them. Finally, is it our job to "rank" these problems (associated in a project)? I'm not sure.  —Preceding unsigned comment added by Frumiousfalafel (talk • contribs) 19:34, 26 October 2009 (UTC)
thread_title:  Performance Section 

message: I am very confused that whether this project is just similar to "Data Mining in heteregeneous Distributed System" or is completely different.  —Preceding unsigned comment added by 116.75.132.149 (talk) 18:20, 11 August 2010 (UTC)
thread_title:  ETL Implementation 

message: The section Transform says: "Translating coded values (e.g., if the source system stores 1 for male and 2 for female, but the warehouse stores M for male and F for female), this calls for automated Data cleansing; no manual cleansing occurs during ETL". The Data cleansing article talks about removing or correcting invalid information, which I would see as different and separate from transcoding. In other words if the data source has "Male" and "Female" and the destination "M" and "F" then this change is transcoding but removing a record that has "Hermaphrodite" in the column would be data cleansing. -- Q Chris (talk) 15:22, 11 April 2011 (UTC)
thread_title:  Dubious 

message: The author claims "Disable triggers (disable trigger ...) in the target database tables during the load. Simulate their effect as a separate step." to gain speed. I don't consider this being a good practice, because triggers ensure database integrity, and if the ETL process fails after the initial step, this might mess up the database completely.  — Preceding unsigned comment added by 88.1.46.185 (talk) 08:12, 20 October 2011 (UTC)
thread_title:  Disable Triggers 

message: A web search backs up my feeling that the alternative 'data model recasting' listed in the section is not notable and the references cannot be considered reliable. Gradientdescent (talk) 02:12, 5 June 2012 (UTC)
thread_title:  ETL Alternatives 

message: This page is not incorrect but it does not contain sufficient citations and is very outdated.  It focuses almost exclusively on ETL from the aughts between relational data structures and traditional data warehouse.   I don't have space to correct it right now but I tagged it as a needing citations and I hope someone can come in and update this, cite it, and make it more modern.   When I have space I will try to come back and  spiff it up.  :-)   Alex Jackl (talk) 13:29, 27 May 2019 (UTC)
replies: text: I was mostly concerned that you inserted what looked like a signed talk page comment directly into the article. I'm also concerned (though less so) that the two sources you added seem to be unsigned, self published websites. - MrOllie (talk) 14:51, 3 June 2019 (UTC)
replies: text:  In hindsight I shouldn't have added the signed note right in the page.   Good catch and I removed it.   I agree that those are not the ideal citations (though they have reasonable content)  but the article is severely lacking in in-line citations so I thought better self-published, yet content-ful citations that supported the material than no citations.   Thanks MrOllie Alex Jackl (talk) 15:12, 3 June 2019 (UTC)
thread_title:  ETL Description Outdated 

message: Social media he ligand reformist sea mist aor 2001:8004:1341:C21A:58E7:95BD:3083:8190 (talk) 03:49, 13 February 2022 (UTC)
thread_title:  Polite technology