Talk:Software testing/Archive 1

message: Tprosser 14:25, 15 May 2007 (UTC)
replies: text: So why don't you fix it yourself? You seem to know the subject. The whole point of Wikipedia is that it's constantly improved, so why don't you fix it yourself instead of complaining? - Jan Persson
I strongly disagree that software testing is to measure quality of software.  Software testing should be a process to verify software quality against required quality.  Required quality can be defined in terms of requirements that can be specified in Software Quality Model such as that in ISO/IEC 9216:1991.  Measuring software quality is a different topic and would involve qualitifying software quality and comparison between software and industries.--Francis Law (talk) 01:36, 20 December 2007 (UTC)
thread_title: Introduction issues

message: The term "axiom" is probably misleading in this context (see Axiom) and should be changed to something more describing, such as "more or less undisputed facts regarding software testing".
Further, each statement should be followed by a short justification (since they are not axioms after all).  —Preceding unsigned comment added by 134.47.109.183 (talk) 08:57, 18 December 2007 (UTC)
thread_title:  Regarding "axioms" 

message: :How about (external) links to Ward Cunningham's wiki? His wiki has a lot of software development stuff: patterns, methodology, etc.
::I added a bit about test driven code; which indirectly refers to the brunt of the discussion on Ward's wiki.JimD 04:48, 2005 Jan 9 (UTC)
I also added a link to Ivars Peterson and a reference to his book.  I feel a little awkward doing so as I'm not trying to endorse it in particular; it just happens to be the resource that I thought appropriate to that juncture.
I'm also too tired to go back through and clean up my edits more and do additional research at the moment; but I feel like the work I did was better submitted, even in rough form, then discarded.  So we'll see what the rest of the wikipedian community as a whole makes of it. :) Edit Boldly
(In particular I know the Latin is awkward.  It sticks out like a sore thumb.  Fix it.  Please)JimD 04:48, 2005 Jan 9 (UTC)
Removed paragraph on being able to prove things in a court of law. Almost all software companies disclaim liability for buggy software, and except for a few life-critical pieces of software, the prospect of being sued isn't a strong motivating factor.  Also, except for life-critical software, most software testing does not seek to eliminate all defects since this is generally too expensive to be worth the cost.
:However I did engage in an assignment once, the objective of which was to test whether the software might leave the supplier exposed to an anti-trust law suit. The software processed data feeds containing financial data and the licence required that this data was capable of being processed by competitors' systems. Without using any of these systems we had to verify that the software and documentation didn't breach this licence. Matt Stan 01:04, 10 Jan 2005 (UTC)
thread_title:  Links to Cunningham's wiki 

message: The current version of the article differentiates "fault" from "failure" in a comprehensive way. However, it does not differentiate both concepts from "error" and "defect". I think this is an important observation that has been overlooked so far. --Antonielly 18:31, 16 March 2006 (UTC)
replies: text: IEEE recommends to use the word anomaly, see here:
":In software testing an anomaly (see IEEE 1044-1993: Standard Classification for Software Anomalies) is everything that differs from expectation. This expectation can result from a document or also from a persons notion or experiences. Also an anomaly can be a feature or an usability problem, because the testobject may be correct regarding the specification - but it can be improved. Another possibility for an anomaly is that a tester executed the testcase wrong and therefore the expected result is also wrong. Like IEEE says, the word anomaly should be favoured instead of e.g. fault, failure, error, defect or problem, because it has a more neutral meaning." --Erkan Yilmaz 15:31, 28 October 2006 (UTC)
thread_title:  Anomaly: Fault X failure X error X defect 

message: It was my understanding that a Heisenbug was a defect that occurred only when run under software in release mode, but stopped occurring when in debug mode. The act of observation changed the nature of the application and the defect disappears. --Walter Görlitz 18:23, 20 October 2005 (UTC)
thread_title: Regarding the ''Custodiet Ipsos Custodes'' section

message: This article is weak. SRE is not mentioned.  There is no mention of any testing practice designed to measure mean time to failure (MTTF).  (The
ambiguous Load Testing is mentioned but with no focus on a MTTF goal.) 
No mention of the Cleanroom-related controversy where the Cleanroomers
advocated SRE as the only testing and no coverage testing.
No mention of the fact that practical coverage testing cannot provide reliability, since it is not practical to cover all the places where
a bug can lurk and coverage testing includes no attempt to measure the
failure rate of the remaining bugs.  (By reliability, I mean a
low MTTF.) (BTW, Dick Hamlet proved that perfect coverage testing is equivalent to a formal proof of correctness, in "Partition Testing
does not Inspire Confidence")
Need a discussion of the goals of testing are.  Need to discuss the strengths and weaknesses of various methods in reaching these goals.
Two kinds of goals: Goals for the testing process, like line coverage,
and goals for the delivered system, like low MTTF.
Need a discussion of how to determine when to stop testing.
*Isn't "mean time to failure" more of a hardware than software concept?
*I have worked for small companies for approx 40 years, where most of the time I report to a manager who is outside of computer expertise.  
** The purpose of testing has been multi-fold
# Does the software do that which was requested by whoever it was, outside of computer staff? (me)
# Do we see room for obvious imporvements, and have those improvements been made successfully?
# Is the software user-friendly and intuitively obvious to operate, such that the risk of human error, using it, as low as we can make it?
# When humans enter bad data, does the software catch that in a manner that is easy to resolve?
# "The time to stop testing" is when the software is working satisfactorily, collectively we not see how to further improve it, other software projects have become more important to be working on.
User:AlMac|(talk) 08:39, 31 January 2006 (UTC)
thread_title:  Software Reliability Engineering (SRE) 

message: I've removed the link to 'testers' from the fifth paragraph of the 'Introduction' section, as it linked to an inappropriate article with no disambiguator. It's difficult to see how a separate section on 'testers' would be justifiable anyway. --Dazzla 15:58, 5 January 2006 (UTC)
thread_title:  Mislinks 

message: Testing analysis can be also  measured the bugs reported from site or from customer in the deployed application for the customer requirement. If bugs are releated to functional then it is better to again review the functional test case because may be there is possibility some testing or functional scanrios are missed out. 
I may be dim today, but I don't see what this means.  It certainly needs to be rewritten in better English.  --David.alex.lamb 20:27, 24 February 2006 (UTC)
thread_title:  Recent addition 

message: This might be a little confusing to readers: Beta testing redirects to this article, yet Beta test redirects to Development stage. They should be more consistent, because I happened upon the latter article after omitting the "-ing" suffix out of curiosity. – Minh Nguyễn (talk, contribs) 07:51, 23 April 2006 (UTC)
thread_title:  Beta test(ing) 

message: I've just removed a large section which was copied straight from . If people could keep an eye out in case it gets added back, I'd appreciate it. Shimgray | talk | 16:32, 23 April 2006 (UTC)
thread_title:  Copyvio 

message: I was wondering what peoples views are about what should, and should not, go in the External links section. There are many Open Source test tools, such as Bugzilla and the test case management tool QaTraq which are very relevant to Software testing. As a contributor to one of these tools I personally feel there is nothing wrong with listing these tools in the External Links section.
Personally I feel it's both useful and informative to add links to tools like these in the External Links section. However, before adding a link again I'd like to know if other people consider this type of link useful and informative under the Software Testing article. William Echlin
:I would not like to see such links added. Wikipedia is not a vehicle for advertising (regardless of whether or not the thing being advertised is open source). Wikipedia is also not a directory of links. Start adding links for a few tools, and pretty soon it everyone wants their favorite tools added to the list — a list that will soon grow to dominate the article. Style guidance for the use of external links can also be found here.
:There are plenty of other sites that provide directory services. Why not just include a few relevant links to such sites, such as the Open Directory Project directory of software testing products and tools, or SourceForge's testing category? These sites are likely to be far more comprehensive than anything that might be added here, and also far more likely to stay up to date. --Allan McInnes (talk) 21:31, 26 May 2006 (UTC)
replies: text: You make some good points there. I see now that this is not the right place for individual links to tools. Perhaps a single link to  OpenSourceTesting.org would be worth considering as well. Thank you for pointing me in the right direction to the 'Style guidance for External Links' too. I had been looking for something along these lines. You make a good custodian of this topic. William Echlin 08:06, 27 May 2006 (UTC)
replies: text: Thank you. I think a link to OpenSourceTesting.org would be fine. I'll add that, and a few of the directory links I mentioned, to the article. --Allan McInnes (talk) 16:58, 27 May 2006 (UTC)
thread_title:  External Links 

message: Some quotes were removed and I cannot seem to find the discussion that went with it, nor can I really see the reason or need for removal. Being new to looking at the wiki editing backgrounds I was wondering if this is common practise. If the removal is done based on a single persons view, could I edit it back (and gain little)? The main reason for this question is the quote "Software Testers: Depraved minds, usefully employed.'" -- Rex Black which I found very accurate and recognizable, which was removed. --Keeton 08:19, 11 July 2006 (UTC)
replies: text: I removed the quotes in question because the quotes section seemed to be getting large (the quote section is largely deprecated on Wikipedia, and is generally supposed to be kept small if it exists at all). The removed quotes were all credited to people who are apparently not even prominent enough to warrant a Wikipedia article (i.e. they were red-links).  The removal was a bold move on my part. If you believe the qujotes in question are useful and important, feel free to revert my changes. --Allan McInnes (talk) 14:29, 11 July 2006 (UTC)
thread_title:  Quotes 

message: Regression testing can be performed at unit, module, system or project level.
What is the difference between unit level and module level? (and may someone explain system level and project level as well?) Thanks, --Abdull 14:34, 27 July 2006 (UTC)
thread_title:  Levels 

message: Hi, I added the category skills, since software testing should be done by skilled, trained people. See here what some authors tell about skills in software tetsing field (with videos). To distinguish with certifications: certifications do not necessarily mean that a person working in the testing field is also skilled. --Erkan Yilmaz 15:25, 28 October 2006 (UTC)
thread_title: Skills

message: I think the section on code coverage is worthwhile information. But maybe it could be moved either to the code coverage article, and this article could still link to it, or it could be moved to the white box testing article. Any input? DRogers 14:41, 3 October 2006 (UTC)
thread_title: Code Ceverage

message: I'd like to see a better explanation of "seeding the database". I also disagree with the sentence "It can also be used of testers who know the internal workings or algorithm of the software...". Doesn't that describe white box exactly? 
Also, how do I suggest a cleanup of the Talk page? DRogers 16:52, 5 October 2006 (UTC)
replies: text: How would you like "seeding the database" explained? Also, you can disagree with that statement but it does not describe a white box tester. A white box tester has access to the code. A grey box tester does not. They only know the algorithms or the "internal workings". If you follow the "box" analogy, if you knew that there were levers and gears, as opposed to pneumatic actuators and hoses, inside the box but did not know their placement, you would be a grey box tester. A grey box tester can only attack the system through the UI or an API whereas a white box tester has the option to set break points or edit variables while the application is in use. --Walter Görlitz 07:28, 13 October 2006 (UTC)
thread_title: Gray box testing

message: Software is defined by a specification -- unless its hacked. Sofware test and measurement consists of stimulus against the product specification to determine defects in the embodiment. If that's not an editorial function, I don't know what is. Perhaps you object to the abstraction? Perhaps you object to the application of pre-checkin evaluation to preclude regression introduction? I've been in the software test engineering game for over 10 years now. The technique I outline has been successfully applied in very large and very small software factories. That curriculum does not exist previously describing it should not interfere with discussion about effectiveness.
Can the material be moved somewhere? I suppose so. But where does it belong? Seems appropriate to outline additional test engineering techniques under the subject of software testing.
--Rmstein 19:37, 5 October 2006 (UTC)
replies: text: I would say that it belongs in an article of the same name if anywhere at all.  The Software Testing article is already getting too large and would be much better just giving an overview of the main aspects of software testing, and I don't believe Editorial Viability Assessment is sufficiently notable to be in the main article.  The (large) image isn't particularly helpful, either.--Michig 19:49, 5 October 2006 (UTC)

text: Given my experience, I'd say I'm still at the introductory level. I haven't come across this terminology in this industry yet. That tells me that, like you said, maybe this is at the wrong level of abstraction. So maybe it belongs in its own article. But I'm not sure. Can you cite some sources or list some references, external links, etc.? DRogers 20:01, 5 October 2006 (UTC)

text: I ripped out the content and stuck it in a separate page, placing a link to it in the software testing page. Appreciate the feedback. I have not encountered any technical discussion of this abstraction in the literature. One sees lot about test-driven development, agile techniques, etc. which may or may not mirror the pre-checkin/post-integration evaluation technique I discuss. Depends on the software factory size primarily -- fewer talented authors are preferred to many mediocre authors. I've applied this methodology in large and small factories to stabilize releases. If the bits aren't stable, you've got a software toxic wastedump to manage. An editorial technique saves a little on kitty litter.  A lot of shops seem to churn and burn their customers, products, and organizational participants. Save for the product monopolies, a broken software factory that cannot embrace a proactive means to enforce continuous release-ready bit maintenance is usually doomed to the bit-bucket (in a globalized economy at least).-- Rmstein 12:24, 6 October 2006 (UTC)
thread_title: RE: Editorial Viability Assessment

message: Did you mean to say Exploratory versus Systematic software testing or even better yet, call Exploratory by its real name ad-hoc testing? The word scripted as it pertains to software testing is the pass tense of writing an automated script or test case. When explaining or describing Madonna life choices in Wikipedia I could understand the use of the word misunderstood. As sergeant Friday said in the TV show just the facts ma’am just the facts. 
MichaelDeady 22:07, 9 October 2006 (UTC)
thread_title:  Exploratory vs. Scripted 

message: Hi everybody, after getting a friendly remind by Pascal.Tesson I added here the roles of software testing, since the term software testers leads here.
The phases and goals of testing can be seen in: 
:Gelperin, D., and Hetzel, B. (1988): "The Growth of Software Testing," CACM, Vol. 31, No. 6
The roles of software testers I have from:
:Thomas Müller (chair), Rex Black, Sigrid Eldh, Dorothy Graham, Klaus Olsen, Maaret Pyhäjärvi, Geoff Thompson and Erik van Veendendal. (2005). Certified Tester - Foundation Level Syllabus - Version 2005, International Software Testing Qualifications Board (ISTQB), Möhrendorf, Germany.   (PDF; 0,424 MB).
I am thinking of adding also the phases of softwrae testing here - let's see, where I can add them at best. Searching...
--Erkan Yilmaz 17:37, 12 October 2006 (UTC)
thread_title:  Roles in software testing 

message: Hi,
Where is the reference in regards to the history paragraph, specificaly;
Drs. Dave Gelperin and William C. Hetzel classified in 1988 the phases and goals in software testing as follows: until 1956 it was the debugging oriented period, where testing was often associated to debugging: there was no clear difference between testing and debugging. From 1957-1978 there was the demonstration oriented period where debugging and testing was distinguished now - in this period it was shown, that software satisfies the requirements. The time between 1979-1982 is announced as the destruction oriented period, where the goal was to find errors. 1983-1987 is classified as the evaluation oriented period: intention here is that during the software lifecycle a product evaluation is provided and measuring quality. From 1988 on it was seen as prevention oriented period where tests were to demonstrate that software satisfies its specification, to detect faults and to prevent faults.
Im doing research on Software Testing and I would like to back this up outside Wikipedia, but im not finding others quoting the same thing (apart from one site, which copied and pasted the above into their page) or even similar.
Hoping you can help..
 
Sully 01:04, 22 March 2007 (UTC)
thread_title:  History 

message: The dates in a seemingly random order (I think as a result of vandalism but I haven't checked ==History=='s history to verify this). I agree with Sully that the statements are unsourced. The section generally doesn't read like part of an encyclopedia entry.  Lumpish Scholar 13:15, 18 May 2007 (UTC)
thread_title:  Call for cleanup of History section 

message: The 'Levels of testing' subsection and the 'Test levels' section need to be merged.  I'll do this at some point in the next few days if nobody else has.--Michig 15:30, 22 June 2007 (UTC)
replies: text: Done.--Michig 09:32, 26 June 2007 (UTC)
thread_title: Levels of testing/Test Levels

message: There is inconsistency on functional vs. non-functional testing and how they divide system testing. Currently, Functional Testing redirects to System testing which names types of tests that are also referenced in Non-functional tests. I am removing the functional/non-functional bullet points for now because it's confusing. Mikethegreen 14:08, 6 August 2007 (UTC)
replies: text:  I'm not sure why there's so much confusion about this. Functional testing tests functionality, i.e. the product is supposed to let me do this, but does it? Non-functional testing is everything else, i.e. performance, security, etc. And both of these typically fall under the black box umbrella. System testing tests the whole system. Some of these tests are functional, some non-functional. Maybe the confusion is because these are all different ways to view the same bunch of things, and not different things all together. DRogers 12:26, 8 August 2007 (UTC)
thread_title: Functional vs. non-functional?

message: In the german version we found an interwikilink from acceptance testing and software testing to the german (adopted from the english word)  translation Akzeptanztest resp. Softwaretest which is now handled in a complex article about system testing within a definition section. So to come back to the english article you may be forced to step back to the redirect! For english peope that should not be necessary, but whats with the other interwikis supposedly installed because of translations from the english version? --SonniWP 09:58, 22 August 2007 (UTC)
thread_title:  Interwikis 

message: Is Component testing analogous to Unit Testing as "each unit (basic component) of the software is tested to verify that the detailed design for the unit has been correctly implemented"?  If so would like to suggest Component testing, which is orphaned, is merged with Unit Testing.  —Preceding unsigned comment added by 87.80.105.132 (talk) 21:55, 3 September 2007 (UTC)
thread_title:  Component Testing 

message: Long references lists are generally useless if they are not sorted because readers will certainly not  click through all references and cannot a reference they like to continue reading without additional information on the subject. I suggest sorting the list using categories such as "Software Development Paradigms" (which emphasize testing), "Testing Methods" (Unit Tests, Smoke Tests), "Component Testing" (GUI Testing, Functional Testing) or "Formalized Testing Approaches" (Formal Verifikation, IEEE).  —Preceding unsigned comment added by 85.178.140.15 (talk) 16:57, 16 September 2007 (UTC)
thread_title:  "See also" Part 

message: I've taken the following to the talk page.  If these are axioms (and I'm dubious about the last two) then they will be written down somewhere.
==Software Testing Axioms==
# It is impossible to test a program completely.
# Software testing is risk based exercise.
# Testing cannot show that bugs don't exist.
# The more bugs you find, the more bugs there are.
# Not all the bugs you find will be fixed.
# Product specifications are never final.
JASpencer (talk) 20:24, 5 January 2008 (UTC)
thread_title:  Removed Software Testing Axioms 

message: I've removed this link to the talk page:
::*Software Testing Specialists - NMQA
If there is anything of interest within the website feel free to reinstate it linking to that particular page. JASpencer (talk) 21:34, 21 January 2008 (UTC)
thread_title:  Removing NMQA