Talk:Generator (computer programming)

message: The article claims that generator were invented in Sather, but they date back significantly further.  Jeremy Hylton gives a short historical overview.  I could track them down as far as:
*Python (cites Icon, Sather and CLU)
**Sather (cites CLU)
***CLU (limited to only one active generator, due to stack sharing) (cites Alphard)
****Alphard (cites IPL-v)
*****IPL-v (this memoir mentions them in passing)
I couldn't find enough information about Alphard and IPL-v to know how they compare to later conceptions of generators.
--Piet Delport 13:55, 7 November 2005 (UTC)
replies: text: Changed article to say CLU instead of Sather.  CLU iterators are clearly generators.  Not sure about Alphard.  From Ms Liskov's article it sounds like Alphard didn't have yield.  But regardless, the new page is at least marginally more accurate.  ;) Jorend 23:01, 5 January 2006 (UTC)
thread_title: Origin of the concept

message: I've backed out this recent edit that introduces the wording "[Generators can be thought of as containing] a continuation (a programming language structure which can be thought of as a "frozen stack frame")".  It was well-intended (thanks!), but a bit misleading, as generators and continuations are quite different.  Tim Peters wrote a most excellent impromptu essay on generators, coroutines and continuations, and how they compare, but i'll attempt to summarize here:
* While a generator contains a single stack frame, a continuation contains a complete call stack.
* Invoking a generator resumes it on top of its caller's stack, and suspending/returning from the generator resumes the caller again (working almost exactly like normal function calls, in both cases (which is why generators are often explained as "resumable functions")).  Invoking a continuation, on the other hand, is like discarding the old call stack entirely, and replacing it with the one stored in the continuation (with no way to return to the "caller", unless its stack was previously saved as another continuation).
--Piet Delport 16:54, 12 February 2006 (UTC)
thread_title:  Generators versus continuations 

message: The generator code is recursive!!! In order to use the example to confirm my understanding of how a generator works is correct, I must first correctly understand ... generators! Clearly, that's a problem!
Could someone please provide a non-recursive example of the code, preferablely with a comment that describes what the expected mathematical outcome should be (ie. which mathematical "sum" is being calculated?). In that way, the reader can confirm for themselves that their understanding of the code matches the author's intent; and that their understanding of generators is therefore correct.
Right now, I *think* I understand what the code does, but since it doesn't include comments and I don't have a python compiler in front of me, I can't be sure that I've got it right... —Preceding unsigned comment added by 216.254.142.195 (talk • contribs) 2006-06-29 19:13
: I agree, the example is nearly uselessly opaque.  I replaced it with an infinite incrementing integer sequence generator (it's not as scary as its name :), which hopefully neatly illustrates generators' laziness and statefulness.  What do you think?  --Piet Delport 14:13, 30 June 2006 (UTC)
replies: text:  That's much better! Thank you! :-)
Just to provide another example in Icon... (using the same "logic" as the python code)..
procedure countfrom(n)
repeat {
    suspend n
    n +:= 1
}
end
# Example use: printing out the integers from 10 to 20.
# Note that this iteration terminates normally, despite countfrom() being
# written as an infinite loop.
every i :=  countfrom(10) do {
    if i <= 20 then
        write(i)
    else
        break
}
Another popular example is a generator for the Fibonacci sequence
procedure fibseq()
    local i, j, n
    i := 1
    j := 1
    suspend (i |              # Generate the 1st value
             j |              # Generate the 2nd value
             |{               # Repeatedly generate the next values
                n := i + j    # This value is returned
                i := j        # i and j to the "next" value
                j := n                           
               } )
end   
...
every write(|fibseq() \20)         # Will generate the first 20 numbers of the fibonacci sequence
CheyenneWills 05:20, 28 November 2006 (UTC)
thread_title:  Poor choice of example code 

message: It was contributed by User:PWnsivander the Great, who, while a nonobjectionable user, was blocked by User:Rspeer as a reincarnation of (unjustly, in the opinion of most of the board) banned user Mike Church. Shouldn't it, as the contribution of a banned user, be removed? Sentinel89 06:07, 13 March 2007 (UTC)
replies: text: I would say no.  The content should be judged on it's own merits regardless of any activity involving the user who initially contributed it.  I've reviewed the code and see nothing wrong with it, and in fact believe it to be an great example for demonstrating Python generators (in terms of ease of understanding, not necessarily efficiency).  My only hesitation would be the policy of WP:BAN, in that if these edits were made while the user was banned, then perhaps they could be reverted.  But the policy doesn't say such edits have to be reverted, only that they may be.  So I believe the content in this case is good enough to keep. Dmeranda 16:56, 16 March 2007 (UTC)

text: I agree, the quality of the content overrules the character of the contributor. However, the Python example's prime number generator is beyond me. In particular, the line "if not any( n % f == 0 for f in p ):" uses the variable "f" which is never created in the code. Any help?
dblanchard (talk)  —Preceding undated comment added 16:22, 5 October 2009 (UTC)
replies: text: Hi, the code within the brackets of the call to any is itself a generator expression. It iterates through p, the list of primes so far, assigning each to f and returns the value of n %f == 0 for each iteration.So it could be read colloquially as:   "if there are not any instances of a multiple of already found primes in n" --Paddy (talk) 17:06, 5 October 2009 (UTC)
thread_title:  Python primes generator contributed by banned user 

message: The definition at the very start of the article is very hard to parse.  It overuses analogy and is quite confusing.  Perhaps later in the intro section it might be comprehensible, but at 0300 in the morning w/ a non-working GRUB2 EFI configuration hanging over my head, I just ditched trying to read the article and leave this short note waiting for the rest of my update to download. 
The definition/intro is ambiguous and seems to muddle up the notion of what is being iterated, what the iterator is, and how it is implemented.    Try to stay at a very non-computer science language level with the initial definition or include the WP article link for every bit of jargon you need to be clear.  You could, however, start by saying that a generator is specific case of an Iterator...  However, nearly every CS topic has, someplace, a fairly clear definition using basic mathematics/logic terms that most reasonable educated individuals without a degree in CS can understand.  
What you should not say is that it is like an array, since it isn't anything like an array.  You might use a generator to access values in an array, but it isn't the array.  You can use a bucket to empty a bathtub full of soapy water, but don't confuse the bucket with the bathwater (and for heaven's sake, take the baby out first!).
Sorry, but most of my programming experience is with Java (w/o generators as a built in, but I am sure I have had to write thousands of them w/o ever knowing what the hell I was doing), or I would try my hand at a new intro.  — Preceding unsigned comment added by 75.73.1.89 (talk) 09:16, 13 January 2017 (UTC)
thread_title: Fuzzy or Hairy?

message: The Icon programming language, while not the first language to implement generators, was one of the first to fully integrate the concept into the language.  Within Icon, generators are not only used within the scope of loops, but are available throughout the language.  In addition while in a certain sense one could say that a generator returns a list of values, a better term is that a generator returns a result sequence.  Specifically, a generator could return an infinite number of values (e.g. a random number generator).
There was a lot of research and many papers written on generators.  In doing a date sorted search of the string "generator" in the ACM portal, the earliest reference I found (outside random number generators, and a couple of references to code generators) was
Mary Shaw, William A. Wulf, Ralph L. London
August 1977 	  	
Communications of the ACM,  Volume 20 Issue 8
Publisher: ACM Press
Full text available: 	pdf(1.28 MB)
	
Additional Information:	full citation, abstract, references, citings
The Alphard “form” provides the programmer with a great deal of control 
over the implementation of abstract data types. In this paper the 
abstraction techniques are extended from simple data representation and 
function definition to the iteration statement, the most important point 
of interaction between data and the control structure of the language 
itself. A means of specializing Alphard's loops to operate on abstract 
entities without explicit dependence on the representation ...
Keywords: abstraction and representation, abstraction data types, 
assertions, control specialization, correctness, generators, invariants, 
iteration statements, modular decomposition, program specifications, 
programming languages, programming methodology, proofs of correctness, 
types, verification 
Of interest, is that immediately following the above article in the same issue of the Communications of the ACM, was an article on CLU.
The next real reference to generators was the article written by the late Dr Ralph Griswold
Generators in Icon
Full text 	pdf formatPdf (1.03 MB)
Source 	ACM Transactions on Programming Languages and Systems (TOPLAS) archive
Volume 3 ,  Issue 2  (April 1981) table of contents
Pages: 144 - 161  
Year of Publication: 1981
ISSN:0164-0925
Authors 	
Ralph E. Griswold 	 Department of Computer Science, The University of Arizona, Tucson, AZ
David R. Hanson 	 Department of Computer Science, The University of Arizona, Tucson, AZ
John T. Korb 	 Xerox Corporation, 3333 Coyote Hill Road, Palo Alto, CA
Publisher 	
ACM Press   New York, NY, USA 
Quoting from the above
5. RELATED WORK
The concept of generators has appeared in various forms in many languages. An
early language that has generators is IPL-V [31], in which a generator is a
subroutine that calls a processing routine to operate on each object in a data
structure. Other languages, such as ALGOL 68 [38] and COBOL, use the term
generator to describe various language features, but this use is unrelated to the
Icon notion.
Languages that support coroutines frequently use the term generator to describe
a particular coroutine usage. SIMULA [1] is the oldest of such languages;
more recent examples include extensions to POP-2 [24] and SL5 [20] and
coroutine additions to PASCAL [27]. There also has been a substantial amount
of work on the "lazy evaluation" approach to the incremental generation of
sequences, usually in LISP or APL frameworks [7, 17, 22].
More recently, ALPHARD [43] and CLU [28] have been developed to support
data structure abstraction. In these languages generators are used to iterate over
the elements of programmer-defined data structures [35]. Included with the
definition of a data abstraction (or "cluster" in CLU) is a procedure for generating
the elements of the abstraction. In CLU, this procedure, called an iterator,
produces its values using the yield statement. When another element is required,
execution of the iterator continues where the yield statement left off. In each of
these languages, generators are only accessible in a specific context: a particular
type of for statement. This is significantly more restricted than in Icon, where
generators may appear anywhere.
Superficially, generators in ALPHARD and CLU appear to correspond to the
every expression in Icon. But while every has the syntactic appearance of a
standard control structure, the nonstandard goal-directed evaluation mechanism
makes its semantics more general than the corresponding ALPHARD and CLU
constructs. It is the restricted applicability and the lack of goal-directed evaluation
that differentiate these constructs from generators in Icon.
ACM Transactions on Programming Languages and Systems, Vol. 3, No. 2, April 1981.
158 R.E. Griswold, D. R. Hanson, and J. T. Korb
CheyenneWills 20:49, 25 June 2007 (UTC)
thread_title:  Suggestion for clarifications, definitions, etc.  

message: In D generators can be done by implementing an opApply function (overload for a 'foreach'). This page shows a good example at the end with primes numbers: .
Could this page include a D example thanks.
Tim M.  —Preceding unsigned comment added by 125.238.96.209 (talk) 09:22, 6 July 2009 (UTC)
thread_title: D Generators

message: I think the current text is messed up to deserve rewrite. I think that generators have little in common with iterators, except a common usage. A generator is a function with a static persistent environment, which is preserved from call to call, cf. local static declaration in C and own in good old Algol-60. Whether used as iterator or not is irrelevant. You cannot create different instances of generators like you can when dealing with iterators, or put it another way: there's only one generator instance ever, and it is "created" at startup. Iterator instances are always created to dynamically produce a sequence, while the generator just preserves a state. When created, the iterators take an argument, often an array or some other enumerable, but not always – this in opposition to the generator, which is not "reset/reinitialized" in a call different from a normal generator call producing the next item. Rursus dixit. (mbork3!) 07:04, 21 June 2010 (UTC)
replies: text: No, I'm confused by the Python generators definition. Forget it! Rursus dixit. (mbork3!) 07:33, 21 June 2010 (UTC)
thread_title: Iterator confusion

message: the article opens by characterizing ICON as a string manipulation language.
I think that if you look at ICON 9.5 this will not seem an appropriate characterization.  Was ICON not the first expressly free language (after accidental SNOBOL 'release'? )
ObjectIcon at code.google.com/p/objecticon is surely more than a string manipulation language
ICON is better characterized as a free expression-based language with goal-directed facilities 
For ICON for pythonistas, see convergepl.org
G. Robert Shiplett 14:34, 29 November 2010 (UTC)
thread_title:  ICON as a language 

message: This article should describe in-detail the most common ways generators are implemented at the low-level. This means information such as if a program with generators is compiled to an assembly language, what would it look like? What happens to the control flow? What happens to the stack? Is a new stack created? Where does it keep the information about where in the generator control flow is resumed to after the yield statement?
--Eboyjr (talk) 17:02, 16 June 2011 (UTC)
thread_title:  Standard or common implementation of generators at the low-level 

message: (Disclaimer: my view of "what a generator is" and "what they are good for" comes chiefly from my experience with Python.)
The article -- and its examples -- are very scattered.  They even include examples of "generating the numbers from 10 to 1".  To someone who doesn't understand the power of or purpose of generators, this will be a highly deluminating example, as they could [correctly] disparage the whole idea: "why wouldn't I just do that in a loop, decrementing a counter?"
My suggestions:
* Point out that generators are really good for generating INFINITE sequences... just a little bit at a time, as the "consumer" needs the "next" item.
* Point out that most useful generators need to store a complicated state.  (It is not just a simple unary test like "is this number even", which -- again -- can be accomplished declaratively and used as a test in a loop.)
* Point out that languages that "support generators" do so by giving the programmer a simple way to do BOTH of those: generate "infinite sequences incrementally" and "save their state between calls".
* Pick a simple example that uses BOTH of those features.  Such as: calculating the Fibonacci sequence.  Let's see a Fib() generator in all the languages discussed.
* Point out that recursion and generation can each be used on certain calculations, but they are NOT synonymous.
* Compare and contrast a generator with a memoized recursive routine.  (You can quickly calculate Fib(10000000) with a memoized recursive Fib, at the expense of the memory for the 1000000 prior entries.  However, with a generator-based Fib(), constant memory (just two state variables) will serve to walk through all the values of Fib forever.) 
If I have time someday, I may attempt to update the article with some of these suggestions.
Dharasty (talk) 13:10, 3 May 2012 (UTC)
thread_title:  Needs common example; Should be an infinite sequence 

message: The dates in this refer to the date the programming language first appeared, which is often *not* the same date as when generators appeared for this language.
For example if you would glance at this article, you would get the impression that:
* Generators were first introduced in Lisp in 1958 (not so, generators are a recent addition to Lisp, and AFAIK not even standardized in Common Lisp)
* PHP has generators since 1995 (It's implemented and available in the `latest source', but available in a general release yet).
etc... IMHO this is rather misleading ... Someone should fix the dates to reflect whenever this feature was added to a language (if it wasn't there from the start) or the dates should just be removed.
Carpetsmoker (talk) 10:00, 11 March 2013 (UTC)
thread_title:  Dates are incorrect. Fix or maybe even remove?