From: celarier@reed.edu (Stuart Celarier)
Newsgroups: comp.os.os2.programmer
Subject: DosSetFHState() and Pipes -- problem
Keywords: OS/2 2.0, API, DosSetFHState, anonymous pipes
Date: 4 Sep 92 17:01:03 GMT
Article-I.D.: reed.1992Sep4.170103.24099
Organization: Reed College, Portland, OR

I have come across a problem using the DosSetFHState() API call in OS/2 2.0,  
which has to do with creating an anonymous pipe "cleanly".  I begin with some  
backgound on the problem, since there may be a better method for accomplishing  
the task which avoids this problem (wishful thinking).

Background.
-----------
In UNIX the fork() and exec() system calls are separate so that one can create  
a pipe using the pipe() call; perform the fork; and then close the parent's  
pipe handles in the child process before performing the exec().  This way the  
child does not inherit the extraneous pipe handles, which [1] would take up  
entries the child's file control structures, and [2] the child should never use  
in the first place.  This mechanism also makes it quite easy to establish the  
child's pipe handles as standard input and standard output so that the child  
knows where to find the pipe, or even better, is blissfully unaware of the  
existence of the pipe -- it is just doing standard I/O.

Creating a Pipe "Cleanly" Under OS/2 2.0
----------------------------------------
Under OS/2 the DosExecPgm() performs the function equivalent to the combination  
of UNIX's fork() and exec() functions, so providing a bidirectional pipe  
between parent and child is more complicated if one wants to do it cleanly.   
This is what I worked out, based on several sources.

	// create pipe0 -- child read, parent write
	DosCreatePipe();	// child read, parent write
	DosDupHandle();		// copy parent's stdin to tmpin
	DosDupHandle();		// copy child read to stdin

	DosQueryFHState();	// get parent write handle state
	DosSetFHState();	// set parent write handle "no inherit"
	
	DosQueryFHState();	// get tmpin handle state
	DosSetFHState();	// set tmpin handle "no inherit"

	// create pipe1 -- parent read, child write
	...			// as above

	// execute child
	//	child should only be inheriting stdin, stdout and stderr
	DosExecPgm();

	// clean up from pipe0
	DosDupHandle();		// copy tmpin back to stdin, closes child read
	DosQueryFHState();	// get stdin handle state
	DosSetFHState();	// set stdin handle "inherit"

	// clean up from pipe1
	...			// as above

That seems like an awful lot of work (comments anyone?), especially compared to  
the similar activity under UNIX.  However, it is necessary because I ultimately  
will be having this parent creating a large number of children, and if each  
child were to inherit its pipe handles, along with all of the handles of all  
the previously created children's pipes and all of the parent's handles for  
those pipes, well that's far too much overhead inheriting and keeping handles  
which _should_never_be_used_ in those children.  (Yes, I do know that I will  
ultimately have to increase the number of handles available in the parent  
process, the default of twenty will run out pretty quickly.)

Are there better, more efficient ways for creating the pipe cleanly?  Are there  
any reasons why this strategy should not work?  I've hit the books and on line  
references (hard), but none of them even mention not having the child inherit  
the parent's handles to the pipe.

DosSetFHState() Problem.
------------------------
This code fragement (with appropriate parameters and error checking) compiled  
just fine, but on execution every one of the DosSetFHState() calls returns an  
error code 87 -- bad parameter.  The actual code looks like this:

	...
	if ((rc = DosQueryFHState (hfParentWrite, &ulState) != 0)
		fatalApiError ("DosQueryFHState()", rc) ;
	if ((rc = DosSetFHState (hfParentWrite,
			ulState | OPEN_FLAGS_NOINHERIT) != 0)
		fatalApiError ("DosSetFHState()", rc) ;
	...

Even if I don't mask on the no-inherit flag, I still get a return code = 87.   
In this case I am just trying to set the state to what the query reported it to  
be!  (The actual state reported is 0x0011, which indicates that it _will_ be  
inherited.)

I have no way of telling _which_ of the two parameters DosSetFHState() thinks  
is bad, but I definitely am using the exact same handle in both cases, so I  
can't see why that should be the problem.  I know I need to only mask on the  
one flag in the state that I am interested in, and so I need to perform the  
query first, and then the set.  What is wrong with this picture?  Am I not  
using the call correctly, or is there a bug in the call?
	
Any answers, or suggestions on how to get to the root of the problem (I've  
tried, Lord knows I've tried...) would be greatly appreciated.

Thanks in advance,
Stuart Celarier
--
Stuart Celarier
celarier@reed.edu	PO Box 82842
503/236-9386		Portland, Oregon 97282