HDTenEightyP/NetNews huggingface dataset:
From: peterw@spaten.sharebase.com (Peter Wisnovsky)
Newsgroups: comp.std.unix
Subject: Re: POSIX update
Date: 14 Aug 1992 13:33:06 -0700
Organization: NCR/ShareBase Corporation
Reply-To: peterw@spaten.sharebase.com (Peter Wisnovsky)
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: peterw@spaten.sharebase.com (Peter Wisnovsky)

In article <16ccqfINNrss@ftp.UU.NET> decot@cup.hp.com (Dave Decot) writes:
>POSIX.1 and POSIX.2 say nothing whatsoever
>about wchar_t.  ISO C makes no requirement conflicting with using Unicode
>(or any 16 bit codeset where there is an all-zero code available for
>the null wide character) in wchar_t.

Perhaps I am mixing up my terminology or not expressing myself clearly
(where is Marlin Fitzwater when you need him?) My knowledge of the
various standards organizations is somewhat sketchy. However, my
impression of what the potential problem is that if the standards orgs
went ahead and used wchar_t as the vehicle for supporting Unicode,
then existing conformant implementations that define wchar_t to be an
8-bit or 32-bit value would be made non-conformant. NT is not
non-conformant in itself...but the solution it proposes to the storage
of Unicode data, that is, the fixing of the size of wchar_t to be
16=bits wide, would not work with existing conformant systems.

Two proposals were discussed at the Unicode conference wrt XPG. One
would be to have the standards changed so that wchar_t would be
defined to be 16-bits wide; the other would be to create a new
datatype, `unichar'. The sentiment at the conference was to create a
new datatype.

Peter Wisnovsky
-- 

UUNET:   peterw@sharebase.com
US Mail: ShareBase Corporation - 1919 Addison Ave. - Berkeley, CA 94704
Phone:   (510) 548-3211


Volume-Number: Volume 28, Number 97
From: gwyn@smoke.brl.mil (Doug Gwyn)
Newsgroups: comp.std.unix
Subject: Re: POSIX update
Date: 14 Aug 1992 23:02:18 -0700
Organization: U.S. Army Ballistic Research Laboratory, APG, MD.
NNTP-Posting-Host: ftp.uu.net
Keywords: posix, unicode, iso10646, microsoft windows nt
X-Submissions: std-unix@uunet.uu.net

Submitted-by: gwyn@smoke.brl.mil (Doug Gwyn)

In article <16b9drINNero@ftp.UU.NET> peterw@spaten.sharebase.com (Peter Wisnovsky) writes:
>In article <166v0lINNpj8@ftp.UU.NET> mckee@imail.compaq.com (George McKee) writes:
>>Not to mention the reports that Microsoft NT will be POSIX compliant when
>>it appears.
>Not really in terms of character set support. My understanding is that
>they are going to fix wchar_t at 16 bits to store Unicode data, which
>strictly speaking is non-conformant.

What's not standard conforming about that?  Indeed, it is not possible
to treat Unicode as a "multi-byte character encoding" according to the
C standard; there is a discussion of this in C Information Bulletin # 1.

>Unfortunately there seems to be no satisfactory way to imbed Unicode
>within the EUC framework used in AT&T's MNLS system, which I understand
>is their implementation of Posix international language support.

The promoters of Unicode were informed of the (numerous) problems in
fitting Unicode into the existing practice, reflected in Standard C
multibyte character support, but basically they responded that
character sets were more fundamental than programming languages and
that they would therefore ignore the impact of their work on
programming languages, it being the languages' responsibility to
use whatever they came up with.  Some attitude!


Volume-Number: Volume 28, Number 99
From: mengel@dcdmwm.fnal.gov (Marc W. Mengel)
Newsgroups: comp.std.unix
Subject: Re: Embedded data in shell scripts?
Date: 14 Aug 1992 14:08:51 -0700
Organization: UUNET Communications
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: mengel@dcdmwm.fnal.gov (Marc W. Mengel)

davidf@mks.com ("David J. Fiander") writes:
>This seems to be attempting to ensure that a script like this
>	foo=`sed 1q`
>	hi there
>	echo $foo
>	echo Done
>When fed to the shell on standard input will generate the output
>	hi there
>	Done

This only works well on tty-style devices, where reads return
on end-of line boundaries.  It is also why the shell has
here-documents, so you can say

	foo=`sed 1q <<EOF
	hi there
	EOF`
	echo $foo
	echo Done

to to the job right in a script, pipeline, etc.

-------
Marc Mengel
mengel@fnal.fnal.gov


Volume-Number: Volume 28, Number 98
From: davidf@mks.com (David J. Fiander)
Newsgroups: comp.std.unix
Subject: Re: Embedded data in shell scripts?
Date: 17 Aug 1992 14:38:26 -0700
Organization: Mortice Kern Systems Inc., Waterloo, Ontario, CANADA
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: davidf@mks.com (David J. Fiander)

According to mengel@dcdmwm.fnal.gov (Marc W. Mengel)
>It is also why the shell has
>here-documents, so you can say
>
>	foo=`sed 1q <<EOF
>	hi there
>	EOF`
>	echo $foo
>	echo Done
>
>to to the job right in a script, pipeline, etc.

I'm quite aware of that.  The problem is that the standard
seems to be attempting to allow my example in scripts and, in
my opinion, failing badly.

- David


Volume-Number: Volume 28, Number 101
From: johnl@iecc.cambridge.ma.us (John R. Levine)
Newsgroups: comp.std.unix
Subject: Re: Embedded data in shell scripts?
Date: 18 Aug 1992 14:20:26 -0700
Organization: I.E.C.C.
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: johnl@iecc.cambridge.ma.us (John R. Levine)

>The standard says (lines 11645--11651)
>
>	When the shell is using standard input and it invokes a
>	command that also uses standard input, the shell shall
>	ensure that the standard input file pointer points
>	directly after the command it has read when the command
>	begins execution.  ...

I don't see how you can have it any other way.  If the shell is reading
directly or indirectly from a terminal I certainly wouldn't be too pleased
to have the shell eat arbitrary amounts of text after every command.  And
I'd be equally unhappy if I saved the input stream into a file and
couldn't replay it.

The efficiency argument seems to me to be a red herring.  When you pipe
large numbers of commands into the shell, if there aren't any loops in the
command list, the time will most likely be dominated by the time spent
running the commands.  If there are loops, the looping commands are read
once and buffered inside the shell anyway.  I agree that the distinctions
between seekable and non-seekable files are ugly, but it's too late to
legislate them out of existence.

Quite a while ago someone (Dennis Ritchie?) suggested that it would have
been a good idea to provide in Unix a version of read() that would read up
to but not past a terminator byte, independent of the data source.  That
would have let programs read a line at a time with reasonable efficiency.

Regards,
John Levine, johnl@iecc.cambridge.ma.us, {spdcc|ima|world}!iecc!johnl

Volume-Number: Volume 29, Number 4
From: davidf@mks.com ("David J. Fiander")
Newsgroups: comp.std.unix
Subject: Embedded data in shell scripts?
Date: 14 Aug 1992 13:22:59 -0700
Organization: Mortice Kern Systems Inc., Waterloo, Ontario, CANADA
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: davidf@mks.com ("David J. Fiander")

All references are to POSIX 1003.2 D11.2, unless stated
otherwise.

The standard says (lines 11645--11651)

	When the shell is using standard input and it invokes a
	command that also uses standard input, the shell shall
	ensure that the standard input file pointer points
	directly after the command it has read when the command
	begins execution.  It shall not read ahead in such a
	manner that any characters intended to be read by the
	invoked command are consumed by the shell (whether
	interpreted by the shell or not) or that characters
	that are not read by the invoked command are not seen
	by the shell.


This seems to be attempting to ensure that a script like this

	foo=`sed 1q`
	hi there
	echo $foo
	echo Done

When fed to the shell on standard input will generate the output

	hi there
	Done

But the standard also says that (lines 5467--5471)

	When a standard utility reads a seekable input file and
	terminates without an error before it reaches
	end-of-file, the utility shall ensure that the file
	offset in the open file description is properly
	positioned just past the last byte processed by the
	utility.  For files that are not seekable, the state of
	the file offset in the open file description for that
	file is unspecified.

This means that the above script is _only_ guaranteed to work
in the case of

	sh <script

since sed is required to ensure that the file offset of the
seekable file script is properly positioned when it exits.

	cat script | sh

does _not_ have to work, according to the standard.  Yes, the
standard ensures that sed will read the right data, but the
shell may miss a large chunk out of the middle of the script
because sed is permitted to perform a buffered read, and not
reset the file offset pointer.

In fact, the only case in which both 'sh < script' and
'cat script | sh' will give the same answer on all systems is
that case in which the embedded data is all at the end of the
file, and that _all_ of that data will be consumed by the last
command executed by the shell.  That is:

	.
	.
	.
	filter -args
	data
	data

Where filter will use all of the data.

What is most unfortunate is that in order to support this one
case, there will have to be a lot of code in the shell which,
when the script is be read from standard input and standard
input is not a seekable file, reads the script unbuffered, a
single byte at a time, in order to ensure that the last command
in the file get the data it is expecting.

Is it really worth it?



Volume-Number: Volume 28, Number 93
From: david@mks.com (David Rowley)
Newsgroups: comp.std.unix
Subject: Re: ISO 10646 files
Date: 18 Aug 1992 14:19:06 -0700
Organization: Mortice Kern Systems Inc., Waterloo, Ontario, CANADA
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net

Submitted-by: david@mks.com (David Rowley)

In article <16p6bmINNs1l@ftp.UU.NET> mskuhn@immd4.informatik.uni-erlangen.de (Markus Kuhn) writes:
>But how may UCS-4 files be identified? Do they always begin with 0000feff
>and are converted if they begin with fffe0000 or other permutations?
>Does ISO 10646 say anything about this or will any future POSIX extension do?

Being relatively new to ISO 10646, I believe the intent is to
use the UCS Transformation Format (Annex F of 10646) as the
standard external representation format (such as file contents,
etc.).  This multibyte encoding supports both the UCS2 and UCS4
codeplanes.

Note that UTF and 8-bit Latin 1 (ISO 8859-1) are identical for
characters 0x00 to 0x9f.  Codepoints above 0x9f are used to
introduce the multibyte sequences.

One problem, though, is that the UTF description in ISO 10646 is
informative, rather than normative.  With this being the case
can implementors safely point to UTF as a standard encoding?

-- 
David Rowley
Mortice Kern Systems Inc.
35 King Street North, Waterloo, ON, Canada N2J 2W9
519/884-2251, FAX 519/884-8861, david@mks.com


Volume-Number: Volume 29, Number 3
From: thorinn@diku.dk (Lars Henrik Mathiesen)
Newsgroups: comp.std.unix
Subject: Re: POSIX update
Date: 17 Aug 1992 14:42:30 -0700
Organization: Department of Computer Science, U of Copenhagen
NNTP-Posting-Host: ftp.uu.net
Keywords: posix, unicode, iso10646, microsoft windows nt
X-Submissions: std-unix@uunet.uu.net

Submitted-by: thorinn@diku.dk (Lars Henrik Mathiesen)

gwyn@smoke.brl.mil (Doug Gwyn) writes:
>The promoters of Unicode were informed of the (numerous) problems in
>fitting Unicode into the existing practice, reflected in Standard C
>multibyte character support, [...]

Why is it so bad that Unicode is not like earlier ``extended
encodings''? Existing code is a large problem, of course, but what is
the problem relative to Standard C itself? The standard provides both
wide and multibyte characters, and Unicode happens to fit the former
definition instead of the latter.

  1) Of course it would be nice to be able to use the good old library
  calls with the fancy new character set. So, if your text encoding fits
  into EUC or whatever, it will work as Standard C multibyte characters,
  but what does that mean? As far as I can see, you cannot use strcat,
  index, or printf("%s") on a multibyte string anyway, you need extra
  magic to make sure that shift states match properly.

  2) For the programmer, it seems easier to just use wchar_t strings;
  the usual string-handling idioms ("while (*cp++)") carry over with
  just a change of declarations, and it is possible to implement direct
  equivalents of every string-oriented function in the library.

  (When multibyte characters are used, some of the idioms will work, but
  everything that tries to interpret the characters, even just to find a
  '/', will have to use mbtowc/mbstowcs or a non-standard routine. If
  you have to convert to wchar_t anyway, what's the point?)

  3) There is a problem in that the standard defines wide character
  constants and wide string literals in terms of multibyte characters.

  However, Unicode does define a ``data interchange'' format (UTF), for
  transmission through channels that like to mung control characters.
  This format happens to be usable as a Standard C multibyte encoding as
  well --- it maps the source character set to itself, uses NUL only to
  code NUL, and has no shift state as such; after editing C source in a
  Unicode editor, you can save it ``as UTF'' and the result will be
  nicely acceptable to an 8-bit C compiler with the right default
  locale. And when you upgrade to a Unicode C compiler, you can convert
  your source again.

If the problem is that POSIX wants to insist on a subset of Standard
C, or C bindings for some system functionality, that cannot work with
Unicode; then I have very little sympathy: The direction that Unicode
was taking must have been clear well before anything within POSIX was
finalized, and disregarding it may well turn out to have been, let's
say, infelicitous.

Lars Mathiesen (U of Copenhagen CS Dep) <thorinn@diku.dk> (Humour NOT marked)


Volume-Number: Volume 28, Number 103
From: al@cvgis.prime.com (Al Parker)
Subject: Embedded SQL ANSI Std - Help
Organization: Computervision GIS (Toronto)
X-Newsreader: Tin 1.1 PL4
Date: Thu, 13 Aug 1992 21:06:10 GMT

Greetings,
I am dealing with various RDBMS packages and am finding that there
seems to be considerable latitude in their implementation of
embedded SQL.
Is this because the standard was not official before the
implementations were put out ?

At any rate, I would like a copy of the ANSI standard which governs
embedded SQL. If this is ONLINE somewhere can someone point me to the
source ?

Thanks in advance, AlP
-- 
-------------------------------------------------------------------------------
Al Parker, P.Eng, Sr Software Engineer - Computervision GIS
al@cvgis.prime.com	(416)508-8020(x881)
These opinions are my own and not those of my compnay.
Newsgroups: comp.std.misc
From: delacour@waxwing.parc.xerox.com (Vincent Delacour)
Subject: Re: Common Lisp is a dpANS in Public Review
Followup-To: comp.std.misc
Organization: Xerox PARC, Palo Alto, CA
	<1992Aug21.163403.7918@iecc.cambridge.ma.us>
Date: 21 Aug 92 15:19:43


In article <1992Aug21.163403.7918@iecc.cambridge.ma.us> johnl@iecc.cambridge.ma.us (John R. Levine) writes:

   >if one were to [put copies of the Lisp standard on CD-ROM], it should
   >probably include beaucoup other ANSI std dox also, for many other prog
   >languages ...

   ANSI gets most of their revenue from selling their copyrighted standards
   documents, which cost about $50 each.  So a CD-ROM with copies of 25
   standards would have to cost about $1000 to give them similar revenue.

Perhaps not so true. Producing paper copies has a cost too, whereas
duplicating CD-ROMs is very cheap. See for example the Oxford English
Dictionary, which costs $3000 in paper, and $850 only on CD-ROM, with
lots of extra functionalities. 

	[vd]
Newsgroups: comp.std.internat,comp.misc
From: mskuhn@immd4.informatik.uni-erlangen.de (Markus Kuhn)
Subject: Re: ISO 10646 the final character set?
Organization: CSD., University of Erlangen, Germany
Date: Thu, 13 Aug 1992 07:00:33 GMT

What means UCS (after ISO 10646) in Erik Naggum's .sig? United Character
Set? What other terms are defined in ISO 10646 (MBP, ...)?

Markus

-- 
Markus Kuhn, Computer Science student -=-=- University of Erlangen, Germany
Internet: mskuhn@immd4.informatik.uni-erlangen.de  |  X.500 entry available
-A distributed system is one in which the failure of a computer you didn't-
-even know existed can render your own computer unusable. (Leslie Lamport)-
Newsgroups: comp.std.internat,comp.misc
From: sakkinen@jyu.fi (Markku Sakkinen)
Subject: Re: ISO 10646 the final character set?
Organization: University of Jyvaskyla, Finland
Date: Fri, 14 Aug 1992 05:16:09 GMT

In article <jenkinsj-130892151628@tseng.taligent.com> jenkinsj@blowfish.taligent.com (John H. Jenkins) writes:
>In article <23303A@erik.naggum.no>, enag@ifi.uio.no (Erik Naggum) wrote:
>> ...
>> That's the abbreviation after the standard's full name, which is (was,
>> at DIS stage, anyway):
>> 
>> 	Information technology -- Universal Multiple-Octet Coded
>> 	Character Set (UCS)
>> 
>> I seem to remember that somebody wanted "multiple-octet" removed from
>> the title.  Anybody else know the details on this?
> ...
>The problem with the name "multiple-octet" is that it implies that 10646
>text can be processed as a series of bytes instead of as a series of words
>or longwords.  This can be disastrous in C because of the NUL octets in
>some character codes.   Changing the name to avoid this implication would
>be nice, but ISO regulations would consider changing the standard's name a
>major revision, which would require a third ballot, and *nobody* wanted a
>third ballot. 

I don't think that the stupid idiosyncrasies of one particular
programming language should be taken into account when a long-time
universal character code is defined.

----------------------------------------------------------------------
Markku Sakkinen (sakkinen@jytko.jyu.fi)
       SAKKINEN@FINJYU.bitnet (alternative network address)
Department of Computer Science and Information Systems
University of Jyvaskyla (a's with umlauts)
PL 35
SF-40351 Jyvaskyla (umlauts again)
Finland
----------------------------------------------------------------------
Xref: sparky comp.std.internat:620 comp.misc:3232
From: enag@ifi.uio.no (Erik Naggum)
Newsgroups: comp.std.internat,comp.misc
Subject: Re: ISO 10646 the final character set?
Date: 14 Aug 92 17:48:44 GMT
Organization: Department of Informatics, University of Oslo, Norway

Markku Sakkinen <sakkinen@jyu.fi> writes:
|
|   I don't think that the stupid idiosyncrasies of one particular
|   programming language should be taken into account when a long-time
|   universal character code is defined.

There are such things as "installed base" to consider when doing things.
Although I share your emotional reaction, I also know that standards
that change too much of the way things work will never fly.  I'd like to
see ISO 10646 fly.

I don't know whether this is relevant, but ISO 10646 suffers from "byte
order" problems when it's a word or half-word (word being 32 bits) code,
whereas well-known multiple-octet coded character sets such as JIS X0208
define a byte order.  ISO 10646 has a "signature" (FEFF) and an unused
(or prohibited, actually) value which corresponds to the byte-swapped
signature (FFFE), and so allow brain-damaged hardware to look like real
hardware.  (Talk about idiosyncracies!)

Aside from the hardware problem which makes it impossible to view raw
ISO 10646 as a word or half-word code to portable C programs, the zero
terminator could be defined to be a word or half-word, and everybody
would be happy.

Best regards,
</Erik>
--
Erik Naggum             |  ISO  8879 SGML     |      +47 295 0313
                        |  ISO 10744 HyTime   |
<erik@naggum.no>        |  ISO 10646 UCS      |      Memento, terrigena.
<enag@ifi.uio.no>       |  ISO  9899 C        |      Memento, vita brevis.
Newsgroups: comp.std.internat
From: jenkinsj@blowfish.taligent.com (John H. Jenkins)
Subject: Re: ISO 10646 the final character set?
Followup-To: comp.std.internat,comp.misc
Organization: Taligent, Inc.
Date: Fri, 14 Aug 1992 15:28:06 GMT

In article <1992Aug14.051609.8673@jyu.fi>, sakkinen@jyu.fi (Markku
Sakkinen) wrote:
> 
> I don't think that the stupid idiosyncrasies of one particular
> programming language should be taken into account when a long-time
> universal character code is defined.
> 

Hearty agreement here.  The problem came from the fact that a lot of
organizations with a lot of clout were fighting 10646 tooth and nail
because of its perceived incompatibility with C.  The name-change
proposal is an indication of how desperate people were getting in trying
to combat this misperception. 


----
John H. Jenkins
John_Jenkins@taligent.com

Xref: sparky comp.std.internat:616 comp.misc:3216
From: enag@ifi.uio.no (Erik Naggum)
Newsgroups: comp.std.internat,comp.misc
Subject: Re: ISO 10646 the final character set?
Date: 13 Aug 92 17:37:59 GMT
Organization: Department of Informatics, University of Oslo, Norway

Markus Kuhn <mskuhn@immd4.informatik.uni-erlangen.de> writes:
|
|   What means UCS (after ISO 10646) in Erik Naggum's .sig? United Character
|   Set? What other terms are defined in ISO 10646 (MBP, ...)?

That's the abbreviation after the standard's full name, which is (was,
at DIS stage, anyway):

	Information technology -- Universal Multiple-Octet Coded
	Character Set (UCS)

I seem to remember that somebody wanted "multiple-octet" removed from
the title.  Anybody else know the details on this?

Best regards,
</Erik>
--
Erik Naggum             |  ISO  8879 SGML     |      +47 295 0313
                        |  ISO 10744 HyTime   |
<erik@naggum.no>        |  ISO 10646 UCS      |      Memento, terrigena.
<enag@ifi.uio.no>       |  ISO  9899 C        |      Memento, vita brevis.
Newsgroups: comp.std.internat
From: jenkinsj@blowfish.taligent.com (John H. Jenkins)
Subject: Re: ISO 10646 the final character set?
Followup-To: comp.std.internat,comp.misc
Organization: Taligent, Inc.
Date: Thu, 13 Aug 1992 22:26:27 GMT

In article <23303A@erik.naggum.no>, enag@ifi.uio.no (Erik Naggum) wrote:
> 
> Markus Kuhn <mskuhn@immd4.informatik.uni-erlangen.de> writes:
> |
> |   What means UCS (after ISO 10646) in Erik Naggum's .sig? United Character
> |   Set? What other terms are defined in ISO 10646 (MBP, ...)?
> 
> That's the abbreviation after the standard's full name, which is (was,
> at DIS stage, anyway):
> 
> 	Information technology -- Universal Multiple-Octet Coded
> 	Character Set (UCS)
> 
> I seem to remember that somebody wanted "multiple-octet" removed from
> the title.  Anybody else know the details on this?
> 

I don't know if it came up at WG2 in Seoul, but it did come up at X3L2 last
month.  
I'd have to actually dig through my list of country comments to see if
anybody in fact suggested the name change (I believe somebody did), but I'm
too lazy to do that right now.  

The problem with the name "multiple-octet" is that it implies that 10646
text can be processed as a series of bytes instead of as a series of words
or longwords.  This can be disastrous in C because of the NUL octets in
some character codes.   Changing the name to avoid this implication would
be nice, but ISO regulations would consider changing the standard's name a
major revision, which would require a third ballot, and *nobody* wanted a
third ballot. 

John H. Jenkins
John_Jenkins@taligent.com
Xref: sparky comp.std.internat:614 comp.misc:3207