From: medawar@prism.poly.edu (Bassem Medawar)
Subject: Re: Some thoughts on Arabic aphabet news reader
Keywords: ARABIC PERSIAN GUI COMPUTER
Organization: Polytechnic University, New York
Date: Wed, 7 Oct 92 21:35:04 GMT


Hi Anoosh and all who replied to me in email,

Thank you for your post and emails, wow!  Down to business, I'll first comment 
on Anoosh's message, then make suggestions of my own.  Anoosh writes a lot of 
interesting things.  I am not going to quote all of them.

In article <ld3jefINN418@spim.mti.sgi.com> anoosh@mips.com (Anoosh Hosseini) writes:

>We had our own escape sequence method, but with new specs coming out, I
>think we will go with Internet standards.

Can you be more explicit, which standards, in this case, you are refering to?

>The English part will always be readable, and required for transport on
>Internet.  Use of Persian/Arabic with be still in Latin characters, but 
>that's for transmission only, they would be representing ISO 8859-6 
>encoding or equivalent. Read below.

Actually what I had in mind was a message in the form:

- Internet header
- Latinized Arabic form of the message automatically converted from the Arabic
  text for those who don't have the capability to display Arabic
- keyword to start arabic coded text
- Arabic coded text

That idea is not only to make the transport possible but to allow those who
won't have the capability to display the Arabic text to be able to read the
Latinized version of it.  Automatic translation from Arabic to Latin may not
be easy because in Arabic Harakaat (accents) are often not represented.  This
would correspond more or less to a Latinized text without vowels.  My name
for example without Harakaat becomes "b\aasm" (assuming that \ is the escape
character for doubling and here for example \aa is the Arabic Aleph.  This
is much less readable than the way I am used to write my name "bassem".  If
we make provisions for using Harakat and I use a kassra under the sin, then my 
name becomes "b\aasim" which is still different from the way I am used to
write it but more readable.  Note, that I am here talking about automatic
translation.  This does not preclude others from using the standard to write
Latinized Arabic (please no flames).


>        we have a demo model, but its not complete, the Persianization 
>needs a little cleanup. My colleague Bahman with a team has developed a
>custom news reader on the PC which works on a file structure similar
>to the one on UNIX.  If one has PC-NFS it can work directly from the
>UNIX tree. I  hope to have my responsibilities done by early 93. 

Maybe at some point, you will make the demo model available to try it and
for comments.  Unfortunately, the installed base of PC-NFS is very small.
Not many will benefit from it.

>                                    Microsoft is working on Arabic
>Windows, (to be released some time this year). With the aid of their
>tools it will be easier to develop such applications.  

That's great, except they're probably only going to market it in the Middle East
and make it hard to get in the US.  I had to have someone buy the IBM arabic
DOS extension from Saudia Arabia and send it to me because I couldn't buy it
here.  That was some time ago and as soon as I upgraded from DOS 3.2, I couldn't
use it anymore.

>                   So X11 will continue to be the path for future work.
>8 bit clean UNIX systems are now standard, before you had to get
>custom versions of UNIX.  Any implementation of SVR4 has national
>language support and a methodology for Internationalization.

I agree that X will continue to be a viable environment in years to come but
many don't have it.  SVR4 will take some time to replace SVR3.  AT&T for example
will not release SVR4 for the old 3B2's (non-RISC).  The non-RISC-3B2's are 
popular in academia because AT&T donated them when they didn't succeed 
commercially.  So a lot of people are stuck with 3B2's that won't be upgraded 
to SVR4.

As far as the ISO 8859-6 (Arabic 8-bit encoding set), is there any possibility 
you can scan it?  I have on the other hand the ASMO-449 and a couple sets 
derived from it.  (ASMO stands for Arab Standards and Metrology organization)

>RFC1341: MIME  (Multipurpose Internet Mail Extensions):
>Mechanisms for Specifying and Describing the Format of Internet 
>Message Bodies.

I need to do some research on this, it was also suggested to by email by 
another person.

>RFC1342: Representation of Non-ASCII Text in Internet Message Headers

And here too, some research is needed.

What is ISO 10646?  Some sort of encoding?  And I take it that RFC 822 defines
7 bit email.

>ON PC/MAC:
>Have a custom/public domain news reader which either has source code,
>or the user can specify his/her choice of  "editor"  for viewing 
>messages. Replace editor with custom bilingual editor. Files are brought
>onto the PC/MAC through several channels. It is even possible to do
>NNTP over serial lines. Lots of options here.


Following is my porposal: we need to formalize a model whereby the solution
can be implemented, in parallel on several environments PC, Mac, X..  To do
so I porpose we copy the idea behind ISO OSI for computer communications, 
namely modularity.  We devide the model into 4 layers:

- Application layer
- Communication layer
- postnews primitives layer
- raw news tree layer

We define (where appropriate) interfaces between layers on different levels.
Once we agree on the interfaces, we can then leave it to each user or group of 
users to further refine the interfaces and implement the modules in each layer
in their prefered environment, then share the results with the rest of us.
Some of the interfaces (that relate to the news tree and postnews) are already
defined so we need not (can not) bother with them.

Here is how I envision the models to be used:



********* Arabic/Persian reader/editor *******************
^         |             |                        ^
| A       | B           | B'                     | A"
|         |             v                        |   
|         |     *********** communication ****************
|         |             | B"                     ^
|         |             |                        |
|         v             v                        |
|        ****** post news primitives ******      | A'
|                 |                              |
|                 | C                            |
|                 v                              |
********************* news tree **************************


If you are reading and posting news on the same host then you virtually bypass
the comm. layer.  Your news read/post path is then: A-B-C.

If you are reading on one host (a PC for example) and posting on another,
then your path will be: A'-A"-B'-B"-C.

If you are on a PC the comm. layer can be something as simple as PC-NFS or
comm. program scripts, or you may even be willing to do it by hand.

Personnaly, I haven't had a programming experience with X, even though I use
it daily.  I am willing to help define the model, and help implement PC based
solutions.


The rest of the message concerns the PC solution.

For the PC (thanks Alex), we have:

- emacs with source for both DOS and Mac. 
- hevga134, author Joseph Gil aka Yogi which is a "Hebrew font & utils for 
  PC's without Hebrow ROM Char".  It is available from:
  vmsa.technion.ac.il, directory: PUBLIC$:[000000.MSDOS.HEBPROG]
- a similar program hebkb110 by Dov Grobgeld available from the same place.
  It is built around the Turbo Editor's Toolbox.
- Jove editor (I am not sure what it does)

(Thanks to Anoosh), we also have:

- graphic routines for manipulating variable size Arabic/Farsi fonts.

Also:

- Microsoft and IBM are coming out with Arabic extensions


The PC implementation:

The PC implementation that I will be helping with is one which is usable by the
widest majority of people: the PC-serial(phone)link-unix (replace PC with Mac
and unix, with vms, etc.., as the case might be).

How do I go about the reader:

   - coding:

        - in text mode, use upper side of the extended ascii table to
          encode Arabic alphabet, or

        - in graphics mode, use some standard code set (ISO)

        - either way, we may have to uuencode/decode to overcome the 7-bit
          Internet restriction.

   - editing:

        - in text mode, use a keyboard utility that switches between Arabic
          and English alphabets (like the Hewbrew programs).  Display both 
          simultanously with the loaded extended ascii table.  We may use an 
	  existing "English" editor.  Or We may create your own.  

          If we use an existing editor, then in English keybord mode, the 
          cursor moves as usual where as in Arabic keyboard mode, the cursor 
          becomes stationary and the sentence moves to the right, as you type
          it.  There will not be any need to read/write Arabic backwards.

          If we create our own editor, then switching between Arabic and
          English togles cursor movement orientation.

        - In graphics mode, we are basically creating our own editor.

   - encoding:

        - Eventhough an 8-bit Internet may be in the works, we can not count
          on it.  It will take sometime until the whole net has converted to
          the new standard.  So we have to encode into 7 bit ascii and produce
          a text message which contains the encoded message (optionally) it
          may include a latinized-readable version of the text following some
          unumbiguous conversion rules.  Agreeing on a coding set standard
          is not a problem as there seems to be plenty of them (ISO, ASMO,..)

   - Communication (read and post):

	- In the final phase, we may create a single program that allows the
	  reading, posting and communication.  In the earlier phases though, we
	  have to rely on off-the-shelf parts.

        - Initially therefore, all posting will be done through some comm.
	  program scripts (telix is my favorite).

          You can write almost effortlessly a (DOS) batch command and Comm. 
          program (for example Telix) scripts to perform the communication
	  functions.  You invoke the batch file to run the comm. prog. with the
          "upload" script which calls the news site, performs the login and
	  postnews procedures, then logs off.

        - To download (read), you can do the same with batch files and comm. 
          prog. scripts.  You can do a (unix) "find | cpio" for example, 
          to get all new files (articles) after a certain date in the groups of
	  choice and download them to your PC.

        - The problem with scripts is that you have to engineer them yourself
          for your site.

	- Again we use comm. scripts in the early stages and a stand alone
	  do it all program (or a set of interacting programs), in the latter
	  stages.


Comments are welcome.  Please no flames.

bassem

Newsgroups: soc.culture.lebanon