From: UPHILG@UNC.BITNET (Philip Gallagher,(919)966-1065)
Newsgroups: bit.listserv.sas-l
Subject: Kathy H. and "scrubbing" unwanted strings
Date: 16 Oct 92 21:11:00 GMT
Reply-To: "Philip Gallagher,(919)966-1065" <UPHILG@UNC.BITNET>
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU
Comments: To: k.holle@DPC.DPC.UMASSP.EDU

> Kathy Holle   UMASS/AMHERST   (413) 545-1465 wrote the msg appended
at the end of this one on getting rid of various character strings.
Kathy, try this one on for size.

It could be done with a tiny bit more elegance with arrays, but
some folks complain so much about "How Hard!" arrays are that I
have given up.

DATA NEW;
   SET OLD;

   *  First replace all offending strings with asterisks;
   IF INDEX(NAME,'Mr.')     GT 0 THEN SUBSTR(NAME,WHERE,3) = '***';
   IF INDEX(NAME,'GDN')     GT 0 THEN SUBSTR(NAME,WHERE,3) = '***';
   IF INDEX(NAME,'Mrs.')    GT 0 THEN SUBSTR(NAME,WHERE,3) = '****';
   IF INDEX(NAME,'Trustee') GT 0 THEN SUBSTR(NAME,WHERE,3) = '*******';
   IF INDEX(NAME,'Custodian') GT 0
                           THEN SUBSTR(NAME,WHERE,3) = '********';

   NAME = COMPRESS(NAME,'*'); *  <===  This deletes all *'s;

    *  This ought to work fairly well.  Just be sure that the
       assignment to the pseudovariable has as many characters (*)
       as are in the word you want to eliminate.
       If it should happen that there actually asterisks in the data
       that should remain in the data, use some other symbol that
       doesn't appear in the data.;

                       Phil Gallagher
                       uphilg@unc

P.S.:
    Kathy, please don't be offended if you knew this already, but,
    if you're a beginner, please remember that upper case and
    lower case letters are different in string comparisons.
    Therefore you if your offending character strings come both
    in mixed case and lower case you will have to explicitly test
    for both, as below:

   IF INDEX(NAME,'Mr.')     GT 0 THEN SUBSTR(NAME,WHERE,3) = '***';
   IF INDEX(NAME,'MR.')     GT 0 THEN SUBSTR(NAME,WHERE,3) = '***';
   IF INDEX(NAME,'GDN')     GT 0 THEN SUBSTR(NAME,WHERE,3) = '***';
   IF INDEX(NAME,'Mrs.')    GT 0 THEN SUBSTR(NAME,WHERE,3) = '****';
   IF INDEX(NAME,'MRS.')    GT 0 THEN SUBSTR(NAME,WHERE,3) = '****';
   IF INDEX(NAME,'Trustee') GT 0 THEN SUBSTR(NAME,WHERE,3) = '*******';
   IF INDEX(NAME,'TRUSTEE') GT 0 THEN SUBSTR(NAME,WHERE,3) = '*******';
   IF INDEX(NAME,'Custodian') GT 0
                           THEN SUBSTR(NAME,WHERE,3) = '********';
   IF INDEX(NAME,'CUSTODIAN') GT 0
                           THEN SUBSTR(NAME,WHERE,3) = '********';

KATHY'S MSG BEGINS HERE
=======================
> I've been fascinated with this discussion!  I've been working on a
> similiar problem and it helped me.  The scenario I've been looking
> at involves data scrubbing a field that is 40 postions long.  It
> originally looked like:
> Mr. John Doe Trustee
> Mrs. Janice Page   Custodian
> Mr. John Alexander, Jr.    GDN
>
> I've mangaged to strip of the titles Mr., Mrs. etc.  But now I need
> to get rid of the following descriptors which now number about 200 varieties,
> can fall anywhere in the field, can be any length.  I was going to try the
> SUBSTR function or SCAN hopefully using a minus counting array to find the
> first word and then a blank.  But I heard that there is a new function
> on 6.07 called TRANWRD that might handle it even better.  Have you heard
> of this function?
> I would be very interested in seeing your homework assignment.  Exercises
> like this speak a thousand words!!!
>
> Thanks for the lively discussion!!
> Kathy Holle   UMASS/AMHERST   (413) 545-1465
> k.holle@dpc.umassp.edu
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU