SECTION III
THE USER
Sandra Browning

In order to develop a user population for SUPARS, many aVenues for advertising the service were explored. These included classified and display ads
in canpus newspapers, as well as large posters placed on canpus. Letters tofaculty and graduate students were mailed, introducing the service and asking
them to announce the service to their classes.
Once the population was developed as much information as possible wasgathered on the user; his problems, his characteristics and his attitudes.
As it is more effective to use several methods and make oonparisons, manymethods for achieving this goal were pursued. During the hours of SUPARS
operation a telephone aid service was maintained by the staff. At the sanetine an unobtrusive data collection program, STAIPAC, was being run. During
the last week SUPARS was in operation, a telephone interview was conducted ofa sanple (100) of registrants, both users and non-users of the system, inorder to learn their use of and reaction to SUPARS. Another sanple was drawn
and administered the semantic differential, aimed at identifying and measuring
attitudes toward SUPARS.
During every phase of the SUPARS operation, the staff attexrpted to assist
the users. First through the publicity canpaign, which reintroduced the service to the canpus oonftunity, and next by sending a User's Manual to each person oonpleting a registration form, During the actual operation of SUPARS,
the telephone aid service attempted to supply answers to the user's problems,
while STATPAC gathered data unobtrusively. The telephone interview and thesemantic differential were used as a neans of gathering inforrtation concerning
attitudes toward SUPARS and suggestions for inprovenent were also accepted.
1.

PIOFILE OF REGISTRANTS

To be eligible to use SUPARS during its six week operating period, allpotential users were required to ooitplete a registration form (Appendix V.)
Questions on the registration form were aimed at collecting demographic-type
information, information regarding registrants' usage of Psychological
Abstracts, and information about registrants' previous experience with computer terminals and retrieval systems.
Data from all oarrpleted questionnaires were entered into the SUPARS
Registration Retrieval System (SRRS), a small-scale retrieval system using
APL which was especially designed for this purpose. Usage of the Inquiry
or retrieval portion of SRRS allowed quick compilation of the descriptive
information presented below.
Descriptions presented in this section deal with the population of57

registrants in terms of a user/non-user dichotomy * Users were identified assuch from STATPAC output, which lists the social security numbers of all
those registrants who signed onto SUPARS and at least once entered a delta(A) to begin a search.
Several problems with this definition of users should be noted. First ofail, registrants who tried to sign onto SUPARS, but for sane reason failed
to reach the point of entering a delta, are not here considered as users*
Responses to the telephone interview (see part 4 of this section) indicatethat nearly one-third of those classified at the time of the interview as
non-users by the STATPAC definition had tried to use SUPARS at least once.
Secondly, twenty social security numbers listed by STATPAC as user numbersoould not be matched with registration forms. And finally, there is evidence
that some users, rather than signing off SUPARS after corpleting theirsearches, left the system operating for others to use. These subsequent lasers
then entered searches which were recorded by STATPAC under the social security
numbers of the original users. Thus, the total number of users (143) referred
to belcw represents a somewhat conservative estimate of the total number ofpeople who used the SUPARS system in some way.
The graphs and tables below are arranged into four sections: (1) cumulative
growth of the registrant population, (2) demographic data, (3) use of
Psychological Abstracts by registrants, and (4) computer experience ofregistrants. Questions on the registration form from which the relevant raw
data were originally drawn are listed preoeeding the appropriate section ofdata description.
a.

Cumulative Growth of the Registrant Population

Figure 9 depicts the growth curve of SUPARS daily registration. (Dateregistered was considered to be the date on which a registration form reached
the SUPARS office). During the 11 weeks of the registration period, which
extended from October 4, 1971 to December 17, 1971, a total of 596 people
registered to use SUPARS. The third and fourth weeks of the period were theweeks of the heaviest registration. By day 16, one-third of the population
had been registered. On day 20, the half-way point of registration wasreached. And by day 23, b«^o-thirds of SUPARS" potential users had registered.
Last year during a 14-week registration period, SUPARS registered 349
potential users. The half-way point of registration occurred on day 30; thetwo-thirds mark was reached on day 37. (3)A oonparison of data for the two periods thus indicates that during thecurrent year more potential users were registered more rapidly and over ashorter period of time than were registered during the comparable period lastyear. This is most likely accounted for by this year's more extensive publicity canpaign, coaled with the certain degree of familiarity with SUPARS
already present in the university population due to last year's operation.

58CF: 10/4 10/11 10/18 10/25 11/1 11/8 11/L5 11/22*11/29

12/6 1 2 A 3

Days of Registration Period•Thanksgiving vacation - no registration forms processed.

Figure No. 9. Cumulative Growth of SUPARS H Registrant Population

59

b.

Demographic Data
(1) University Status

Question 6:

Are you primarily: Faculty, Undergraduate student, Graduate
student, Staff, Administrator, Visitor, Other?

Table V lists the number and percentage of users and non-users in each of
the seven categories of university status. Graduate students accounted forthe largest proportion of both users and non-users, with 62.4% of the users
and 45.6% of the non-users being graduate students. The undergraduate andfaculty groups were the second and third largest groups in both the user andnon-user populations. Of the three university status subgroups mentioned,
only the graduate student group contained more users than non-users.
In last year's registrant population, the three major academic subgroups
were also graduate students, undergraduates, and faculty. Hcwever, thedifferent methods used in accounting for multiple responses to the academic
status question (See Table V) make exact between-year comparisons somewhat
tenuous(2) Departmental Status
Question 7: What is your academic major or department?
v

As indicated by Table V I , the three individual departments accounting forthe largest nuntoer of both users and non-users are, as would be expected fromthe nature of the data base and of SUPARS' affiliation, Education, Psychology,
and Library Science. In the user group, Library Science accounted for 24.7% of
the total, Education for 21.3%, and Psychology for 20.7%. Among the non-users,
nearly one-third of the total was accounted for by Education registrants andone-quarter by Psychology registrants.
Approximately one-third of both the user and non-user population falls
into the category, "Other." Table VII lists the full range of departments
represented in the total SUPARS registrant population.
For all major departments, there were more non-users than users.
A comparison of this year's data with last year's indicates that for
both operating periods, the same major departments reported the heaviest
rate of registration. However, for last year's group of registrants, theproportion of registrants from the Psychology Department was higher (39.5%),
accounting for nearly t^o-fifths of all registrants. This difference is
probably due, not to any marked decrease this year, in interest on the partof those affiliated with the Psychology Department, but rather to the broader
range of departments contributing registrants this year and to increased
registration from such departments as Business, Journalism, Hone Economics,
and Social Work,(3) Time Spent on Teaching, Research and Other Activities

60

TABLE V
NUMBER OF SUPARS REGISTRANTS BY UNIVERSITY STATUS*NUMBER OF REGISTRANTSSTATUSi

Users

Nonusers

Total! /klroinistrator
1

2

4

6

1.3%

.8%

11

Faculty

82
7.4%

i

216
62.4%

*'

La ,.aj Staff

i
14.9%

309
45.6%

49.6%

'

•

2

30
1.3%

t.

1

93
17.3%

93

Graduate Student

1.0%

32

129
21.5%

Undergraduate

32
6.3%

5.1%
161

!
25.8%

27.2%

i

i

Visitor

15

9

6

2.4%

1.9%

4.0%

Total

1.1%

.8%

2.0%
Other

7

4

3

623

474

149

100%

100%

100%

*Note: registrants were permitted to answer in more than one category. Whena respondent answered in more than one category, each answer was tabulated asa separate response, reflected by the inflated oolurm totals* For last year's
data analysis, all multiple responses were totaled into the category "other".

61

TABLE VI
NUMBER OF SUPARS REGISTRANTS BY MAJOR DEPARTMENTAL CATEGORIES*NUMBER OF REGISTRANTS
DEPART EOT
Users
Education

32

Nbnusers

Total

149

181

21.3%

Psychology

31.0%

31

Library Science

Other**

23.7%

24.6%

37
-24.7%

,45-

47

156

82
13.0%

9.4%
203
32.5%

31.3%

Jo Answer

149

118
20.7%

28.7%

12

3
2.0%

32.2%
15
2.4%

2.5%

^i

150
Total

480
100%

630
100%

100%

*Registrants were permitted to answer with more than one response. In
cases of multiple responses, each response was tabulated separately, giving
the inflated coluran totals.
**See Table VII for a detailed listing of all departments.

62

TABLE V I I
NUMBER CF SUFARS REGISTRANTS BY DEPARTMENTAL CATEGORIES

Department

Users

Non-Users

Total

2

i
i
i

5
4
3
5
13
1
1
149
2
5
6

2
6
5
4
7
16
2
2
181
4
6
9
1
2
15
2
21
1
1
6
1
82
2
1
2
1
6
1
3
149
1
5
1
1
6
2
9
33
11
1
1
1
1
1
15
630

Accounting
Anthropology
Architecture

Art

2
3
1
1
32
2
1
3
1

BiologyBusiness
Chemistry
Economics
Education
Engineering
Sfliglish
Forestry
Geology
History
Home Bconaaics
International Relations
Journaliam
Journalism Library

2
14
2
15
1
1
6
1
45
2
1
2

—

1
—

6
—
—
—
—

Law
Liberal Arts
Library Processing
Library Science
Math
Music
Philosophy
Physical Education
Political Science
Pre~Law
Pre-Madical
Psychology
Psychological Research Center
Public Administration
Rcmance languages
Russian Studies
Systems Information Science
Social Sciences Program
Sociology
Social Work
SpeechStudent Affairs OfficeStudent Personnel Administration
Syracuse University Research Cbrp
Upstate Medical Center
Urban Studies
No Answer/&iestion does Not Apply

—

37
—
—
—

—

1
2

4
1
2
118
1
4
1
1
3
1
8
25
7
1

—

1
31
—

1
—
—

3
1
1
8
4
—

1

—

—
—

•

—

3
150
63

1
1
1
12
480

Question 8: What percent of your tine is engaged in: Teaching
and/or learning, Research, Other?
Tables VIII through X present data on the percentages of time spent byregistrants in teaching/learning, research, and other activities. As shewn
by Table VIII, approximately one-third of the users reported spending between
76 and 100 per cent of their time on teaching and/or learning activities.
Another quarter of tlie users indicated spending 26-50% of their time on these
activities. The same categories also accounted for the largest proportionsof respondents in the non-user group.
According to Table IX, a little over one-half of both users and non-users
reported spending between 0-25% of their time on research activities.
Similarly, as shown in Table X, over one-half of both users and nonusers reported spending 0-25% of their time on activities other than teaching,learning, or research.
It is to be noted that, in answering this question, respondents were notrequired to total their answers to 100%.
Conparable data was not presented in last year's final report.
(4) Previous SUPARS Registration and Use
Question 9:

Did you register to use SUPARS last year?

Question 10: If yes, on the average, how often did you use
SUPARS — more than once a day, once a day,several times a week, several times a month,
once a month, less frequently, no time?
As indicated in Table XI, the greatest majority (76.9% and 85.9% respectively) of both users and non-users did not register to use SUPARS last year,
although there is a slight tendency for a larger proportion of the user groupthan of the non-user group to report registration.
Data from Table XII indicates that the majority of users who had registered last year reported using SUPARS several times a week or several times
a month. Together these two categories account for two-thirds of the responses.
Non-users who had registered with SUPARS last year most often reported having
used SUPARS several times a month, with a somewhat higher percentage of (thisyear's) users reporting such usage. More non-users than users (19.4% as
oenpared with 9.1%) reported using SUPARS no time.
c.

Use of Psychological Abstracts by Registrants(1) Reoent Use of Psychological Abstracts

Question 11: On the average, over the last 2-3 months, how frequently
have you used Psychological Abstracts — more than once a

64

TABLE VIIIPERCENTAGE OF TIME ENGAGED IN TEACHING AND/OR LEARNINGNUMBER OF REGISTRANTSI Peraent of TimeUsers

1 0-25%19

Nonusers

Total

54

73

13.3%

'i

j
1 26-50%

11.9%

36

129
25.2%

12.2%
165

28.5%

27.7%

i

i 51-75%

j 76-100%19^13.3%

80

49

123
34.3%

No Answer

ii

•

143
•

•

•

•

•

,

28.9%
87

14.8%

453
100%

i

172

67
14.0%

65

14.6%

596
100%

I

|
16.6%

27.1%

20

TOTAL

99
17.7%

100%

TABLE DC

PERCENTAGE OF TIME ENGAGED IN RESEARCH^ —i

Number of Registrants
Percent of Timei 0-25%

Users

Nonusers

Total

79

228

307

55.2%
i 26-50%

27

76
18.9%

[

51-75%

13

* .-

32

1 •

!

18

123

100%

66

3.0%
155

27.1%
453

143

2.2%

2.9%

22.4%

TOTAL

13

13

5'

17.3%

2.9%

3.5%
j No Answer

103
16.8%

0.0%
76-100%

51.5%

50.3%

26.0%

596
100%

100%

J

TABE£ X
PERCENTAGE OF TIME ENGAGED IN OTHER ACTIVITIES• " • • "-•• • ' • "Nunfoer of Registrants!

i Percent of Time
',

!

0-25%

Users

Nonusers

Total

82

245

327

i .
i

57.3%

| 26-50%

13

54.1%
32

9.1%

l
j 51-75%5

76-100%132•i No Answer

"

•

'

.

10

155

143

67

1.7%
196

34.2%

453
100%

3.0%

1.8%

28.7%

Total

18

8

41

7.5%

2.9%

1.4%

"

45
7.1%

3.5%

1

54.9%

|
32.9%

596
100%

100%

TABLE XIREGISTRATION FOR SUPARS LAST YEAR

f

Nurrber of Registrants

Indication of\ Registration
Users

33

! Yes

Ifonusers

Total

62

95

23.1%

1
15.9%

13.7%

1
Nb

110

389
76.9%

83.7%

2

No Answer
0.0%

Total

499
85.9%

143

2
0.4%

453
100%

0.3%

596
100%

100%
'••••-'••iday, once a day, several tines a week, several tines
a month, once a month, less frequently, no time?
Table XIII shews that both the user and non-user groups most frequently
reported no usage of Psychological Abstracts in the 2-3 month period preoeeding registration, ihis response accounted for 37.1% of the users and 33.1%of the nonusers. Approximtely another one-fifth of both groups reported
their previous usage of Psychological Abstracts to have been several tirresa month. These data are ocnparable to last year's data.
(2) Future Use of Psychological Abstracts
Question 12:Estimate how frequently you see yourself using
Psychological Abstracts in the next 2-3 months —1
more than onoe a day, once a day, several times
a week, severed, times a month, onoe a month, lessfrequently, no tine?

68

TABLE XII
USAGE OF SUPARS BY LAST YEAR'S SUPARS REGISTRANTS
1

1

.

.

.

1

1

! ! • Ml

••

1

1.

• —

"II"

'

1

l »

...

• 1

!••

lJurttoer of RegistrantsjFrequencyof Usage*
Users
A

Nonusers

Total

1

1
3.0%

0.0%

1.1%

0.0%

0.0%

0.0%

B

':

c

12

10
36.4%

1

D

J

E

1

F

4

10

19

G

No Answer

Total

30.7%

6.1%

17.7%

30.5%
13

10

13.7%
14

3

12

]
14.7%

16.1%
15

9.1%

19.4%

3.0%

0.0%

1

15.8%
1

33

100%

More than onoe a day
Onoe a day
Several tines a week
Several tines a month
Once a month
Less frequently
No tine

69

1.0%
95

62
100%

*A
B
C
D
E
P
G

23.2%
29

30.3%

12.1%

1

22
16.1%

100%

TABLE XIII
AVERAGE USAGE OF PSYgPLOGICAt, ABSTRACTS
IN THE 2-3 MOMEH PERIOD PRECEDING REGISTRATION

Number of Registrants
J Frequency| of Usage*

A

|

u

II

• » — •

Users

Nonusers

Total

1

3

4

0.7%

0.7%
'B

1

1

0.2%

0.3%

8.4%

4.6%

33
* 5.5%

19.6%

98
21.6%

126
21.1%

62
13.7%

76
12.8%

12.6%

61
13.5%

79
13.3%

37.1%

150
33.1%

203
34.1%

11.2%

57
12.6%

73
12.2%

453

596

12

D

2

1
0.7%

C

0.7%

21

28

V

E

, ,-..

FG
No Answer

Total

14
:'" * 9.8%

;

18

53

16

143
100%

100%

70

100%

Table XIV indicates registrants' anticipated use of Psychological
Abstracts in the 2-3 month period following SUPARS registration. The data
shew that, although previous usage of Psychological Abstracts was low(Table XIII), both users (38.5%) and nonusers (44.4%) most frequently responded that they anticipated using P^chological Abstracts several tiroes amonth for the next 2-3 months. A slightly higher percentage of users thannonusers anticipated using Psychological Abstracts more often than several
times a month. Again, these oata are conparable to last year's.
(3)

Recent and Future Use of Psychological jfostracts for
Preparation of a Term Paper, Thesis, or Speech

Question 14: Have you used Psychological Abstracts in the last
2-3 months to help you prepare a term paper f thesis,
etc.
Question 15: Do you need Psychological Abstracts in the next 2-3
months to help you find references or abstracts fora term paper, thesis, speech, etc., you are currently
preparing or planning to prepare?
Consistent with the data reported in Table XIII, data in Table XV
indicate that over two-thirds of both the user and nonuser groups had notused Psychological Abstracts in the 2-3 months preceding SUPARS registrationin preparing a term paper, etc.
Similarly, consistent with Table XIV, Table XVI shews that over threequarters of both groups of registrants intended to use Psychological Abstracts
in the months following registration to help prepare a paper, thesis, or
speech.
(4)

Specific Need for Psychological Abstracts

Question 16: What is your specific need for Psychological Abstracts
in the near future — to find a specific referenceor abstract, to keep up to date in one or twocontent areas, to survey the literature in general,
to exhaustively review a specific topic or area, tofind several current references in an area, other?
As shown in Table XVII, approximately one-third of all SUPARS users
responded that their immediate need for Psychological Abstracts was forexhaustively reviewing a specific topic or area. Approximately one-quarterof all nonusers also indicated this choice. Among the users, another 38%
indicated that they would use Psychological Abstracts to find several
current references in an area or to keep up to date in one or two content
areas. Forty percent of the nonusers indicated that they needed Psychological
Abstracts to find several current references in an area or to survey theliterature in general.
Data from last year indicated that the mjority (61%) of registrants

71

TABLE XIV
ANTICIPATED USAGE OF PSYCHOLOGICAL ABSTRACTS IN THE 2-3

MO!t*TH PERIOD FOLLOWING REGISTRATION
Number of Registrants!

Frequency ofUsage*

i

A

Users

Nonusers

Total

1

3

4

32

D

201

55

13

F

73

11

i

More than onoe a day
Onoe a day
Severed times a week
Several times a month
Onoe a month
Less frequently
No time

72

7.0%
25

|
4.2%

3.5%
596

453
100%

*A
B
C
D
E
F
G

42

16

143

11.4%

6.8%

6.3%
Total

68

31

9

14.4%

11.0%

7.7%
] No Answer

86

50

18

43.0%

16.1%

12.6%

G

256
44.4%

9.1%
1

18.3%

17.0%

38.5%

E

109

77
22.4%

J

1.0%

0.4%

2.8%

c

6

2

4

!

0.7%

0.7%

0.7%

100%

100%

TABLE XVPAST USE CF PSYCHOLOGICAL ABSTRACTSFOR PAPER, TBESTS, AND SPEECH PREPARATION
t

1

1

•

•

.

•

.1

••

••-.

.

Users

Nonusers

Total

YES

39

105

144

27.3%

j

NO

318

11

30
7.7%

TOTAL

411
70.2%

65.0%

68.9% 1
41

6.6%

143

453
100%

[

1

24.2%

23.2%

93

|

i No Answer

II

Number of Registrants

Indicationof Usage
i
5

•

596
100%

•

73

6.9%

100%

TAELE XVIANTICIPATED FUTURE USE OF PSYCHOLOGICAL ABSTRACTSFOR PAPER, THESIS, AND SPEECH PREPARATION
Indicationof Usage

Number of Registrants
Users

Nonusers

Total
1

YES
NO
!
_,

•No Answer

108
75.5%

351

29
20.3%

70

6

32

'

77.0%
99

15.5%

16.6%
38

7.1%,

143

453

,. loot

<:

74

""•""

459
77.5%

4.2%
TOTAL

V

6.4%

596
100%

100%

TABLE XVIIANTICIPATED TOPES OF USE OF PSYCHOLOGICAL ABSTRACTS*r•'

Number of Registrants
Type of Usage**

A

Users

Nonusers

Total

32

118

150
14.0%

12.3%

B

48

165
13.8%

D

224
31.0%

E

7

23
2.1%
13

9

1102

841
100%

100%

[
1.2%

1.1%

261

i

20.5%

1.9%

1.5%

L_

226

16

4

j
27.7%

20.6%

2.7%

Total

305

173
20.3% -No Answer

18.2%

26.6%

53

F

201
19.6%

81

|
16.7%

16.2%

36

c

184

136
18.4%

13.6%

100%

*»espandents were permitted to indicate multiple responses to this question.
Column totals reflect these multiple responses.
**A To find a specific reference or abstract
B 3b keep up to date in one or two content areas
C To survey the literature in general
D To exhaustively review a specific topic or area
E To find several current references in an area
F Other75

TABLE XVIIIPREVIOUS EXPERIENCE WITH COMPUTER TERMINALS

J

Number of Registrants
Amovint of Experience

A lot

Users

Nonusers

Total

16

23

39

11.2%
Some

67

5.1%
159

46.8%
None

60

6.5%
226

35.1%
271

37.9% 1
331

42.0%

59.8%

55.5%

0.0%

0.0%

0.0%

No Answer

Total

143

453
100%

596
100%

100%

anticipated using Psychological Abstracts to exhaustively review a specifictopic or area. Although this category was the most frequently reported thisyear also, responses tended to be less concentrated in this category, instead
being more evenly distributed among categories A-E.
d.

Computer Experience of Registrants(1) Previous Experience with Computer Terminals

Question 17: What previous experience do you have with computer
terminals — a lot, some, none?
As indicated by Table XVTII, users as a group have had more experience
with ocnputer terminals than have nonusers. Over half (58%) of the users
reported a lot or some experience with computer terminals, while only 40.2%of the nonusers reported such experience.
A greater proportion of this year's registrants reported some experience
with computer terminals; only 25.8% of last year's registrants reported someexperience with terminals.

76

TABUF! XIX
PREVIOUS EXPERIENCE WITH COMPUTER-BASED RETRIEVAL SYSTEMS

Amount of Experience
User's"'
A lot

Number of Registrants
Nonusers

9

8
6.3%

Some

17
1.8%

38

97
26.6%

None

135

342
67.1%

22.7%
438

75.5%

No Answer

6
0.0%

73.5%
6

1.3%

;'
100%-

2.8%

21.4%

96

143

Total

453

1.0%

596
100%

100%

I
(2) Previous Experience with Conputer-Based Retrieval Systems
Question 18: What previous experience do you have with ocnputerbased retrieval systems — a lot, sate, none?
The majority of both the user and nonuser groups reported no experience
with cxatputer-based retrieval systems, A greater percentage of lasers (32.9%)
than nonusers (23.2%) have had a lot or some experience with retrieval
systems. These data are similar to the comparable data from last year's
registrant population.
e. Surcmary
In summary, the total nunber of registrants this year was 596, with
143 users and 453 nonusers. Approximately 16% of this year's registrants had
registered with SUPARS last year. Registrants from Education, Psychology,
and library Science accounted for the largest proportion of the registrant
population.
In general, this year's data revealed few major differences between users
and nonusers, with the exception that users tended to have had more previous
experience with cxuputer terminals than nonusers.

77

Finally, coirparison of this year's registrant data with last year's, showsthat, except for size and rate of registration, data configurations forboth groups are highly similar, A larger proportion of current registrants,
however, did report some previous experience with ooqputer terminals. This
last trend may be accounted for by the re-registration of 95 users from last
3ar.

2.

PUBLICITY - Margaret Mucia

Publicity this year was aimed at the canpus community only, whereas lastyear the Syracuse netropolitan area was covered. A total of ten display ads(6 column inches) were ran in canpus newspapers (The Daily Orange and the
Syracuse University Record.) Five classified ads were also run but additional
ones were discontinuea on October 29, 1971 as they were deemed non-effective.
M l these ads were similar in scope and composition to those of last year.
Large posters (22" x 28") were placed in six key locations on canpus with anattached box of registration forms. Approximately 1,000 forms were madeavailable in this manner. Upon request, a registration form was mailed toanyone who could not conveniently dbtain one from any of the canpus locations.
Upon receipt of the form by the SUPARS office a Users Manual was sent to eachregistrant. Another avenue of advertising pursued was the distribution ofletters to faculty >nd graduate students in those departments most likely to
be interested in SUPARS. TVto seta were distributed, one being an introductory
letter briefly explaining SUPARS, and the second requesting faculty members
to announce to their classes the availability of SUPARS. Attached to all letterswere registration forms. Nine slide-tape demonstrations were given by the
SUPARS staff to various classes such as Introduction to Information Technology,
Social Psychology and Foundations of Instructional Technology.

3.

4220 I0GS - June Brewer

During the six weeks that SUPARS was operational, a telephone number
was known to the users for them to call for aid with problems. This number(extension 4220) was iranned from 8 a.m. - 10 a.m. and 7 p.m. - 9 p.gu (Monday
through Friday) by various staff members who were trained in the use of thesystem.
Each incoming call was logged on a special log sheet. The problems
were numbered in sequence as they were received; and a description of the
problem and the action taken to solve it by the staff member were included.
Data from the original log sheets were categorized and summarized by day
in Log Summary A (Table XX)
Log Surrmary B (Table XXI) categorizes and summarizes the data from
Summary A by week and gives totals for all categories. Log Summary B will
oe described in this section, column by column, as to the patterns of calls
and the information exchanges carried on via the telephone aid service.

78

a.

Description of Log Summary B.

Coiuim 1: Vfeek
In this coluron, each of the six weeks of SUPARS university-wide operation
is designated by the dates of that week.
Column 2: Total Nurtfeer of Calls'Ihe total number of calls received per week on extension 4220 is logged
in this column. It can be seen that telephone calls ranged from 32 (a three
day week) to 30. During the six weeks of operational service a total of 402
calls from users were recorded. The mean nuntoer of calls per week was 67.
Tfae totals across the week are often greater than the nurtber of calls becauseseme users had more than one problem.
Column 3; General Information
All calls for operation hours and requests for registration forms areincluded in this column. The total nurtber of calls reaeived in this category
was 86, the second largest total for any category.
Coluron 4: User Problems ".*•••
A user problem was defined as a problem originating with the user
rather than one originating with the SUPARS program or with the Syracuse
University Confuting Center and its equipment. Examples of user problems
are: a user needs help in obstructing his search properly; a user cannot
sicyi onto the system because he has omitted a line of input. Column 4
indicates no particular trend over the six week time period in the frequencies of this type of call. Cadis ranged from a lew of 5 (during the firstweek) to a peak of 13 per week. The total nuriber of calls in this category
was 67.
Column 5: PA UNDEFINED
When the staff at the carrputing oenter failed to load the SUPARS disk
packs at the time the system was scheduled to begin its daily operation a"PA UNDEFINED" message was printed out to users attenpting to sign onto thesystem. Also, after an ABEND (defined later) the "PA UNDEFINED" message
would go out until the ABEND was corrected. Column 5 logs the nunber ofcalls per week pertaining to this problem. The frequency of calls in thiscategory was faiily low and it is assumed that the oonmand was self-explanatory
and thus users did not need to call and find out what it meant. This category
accounted for 37 calls*
Cojuran 6:

ABEND

An ABEND is a system error which causes the SUPARS oonputer program tostop processing. Cadis in this category were inquiries requesting information of system malfunction resulting from ABENDS. The second week of operation resulted in the greatest number of cadis, 19, but there were continuous

79queries throughout the six.week period and resulted in 38 total calls.
Column 7: TOO MANY USERS
The "TOO MANY USERS" command was printed when all slots in the queues.%*ere filled; the user had to keep trying to sign on. Calls of this nature
resulted in 19, the srrallest total for all categories. It can only be
assumed that the command was understood by the users that did not call# because
this was one of the mo6t frequently printed commands to carte from the system.
Column 8: System Status
When a user inquired if SUPARS was or would be in operation that particular day, a call was logged into this category. The first two weeks of theuniversity-wide servioe resulted in 68 calls in this category because of tha
problems SUPARS had in debugging the system. If you eliminate the first two
waeks, the x was 7.5 calls, from 30 total calls. However, by including thefirst two weeks for a total you reach 98 calls, which was the largest numberof calls in any category.
Column 9:

SUOC Problems

Figures in this category refer to user calls pertaining to system nonfunctioning which was attributable to cxxiputing center problems, i.e.
equipment trouble, social security numbers not accepted, or general system
failures. Frequency of calls per week ranged from 0-22; the total number ofcalls in this category was 63, the fourth highest number of calls in any
category.
Column 10: SUOC Messages
Calls made to the SUPARS office by the oonputing center staff members
were logged in this column. Very infrequently made, these calls usually
informed SUPARS of impending delays in the loading of the SUPARS disk packs
or of operating system trouble. Only four such calls were received during
the six weeks of SUPARS university-^tide operation.
b.

Summary

By allowing a user to know that there is "scneone out there who will help"
is of primary importance in a retrieval system which can seem to be too
mechanized for the uninitiated.
During the last three weeks of SUPARS operation the largest group of
calls were for information on why things weren't working properly. The
problem was with the Syracuse University Computer Center, which was beginningthe change-over from the IBM 360 to the IBM 370 conputer. Nothing could bedone about the situation but users were able to relieve their frustrations bycalling extension 4220 and hearing what had caused the bed conditions.
It is felt that without the telephone aid servioe the premise of
SUPARS — to make available a service to individuals — would not have been
as effective.

80

r

"T
CO

w

3

CM

P

in

ro

vx>

00

CM

r*

in

CO

w

B
ro

CM

£fe

CNJ

CM

ro

I

CTk

$

h

in

CM

O
CN

CM
CO
I

VD

11
81

in

1
I

CO

5 H

ro

in

co

CM

CM

00

oo

in

CM

CM

a

Is
13
£3 £3
05 co

I

CM

m

V£>

00

ft)

CM

CM

in

CM

LO

m

CM

m

CM

if)

00

en

55

CM

CM

ro

CO

CO

<\
in
t-1

CM

I

CO
CM
I

CM
I

vD

£^5

in

<7*
CM

CM

KD

cr\

CO

CM

&

O
CO

M CM
I
CM

I

82

£2

3?
£2

CO
i
CM

CM

gs

CM

CM

i

*y

CO

en

CO

co

in

co

CM

CM

rH

rH

^*

rH

rH

ff

H

ro

co

CO

rH

CM

•

co

CO

rH

rH

Friday12-10

Monday12-13

Thursday
12-9

co

CM

rH

CO

rH

rH

rH
CM

•H

CM
rH

a\

rH

CN J - oo •
^

9

CM

Friday12-17

CM" --. •

1 Thursday
i 12-16

CM

VP

i-

Wednesday| 12-15.?•r*| 12-14

rH

H

rH

^»

83

8
CN

CN

CO

CN

ex

en

5,3

O

CN

CO
CM

Csi

to

c^

CN

OJ rH

in

CN

m

V£>

in

in

cr\

ro

<^>

^O

CO

LO

Lf)

VD

kO

CO

CO

CO

rd
CO

CN

CN

CO

id
CO

o
CO

in o>
l
1
rH rH
rH rH

rH rH
!
I

(N
CO

CN ^
CN CN

&3

CN

o

as
CN CO

O
VD r H

I

CN CN

t

r H CN

84

co r^
rH rH
I
I
CN CN
rH rH

4.

STATPAC - Margaret M u d a and Michael McMullen

In cxaijunction with the telephone aid service, the telephone interviews
and the semantic differential, STATPAC was used to gather information on theuser. STATPAC is a specially developed computer program used to collect,
store, and retrieve all elements of user interaction. A flow chart of thedata flew and accompanying documentation is given in Appendix IV. STATPAC
was distinguished from the other three methods because it was an unobtrusive
general izable data gathering retrieval program. Because STATPAC was anexperimental program under continual revision, data recorded was not always
100% complete. As validation checks of output data were made with actualuser interaction, program improvements were made to correct the system.
STATPAC was designed to show growth patterns as well as usage patterns.
Growth was most evident in the nurrber of lasers who entered their social
security number to sign onto the system and the nunber and types of searchinquiries entered. As shewn in Table XXII, the data indicate a definite
increase in the user population and the nunber of searches. (This will be
discussed in greater depth below.) It should be noted that because of theexperimental nature of SUPARS, fluctuations in usage due to prograrmtingchanges, 06/360 computer difficulties, and variations in actual use patterns
were reflected in ah uneven growrth rate.
•*.

STATPAC consisted of two summaries, Summary 1 and Summary 2. The
summaries were developed by the project staff to monitor data about the mostimportant parameters of system operation. These summaries are only two of
the many configurations available. The distinguishing feature between thetwo was the different aarnbinations of fields included. For exanple, Summary
1 included the following fields: Social Security Nunber, Log Nurrber, Typeof Search (A, AS, A10, Date, Clock Tine, Terminal Number, Input Lines and
Output Lines.
A A is the character on the terminal used to initiate the different
types of searches. A A gives access to the document data base; a LV to the
vocabulary data base; and a AS to the search data base. The primary sort
was by social security number, with the secondary sort by log number, eachin ascending order.
a.

Summary 1

Exanple of a line from Sunmary 1
Sgj
36XXXXXX0*

LOGN DELTACALENDAR

OflCK

TER

INPUT LINES

DEL

11/18/71

0.00

52

Ll PSYODLOGY
L2 COMPUTER (?)
L3 ATTITUDE
L4 Ll and L2 and
L3;

7444

OJEFOT LINES

13399 44

*The social security number is not given in its entirety in order to insure
the user's security and to protect against invasion of privacy.

85

In Table XXII weekly totals are given for the five weeks SUPARS wasavailable for canpus use. The contents of each column are discussed indetail below. The letter preceding each section refers to the corresponding
letter for each column in Table XXII. The data was collected on the IBM 360/
50 OS.
(a)Social Security Nurrber (SSN)

Each tine a user signed onto SUPARS his social security nuntoer wasrecorded along with his interaction with the system. An increase in the number of people who used the system is evident from this column. Although sate
people may have used the system in more than one week, the overall userrate increased except for the last week. The decrease evident in Week 5 canbe explained by difficulties arising from the Syracuse University Confuting
Center, and not from the SUPARS program itself. The mean over the total
period was 50 users per week. The total number of times social security
nunbers were entered durings its 5 weeks of operation was 250.
(b)

Nuntoer of Log Numbers (LOGN)

Each timfe a user typed a delta (A) a log nuntoer was generated. As
there were wore users on the system during the second, third, and fourthweeks, there were an increased nuntoer of log nunbers generated. The mean was
502 per week.
(c) Number of Terminals (TER)
Syracuse University has 100 IBM 2741 terminals located in buildings on
canpus, but the highest use during any one week occurred during Week 4. There
were a total of 42 in use at that tine. This can be explained in part by thefact that some users found it difficult to sign on due to the high use of
the system, and consequently as one user conpleted his searching, anotherwould begin at the sane terminal, eliminating the sign on procedure. The
mean was 37.
(d), (e), (f) Nuntoer of Deltas (DELTA)(DDB-Document Data Base,
SDBr-Search Data Base,VDB-Vocabulary Data Base)

As a query was begun, STATPAC logged the type of search A, AS, LV. The
nuntoer of A searches far outnumber either of the two other options, AS or bV.
During the first four waeks the LV data base was rot in operation and therefore could not be searched. The mean of A searches was 461 and AS searches
was 32 over the five week period.
Additional data in S T M P A C includes (a) the nuntoer of search inquiries
initiated by a user, (b) the ratio of the nuntoer of inquiries made by one user
to the total nuntoer of inquiries. For exanple, one user initiated 120 inquiries
and accounted for .05 of the total searches. The total nuntoer of searches
initiated was 2399, and the total nuntoer of different users who initiatedsearch inquiries was 143. There were 17 users who initiated one searchinquiry. Of the 143 users of SUPARS the mean number of inquiries initiated

86

*

> 9
$
CM

o

CD

in

ON

ON

ON

CM

3

rd

13

<§

m

o>
in

ON

in

in
in

o
CM

8 i
CO

•8.

fti
O

m

CO

o

CM

ON
VD

ON

in

in

00

en

i i
3I

00

i

O
CM
ON

ON

CM

I

en
in

an

0

c^
o

6

in

CM

i
V£>

«s

O
in

\D

rd

o

CM

oc

5
o
CM

<

I
CM CM

I
CO < N

*r in

in

i !
87

I

5

oo

CM

a

§

S

<3

was 17, although there were wide ranges of use, from those who used the system
only onoe to twice to one user who initiated 120 search inquiries.
b.

Sumtary 2

Sumtary 2, which sorted all other fields by date, consisted of thefollowing fields; date, log number, type of search, social security nuntoer,
terminal nutter, clock time, CPU time, cost, maximum items possible, nutterof items printed, and input and output lines.
The data presented in Table XXIII is the most current and was gathered
while SUPARS/DPS was under the operational control of the IBM 370/155 OS.
The fact that the data was gathered from search inquiries performed by staff
search experts and not the canpus users distinguishes this table from Table
XXII.

2.

The following explain the column headings that are shown in Example
Those which are the same as the one in Example 1 are duly noted.
Exanple of a line from Summary 2

DATE
11/30/71

LOGN DELTA
7454

A

SSN

TER

CLOCK CPU POST MIP

31XXXXX00

52

2101

1

4.45

43

NPRINT20

Date
ihe date for each day of operation was recorded in ascending order, inthe first oolunii.
Social Security Nuntoer (SSN)
Ihis column is the same as in Summary 1.
Terminal (TER)

This ooliitn i s the same as in Surtmary 1.
Clock Time (CLOCK)i

ihe clock time was measured in the same manner as CPU time, from thetime a user struck a A to the time he hit another A or typed )CFF to disconnect from the SUPARS program. It is the total time a user was connected.
CPU Time (CPU)The Central Processing Unit time of the ocnputing system was recorded.
Cost (COST)
Cost is the total cost of each particular log nuntoer.

88

TABLE XXIIIMEAN CPU TIME AND COST
OVER THE THREE DATA BASES
1(Delta[^Number of initiatedsearch inquiries

Mean CPU
Tine

Mean EXCPCount*

Mean
Cost

Document
Data Base

824

5.45

407.92

1.16

AS
SearchData Base

216

0.24

67.45

0.13

318

0.11

41.37

0.08

LV

Vocabulary
Data Base* EXCP count is a measure of I/O activity plus any internal channel

89Maxiinum Items Possible (ME?)
The term Maximum was used in the Maximum Items Possible message because
DPS limitations cannot handle searches with complex operator's. Clhose otherthan AND, OR or NOT), without processing the search a second tine. Therefore
this number is the initial count and is not always the exact nuiriber a search
may retrieve. If very broad inquiries are input, this column will reflecta large number of Maximum Items Possible, whereas if a very specific topic
were being searched, the Maximum Items Possible would be very small. Therefore the number of searches during any one week may not always correspondwith the number of naximum items possible. For exanple, if in one week the
topics searched were of a very specific nature, the total surmary of thenumber of items possible may be low.
c.

Summary

By means of STATPAC many variables could be studied regarding the userand his interface with the SUPARS system in an unobtrusive manner without
having to observe the individual directly. These variables were printed in
fctfo different summaries, Summary 1 and Suninary 2. Summary 1 shows growthby increased number of users signing on, and the increased nunber and variety
of searches. Summary 2 indicates growth by nunber of users in relation totime of searches and the costs of those searches.
STATPAC is a good example of the wide range of variables and combinationsof those variables which can be studied and analyzed without confounding user
data by the reactions an individual generally makes or withholds when his
behavior is being observed. This study and analysis will hopefully lead toa better understanding of the user and his needs.

90

5. TELEPHONE INTERVIEWS OF SUPARS REGISTRANTS - Jeffrey Katzer
One of the techniques we used to assess reactions to SUPARS is the
structured interview (other techniques are discussed elsewhere in thisreport.) The major objective of this study was to obtain reliable estimatesof the reactions of users and registered nonusers toward SUPARS. The survey
was planned to shed light on several specific questions.
1. Can we identify the variable (s) which differentiate
the population of registrants into users and nonusers?
2. What mechanisms are effective in the education of
SUPARS users?
3. How often and to what extent is the system wanted?
4. What are the major problems encountered by users?
5. How do users judge the quality of their researdi output?
6.

How do users and-nonusers characterize their reactions
toward the -system?
/••*.

In an atterrpt to obtain user's reactions during the first year's
operation, Frierson aonducted lengthy personal interviews with a random
saitple of 20 registrants (SUPARS 1970 Final Report). A major goal of thatstudy was to develop and test the interview guide, and for that reason wewere willing to tolerate the small sarrple size. This year, we wanted toincrease the sanple size to increase the reliability of the information. For
practical reasons it was necessary to change the format from a personal interview to a phone interview. The interview gii.de, pretested last year,
was also changed in order to incorporate the differences between the 1970
and 1971 SUPARS system as well as the new phone format. Several draftsof the interview guide were prepared and pretested. All of the interviewers
were trained in the use of this instrument.
The phone interview was conducted of SUPARS0 registrants during theweek of Deaentoer 13, 1971. By delaying the survey until the last week ofservice to the university,1 we hoped to maximize the nuraber af registrants
contacted who would be users. ttofortunately this delay affected our completion peraentagb because several of our potential interviewees oould not
be contacted because they left town early far the upaorreLng holidays.

X

A later decision extended SUPARS service after the University's Christmas
recess.

91

TABLE XXIVFREQUENCY OF CCMPLEIED INTERVIEWS

Conpleted

Nonoonpleted

Total

Users

35

18

53

Nonusers

28

19

47

Total

63

37

100

Our first oonaern was to try to determine if a response oonpletion bias
was present, and if so, the nature of the bias. Within each category (userand nonuser) the oonpletion response rate was at least 60%. While not ashigh as desired, this percentage is a minimal rate to give us same assuranaethat those who conpleted the interview represented the total sanple. It
should also be noted that the percentage of users and nonusers with conpletedinterviews (55% and 45% respectively) corresponds closely with the overall
percentage of users and nonusers in the sairple (53% and 47%). Lastly, it
should be noted that none of the 37 nonoonpleted interviews were due to an
active refusal to be intervievted. Every person in the sanple contacted,
conpleted the interview.
As a further check on possible response oonpletion bias, we conpared
the information reported on the registration forms of those oonpleting theinterview with those whom we could not contact. Among the users we foundno disoernable differences in terms of academic status, departmental affiliatioa, or anticipated use of PsycholoyLcal Abstracts. Similar results were
found among the nonusers with one major exception,""academic status. Allof the seven faculty nonusers conpleted their interviews. This probably
reflects the fact that the faculty could not leave town for the holidays
before the official last day of classes (which was the last day for internviewing) . Students usually leave for the holidays earlier; in fact 37% of
ail non-respondents were students known to have left town.
A response bias is evident: faculty are over represented in the groupof nonusers who conpleted their interviews. This should be kept in mind
when interpreting our findings. However, it will not seriously affect theinterpretation of any differences between users and nonusers — our majorarea of interest.
The remainder of this report of the telephone survey will be based only
upon the aonpleted interviews (N=63).

92

a.

Ihe Sanple

A random sample of 100 registrants was drawn from two separate populations.
Using STATPAC (a log of the system which collects, stores and retrieves alluser interaction with SUPARS) we obtained a list of users2 of the system. At
that time 97 registrants used the system at least once.3 From the list ofusers a random sample of 50 were selected for the phone survey. Anotherrandom sanple of 50 were selected from the remaining 470 registrants (nonusers.)
During each interview several questions were asked to determine if
the original classification of user or nonuser was correct. Because several
days had elapsed between collecting the list of users from STATPAC and theobnpletion of the interviews, and because of the limitations of STATPAC3,
we decided to define a user as a registrant who signed onto SUPARS, as
ascertained by the interview. That is, a user is defined as one who
responded positively to Question 3 (in Section I) of the interview:H They
were able to sign onto SUPARS at least once. Ctoss-echecking the interview
with STAEPAC we found three registrants originally designated as nonusers,
who had used the system. Ihe final survey, therefore, was ocnposed of 53
users and 47 nonusers.
b.

Completed Versus tfanoonpieted Interviews

Of the 100 possible interviews, 63 were cotpleted. The breakdown ofcompleted and nonccnpleted interviews between the users and nonusers ispresented in Table XXIV.

2

STAH>AC defines a user to be anyone who signed onto SUPARS successfully.
We did not check STATPAC to see if the user actually made any SUPARS
searches. However, a check was made during the interview to see if the
STKEPAC-designated user actually made searches — see text.

3

This matter is a conservative lower limit. As discussed elsewhere inthis report, we have evidenae to suggest that once seme users signed onthey left the system operating upon completing their personal searches.
Other users then entered searches which were collected by STATPAC under
the social security number of the original user.

*A copy of the phone interview schedule and a registration form are includedin Appendix V.

93

TABLE XXVACADEMIC STATUS OF USERS

Faculty

Undergraduate
Student

Graduate
Student

Staff

Frequency of
Users

1

8

23

1

1

1

35

Percentage of
Users

3%

23%

65%

3%

3%

3%

100%

Percentage ofRegistrants1 ! 15%

26%

50%

5%

1%

3%

100%

1

AdminisOther
trator

Total

Since registrants were permitted to indicate more than one status position,
the frequency of registrants in these categories totals to mace than the
567 registered when the phone survey began.

c.

Users;

Description

Thirty-five users ooirpleted the phone interviews. Academic status of theinterviewed SUPARS users is presented above in Table XXV. While it is not
surprising that the great majority (88%) of users were students, we anticipated a greater percentage of faculty users. If SUPARS presents an information service for those doing research, it seemed reasonable to expect more
usage by faculty. As shown in Table XXV, approximately 15% of SUPARS
registrants were faculty but they made up only 3% of lasers.
We have aollected seme hearsay evidence which suggests that for at least
some faculty, graduate students were used to test the system's availability
and usefulness. Moreover, it is highly likely that graduate students
performed SUPARS searches for seme faculty members. This pattern ofresearch assistants (in this case graduate students) making searches forresearchers is understandable, and occurs frequently in at least one otherinteractive bibliographic reference retrieval system (SUNY Biomedical System)
Users were affiliated with 10 different academic departments or area
specialities. The largest group of users came from Library Science (10).
Ihis probably reflects the fact that the SUPARS project was administered by
the School of Library Science. Severed instructors in the school used SUPARS

94

in their classes as a pedagogical tool — rather than a research tool. Boththe Psychology Department and the Education Department had seven users each.
Social Work, the next more frequent category, had three users. All otherareas had one user each.
Users found out about SUPARS in a variety of ways, Forty-one percent of
the users mentioned learning about the system directly frcm our publicity
efforts (newspaper notices, posters on caitpus, slide presentations.) Six
users remenbered SUPARS from the service offered during the Pall of 1970
(S3Ql).b
All of the users received a copy of the 1971 User's Manual upon registering.6 In general, they found the Manual well written and quite useful. Ninety-seven percent of users took their Manual with them to the terminal. The
absence of an explanation of common error messages was the major oonplaint
about the Manual (S3Q4 - 13).
Thirty-one users remenbered receiving a Pocket Reminder Card (a folded
3 x 5 sized svranary of the main sign on and search procedures.) All registrants should have been sent one. The Pocket Reminder Card seemed to be lessuseful than the Manual; 39% of the users reported they did not find it helpful. Except for twb users statlhg that the print was too small on the card,no other ooftplaints were made. It is difficult to determine why the card was
not used more frequently (S3Q15-19).
As a backip for the Manual and the Pocket Reminder Card, the SUPARS staff
manned a user-service phone (extension 4220) during the hours of SUPARS
operation and for most of the remainder of the working day. Ttoenty-threeusers balled 4220 at least once; most users (56%) called less than four times
during the entire SUPARS operation (S3Q22-23).
Users judged the information supplied by SUPARS staff via extension 4220
to be helpful (78%), (S3Q24)7d.

Users: General Usage Patterns

A breakdown of the frequency and duration of attenpted and actual terminal
use is presented in Tables XKVI and XXVII.

5

Items in parentheses identify tiie interview question applicable to theresults discussed. In this case it was Section 3, Question 1.

^Ihqugh we offered instruction of the use of the terminal and constructionof searches, very few users requested such add. Our instructional
plan was to rely on the User's Manual as the primary means for user
self-education.
7

A nore conplete description of users, based upon infonration contained on
the registration forms, and a log of the calls reaeived on extension 4220
have been presented in this section.

95

TABLE xxvi
FREQUENCY OF ATTEMPTED USE OP SUPARS (S1Q1)

Daily

SeveralTimes a Week

Once aWeek

Less Than
Once a Week

Total

Frequency

2

17

7

9

35

Percentage

6%

49%

20%

26%

101%

TABLE XXVIIAVERAGE TIME SPENT USING SUPARS IN ONE SESSION AT THE TERMINAL
• —

*.„

«

II

lllml

(S1Q18)l

1 II

Less lhan
15 Minutes

15-30 Min.

30-60 Min.

Frequency

0

4

8

21

332

Percentage

0%

12%

24%

63%

99%

60-120 Min.

Total

lf

Ihough SUPARS was available for four hours each weekday, these hours were
distributed into two, two-hour sessions.

2This question was only asked of the 33 users who were able to obtain output
from at least one of their searches.

The major lindtation in interpreting these data is the lack of control
over the length of tiiae each respondent used SUPARS before the phone interview.
Seme respondents might have been using SUPARS from the first day it was available. Others might have begun the day before the interview. It is possiblethat respondents in either of these groups oonpleted the sane number of searches.8[

0nae the output from STATPAC is available and tested, we will be able todetermine the actual use (number of days# number of searches, and amount oftime at the terminal) for each user.

96

Taking the information in Tables XXVI and XXVII at face value, itappears that three-quarters of SUPARS users tried to use the system at least
onpe a week, and once they managed to sign onto the system, they stayed on it
far at least 15 minutes — most of the time longer* A more detailed look at
the responses of these twenty-one users who reported spending about one totwo hours at the terminal each tine they used SUPARS raises sone questions*
Ten of the 21 users in this category also claim to make only 2 to 5 searcheson the average in one terminal session* Under good conditions it is not
unusual for a user to oonplete several times that number of searches in a
two hour period. It is possible that these users were doing a great amountof thinking about search strategies between searches. Or they might have had
to wait for a large amount of output to be printed. Their likely explanations
would be that either they ware teaching themselves how to use SUPARS andprooeded cautiously with errors, or they might have experienced oonputer
problems with SUPARS per se or with the total operating system.
Table XXVIII presents the number of searches users estimated that they
made in each terminal session and across all terminal sessions.
TABLE XXVIIIFREQUENCE AND PERCENTAGE OF ESTIMATED NUMBER OF SEARCHES MADE

In oneFrequency
terminal
session.
(S1Q17) Percentage
Totaledfor allFrequency
cerminal
sessions. Percentage(S1Q11)

1

2-5

5-10

10-20

20-50

over 50

total

2

17

9

4

1

0

33

6%

51%

27%

12%

3%

0%

99%

1

8

7

3

10

6

35

3%

23%

20%

9%

29%

17%

101%

In general, the median number of searches made at each terminal session
is in the 2-5 range.. Over all of his terminal sessions the user makes anaverage (median) of 10-20 searches. Both of these figures suggest a lessfrequent use of SUPARS than had been anticipated, (though sane informationneeds could be met with a few nuttoer of searches.) Certainly the delays instarting the system each day and the frequent software failures did not help
increase the usage of the system.
While all of the users vaere able to sign onto the system, 31 of them (89%)
experienced some difficulty in signing an at least once. The different types97

of problems experienced by users in signing on are listed in Table XXIX*TABLE XXIX
PROBLEMS EXPERIENCED B¥ USERS WHICH HAMPERED ATTEMPTS TO SIGN ON (S1Q8)

ProblemFrequency

No terminal available

1

Keyboard frozen

1

Unexplained Messages (Other than those below)

4

SUPARS not available when it should have been
(PA UNDEFINED message)

6

Complete IBM/360 Operating system failure

9Area storing user's search filled1(TOO MANY USERS message)

21
42

2

As each SUPARS user signs on, a 1200 byte storage area is set aside to holdeach search as it was entered. Given a limited amount of oore available it
was impossible to have an unlimited number of user storage areas. Eight ofthese areas were unavailable. Users who signed an after the last storage
area was allocated were sent the "TOO MANY USERS" message.

Forty of the forty-two problems mentioned by the users are software
problems. T\*enty-five of the forty-two problems were SUPARS problems. It isamazing that users had the persistence to continue trying to use the systemafter so many problems. The frequency of occurrence of the "TOO MMflf USERS"
message helps explain the "corrective" action taken by some of the users(described in footnote 3.)
After successfully signing on to the SUPARS system other types ofproblems became possible. TWenty-one users (60%) experienced one or more
problems while searching. The twenty-six problems reported by these users
are classified into the four categories presented in Table XXX.

98

TABLE XXXPROBLEMS EXPERIENCED BY SUPARS USERS AFTIR SIGNING ON (S1Q10)

Problems

Frequency

LSbn-operation; such as SUPARS or operating
system failures

6

Free text; such as user supplied keyword notin dictionary1

4

SUPARS interaction; such as uninterpretable
output or operators not used or not working oorrectly

12

Other problems; such as typing errors, keyboard jamming

4

26
1

Pursuing this type of problem further, we asked 33 users if they preferreda free-text system or one in which the vocabulary was more controlled.
42% preferred the free-text; 36% preferred the controlled vocabulary;
12% wanted both possibilities and the others had no preference (S2Q11.)

Of these problems, 69% of them are software related and 58% specifically
due to difficulties with SUPARS.
e.

Users: Reactions

One of the purposes of the phone interview was to get users1 iitpressions of the quality of output retrieved by SUPARS. Thirty of the 33 users
asked, felt that SUPARS generally retrieved the information they wanted.
(S2Q1).
We also asked those 29 interviewees who have used the printed versionof Psychological Abstracts to ccnpare it with SUPARS in terms of speed, easeand quality of search output. Sixty-nine percent judged SUPARS to be faster,and 3 users who did not agree with this majority said that SUPARS would befaster if it worked all the time, or if he could always get on the system —see footnote to Table XXIX (S2Q5.) Most of the users (76%) considered SUPARS
to be easier to use then Psychological Abstracts (S2Q6) • Responses to twoquestions asking lasers to judge the relevance of SUPARS retrieved output arepresented in Table XXXI.

99

TABLE XXXIFREQUENCE AND PERCENTAGE OF RESPONSES:"Have you located more relevant information with SUPARS"

than you wofcLd havefound using
Psychological
Abstracts (S2Q7)

which you probably
would not havefound without
SUPARS (S2Q8)

Yes

No

Don't Know

fjrpi^l

Total

Frequency

15

8

3

2

28

Percent

54%

28%

11%

7%

100%

Frequency

18

4

6

0

28

Percent

64%

14%

21%

0%

101%

One user felt that "quality" of abstracts from Psychological Abstracts
is better than frcm SUPARS, but for "quantity" the situation is reversed.
Another user, giving an affirmative response to the second question, felt that
he would not have expended the extra amount of effort to make manual searches
in Psychological Abstracts.
While responses such as shown in Table XXXI are encouraging, it mostbe remembered that these data are based upon user estimates, not behaviors suchas actual use of retrieved abstracts in term papers. Parenthetically, it isinteresting to note that cognitive consistency theories would predict a favorable user response, such as shown in Table XXXI, if the retrieved abstracts
were nonrelevant and user-SUPARS interaction was a negative experience. As
described below many users found SUPARS frustrating.
(Also see Table XXXIII)
Users were asked to state those aspects of SUPARS they liked irost andliked least. Of the 46 positive responses, about one-third mentioaned speed
or quickness of using SUPARS. Other representative positive responses were"fun," "thorough," "don't have to copy printout," and "dig ocnputers." (S2Q9).
Of the 46 least-liked items, 22 of them (48%) were related to therelative unavailability of SUPARS, (poor service hours, too few days, too
many users, too many system failures, etc.) Other typical negative responses
were "irrelevant output," "weird commands," "more expensive," and "don't
knew what words to use in a search." (S2Q10)100

In order to get an overall assessment of user's reactions to SUPARS,
much later in the interview, wa asked each user to state two or three adjectives \vhich best described his experiences with the system (S3Q27). Eachuser, of his cwn aoaord, gave at least one negative and one positive responseto the question. Of the 78 total responses, 21 were the sane adjective,"frustrating." The remaining responses varied greatly, e.g. "cantankerous,""challenging," "helpful," "tine consuming," "notivatinj," and "useful."
The picture that seems to be painted by user responses is a conflicting
one. In general, they find it frustrating to use, but persist because they are
intrigued with its speed, novelty, and potential. There is no discounting
the negative experiences users had with the system — especially the difficulty of getting signed on when they wished to. However, most of the users(94%) said they would register for SUPARS if it became available again*S3Q26.} And, at the end of the interview, vfoen the user was asked if he
had any questions, eleven of the 23 who had, wanted to know if the SUPARS
service oould be extended into January, 1972 (S3Q28). Several users
volunteered to help SUPARS keep running past its deadline by asking theirchairmen to contribute deparbnental computer funds to the project.
f.

Nonusers;

Description

Ttoenty-<eight nonusers were interviewed. Table XXXII gives the frequency
and percentage of nonusers according to academic status.
TABLE XXXIIACADEMIC STATUS OF NONUSERS
1

Faculty

Undergraduate
Student

Graduate
Student

Staff

Frequency ofNdnuser

7

7

11

2

1

Percentage ofNonuser

25%

25%

39%

7%

4%

Percentage ofRegistrants

15%

26%

50%

5%

Administrator
Other

j

%
1
i

Total

0

28

0%

100%

1

3% 1 100%

The major difference in status between users and nonusers occurs in thefaculty and graduate student categories. Nonusers have a larger percentage
of faculty and a smaller percentage of graduate students than users9.
^Tto get a rough idea of the "randamess" of the distribution of observed
frequencies for users and nonusers, we collapsed the 3 least frequent status
categories (staff, administrator, other) into one category. The frequencies
then were cast into a 2 x 4 contingency table (i.e. user/nonuser by the 4101

Nonusers represented 13 different academic departments. The largest
gxxxp of nonusers came from the School of Education (6).
Nonusers as a group first learned about SUPARS in ways similar to thatof the users. Four of the nonusers knew about the system from the previous
year, and 19 other users were introduced to SUPARS by seme of our publicity.
One might have guessed that one of the variables vrtiich separated
registrants into the user and nonuser category was the convenienoe of SUPARS1
operating hours. Tliis did not turn out to be true. In fact, a slightreversal of what was expected occurred. While most of the users did notprefer the servioe hours, most of the nonusers did (57%). Occasionally it
became useful in this analysis to separate nonusers into ta*o grovps: those
who tried to use the system at least onae (but failed), and those who nevertried (see page 1 of interview face sheet, question 2.) Hie 28 nonusers split
into a groip of 9 (who tried) and a group of 19 (who never tried.) Six ofthe 9 who tried SUPARS preferred other hours, while 13 of the 19 who didn't
try found the scheduled hours satisfactory. Though there is no apparent
explanation it seems as if the more a registrant tries to use the system(from "nontryer," to "tryer," to user) the more he finds the operating hours
unsatisfactory.
All but three of the nonuSters received a Users Manual;l ° two of these3 reported having access to a copy of the Manual. n Most of those who readthe Manual found it well written (91%). Though it was possible that several
registrants did not receive a Manual, we did try to send a copy of the Pocket
Reminder Card to all. Even so, eight of the nonusers reported not receiving
one. Most of those who received it found it helpful. Ttoe SUPARS telephone
helping service was used by 7 of the nonusers. Of these 7, 5 were "tryers."
There is an interesting difference in the adjectives given by the"tryers" and the "nontryers." Those who tried SUPARS (9) had mainly negative
adjectives to describe the system. The irost common of these was "frustrating."
Other adjectives reflected a similar feeling: "maddening," "disappointing,"and "discouraging." Given that these people tried to get onto the systemand failed, these attitudes are understandable. It is interesting to note,however, that several of these respondents asked if SUPARS would be available
after the Christmas recess. One faculty mentoer in this group volunteered to"apply pressure" so that SUPARS aould be continued.

types of academic status.) A Chi Square test for independence was calculated.
The observed value of Chi Square equals 7.92. Tto be significant at .05 level,
the Chi Square needed to exceed 7.81. Therefore, the distribution of scoresin the underlying populations are not proportional to their totals, (i.e.
not random.)
10uThis is possible. The number of registrants was considerably higher than
anticipated, and some of the later registrants were not sent a manual because we had distributed all of them to earlier registrants.

T o attenpt to alleviate the difficulties caused by not sending registrants

102

to

3

8 -3
J8 3
3 •8 S

CO

CO

3 p

• co

%

o5

to 13 gIf!
IS

3 &• S? .. « 3 d

5
8,8-i

8S§3

>.

3

5

a2.

1

H

>..

ghl

ghl pos

2
3
-H

3
O

«w S ai.3 -P

8

Mi

2

Ms
H"BS|

$ mB o

8

to

O

I

CO

4

2

10
ON

u

ON

"^ 3 S103

In contrast to the "tryers", the nonusers who never attempted to sign
on to the system (N=19) had uniformly positive adjectives about SUPARS:"accessible/' "innovative/1 "personalized/1 "good/" "great service/1"interesting/1 and "exciting idea."
It seems that all pre-use contact with SUPARS via publicity, Users
Manual, etc. helped produce the attitude we wished — and thought necessary
for a openly-accessible/ computerized system without intermediaries. This
positive feeling, however, was changed by actual initial contact with thesystem. Inability to sign on successfully when the potential user wants
to and the inability to remain on the system once having signed on, seemed
to be the major contributors of the negative attitudes toward the system.
These data are not of the kind necessary to shew a causal relationshipbetween attitudes toward the system and contact with it. Nonetheless,
Table XXXIII is interesting to consider.
g.

Summary

Listed belcw are the conclusions drawn from this work in response to
the six questions of interest identified at the beginning of this subsection. To help put these conclusions in some perspective, each will bepresented with the corresponding findings from,last year's study conductedby Frierson. As mentioned earlier, only 20 registrants were interviewed
last year. With such a small sanple size it is difficult to identify reliable
findings. For this reason we will only be concerned with broad "trends"
apparent over several variables or highly discrepent findings.
1. This year, the only observable difference between users and registerednonusers was academic status: faculty members are more likely to be nonusers.
An analysis of several variables from last year's study could not differentiate
users from nonusers.
2. In both years of SUPARS service, the reactions of registrants tothe publicity-education-aid package (e.g. Users Manual, phone extension 4220,
etc.) was, for the most part, highly favorable.
3. Last year, SUPARS was available to the public for one four hour
time block each day (this year SUPARS was offered for 2 hours in the morningand 2 in the evening.) Even with longer terminal sessions none of the 12
users interviewed last year made more than 4 SUPARS searches per session.
Over all terminal sessions, 58% of last year's users made 10 or less searches.
The corresponding figures for this year are, 57% of users made 5 or lesssearches per terminal session, and 46% made less than 10 searches over all
sessions (See Table XXVIII). Both figures are similar over the two years,
alley are puzzling because they are much lower than we anticipated andcausal obervation of user behavior indicates. Several possible explanations

a personal oopy of the Users Manual, we deposited several copies in various
high-usage locations around canpus. Registrants without manuals were told
the location of these referenae copies.

104

for this low USG v**re mentioned earlier. It will take a thoroughanalysis of STATPAC data and seme systematic observations of users at
terminctLs before v*a are confident in these data, and possibly have seme
e-rfplanation for their size,
4. Problems encountered by users were, for the most pari:, the samein last year's operation and this year's. While this seems to indicatethat the system was not inproved between operations, that is not the case.
/Approximately 32% of all problems experienced by users this year were notdue directly to SUFARS (e.g. operating system failure, or SUPARS not loaded
when user attempted to sign on) • The major problem encountered both years
was the inability of a user to sign on because of high use (the TOO MANY
USER message.) While this is counted above as a SUPARS problem it is afunction of the size of core portion available to SUPARS which in turn is afunction of demand on the aonputer by other interactive and batch systems.
Inprovements were made in this year's system which decreased the numberof SUPARS generated error messages. Offsetting this improvement was thefact that this year's operation could not get completely debugged before thesystem was offered to the public. Prograntrdng problems and drasticallydecreased turn-around time prevented us from meeting our planned time tablefor dewlqament.
5. A difference is apparent between quality of SUPARS* output asjudged by last year's and this year's users. Last year only 50% of the users
felt the output contained information they wanted. This year the corresponding figure was 91%, a sizeable increase (if reliable). It is possiblethat iirprowaments in the system accounted for this difference. Anotherpossibility suggests that users rated output quality high in order toresolve an inconsistency between their registering for the system and thefrustration using it. We have no evidence bearing on either of these
possibilities, one way or the other.
6. General reactions of this year's users toward SUPARS are sumBrizedin Table XXXIII. Attitude may be thought of as a function of the degree ofcontact with the system. Our public relations-educaticn-aid package createsa favorable, inviting view of SUPARS. Initial use of the system is often
frustrating, and unless a user persists, he frequently acmes away withstrong negative attitudes. Users who do persist recognize both the positive
and negative aspects of the system.
Comparable data from last year's interviews are not directly available.
While users voioed praise and conplaints, there did not seem to be the intensity of feeling then as now. If we aonsider user's attitudes as measured
by an attitude scale, last year's users, in general, liked the system.
(Frlerson & Atherton.)

105

6.

SEMANTIC DIFFERENTIAL - Patricia Moell

One objective of the SUPARS research project has been to measure user
reaction to an on-line interactive bibliographic retrieval system. Frierson
and Atherton (SUPARS Final Report, 1970) reported on a survey of attitudes
toward SUPARSvvrtiich indicated that registrants exhibited "a definite positive
reaction toward SUPARS and catputerized information retrieval systems ingeneral." The present report describes a replication of that study to determine
if user reaction to SUPARS remains positive.
A Semantic Differential was employed to measure user reaction both to
SUPARS as a system and to the services provided by the SUPARS staff. This
instrument is considered to be "a reliable means of measuring user reaction
to the SUPARs system and perhaps to ccmputer-based information retrieval
systems in general/1 (Frierson and Atherton)
The Semantic Differential (SD) was used because it could identify independent dirnensions of users' attitudes toward the system. There were 20
SD's (one for each concept — see Table XXXIV); each SD consisted of 10
bipolar adjective scales.1
Seven equal intervals separated the ends of each adjective pair. Each
respondent was instructed to readt to the concept by marking an interval
between each set of adjective pairs. If his reaction to the concept wasstrongly reflected by an adjective, he was to mark an interval close to that
adjective. Each adjective pair, therefore, became an attitude scale. The
ten adjective scales and an example of a SD are presented in Table XXXV.
a.

Description of Procedure

A SD package plus a preliminary information page and a direction sheet(see Appendix VI) was constructed for each respondent. In each packet theconcepts were randomly ordered. To avoid response set, the bipolar adjectives
were reflected (e.g.; good-bad became bad-good) in four different combinationsf
so that 5 concepts had the same page layout of bipolar adjectives. For easeof scoring, however, the adjective pairs were presented in the same order on
the page, perhaps creating a sequence effect.
A SD packet was given to 29 students in the School of Library Scienceduring regular class sessions. In addition, 73 SD packets were nailed to 21
Library Sctiool students and 52 Syracuse University students in other disciplines.
It was planned that only students who ware users of the system would receive
the packets but 10 recipients previously identified as users (through class1

Though each respondent was asked to react to each concept with 19 adjective
scales. The. additional 9 scales are being used in a replication of theoriginal factor analysis (see 1970 Final Report, Section IV-B-5). These
9 scales were not analyzed in this report.

106

TABLE X X X W
CONCEPTS USED IN THE SUPARS SEMANTIC DIFFERENTIAL PPCKET
1. Psychological Abstracts
2.

PA UNDEFINED

3. Computer
4.

TOO MANY USERS - TRY AGAIN SOON

5.

SUPARS

6.

MAXIMUM DOCUMENTS FOUND

7. Extension 4220
8. How I fourri out about SUPARS
9. LI
10. Myself and computers
11. OUTPUT

12. Usefulness of SUPARS to roe
13. Constructing my search logically
14. Computer Search
JL5; Free Retrieval Service
16-. A
17.
18.
19.
20.

Ttetfninal)EXEC PA
Keyword
LIST RECORD

Concepts 2 and 4 are default messages which tell the user he isunable to use SUPARS. Concepts 3, 9, 10, 13, 14, 15, 17 and 19
are related to ocxrputer-based search systems in general. Concepts
5, 7, 8, and 12 relate to general SUPARS features. Cbnoepts 6,
U , 16, 18, and 20 are oonnnn oonponenets of the SUPARS usersystem acnrounications language.

107

TABLE XXXVEXAMPLE OF A SEMANTIC DIFFERENTIAL
USED IN THE PRESENT STUDYTOO MANY USERS

slow

;

:

:

:

:

:

fast

active

;

:

:

:

:

:

passive

good

:

:

:

:

:

:

bad

successful

;

:

:

:

:

:

unsuccessful

hot

:

:

:

:

:

:

cold

colocful

;

:

:

:

:

:

colorlessbeautiful

;

:

:

:

:

:

ugly

heavy

:

_:

:

: •'- :

__:

light

serious

:

:

:

:

:

:

humorous

big

:

:

:

:

:

;

small

108

o

m

vo

o

in

i
is

£

8&

H
«

i«-*~«

8

ro

V£>

ro

ft
1B
<D

It
is
3°«

it

en

if

0)

"8

•H

(1)

$&

6

1

*8
0) 4^ O
D H

I

Cfl

i s
0)

109

Q

I

00

1

CN

to

u>

m

CM

<N

CN

CN

CN

CD

U)

d

vo

CM

ro

CM

W

0
in
I

in

l

O
CN
J

in
CM
CM

no

O
CO
I

o

o

CM

CO

in
1
rH

vo

in

i

participation or through a STATPAC listing of users' social security numbers)
were not users. Pour nonusers catnpleted the packet. Of the 102 recipients,
34 Library School students and 18 students in other disciplines responded.2
Of the respondents, 27 Library School students and 17 other students filledin the packets.
Since the total number of users identified by the STATPAC printout wasless than 100 (although there were other users of the system — sc*ne identified through the Library School, some identified through telephone interviews, and others not identified), selection of recipients of the SD packets
was not randan. Very few identified users were not sent packets. Whenever possible, users were not sent packets if they were known to be scheduled
for a SUPARS phone interview, because of the rather large catitdtment ofstudent time entailed by each measure. Thus, the selection of users forthe study was randan only in the sense that after the interviewees had
been identified, they were selected at randan to fill out the total list of
102 recipients. If the sample is not considered randan it may still be
representative because we almost used the entire population of known users*
Bable XXX/I shows the classification of respondents by discipline.
Table XXXVII classifies respondents by number of searches made, andTable XXXVIII classifies them by the nufriber of days on which they used
SUPARS. Interestingly, four students made fewer searches than the numberof days they used the system. This could be an indication of failure to geton, system failure, time at the terminal, or length of search.
Not all respondents completed all 20 SDs. Several left sane concepts
blank, seme inadvertently, others apparently not. These SDs were not analyzed.
SDs for a given concept were also discarded fron the study if lines waredrawn vertically down the middle of a page to indicate that no reaction or aneutral reaction was present. It was felt that this response indicated that
adjective pairs had not been examined individually for the concept.
The ten adjective pairs were grouped into three separate dimensions ofreaction to SUPARS and computerized information retrieval systems based on
the findings of last year's final report.
The first dimension, "evaluative-specific," was composed of the first
four adjective pairs:
slow-fast
active-passive
good-badsuccessful-unsuccessful^Students in other disciplines represent the fields of Psychology, 40%;
Education, 20%; and others (such as Journalism and Social Work), 40%.
Only 2 persons responding (one a Library School student and one not)relfused to complete at least the first page (see Appendix VI) and thus
are not represented in any tables, although one was a user of the system.

IllTABLE XXXVIIINUMBER OF DAYS STUDENTS USED SUPARS

Library SchoolStudents

Other
Students

Total

0

4

0

4

1-5

17

11

28

6-10

5

5

10

11-15

1

1

2

TOTAL

27

17

44

Number of Days

The second dimension, "desirability/1 was composed of the fifth, sixth,
and seventh adjective pairs:
hot-coldcolorful-colorless
beautiful-ugly
The third dimension, "enormity," was composed of the final three adjective
pairs:
light-heavy
serious-humorous
big-smallb. Data Organization
Forty-four conpleted or partially completed Semantic Differential packets
were analyzed in the study. On each concept sheet the respondent's reactions
were coded using a one indicated the most positive reaction possible, a scoreof four indicated a neutral reaction, artd a score of seven indicated the mostnegative reaction possible.
Oontrary to the Frierson-Atherton report, the score for negative concepts(scores larger than 4) were not reflected since it was felt that although theconcepts in themselves are intuitively negative, they remain negative nevertheless, and their contribution toward the overall mean for the dimension

112

should indicate their negative quality.
The means of the total concept scores were calculated by dimension(Table XXXDC). Hie twenty concepts cure listed separately and arranged into
the groups suggested in XXXDC. Table XL shows the corresponding standard
deviations.
c.

Results

The concepts were divided into five groups based on their relation toa group concept or topic. Hie five groups are: (1) Psychological Abstracts,(2) computers in general, (3) SOPARS in general, (4) SUPARS computer languageand (5) SUPARS default language.
Fran Table XXXIX it can be seen that users reacted most positively toGroup II, computers in general, and most positively to the individual conceptsof free retrieval service and computer in Group II, and Li, A, and "How I
fbund Out About SUPARS" frem other groups (especially on the evaluativespecific dimension.)
Generally, reactions to individual and group concepts (with the exceptionof the SUPARS default lai^uage concepts) ware slightly positive. The means
regressed from positive toward neutral from Dimensions I to III to II. The
evaluative-specific dimsh^ion indicated the most positive reaction, andthe desirability dimension the least. Standard deviations moved from high(the highest a 1.59 for Dimension I — Terminal) to low (the lowest 0.66 forDimension III — Li) along Dimensions I to III. Standard deviations werehighest for Group 5, the negative concepts, and lowest for Group 4, thecomputer language group.
d.

Comparison with the Results of Last Year's Study

Overall, the respondents1 reactions to SUPARS this year were not as
positive as those last year. In fact, both the dimensions of desirability
and enormity had coluun means greater than 3.50, indicating that the reaction
was only slightly more positive than neutral. Last year only the dimensionof enormity exceeded 3.50. This year enormity was shown to be more positive
than desir<ibility, vdiereas the opposite was true in the 1971 stixfy.
As in the previous study, Group II was regarded most positively, withfree retrieval service again the most positive concept and computer thesecond most positive.3
The more neutral reactions of this study as compared to the last are
not merely a result of the difference in reflecting last year1 s scores andnot reflecting this year's for the negative concepts. Reactions are moreneutral across all individual concepts and across all groups.
3

But as one respondent conmented at the top of the page - "free to Whom?
ttie taxpayer?"

113

CO

00

00

m

CO

00

ess
•

0)

IS

2

en

00

CM

00
O

co

oo

H

CM

co

H
H

CO - Hsnojoum]4c-snoTjagg g

AAPdH-^BTTg £H

O rH 0 0
CM VO TJ«COro ro oo

co

O

w

N

H

00 00

m CM m

o
r>

CO CO

CO CO

CO K?

CO

*<# in TJ*«3* co inCO-Pssax j Q T°3lI a) - H-J£i£°I£dg

VO CM

LO VO

co

VO VO

r>

vo 00

Ch

in in CMvo

VO r H

O CM
VO ^

CO

CO CO COCO

CO (*S

CO COo co &
m o o

m
co

CM 00
CM ^r

co co

CM ^0 co

ro

CO CM

CO CO

• • •

• • •

oo

CO CO CO

9
M
EH,

o
m

4M

TnjssaDonsunl-Tn^s^aoongI

p g g - p o o g f . 4J

9ATSSg3>9AT^ w

o

(dm
3-H
^ o x s - ^ s r e j l •H U
CO 0)
. H > O4P W CO
G

l-P

c p

CN

rn

rd
U
•H
DlcO
OJ-P
O
rd

u

-p

w

•H U

oo.m inco H

CO CO CMc o a) a)
M

M+J

Q) 6

^

E-PwOGwgcuwSi*

m 8 o g o a m 4J
3

>i

o

on

^
TVH3N3S N I SHSindWOD

114

<D<U

M

1
o

c 3 co o 4->
CO
o
CO p$

OH

Q)
3 §•6 <D a) - H <d <d+ J 4 J M - I Q I M M 4 J M > C M
3 3 rH E f + J <d 3
M-H O

a D 4 ( U O U) 0) a d )

ro

-P
o
2 (A
CM O 04CM(dtj> 0* <d >g a) -p

00

H
co -w

«
CM

CO
G
Q)
-P

CO

H

D
CO

CD

_y
m

Z

<

a*
5
co

SHVdflS

^
*e

oo

(N

ro

<o
C

in

Q)

VD

en

ro

§•

in
oo

2

m

ro

o
M

c
o
-H
CO

in o CM rH
Ch Gh 00 00

inft• • • •roro ro ro ro

co co

co

CO CM

• •
CO CO

CO

o
-H
CO

c

O vo ro oo
in vo eo in• • • •ro ro ro ro

vo

1 ro
Q

^

CO

o

• •
co

o
CO

c

1

GO
CO

vo o\ »H r*
r- co r^ rH

GO C OcoC M CM C O C O

r* x*

CO

-H

CO

n3

0)
4 J CD

< 8

04

6

•H

3

e

W
CO

8.rH

u 4J

a

CO

^. « - H
hi < ~ J

CD

CD
CO

c

3
o
o

CO

£
o
u

aDvnoNVT
innvaaa
aaindwoD •* snvdns's
115

<**

00
CO

rH

o

o

CM

o

rH

•

LO
OD

H

anda;
tion

14

M
H

1

•

CNJ

- p <d H
CO - H H

rH

rH

2 a

vp
CO

o

•

•

•

rH

rH

rH

>

0
M

H

o

•

•

rH

^r ^
oo r>*

00 00
r > enro
VO VO
o LO r >

VO
00

rH

rH O rH

O

O O

o

rH o

o

o

VO

o

•

o

*

0.99
^

in o
*a* rorH rH rH

rH

rH rH

0.98
1.13
1.11

1.18
1.06
1.28
o oo r oro r o *a*

1.24
0.88

1.27

>i

imension I
esirabilit

'AIRS

H

a*
w
>

o\
vo

1.06 .
0.90 :eo•H C^Tfefl
-TnjT^nHag
ssaxJcoxoo-XnjjeoxoQ
PTO0-1OH

oo r f
oo o

TJ*

1.36

TT^^S-Bxg
snojcouinH-snoxjcas^A^aH-m^TT

ision II
dty

H

rH

VO ( N
LO OJ

00
«3*

•H rH rH

rH

*

,Q a

H
U

w
h3

1

ion
tive

H

rnjssaoonsunO< 1 -Xnjssaoons
•H
to
rd
M-i
ppg-pooo-H > 04
Q W CO

OJ

rH rH

*

1
o3
1 G O
O w

u o

tii CO
0 +>
rH O
o CO

C <D -P
H 0)
H
3 +3 O Sl W -P O rHQ)
3 0 U ( D < D * H < d r a- P - P ^ O ^ H - p H > C H
3 P H 6 4J «J D
H -H 0X 1 COm04

6 + J CO 0
0 2 >1O 0 £+*\•s1\u\

IS!

•

.

rH

TOH3N30

116a) 6 £C CO S <D
CO H
>,
a) a)
O
O H
- p ^
O
OIH

nd out

CU CO<D 0 4

4220

LP

s of
to me

rH

UPARS

H>5

rs
ing ray
logicasearchieval

MOXS-^SPJ

Q) rH Od~Oi\ T S S ^ c J - . a A T ^ O V

ical

Q

G P CO COQ) CO
0 0- H «W 4J C tfCO
3 H <C H 0 3 Oi
M-l D
<
(D CO
04
X 0
CO < D x :s+J £ "93y.

N I saa^ndwoo *zsHVdns #e

00

00
O

O

ro

15

03

9*3
2

C

o

-H

c
e

«0

VO ^

CN «*fvo o r^ o>

•H
Q

• » •

*

O r4 O

O

CO 00
G\ CN

en

O H

o

<<* inrH i n

00

• •

C
0
-H

c

8

ro
o

CN iH VO 00
0 0 O KO O

• • • •O H O H

Q

M

O
co
C

I

oo r^ CN ^

o

rH ^

CM

o

CM CN

in

C
0
•H

g
T>

<D

4J <D
•H rH

M

S3

g -H

«8

•H
M-l

04 0)

•H 0

U
tq 4J

§

XI

P CO
£ CO

x
03 a rH

X

CO

W -H

~ r H

BDYQDNY1

-p

>,

41S
g
<

04

•5!
T3
T3

M

O

CdrH
'OrH

0

H

§

aDYnDNVT
JiTflYJaa

kajindwoD •* snvans *s
117-p <*HCO 0

e. Conclusions
The results of this study indicate a slightly positive reaction toward
SUPARS and ccmputerized information retrieval systems in general. They also
indicate a more negative reaction to SUPARS and other systems by this year's
registrants in comparison with last year's. The most positive reaction wasrecorded in the evaluative-specific dimension, lending credence to the assumption that the dimensions of desirability and enormity are less specif icallyapplicable to the concepts used.
The standard deviations of all the means are relatively lew. This may
indicate similarity of opinion concerning SUPARS and other corputer retrieval
systems. Since most of the means tend to be near neutral (or 4), however,the low standard deviation probably reflects a high concentration of neutral
reactions. An examination of individual scores would lend support to thisconclusion.
The results are generally in accord with those of last year's study,
indicating reliability of the testing instrument. The overall less positive
reactions recorded may be a result of actual differences in attitude toward
SUPARS and computerized retrieval systems this year as compared with last.
It would be helpful to compare the results of user reactions via the interview
method for the two years to see if this conclusion holds true. (This isincluded elsewhere in this report.)f.

Discussion

Several factors nust be considered in relation to the results of thestudy. First, for purposes of scoring, bipolar adjective pairs were notrandomly distributed on a page. Since an examination of individual concept
pages would show that scores tend toward 4 in the middle and lower rangesof the page, it would seem that the dimensions of desirability and enormity
might suffer fron the location of adjective pairs. Attitude pairs could be
distributed randomly on the page with no loss of scoring ease by assigning
numbers to each attitude pair in coribination with a 1 or 2 to indicate
which way the score should be read. These numbers could be keypunched alongwith the score and then read by computer.
Second, the difference in user reactions to concepts last year and thisyear might be a function of differences in service. Of interest is the factthat for both years computer is a more attractive concept than SUPARS, despite
the fact that free retrieval service is the most positive concept of all.
Perhaps the more negative image of the concept SUPARS has some causal factors.
Third, the many neutral reactions might be an indication of lack of useof a concept rather than a lack of interest. Extension 4220, for example,
which scored a 4.02 in enormity brought forth statements such as "never useit." In other words, neutrality may be a result of a lack of familiarity
with the concept rather than neutral feelings about it.

118

Fourth, the fact that this year's division of concepts into dimension"as a result of last year's rotated factor matrix might render the results
doubtful if a factor analysis of this year's results alter the ocnpanentsof these dimensions.
Last, the similarity in structure of responses over the two years indicates that the Semantic Differential tends to measure these concepts inthe same ways. The less positive results of the study might well indicate,
therefore, a greater dissatisfaction with SUPARS this year than last.

7.

IMPLICATIONS AND PROJECTIONS - Kenneth Cooka.

User Orientation

The necessity for the integration of the user as a major part of designing interactive retrieval systems cannot be overenphasized. For the systemthat does not use intermediaries, user integration becomes of paramount concern. Consideration for the user goes well beyond just the design stage.
Squally important is a need to inform, instruct, and educate people vdho areusers and potential users. The final, and perhaps the most critical aspectof user orientation, is the necessity of determining user reaction in termsof attitudes, opinions, and behavior.
The various technicjies used to evaluate user reaction to the SUPARS/BPS
system indicated that problems related to the user and the user interface
with the system were almost as prevalent as problems related directly tohardware and software component malfunctions or interruptions. A study
dealing with free-text retrieval (4) found that over 85% of the problems
and retrieval failures resulted fron either the search process or the user/
system interaction.
What these findings seem to imply is that an important overlookedfunction in the development of a user-oriented retrieval system is theinclusion of a strong publicity/information and educatiorv^instruction
effort. Tto make the assumption that people will be eager to approach vrtiat
they might consider to be a mysterious system controlled by an unseen computer simply because the designers were enamored with the system is, at best,a guess, and at worst, the basis for potentially maintaining a little used orunderstood system.
b.

Publicity and Instruction Functions

Because the boundary between educatiorv/iiistru^
and publicity/information is difficult to draw, and seme activities encompass both areas, adistinction between the two is itade here simply for descriptive purposes.
The techniques mentioned below were found to have a positive effect on
increasing the number of SUPAPS/DPS users and instructing users to be selfActivated and independent of an intermediary.

119

In the area of publicity/information, some of the most useful techniques
related to making the existence of the system known and popular were (1)personally addressed letters announcing the upocming system to potential
opinion leaders, (2) short, descriptive brochures explaining vdiat the system
can do, when it will operate, how to get more information, who is in charge,
and why the system is being operated, (3) posters, notices, or advertisements in-house organ, (4) small-group demonstrations and talks, preferablywith hands-on terminal experience, augmented either with slides or moviesand (5) locally postered examples of computer generated input and output
to prepare users with examples of what to expect when they use the system.
In the general area of education/instruction, demonstrations and handson experience with a terminal can be helpful when slanted to emphasize how
to actually use the system. In addition, the education/information functions
most useful included: (1) a special telephone number that users could callwhenever they needed help with use of the system, and provided a searchexpert to talk with, (2) an easily available user's manual emphasizing
self-instruction, with extensive use of input and output interaction, anda step-by-step procedure for developing search inquiries. The Users Manual
was found to be one of the most important and helpful devices in developing
a user population that could independently learn hew to use an interactive
retrieval system, (3) a small, "Pocket Reminder Card" containing a summaryof the essential elements of using the system that could be used in place ofthe manual when a user had acquired the basic skills of searching.
c.

User Control of Interaction

In addition to the areas of publicity/information and instruction/
education, a user approach to. system design has to emphasize a maximum ofuser control in the interactive dialogue. User control implies that theinteractive dialogue should provide an individual with a chance toconstantly switch tactics at any point in the dialogue. For example, thismight mean the ability to use a single keystroke to cancel an inquiry thatwas being developed and begin another, or the chance to use a single keystroke and junp from one data base to another. This ability to switch tactics would have as its goal the opportunity for an individual to make adecision and implement it at a terminal as quickly as he mentally makes thedecision.
••

The SUPARS/DPS system was implemented to provide as many decision/implementation points as possible in the user interaction. In effect, theuser remained in control as much as possible and did not have to wait for
lengthy oenputer replies or type more than one or two keystrokes to implement a change that he has just decided on.
Another aspect of the user orientation to free-text searching without
intermediaries means that an individual should have additional supply ofpotential search keywords available at any point in the interactive dialogue.
The ability to easily obtain new words through a free-text vocabulary, or
through previously stored search inquiries was a key user-related improvement in the current research.

120

d.

Obtaining User Response

One of the final requirements for developing a user orientation indesigning and redesigning is the use of techniques to obtain user responsethat can be used as feedback to modify aspects of the total user/softwaresystem.
An in£artant facet of obtaining user response is the need for using morethan one method of data collection to insure that a single method, in itself,
wauld not be a biasing factor in obtaining a certain type of response.
The other related facet is the need for gathering more than one type of
response or behavior in order to obtain more than one dimension of a user's
response to the system, ftiis multi-method, multi-trait technique was
employed in the current research and provided the advantage of avoiding amyopic viewpoint when analyzing the complexity of the user's response to
an interactive system.
Several generally accepted techniques of the social sciences have been
employed to obtain abjective and verifiable data, such as a Semantic
Differential Attitude Scale, personal, in-depth interviews, records oftelephone inquiries, and unobtrusive measure of user interaction.
e.

Conclusion

The difference between the ability to design a highly adaptive retrieval
system and a highly adaptive system that is well known and maximally used
might lie in the designer's ability to shift his focus from defining the"system11 as the hardware and software to an enlarged concept of the systemthat puts the user on an equal footing.
Projected interactive systems of the future not able to support highly
trained intermediaries will have to follow the axiom of "beginning withthe user and designing outward." The beginiiing-with-the-user of SUPARS/DPS
and its concurrent high use has been in great part a function of the techniques discussed in the previous sections.
As Paisley (1968) points out "...Computer based storage and retrieval
systems that sure not integrated into these social, political,
and economic systems will be expensive, unused novelties." (5)Paisley's statement could also be viewed from the other side of thecoin to emphasize that "social systems of users that are not integrated
into computer retrieval systems can result in uninterested, little informed
nonusers.

121

APPENDIX IVPROGRAM DESCRIPTION OF SUPARS STATPAC
Michael McMullen
The SUPARS STATPAC was written to fill the need for a package of programs
to analyze and sunmarixe the activity that occurred through the Search
Monitor,
The major considerations in the writing of the package were:
1. The package must be usable by relative non-prograimers *2.

The diverse data types and forms demand a flexible
package Which could handle rather variable records.

The approach which was chosen was to make use of the catpile-time
preprocessor available in IEM Pl/U What was written was a large body of
pre-processor statements which would conditionally create the source code
for any program which was desired for analyzing the Monitor statistics,•Dais program is henceforth referred to as MACRO.
This approach was taken because of the facility with which it fulfilled
the t**> major criteria stated above. The package is relatively easy to use
since the input to MACRO consists of simple keyword assignment statements.
The user nead have very few concerns about the data types being handled
since once the basic data description is inserted in the program it
will take care of all data conversions.
Data Flow of SUPARS Log
The activity log as produced by the Monitor consists, for each searchnumber generated, of a possible 14 different types of records. Each ofthese different types of records are produced by a different type ofactivity within the monitor. One record is produced vtfien the Monitor
is brought up and records the date and time thus producing a date for all
records that follow. For each search a separate record is produced for eachof the following things: the entry of a delta (time and type), the input
of a search line or the production of a line of output (the text of thatline), the entry of a GO or STAT or. CFF or the pushing of attention to halt
output, etc.
Doe to factors such as the greatly increased handling and preparationneeded for each use of the log data it was elected to combine the manysnail records for each search into one large record. To accomplish this a
PL/1 program was written which merged the records for each search. This
program is henceforth referred to as CONVERT.
Since the log records as produced by the ffonitor were not strictly inorder by search, due to the fact that several searches could be in progress122

simultaneously, it was first necessary to sort the records as to searchnimber. This was done through the use of the IBM SORT/MERGE package. These
sorted records were then used as input to CONVERT.
Before beginning the actual processing of the records several parameters
are first set in the program. The first of these are conversion factors to
be used in the ocnputation of cost figures for each search. Also set were
two parameters which determined the first and last search to be processed.
The records produced by CONVERT consist of a constant header followed bya variable portion. The constant header contained such information that
had a oonsistant length and form, such as search number, user number, terminal number, etc. The variable portion of the record contained those items
which were conditionally produced by the monitor and which could occur invariable numbers and lengths, these were such things as the input and output
lines and search error codes.
!

At the same time that the records were being reformatted for use instatistical analyses another set of records were being produced by CONVERT.
These records were the ones that served as input for the establishment of
the data base for later delta S searches through the Monitor.
CONVERT also did a certain amount of data cleaning and supplied default(usually zero) values fbr toy record types which were not received for aparticular search.
The flow of data through STATPAC programs, MONITOR, SORT, and CONVERT,is graphically shown in Figure No. 1.
A detailed description of the STRTPAC program follows. The programsreferred to are all explained in the flowcharts labeled Figures 2 through5.
Figure No. 2 describes the overall logic flow for producing STAT output.
Figure No. 3 is a more detailed description of the conversion program vdhich
produces the records STATPAC operates on.
The actual STATPAC processing programs, MACRO and PROCESS are explainedin Figures 4 and 5, respectively.

123

START
MQUTTARPERFORMS
SEARCHESAND PRODUCES LOG

RAWMONITORLOG
QRZ.
SORT LOGINTO
INPUT ORDER
i* CONVERTSORTEDI MONITORLOG

CONVERT'COIl VERT ED
LOG FORAS SEARCil
DATA BASE

<r

MERGE
RECORDSFOR EACH
LOG NUM.

>

CONVERTEDLOG FORSTAT.
ANALYSIS

STOP

Figure No. 1

data Flow of SUPAPS Log

124

/USERORIENTED
INPUT

rmatu
ORIENTEDINPUT

1IEFORMATCONTROL
STATEMENTREFORMAT

yU'.ACRO
INPUT
DATA
,SET

U CRO
PL/1

PREPROCESSORPROGRAM<r

START 3

PROGRAMLIBRARY

CONVERTED,
LOG
DATA

>

PROGRAMFROMLIBRARY- >PRINTEDREPORT

STOP

Figure No. 2 Program Flow for Producing SD\T Output

125

COW VERT
START
ORTED
LOG
FILE

READ
SORTED
LOG
RECORDSET
PARAMETERSJ[/READ UPTO FIRST
RECORD
WANTED^' CONVERTEDLOG STATFILE

SET
END-OF-FILE\
FLAG

1EV
/
LOG
^JH UMBER/WRITEj/ A S SEARCH-^-/CONVERTEDN { DATA
1
RECORDS I\ BASE\SOURCEINITIALIZE
RECORDBUFFERS>

Figure No. 3

STOP

Program f o r Conversion o f Log t o STAT Usable Form

126

SFT CLOCK
COST,CPU,
EXCp IN
OUT-PUTSET"AXTWn\PRINTED ,AND\REPEA TED©
CATENATE
INPUT LINETO OUTPUT
RECORDCATENATE
VO-VTPUT
ITEM

DELTA
VARIABLES

Figure 3(Continued)
127

' GO
H7A TEMF.ViT

Figure 3(Continued)

128

MACRO\SECTIONSOF CODEj MARKED BY BRACKETSJ ARE CONDITIONALLY
i INCLUDED IN! PROGRAMI CREATED

START
SET
PARAMETERS
ALL = 1CALL SORT-[CONVERTEDI LOG

I FILE

<

•

'READ
LOG
RECORD>

CREATE
NEl'
N
OF FILE J? i > M VARIABLES'END \
ALL = 0Figure NO. 7
PROCESSPRINTEDREPORT* £fifi£££2 SECTION^TWTNH~FINAL
COUNTS

OF CODE MAY
DF REPEATEDANY NUMPER
OF TIMES(

Figure

No. 4

STOP

J

MACRO - General O u t l i n e o f STAT Programs

129

PROCESSENTER<rSTATISTICALPROCESSINGSECTION<r

PRINT
VARIABLES,DESIRED

PRINTEDREPORT

RETURN

Figure No. 5 Process - Main Working Section of STAT Programs

130

When using the SUPARS STAIPAC there are 3 ways in which a STAT run canbe initiated.
The first is where a series of control statements are entered into
MPCBD through the use of reformat. Reformat is an assembler language
program which takes control statements in a user oriented form and reformats
than for entry in MACRO ina program oriented form. KEEX3RMAT allows the user
to utilize several alternative characters (e.g. 4* for / ) , takes care of theinsertion of certain details demanded by MACRO arid also does a certain
amount of error checking such as looking for unmatched parentheses. REFORMAT is intended for use with input from an IBM 2741 terminal but isusable with sixiple card input.
If it is not desired to use REFORMAT the second starting point for a
STAT run is used. Here input is made directly into MACRO. Here however,the control statements must already be in a program oriented form.
Whichever form of input was used MACRO at this point takes the control
statements and uses them to produce a PI/1 source program which is thencompiled. The object program is then placed in a program library.
This object program created is loaded and; executed to produce the final
printed report.
This is the third entry point into the sequence. If the program that isdesired has already been created then one need simply call it up from theprogram library thus bypassing the earlier creation steps.
A Sanple Set of Control Statements (As to be input to REFORMAT)
TITLE"SAMPLE

COiiTKUL

STATEMENTS*

CREATE"iiEWVAR= CFU * 0.1 +LIMITS"VATE~ < PARM01

SORT"SSU,.AtDATE\D~

PRINTS" SSU, LOGU, CALENDAR . UEWVAR"STATS"SUii (ALL*1.,NEWVAR)XLISTER

EXCrf* DATE ~< PAR!
^ ^(SEN,CALENDAR)(As to directly input or as output by REFORMAT)
TITLE

=

'SAI1PLE

CONTROL STATEMENTS'

CREATE-'UEWVAR = CPU * 0.1 + EXCP',LIMITS=.'DATE~< PARM01;'[* DATE ~ < PAR 1102' ;

131

=

f

P/tf/IKP='/7Z7// • L 0 G i l •CALENDAR , NEWVAR f ;STylTS=?

S0RT

SSil, A , DA TE.D' ;5Ui! <ALL*l9i!2WVAH)XLISTER (SSll.CALENDAR)f;

The resultant program would have the following characteristics. At the
top of each page of output would be printed the line 'SAMPLE CONTROL STATEMENTS' . A new variable would be created which would contain the value obtained by nultiplyiixf the variable CPU by 0.1 and adding the variable EXCP.
The cases that would be accepted for analysis would be those in which
the variable date was not less than the variable PAPM01 and not greater thanthe variable PAFM02.
The input data set would be sorted first by the variable SSN in ascendingorder and secondarily by the variable date in descending order. For eachcase accepted under the conditions set by limits the variables SSN, L0GNf
CALENDAR, and NEWVAR will be printed.
STATS would cause the following statistics to be accumulated:
SUM ASKS THAT THE SUM9MINIMUM9 MAXIMUM, AND MEAN OF THE VARIABLE+NEWVAR BE
ACCUMULATED WHILE THE, VARIABLE ALL CONTAINED THE VALUE U

XLISTER asks that for as long as consecutive cases of input contain thesame SSN a frequency count for the variable calendar* be kept and when thevalue of SSN changes this frequency count be printed and a new one started.
MACRO as stated in the introduction the heart of the SUPARS STATPAC
system is the PI/1 pre-processor program referred to as MACRO through thesetting of a series of pre-processor variables a processing program for thelog is written. This program will conditionally include the code necessary
to perform the tasks that you desire. The pre-processor variables that wereused were called SORT, LIMITS, TITLE, CREATE, STATS, and PRINT.
The variable SORT allows you to specify that the input data set is tobe sorted into whatever input order you specify. The value assigned to SORT
consists of a series of variable name, sort direction pairs.
The variable TITLE is siirply used to define a title header which is
to be printed at the head of each page of output.
Create allows the creation of new variables frem old variables foruse in later print and STATS sections.
LIMITS is used to define criteria for the selection of cases to beprocessed by the program created.
PRINT is a list of variable names, a PRINT statement will be created
to print the value of these variables for each case selected for processing132

by the program. Die creation of this PRINT statement makes use of two
tables which are set up to describe the data base that the programs created
are to process. The first of these tables is a list of print format items.
They describe how each of the variables is to be printed. The second tablecontains the header field which is to be associated with each variable.
This table is used to construct a header for the top of each page of output.
New variables from any create statenents autanatically have entries generated
in these tables for than.
Most important of all is the STATS pre-prooessor variable. Thisvariable is used to create the segments of code which will do any statisticalanalysis desired. The value assigned to it consists of a series of keywords
each followed by a list of arguments. According to which keyword isspecified code is generated utilizing the arguments to perform the analysis
desired. Seme of the types of analysis we had available were: count the
number of oocurances of a stated logical condition, find the son, minimum,
maximum, and mean, or accumulate a frequency count for a variable.

133

APPENDIX VSUPARS TELEPHONE SURVEY QUESTION AND
REGISTRATION FORM

134

SUPARS TELEPHONE SURVEY QUESTIONNAIRE
REGISTRANTS UWE:TELEPHONE:

ACADEMIC STA3US:

DEPAR3T1EOT:

12/71

SOCIAL SECURTIY NUMBER:

TIME

DATE

COMMENTS(LIST ALL ACTION FOR EACH TELEPHONE CALL. IF INTERVIEW ISCOMPLETED, ENTER "INTEFVIEW COMPIH7EED.")INTRODUCTION
Hello, my name is _^_^and I am calling for SUPARS. We're doing a survey of people vdho registered to use the SUPARS system to get Psychological
Abstracts by ccraputer. Vfe would appreciate it if you could give us a few
minutes of your time. This won't take more than six minutes and in semecases it will be much shorter.
(IF TEE RESPONDENT DOES NOT HAVE THE TIME FOR AN BUKW/lfcW, ASK Hlty/HER WHEN
IT WOUID BE MORE CONVENIENT FOR H D 0 E R , AND HE SURE TO NOTATE THE TIME UNDER -CaMENTS.")
1. Do you remember registering to use SUPARS?
Yes
No

135

(GO TO QUESTION 2)(PROBE, IF RESPONDENT STILLDOESN'T REMEMBER OR STATES THAT
SHE/HE NEVER REGISTERED, THANKHER/HIM FOR THE TIME AND DISCONTINUE INTERVIEW.)

2.

Have you ever gone to a terminal to try to use SUPARS?
Yes
No(ADMINISTER SECTION 1)(GO TO QUESTION 3)

3. Why havenft you ever gone to a terminal to use SUPARS?

(ADMINISTER SECTION 4)

SECTION 1: GENERAL USAGE1.

How often do you go to the terminal to try to use SUPARS?

(PROBE)

Daily
Several times a week
Once a week
Less than once a week2.

Were there any days when you went to a terminal more than once to try
to use SUPARS?
Yes
No

3. When at the terminal, have you been able to sign-on to SUPARS at least
once? By signing on I mean, being able to reach the point where you cantype in a delta to begin a search, after having typed in EXDC PA andyour social security number ?
YesNo(GO TO QUESTION 7)(GO TO QUESTION 4)

4. Why haven't you been able to sign on to SUPARS?
5. liow much time do you estimate that you have spent in trying to use
SUPARS? (PRDBE)
Less than 1 hour
1-2 hours
2-5 hours
5-10 hours
over 10 hours(STOP HERE: ADMINISTER SECTION III)

136

7.

Have you ever been unable to sign-on to SUPARS?
IN QUESTION 3)
Yes
No(SIGN-ON IS DEFINED(GO TO QUESTION 8)(GO TO QUESTION 9)

8. Vtiat kinds of problems have prevented you from getting signed on?
9.

Have you ever had any problems searching SUPARS after you have signed
on? That is, after you have typed in your first delta to start yourfirst search?
Yes
No(GO TO QUESTION 10)(GO TO QUESTION 11)

10. What kinds of problems have you had?
11 • How many searches do you estimate that you have started using SUPARS?
(PROBE)
1 search
2-5 searches5-lf) searches
10-20 searches
20-50 searches
over 50 searches
12.

Have you been able to complete—tliat is , enter and receive output froma search—any searches using SUPJVS? (BEING TOUD THAT THERE ARE NO ABSTRACTS FITTING YOUR SEARCH IS AlflO CONSIDER A (XMPLETED SEARCH.)
Yes
No(GO TO QUESTION 16)(GO TO QUESTION 13)

13. VBiat problems have you had?
14.

How much time do you estinate that you have spent in trying to use
SUPARS at the terminal? (PROBE)
less than 1 hour
1-2 hours
2-5 hours
5-10 hours
over 10 hours(STOP HERE: ADMINISTER SECTION 3)

137

16.

Roughly, what percentage of all searches that you've tried have been
completed? (PROBE)
0-10%
10-20%
20-40%
40-60%
60-80%
80-90%
90-100%17.

Generally, how many searches (or how many deltas) do you attempt in one
visit to the terminal? (PROBE)
1 search
2-5 searches
5-10 searches
10-20 searches
20-50 searches
over 50 searches

18.

Approximately how much time do you spend in one session at the terminal?
(PROBE)Less than 5 minutes
5-15 minutes
15-30 minutes
30-60 minutes
60 minutes to 2 hours_____(STOP HERE: ADMINISTER SECTION 2)

SECTION 2: REACTIONS

In general, do you feel that the output you have gotten from SUPARS
contains the information you want?
Yes
No
2.

(TO TO QUESTION 4)(GO TO QUESTION 2)

In what percentage of your searches would you say that the output has
not contained the information you wanted? (PROBE)
10-20%
20-30%
30-40%
40-50%
50-60%
60-70%

138

70-80%80-90%
90-100%3*

In what way is the output unsatisfactory?

4. Have you ever used the printed Psychological Abstracts?
Yes
No(GO TO QUESTION 5)(GO TO QUESTION 9)

5. Do you find SUPARS faster to use than the printed Psychological Abstracts?
Yes
N6
6.

Do you find SUPARS easier to use than printed Psychological Abstracts?
Yes
No

7.

Do you find that SUPARS helps you to find" more relevant information
than you would find.ttsing printed 'Psychological Abstracts?
Yes
No

8. Have you located relevant articles through SUPARS which you probably
would not have found without SUPARS?
Yes
No
Don't knew
9,What have you liked most about using a computerized infantation retrieval system?

10.

Vhat have you liked least about using a oonputerized information retrieval system? (PROBE: Has anything in particular frustrated you
about SUPARS?)

11. As you know, when you use SUPARS, you can choose any word or words thatyou like as your keywords. Do you think you would prefer instead having
a list-of words froa-vdiich you could.choose your keywords?
Yes having a list
Nousing own words(STOP IERE: ADMEJISTER SECTION 3)

139

SECTION 3: OTHER EiFOWIMTON
1. Itowdid you first find out about SUPARS?

2.

(PROBE)

Daily Orange
RecordSyracuse New Times
Posters (GIVE LOCATION)
SUPARS Classroom Demonstration
Professor
FriendOther (SPECIFY)
I lave SUPARS Vs operating hours been convenient for you?
Yes
No

(GO TO QUESTION 4)

3. What hours voald you prefer?
4. Did you receive a User's r1anual after you registered?
• Yes
No

(C$0 TO QUESTION?)

5. Were you able to gain access to a manual?
Yes
No(00 TO QUESTION 15)

6. I low did you gain access to a manual?
A friend
Library School Resource Center
Psychology Department
Carnegie Library
Other (SPECIFY)
7. Did you read the manual before trying to use SUPARS?
Yes
NO
H.

(00 TO QUESTION 9) .

Have you read your manual since then?"
Yes
No

140

(GO TO QUESTION 15)

9.

Do you take your manual with you to the terminal when you search SUPARS?
Yes
No

10.

Did the manual present the material clearly?
Yes
No(GO TO QUESTION 12)

11. What do you feel was not presented clearly in the manual?
12.

Did you feel tliat anything was left out of the manual that you wanted
to know?
Yes
Ifo(GO TO QUESTION 14)

13.

VJiat do you feel was left out?

14.

Did you learn more fran the manual or fran actual practice at the terminal?
•••'•—..•.
..

'

'

'•

•*••'•'

15.

/.v'r.

"anual-.
Terminal
Wasn't able to learn anything

Did you receive a Pocket Reminder Card. (PROBE: A small folded rocketsized card, summarizing the main steps in using SUPARS.)
Yes
No

16,

(GO TO QUESTION 20)

Did you use the Pocket Reminder Card?
Yes
ifo

17.

_____

(GO TO QUESTION 19)

Did you find it helpful?
Yes
No

18.

What would make it more helpful?

'.9. Do you take the Pocket Reminder Card with you to the terminal when you
search SUPARS?
Yes
No

141

20.

Have you ever iiad a demonstration of the SUPARS system?
Yes
No

21.

(GO TO QUESTION 22)

Were you av/are tliat demonstrations were availal)lc?
Yes
No

22.

Have you ever called extension 4220 for help vdth questions or probleros
about SUPARS?
Yes
I to (GO TO QUESTION 25)

23.

Itowoften have you called extension 4220 for help?

24.

lias the person answering been able to help you?

_

Yes
No.
25.

Do you intend to use SUPARS again this foeek?
Yes
No
Don't know

2G. Would you sign up for SUPARS if it bccaine available again?
Yes
NO
Don't know
27.

Give ne tws or tliree adjectives or other words which might best describeor characterize your experience with SUPARS.

28.

Do you have any questions now about SUPARS v M c h you v;ould like to ask?

Ihank you for your tijne and cooperation.
SECTION 4: OTHER EtfTMlATION

1.

Itowdid you first find out about SUPARS?

(PROBE)

Daily Orange
Record142

Syracuse New Times
Posters (GIVE LOCATION)
SUPARS Classroom Demonstration
Professor
FriendOtlier (SPECIFY)
Have SUPARS's operating hours been convenient for you?
Yes
Ito

(GO TO QUESTION *)What hours would you prefer?
Did you receive a User's ^lanual after you registered?
Yes
No(GO TO QUESTION 7)

Were you able to gain access to a manual?
.,*,-,••

Y

e

s

Ito
How did you gain access to a manual?

(06/T0 QUESTION 12)(PROBE)A friend
Library SCITOOI Resource Center
Psyctology Deparbnent
Carnegie Library
Other (SPECIFY)
Did you read the manual?
Yes
No(GO TO QUESTION 8)(GO TO QUESTION 12)

Did the manual present the material clearly?
Yes
No(GO TO QUESTION 10)

What do you feel was not presented clearly in the manual?
Did you feel that anything was left out of the manual that you vented
to knotf?
Yes
No
What vjas left out?

143

(GO TO QUESTION 12)

Did you receive a Pocket Rominder Card? (PROBE: a small folded pocketsized card, summarizing the main steps in using SUPARS.)
Yes
No(GO TO QUESTION IC>)

Yes
Ifo

(00 TO QURSPinN 14)(GO TO QUESTION 16)

Did you use it?

Did you find it helnful?
Yes
Ifo
What would make it more helpful?
I lave you ever liad a demonstration of the SUPARS system?
Yes
Ifo(GO TO QUESTION IB)

Were you aware t h a t demonstrations were, available?
Yes
Ifo
I lave you ever called extension 4220 for helo with questions or problems
about SUPARS?
Yes
Ifo(GO TO QUESTION 21)

How often liave you called extension 4220 for help?
Has the person ansvering been able to help you?
Yes
Ifo
Do you intend to use SUPARS again this week?
Yes
Ifo
Don't know
Vtould you sign up for SUPARS if it became available again?
Yes
. Ifo
Don't know

144

23.

Give me UJD or three adjectives or other vorcls which might l^est describeor characterize your experience *dth SUPA^S.

24.

Do you have any auestions now about SUPfiRS **tich y°u v.ould like to as];?

Thank you for your time and cooperation.

145

1. Name
2. Date3. Social Security^
4. Campus/Home Address
5. Cantpus/ISome Phone
6. Are you primarily: Faculty Undergraduate student Graduate Student^
Staff Mministrator
7. What is your academic major or department?
Question does not apnly
3. What percent of your time is engaged in: Teaching and/or learning
Research
Other
9. Did you register for SUPAPS last year?
YES
NO
10. If YES, on the average, how often did you use SUPARS?
a. more than once a day c. several times a weake. once a monthb. once a day
d. several tines a month f. less frequently
11. On the average, over the last 2-3 months, how freouentlv have you usedPsychological Abstracts?
12. Estimate how frequently you see yourself using Psychological Abstracts inthe next 2-3 months (assuming computer service were !HT available)?
Select ONE choice from 'a-g' above.
13. Vftiat recent interest areas are important to your use of Psychological
Abstracts?
(Most)
(2nd)
"
(3rd)
— - \
14. I lave you used Rsycliological 7\bstracts in the last 2-3 months to heln you
prepare a terra paper, thesis, soeech, etc.
(If YES, nleasex
give title or topic)
,
:
•.
__
15. Do you need Psycliological Abstracts in the next 2-3 months to help you
find references or abstracts for a term paper, thesis, sneedi, etc., you
are currently preparing or planning to prepare?
(If YES,please give title or topic)
16. Wliat is your specific need for Psycliological Abstracts in the near future?
To find a specific reference or abstract? To keep up to date in one or
tv.o content areas? To survey the literature in general? To exhaustively review a specific topic or area? Tb find several current referencesin an area (but not necessarily exliaustively review that area?)_
Other (Specify)
17. What previous experience do you have with conputer terminals? A lot?
Same?
None
If youliad oj<oerienceplease identify
typesystems
18. Whatprevious
experience
doliaveyou havewith catputcr-l^ased
retrievalofterminal,
when
you
used
it,
where
used?
___
A lot?
Some?
None?
: If you have had experience please identify
the system, and where and wlien you used it'

Figure No. 1

SUPARS Pjegistration Form

146

APPENDIX VI
INnCOUCTICN TO SEMANTIC DIFFERENTIAL OUESTIONNAIRE

147

APPENDIX VI
iraRODUCTION TO SEMANTIC DIFFEJREtWIAL QUESTIONNAIRE

Hi,
SUPARS is interested in your reactions. Please fill in tiie enclosed form
today and return it in the enclosed envelope to me via campus mail. Thankyou.
Pat Moell
SUPARS

Have you registered to use SUPARS this year?

YES

110

Did you register to use SUPARS last year?

YES

NO

J lave you read the SUPARS User's Manual '71
edition?

YESNOIF YES : How many different days did you use SUPARS?
are sufficient)
How many SUPARS searches did you make?
begins every time you type in a A.)(Approximate answers(NOTE: A SUPARS search(Approximate answers are sufficient.)

IF YOU ARE FILLING OUT THIS QUESTIONNAIRE IN CLASS, DURING A REGULAR* CLASS PERIOD, PLEASE GIVE THE COURSE TriLE AND NLT'BER IN THE SPACESPROVIDED BELOW.
COURSE TITLE:
COURSE NUMBER:

143

INSTRUCTIONS
The purpose of this study is to measure your reactions to certain tilings byhaving you judge them against a series of descriptive scales. In taking thistest, please make your judgements on the basis of hew you react to them. On
each page of this booklet you will find a different concept to be judged andbeneath it a set of scales. You are to rate the concept on each of thesescales in order. Here is how you are to use the scales:
If you feel that the concept at the top of the page is very closely related
to one end of the scale, you should place your check mark as follows:
fair /* :

:

:

fair

:

:

:

:

:

unfair::: ^unfairor:

If you feel that the concept is quite closely related to one or the other endof the scale (but not extremely), you should place your check mark as follows:
strong: JT • :m + ,,•

— !

:

:

:

or
strong

:

:

:

weak

u

b.

:

:

: X

:

weak

If the concept seems only slightly related to one side as opposed to the otherside (but is not really neutral), then you should check as follows:
active:: %:active

:

:

:

:

:

:

passive: ^::passiveor
The direction toward which you check, of course, depends upon which of thetwo ends of the scale seem most characteristic of the thing you're judging.
If you consider the concept to be neutral on the scale, both sides of thescale equally associated with the concept, or if the scale is completely
irrelevant, unrelated to the concept, then you should place your check-markin the middle space:safe::: \^ :::
dangerous
7T5 Place your check marks in the middle of spaces, not on the boundaries.
(2) Be Aire you check every scale for every concept — do not omit any.
(3) Never put more than one check-nark on a single scale.
Sometimes you may feel as though you've had the same item before on the test.
This will not be the case, do not look back and forth through the items. Do
not try to ranwmber how you checked similar items earlier in the test. Makeeach item a separate and independent judgment, Wbrk at fairly liigh speecP
through this test. Do not WDrry or puzzle over individual items. It isyour first impressions, the iimediate "feelings" about the items, that we want.
149