Approved-By:  James Powell <JPOWELL@VTVM1.BITNET>
Newsgroups: bit.listserv.vpiej-l
Date:         Thu, 7 Jan 1993 12:57:39 EST
              and Access" <VPIEJ-L@VTVM1.BITNET>
From:         Stevan Harnad <harnad@Princeton.EDU>
Subject:      ELECTRONIC PUBLICATION AND THE KNOWLEDGE GLUT


            ELECTRONIC PUBLICATION AND THE KNOWLEDGE GLUT
    Part I: New Electronic Search/Retrieval Tools Such As VERONICA
          Part II: Is There a Substitute for Peer Review?

                        [Posted with permission]

Despite the eulogy at the beginning, this letter actually raises some
points of criticism, which is why it is being posted, with replies, to
invite further discussion. -- Stevan Harnad, PSYCOLOQUY

aa> From: ashley@cs.unsw.oz.au (Ashley Aitken)
aa> Date: Fri, 1 Jan 93 23:44:29 +1100
aa> To:   psyc@pucc.bitnet (PSYCOLOQUY)
aa>
aa> I write to highly commend the APA, particularly those involved with its
aa> initiation, and Stevan Harnad, for the most interesting, appropriate,
aa> and modern form of scholarly discussion you have produced through the
aa> PSYCOLOQUY electronic refereed journal.
aa>
aa> I find PSYCOLOQUY very useful, it gives me the ability to easily (from
aa> the comfort of my office or living room chair) become a part of a large
aa> and international academic community.  There are, however, a number of
aa> points which I would like to make - most of which relate also to the
aa> traditional form of published journals.

Thanks for the kind words. I have some replies to your points below:

aa> 1. Thought needs to be given to ways to reduce the amount of material
aa>    published (in number and size).

Odd suggestion! I would have thought just the opposite. PSYCOLOQUY
published about 30 nonarchival Newsletter issues and 70 archival
journal items (refereed target articles, commentaries and responses) in
1992, and my explicit goal is to double or even triple the archival
portion in 1993!

If the submission volume gets high enough, PSYCOLOQUY may split into
separate sections that can be subscribed to independently, but it is
too early for that. For now, all it takes to discard an article in
which you are not interested is a keystroke. That means PSYCOLOQUY
could have cost you a MAXIMUM of 100 keystrokes all last year! And
surely reducing the publication volume for everyone is not the right
remedy for saving oneself (or anyone else) those keystrokes! PSYCOLOQUY
should of course continue to grow.

Currently, PSYCOLOQUY postings are set up to contain the most salient
portion at the top, so it is not necessary to scroll further if you find
that an item does not interest you. If we ever switch to posting only
abstracts, and archiving full texts so that only those who are
interested in retrieving them need do so, this still won't save that
many keystrokes, as an abstract takes the same space as the first
half-screenful of the full text. I don't think publishing only periodic
lists of titles will be enough for those who take a more active interest
in the contents, but if this is voted in by the PSYCOLOQUY readership,
I am of course ready to comply.

An even more extreme solution would be to have no subscribership at all
(hence no unnecessary keystrokes). This is not as odd it may sound. At
the end of this message I will demonstrate how anything in the
PSYCOLOQUY archive can be scanned (browsing contents, titles),
full-text searched, and instantly retrieved any time, anywhere, with a
very small, simple number of menu-driven keystrokes using a remarkable
new public-domain search tool called Veronica (which will no doubt soon
be superseded by even more powerful tools). A demonstration of Veronica
follows this message.

So, in principle, the days of direct subscription, even electronic
subscription are already over! You always have the option of just doing
occasional searches on your own initiative, like browsing a journal's
contents in the library (except that that you no longer need to go to
the library -- or scan Current Contents).

Some people will no doubt continue to prefer to subscribe, however,
despite the few extra keystrokes it entails to discard the items that
don't interest them.

aa> 2. Thought needs to be given to more powerful indexing and referencing
aa>    techniques.

Look at the sample of Veronica below and you will see that not only has
thought been given, but some unprecedentedly powerful tools are already
available! Boolean search is next, on SGML fields or full text, and even
smarter tools (such as Bellcore's Latent Semantic Indexing, a way of
searching conversationally in ordinary English, now being adapted to
PSYCOLOQUY by Assistant Editor Malcolm Bauer) are not far behind.

aa> 3. The refereeing procedure needs to be made more explicit and open to
aa>    discussion.

I don't know how familiar you are with peer review for scholarly and
scientific publication. An Editor normally consults 1 - 3 experts, who
have the option of being anonymous. That system is medium independent.
As it happens, because of the additional goal of eliciting Peer
Commentary, PSYCOLOQUY often consults even more referees than that (the
recent target articles by Garnham and by Glenberg & Mathews were
reviewed by over TEN referees each, and underwent a major round of
revision before acceptance). Commentaries and authors' responses are
also refereed (although often only by the Editor); and of course the
peer commentary is itself a form of open "refereeing." So I would say
PSYCOLOQUY's refereeing system is not only at least as rigorous as
standard peer review, but it is more broad-based and answerable to the
peer community.

aa> For all of these, although I have ideas, I naturally don't have complete
aa> solutions.  My one comment would be that with these times of change one
aa> should not hold back - be bold, incorporate new ideas and technologies. I
aa> personally feel we should make a major break from the traditional pub-
aa> lication process.
aa>
aa> Lastly, as with Stevan, I am strongly opposed to charging for individual
aa> issues.  I think in the electronic form the distribution (and media) cost
aa> should (eventually) be so minimal not as to warrant an individual charge.
aa> Unfortunately, in other publishing arenas even though the cost of dist-
aa> ribution has been (or could be) dramatically reduced the cost to the in-
aa> dividual has not come down.
aa>
aa> Once again I thank you all for PSYCOLOQUY and trust you take my suggestions
aa> in the good spirit that they are given.
aa>
aa> Ashley Aitken
aa> E-MAIL      :               ashley@cs.unsw.oz.au                     AARNet
aa> Academic Address:                   Residential Address:
aa> Schools of EE and CS&E, (AI Lab)    c/o Basser College, (Flat 7A)
aa> University of New South Wales,              The Kensington Colleges,
aa> Box 1,PO KENSINGTON,N.S.W.,2033.    Box 24,PO KENSINGTON,N.S.W,2033.
aa> AUSTRALIA.                          AUSTRALIA.
aa> Ph. (02) 697-3940 Fx. (02) 663-4576 Ph. (02) 663-8117 (Preferred!)

--------------------------------------------------------------------

             HOW TO USE VERONICA TO DO FULL TEXT SEARCHING
             AND RETRIEVAL OF ALL BACK ISSUES OF PSYCOLOQUY

(1) Call up gopher, by typing "gopher". You will get a screen like this:
-------------------------------------------------------------------------
                    Internet Gopher Information Client v1.1

                               Internet Resources

      1.  Apple Computer Higher Education gopher server/
      2.  FTP Searches/
      3.  Federal Information Exchange <TEL>
      4.  Geographical Name Service <TEL>
      5.  Internet Mail Guide <TEL>
      6.  Internet Resource Guide <?>
      7.  Knowbot Network Email Directory <TEL>
      8.  Sun Managers Archive/
      9.  The Cleveland Freenet <TEL>
      10. The Whole Internet: User's Guide & Catalog/
 -->  11. Veronica (search menu items in most of GopherSpace)/
      12. WAIS Based Information/
      13. Weather/
-----------------------------------------------------------------------


(2) Select Veronica (11.). You will get a screen like this:
-----------------------------------------------------------------------
              Veronica (search menu items in most of GopherSpace)

      1.  About veronica.
 -->  2.  Search many (300) Internet gopher menus by _SINGLE_ keyword  <?>
      3.  Experimental :)   ( specify desired item types ) <?>
      4.  Help on how to use Experimental veronica.
      5.  Proposals for veronica Development.
      6.  gopher.protocol.
------------------------------------------------------------------------


(3) Select Search 2. You will get a screen like this:

 +--------Search many (300) Internet gopher menus by _SINGLE_ keyword --------+
 |                                                                            |
 | Words to search for  psycoloquy                                            |
 |                                                                            |
 |                                       [Cancel ^G] [Accept - Enter]         |
 |                                                                            |
 +----------------------------------------------------------------------------+


(4) Enter psycoloquy  in the blank space (as above). Next screen:
------------------------------------------------------------------------
    Search many (300) Internet gopher menus by _SINGLE_ keyword : psycoloquy

      1.   About the PSYCOLOQUY Archive.
 -->  2.   Psycoloquy Searchable Archive <?>
      3.  Volume 1 (1990)/
      4.  Volume 2 (1991)/
      5.  Volume 3 (1992)     [last change Dec.22 1992]/
------------------------------------------------------------------------


(5) Select Psycoloquy Searchable Archive (2.). Next screen:
 +----------------------- Psycoloquy Searchable Archive-----------------------+
 |                                                                            |
 | Words to search for  reading                                               |
 |                                                                            |
 |                                       [Cancel ^G] [Accept - Enter]         |
 |                                                                            |
 +----------------------------------------------------------------------------+


(6) Select any topic or author (e.g. "reading," as above). Next screen
    yields every issue of PSYCOLOQUY that has "reading" in its text:
--------------------------------------------------------------------------
                     Psycoloquy Searchable Archive: reading

      1.  psyc.arch.2.9.91   /archives/Psycoloquy/91.V2/.
      2.  psyc.91.reading.1.skoyles   /archives/Psycoloquy/91.V2/.
      3.  psyc.arch.2.8.91   /archives/Psycoloquy/91.V2/.
      4.  psyc.92.3.06.reading.7.skoyles   /archives/Psycoloquy/92.V3/.
      5.  psyc.91.reading.2.coltheart   /archives/Psycoloquy/91.V2/.
      6.  psyc.91.reading.5.skoyles   /archives/Psycoloquy/91.V2/.
      7.  psyc.91.reading.3.skoyles   /archives/Psycoloquy/91.V2/.
      8.  psyc.92.3.13.reading.8.cassidy   /archives/Psycoloquy/92.V3/.
      9.  psyc.92.3.61.reading.10.small   /archives/Psycoloquy/92.V3/.
      10. psyc.92.3.63.reading-inference-1.1.garnham   ..ves/Psycoloquy/92.V3/.
      11. psyc.92.3.05.reading.6.cassidy   /archives/Psycoloquy/92.V3/.
      12. psyc.92.3.64.reading-inference-2.1.glenberg-mathew   ..oloquy/92.V3/.
      13. psyc.91.reading.4.reilly   /archives/Psycoloquy/91.V2/.
 -->  14. psyc.92.3.14.reading.9.skoyles   /archives/Psycoloquy/92.V3/.
      15. psyc.92.index   /archives/Psycoloquy/92.V3/.
      16. psyc.91.index   /archives/Psycoloquy/91.V2/.
      17. psyc.arch.1.11.90   /archives/Psycoloquy/90.V1/.
      18. psyc.arch.1.16.90   /archives/Psycoloquy/90.V1/.
      19. psyc.arch.1.13.90   /archives/Psycoloquy/90.V1/.
      20. psyc.arch.1.8.90   /archives/Psycoloquy/90.V1/.
      21. psyc.arch.1.15.90   /archives/Psycoloquy/90.V1/.
      22. psyc.arch.2.3.91   /archives/Psycoloquy/91.V2/.
      23. psyc.arch.2.7.91   /archives/Psycoloquy/91.V2/.
      24. psyc.arch.1.14.90   /archives/Psycoloquy/90.V1/.
      25. psyc.92.3.11.mood.10.plutchik   /archives/Psycoloquy/92.V3/.
      26. psyc.92.3.16.space.1.bryant   /archives/Psycoloquy/92.V3/.
      27. psyc.92.3.23.consciousness.8.laming   /archives/Psycoloquy/92.V3/.
      28. psyc.92.3.27.consciousness.12.noble   /archives/Psycoloquy/92.V3/.
      29. psyc.92.3.07.mood.6.morris   /archives/Psycoloquy/92.V3/.
      30. psyc.92.3.08.mood.7.nesse   /archives/Psycoloquy/92.V3/.
      31. psyc.newsletter.Dec.16.1   /archives/Psycoloquy/92.V3/.
-------------------------------------------------------------------------


(7) Finally, select (say) item 14 above, which actually retrieves the
    article itself from the archive (only the beginning is displayed
    here for illustration):
     This section is from the document
     '//archives/Psycoloquy/92.V3/psyc.92.3.14.read ing.9.skoyles'.

--------------------------------------------------------------------------
Date: Mon, 02 Mar 92 00:29:36 EST
PSYCOLOQUY   ISSN 1055-0143     Mon, 02 Mar 92

From: John R Skoyles <ucjtprs@ucl.ac.uk>
Subject: psyc.92.3.14.reading.9.skoyles: Logographic Reading: Are We So Differen
t?

        Logographic Reading: Are We So Different?
                Reply to Cassidy

                John R. Skoyles,
                ucjtprs@ucl.ac.uk
                Department of Psychology
                ucjtprs@ucl.ac.uk
                Department of Psychology
                University College London
                London WC1E 6BT

I do not see where Cassidy and I disagree on children's logographic
reading. I agree that phonological reading follows logographic reading.
I mentioned in my prior reply (PSYC 3.1.2.2 1992) that I had previously
proposed this in (Skoyles 1988): "Children pass through several stages
in learning to read English. Initially -- often before entering school
-- children recognise some frequently occurring words using visual
`logographic' characteristics. Most children then quickly learn to use
the phonological information present in written words. The limited
vocabulary of logographically recognisable word could provide the
examples needed to train the phonological network." I think Cassidy and
I are in agreement.

Cassidy claims that I deny children use an abstract letter-to-sound

--More--(32%)[Press space to continue, 'q' to quit.]

Press <RETURN> to continue, <m> to mail, 
 to save, or <p> to print:

-------------------------------------------------------------------------

(8) The file can then be printed out directly, or you can save it in
a local file, or have it emailed to you.

So if you have Gopher/Veronica at your site and you don't want to
receive articles you are not interested in, cancel your PSYCOLOQUY
subscription and simply use Veronica as often as you like to browse
the contents. Veronica will also call up the constantly updated
cumulative Table of Contents for each volume/year of PSYCOLOQUY.


---------------------------------------------------------------------

           ELECTRONIC PUBLICATION AND THE KNOWLEDGE GLUT
          Part II: Is There a Substitute for Peer Review?

ashley@cs.unsw.oz.au (Ashley Aitken) wrote:

aa> Although I do not mind your posting my suggestions and your replies, I
aa> cringe a little at how they make me look, and at how my suggestions
aa> were interpreted - due completely to the cursory way in which I put them.
aa>
aa> So if you will bear with my attempt to explain in a bit more detail the
aa> real essence of my suggestions, I will rest easier :-) with the
aa> original posting, knowing that I have at least tried to make my points
aa> clearer to you.
aa>
aa> In general, my hope was for PYSCOLOQUY to take bold new steps in
aa> different directions, rather than more electronic steps in the same
aa> direction as the current printed journals.
aa>
aa> In particular then, with regard to each specific point -:
aa>
aa> aa> 1. Thought needs to be given to ways to reduce the amount of material
aa> aa>    published (in number and size).
aa>
>   sh> Odd suggestion! I would have thought just the opposite.
aa>
aa> Here, my suggestion must have been unclear - as your amazement shows. I
aa> was suggesting that we need to move away from the idea of everyone's
aa> having their say on an issue directly in the form of an article or review.
aa>
aa> As I said before I don't have all the solutions but generally what I
aa> was thinking of was a more constructive approach where we EXPLICITLY
aa> build some sort of 'knowledge' tree (with all its ambiguities and
aa> branches in so many different directions) which is electronically
aa> accessible (as a hypertext document, for example).
aa>
aa> With the approach that is currently being taken we will end up with
aa> hundreds (if not thousands) of electronic journals publishing tens (if
aa> not hundreds) of thousands of editions a year. Yes, they will be
aa> extremely well indexed and on-line but will that make us feel any less
aa> overwhelmed? The knowledge is only implicit in this juggernaut of
aa> publications, not anywhere near explicit.
aa>
aa> Under an 'explicit constructive' approach, research publications would
aa> instead be auxiliary justification for alterations to the 'knowledge'
aa> tree. Note here that I am not operating under any naive impression that
aa> there is one true tree. There would be many branches on this tree
aa> representing the many beliefs and theories - but each explicitly stated
aa> and perhaps with some weighting.

These suggestions are all commendable, and no doubt many new epistemic
forms will evolve in the new medium. Here I wish only to point out that
there IS a means of sorting the wheat from the chaff in (if I may
change metaphors) the expanding grain bowl of knowledge, and that
means is PEER REVIEW. That quality control mechanism, which consists of
nothing more than qualified peers EXPLICITLY reading and evaluating
material PRIOR to publication, is medium independent. Until you find a
means of accomplishing this automatically, without human intervention
(I doubt that I will live to see the day -- especially in cognitive
science, where to suppose that we already have such a mechanism is
tantamount to supposing we have already solved most of the problems of
cognition!) we are stuck with this "traditional" tool.

No need to despair, though, because most of the information that is
burgeoning on the net does not ASPIRE to be refereed. For that small
subet of it that does, there are still, and will continue to be, enough
peer reviewers to go around. And once they have exercised their
gate-keeping function for us all, we are free, if we wish, to use this
authenticated archive to help filter our reading. In other words, as
long as there is peer review, and a clear means of distinguishing what
in the archive has and has not been peer-reviewed (and how rigorously,
for this too varies from journal to journal), there is still a
medium-independent way of calibrating our knowledge consumption,
despite the epistemic explosion.

There are many exciting things to be done beyond the scope of this
quality-controlled canon, in the vast vanity press that will soon
materialize, but that is not the terrain to which PSYCOLOQUY is
dedicated. There is also room for revolutionary innovation WITHIN the
canon, as it is implemented electronically. That is the domain that
interests me: How to implement peer review electronically, how to speed
it up and make it more even-handed, and how to couple it with the
subsequent phase of open peer commentary.

I wish the knowledge tree well, but I already know (from experience
WITHIN peer review) that mere box scores, publication counts, and citation
counts (in other words, the kinds of variables that would govern the
arborization of the knowledge tree) will never substitute for the acute
insight of a knowledgeable individual. Or not for some time, anyway.


aa> aa> 2. Thought needs to be given to more powerful indexing and referencing
aa> aa>    techniques.
aa>
>   sh> Look at the sample of Veronica below and you will see that not only has
>   sh> thought been given, but some unprecedentedly powerful tools are already
>   sh> available!
aa>
aa> Yes, you are correct - there are powerful tools. I knew of Archie (for
aa> software) and read of Veronica and Gopher in one of the five or more
aa> editions of PYSCOLOQUY still remaining in my mail box after I had
aa> posted to you (BTW, I wonder if there is a universal physical constant
aa> pertaining to the number of issues of a journal which are always
aa> present in one's mail box :-).
aa>
aa> However, no matter how powerful these methods may be (and at present I
aa> believe they are limited to indexing on the file name) at some time in
aa> the future they will surely produce far too many matches to be
aa> useful. If everyone is having their say in the same old manner, and
aa> 'everyone' is growing exponentially, then this path is always
aa> doomed.
aa>
aa> I don't think we should have to do the searches to find the papers to
aa> find the theories and facts to piece together the present state of
aa> knowledge.

Electronic search tools will become ever more subtle and sophisticated,
however it will be some time before they (or hypertext links and
arborization paths) will substitute for informed judgments of quality.
So I predict that the most valuable single source of information about a
retrieved item for some time to come will be whether or not it passed
peer review.

As to unread email: When you will have redirected most of your
telephone, paper mail, face-to-face, conference, and even paper-reading
traffic to your email, as I have, you'll find that "unread email" is not
an informative descriptor, for virtually all your information will be
electronic, and how quickly you attend to it will depend on what
indicators you have of how important it is. Just as in paper days I read
refereed journals before looking at unrefereed preprints, there will be
ways of ordering priorities on the Net too.
aa> aa> 3. The refereeing procedure needs to be made more explicit and open to
aa> aa>    discussion.

>   sh> I don't know how familiar you are with peer review for scholarly and
>   sh> scientific publication.
aa>
aa> I am familiar, to a point, with the process for peer review for
aa> scholarly and scientific publications. I am not sure, however, how much
aa> justification there is for it as the best way to choose the best
aa> publications. Again, I read in a later edition of PYSCOLOQUY of more
aa> radical ideas, viz consensus review. This type of rethinking of the
aa> refereeing procedure is what I was hoping for.

The article advocating consensus review was refereed and accepted as a
speculative hypothesis. But what is needed before we abandon peer review
in its favor is empirical research on the matter, with its results
submitted to refereed journals for peer scrutiny.

aa> Perhaps, I am just naive about the real nature of scientific research
aa> and the reasons for publication, or perhaps I am looking too far into
aa> the future. Anyway these suggestions are just some that linger in my
aa> head when I should be instead concentrating on my graduate research.
aa>
aa> As before, I am very grateful for the PYSCOLOQUY electronic journal and
aa> offer this discussion only as an attempt to make it better.

I have posted your thoughts because I think many others are thinking
along similar lines and because the thoughts are worth discussing. There
are, however, certain prima facie answers that should also be taken into
account, and I have tried to provide some of them here.

With thanks for your comments, which will, I am sure, generate still
further discussion.

Stevan Harnad
Editor, Behavioral & Brain Sciences, PSYCOLOQUY

Cognitive Science Laboratory |    Laboratoire Cognition et Mouvement
Princeton University         |    URA CNRS 1166
221 Nassau Street            |    Universite d'Aix Marseille II
Princeton NJ 08544-2093      |    13388 Marseille cedex 13, France
harnad@princeton.edu         |    harnad@rrmone.cnrs-mrs.fr
609-921-7771                 |    33-91-611-420