From: stamm@MasPar.COM (Richard Stamm)
Newsgroups: bionet.molbio.genbank
Subject: BLAZE survey
Date: 5 Jan 93 02:03:42 GMT
Distribution: bionet

This posting is in response to Michael McKenna who writes:

> ...I grew kind of fond of the BLAZE server at IG during the short period
> it was available (for free). And does anybody have any comments regarding the
> _practical_ advantages of the various search strategies- ie FASTA, BLAST,
> BLAZE.

I recently compiled the responses of a survey taken of all users
of the GenBank BLAZE server.   We will post a tally and summary of 
all the responses shortly.  What I am posting here is a listing of 
the qualitative comments contained in the responses -- that is, 
comments where the person had actually used the server and took 
the trouble to write down something other than "yes", "no", or 
"couldn't tell".  We felt that some of these qualitative comments 
reveal the kind of information that Michael asks for.

A scientific comparison of the various search methods is in also
in preperation.  

Rich Stamm
stamm@maspar.com

GENBANK 1992 BLAZE USER SURVEY -- LISTING OF QUALITATIVE COMMENTS
=====================================================================

No better sensitivity -- I still have no match for my novel gene.

The Blaze algorithm seemed as sensitive. Major advantage was speed.

Blaze was routinely the most sensitive.

It picked up matches that were significant but very difficult to see 
other than by such an algorithm

Maybe fastdb could be tweaked to get better sensitivity but Blaze is 
easier.

YES. THE SENSITIVITY WAS BETTER WHEN COMPARED TO PROGRAMS LIKE TFASTA 
AND WORDSEARCH FROM THE 'GCG' PACKAGE.

Yes, over FASTA and BLAST

the default values gave me the results I needed.

I would very much enjoy being able to continue to use it either on the 
Genbank or NCBI servers as an on-line service. Will this be possible in 
the near future?

I liked the algorithm. It did the job.  I did get different results 
sometimes and the BLAZE results were more in line with what I expected.

I don't know about sensitivity. Most of the times, I did not run the 
same search on both FASTA and BLAZE.

Generally (better) yes.

I liked BLAZE better because it was fast and very sensitive for the 
alignments of very distantly related Drosphila vs yeast protein 
comparisons but that is not my usual goal here at the FDA...but when an 
unknown sequence comes up in a sequence it is great.

very sensitive for the alignments of very distantly related Drosphila vs 
yeast protein comparisons...

Defaults are usually the best for me unless I have a lot of time to play 
with the program and sequence...not really often does a bench scientist 
have the time/knowledge/inclination to massage alignments to that 
degree.  A person interested in 2ndary, and tertiary  structures is a 
different story and is more familiar with computer work than test tubes.

Fewer false positives seemed to be the case  -finding matches with large 
gaps Yes this seemed to be better than Fasta...of course much better 
than blast

did not find improved sensitivity.  I only ran a few searches... for 
unique proteins.

No, but this is hard to judge.  the protein I was looking for homologies 
too appears to be unique, under all search algorithms.

I think the overall senstivity is similar-I really don't know enough  
specifics to judge this very well.

We were quite impressesd with BLAZE.

BLAZE appeared to be more sensitive than FASTA and BLAST.

We noticed a small margin in improvement in biological significance, we 
had greater similarities amongst the bacterial sequences, with fewer 
false hits.

revealed matches that were missed by the other programs.

greater sensitivity than FASTA & BLAST

yes; fewer false positive and negative hits

Yes. (better than) Fasta, Blast

Previously insignificant hits elevated to a level where they bacame 
significant.

YES, MUCH BETTER THAN WITH FASTA AND BLAST.

-previously insignificant hits having their scores elevated to where 
they became significant, when these hits actually were biologically 
significant  YES, REALLY

I found blaze to be more complete and allowed me to find more 
homologies.  I particularly liked the annotation that appeared with the 
search.  Very useful.

I did find better sensitivity than other searches... from time to time 
the blaze search did not come up with significant homologies found in a 
Blast search.

There did appear to be fewer negative hits.

Since I never got an alignment I'm not sure if I was impressed or not.

I like the results and was impressed.

we found some additional sequences with BLAZE, and we felt that some of 
the ones we found were significant.

Other than speed, the results were pretty similar for comparable 
substitution matrices using otherwise default gap penalties.

Yes.  I've carried out some of my own tests with SSEARCH versus BLAST 
and FASTA.  Given similar substitution matrices,  the Smith-Waterman 
algorithm  gave better performance overall... Since August, BLAST has 
been reporting P-values for multiple hits, and combined with the use of 
word-size=3 sensitivity seems improved.  My earlier tests  compared it 
to the older version of BLAST, in which BLAZE did better overall. I'm 
less sure now.  In any case, the BLAST e-mail server uses PAM120 as a 
default, and this greatly limits its potential.  Since FASTA (or FASTDB) 
tries to model the Smith-Waterman algorithm with smart heuristics, it's 
hard to see how these programs can do as well; certainly not at ktup=2.

Mostly  I used BLOSUM 60 with the default gap penalties for anything 
important... the matrix (PAM vs. BLOSUM for example) had more of an 
effect in my tests, such as for the  G-protein coupled receptors, and 
other families... I might be  prejudiced, but I think that the 
substitution matrix makes more of a difference than FASTA ktup=1 vs S-W, 
and maybe even FASTA ktup=2 vs S-W.

I would find the results of this survey interesting if people have 
comments about relative performance of substitution matrices using 
BLAZE.  I already have a copy of the Brutlag et al manuscript describing 
your tests.

your implementation of s-w seems to give comparable output to other 
versions i have used

Yes (better than)  FASTA, BLAST

-returning grouped superfamilies ranked by their evolutionary distance 
from the query  Yes -fewer false positive and false negative hits  Yes

My impression was that BLAZE worked very well.

Blaze is great a vast improvement.  The format of the output was 
particularly impressive.

I could not really judge if the program would generate better matches 
because I did not do side by side searches.

Yes, I did vary the pam scores and achieved differnt results

My use of BLAZE was relatively limited so I not able to provide any 
useful input for your survey.  However, I would  like to be able to 
continue to use BLAZE in the future and hope it will continue to be 
available to us.

I personally like having the option of doing BLAZE searches.I find them 
easier and I also like the fact that a different algorithm  can be used 
to find any sequence similarity to what I'm looking for.

I was impressed with speed even via email server.

Yes - but the sequences used were not designed to best show this - an 
Actin  sequence which of course dredged all the actins - the setting of 
different PAM's definietely had an effect on the ordering.  The other 
sequence was a Varicella zoster virus sequence which we knew only had 
two anywhere near homologous  sequences.  If the advertising speak is to 
be believed with the distantly related haemoglobins and plant 
leghaemoglobins than it is impressive!

Not in general, although in a single case BLAZE found a similarity which 
had been missed by FASTA and BLAST

I varied the PAM matrix over a broad range and found considerable 
differences. The example mentioned under 2) was only found with PAM250 
(if i remember correctly).  I did not experiment with varying penalties.

I had the impression of a less clear significance threshold in BLAZE 
compared to BLAST. Positive and negative hits scored closer toghether. 
In general, high BLAZE scores, although not clearly separated from the 
main field, in most cases had a biological meaning. --Clearly fewer 
false positives

-returning grouped superfamilies ranked by their evolutionary distance 
from the query       sure.  this is what we really want sometimes. --
finding matches with large gaps sure.  No gaps introduced in alignment 
might be one of the disadvantages of BLAST (so it is faster?).

Yes, (better than) both FASTA and BLAST formats.

I varied the gap and gap extension penalties as well as the PAM matrix 
but observed no significant difference.

My searches yielded gapped matches that otherwise might not have 
appeared.

I would like to see BLAZE offered thru NCBI as it is the only up to date 
database to which I have access.

our overall impression : BLAZE is significantly more sensitive than 
FASTA or BLAST.

We would be very interested in access to a MasPar computer running 
BLAZE, either interactive access or via a mail server.

I cannot really compare sensitivities, because I (still) do not know 
which alignments were significant in the first place.  I was using BLAZE 
to try to find alignments to my (still) unknown new clone.

noticed increased speed, did not assess sensitivity.

The algorithm identifies useful sequences on this system very quickly.

I think Blaze may be a little more sensitive than FASTA

I used a complement of FastA, Blast and Blaze.  However, the search that 
worked the best is the Sensitive Search of Pat Argos at the EMBL.

Yes:varying the Pam matrix and providing a user programmable input is 
essential.

-returning grouped superfamilies ranked by their evolutionary distance 
from the query -- that was great.  -previously insignificant hits having 
their scores elevated to where they became significant, when these hits 
actually were biologically significant -- Yes, we did see improvement

 I have only performed a database search once using the BLAZE program.  
In comparison to BLAST available through NCBI/NLM/NIH, the search took 
much longer (based on the amount of time required to receive the 
results)

no more sensitive than BLAST.

I did not try to optimize my search parameters for either program, 
however, the results in both instances were satisfactory.

My favorite would have to be Blaze... I was in fact becoming quite 
dependent on Blaze, and am quite interested in gaining access to it 
again, though cost is certainly a factor.  I like it more for it's 
exploritory nature.  when one is lost with no related proteins that one 
knows of, the ability to spin the knobs and look through long lists of 
mostly crap is essential.  When there are close homologies in the 
databases it doesn't really matter, and Blast is probably best at giving 
an uncluttered list of family members.  But I have been dealling with 
proteins that only have distantly related homologs in the data base and 
the "touchie-feelie" nature of Blaze has been very usefull in pulling 
then out.

Yes, (I varied the search parameters) quite a lot, and the ability to do 
that quickly was quite important.  First because the default values are 
very poor for my particular family of proteins.  I got much better 
results with increased Gap/length penalties, and with Blosum rather that 
the PAM matrix.  Of cource I didn't know that at first since I didn't 
know what my protein was related to! It was the ability to fire off a 
bunch of searches and see what hits were common to all of them, even 
though far down the list, that saved me.  And some luck...  WORD might 
have done OK, but is to slow on the VAX to do many variations on the 
search (more later) and so wasn't heplfull.  The usefullness of this 
tech. can be seen in some of Steve Henikkoff's work with BLOCKs.

False hits change dramatically with (search parameter) conditions, while 
the real hits, even if weak, tend to hang in.

IT APPEARED ADEQUATE.

I haven't noticed better sensitivity using BLAZE over FASTA or BLAST.  I 
may not have come up against a good test case to judge the differences.  
I.e., my cases were either substantially homologous or not homologous at 
all.

This algorithm is a significant improvement over FASTA in speed, ability 
to account for gaps, usefulness and flexibility in the way the search 
data is formatted and expressed.

This is hard to say.  there were no matches in any of the databases for 
the sequences in which I was interested.

far better results than the Blast program

Blaze found more significant areas of homology than blast, some of which 
showed other genes in the same family with similar functional areas.

I can't tell - The sequences I used for the BLAZE searches did  (so far) 
not show any significant similarity to any sequence in the  databases 
with any of the search algorithms. Other search algorithms used were 
FASTA, TFASTA, BLAST, WORDSEARCH (UWGCG) in a complementary manner to 
convince myself that there is indeed no similar sequence in the 
database.

Yes, I used different parameters - yet nothing significant came up.

It (BLAZE) is very useful and quick to check homologies  of sequences as 
they are sequenced in the laboratory, but only in the one direction.

I didn't necessarily pick up any matches in the database that I didn't  
see with FASTA but the alignments achieved were extended over greater 
distances.

It did seem to eliminate the usual false positives like gamma globulin.

very pleased with the algorithm and decided to use blaze instead of 
blast or fasta.

The sensitivity was probably somewhat better with blaze, but we did not 
make a lot of comparisons between the different programs.

No, we did not see any new protien matches that were not picked up by 
fasta.

We liked the output format for alignments much better than that produced 
by fasta.

It was quite useful and very fast.

Yes.  (better than) FASTDB.

Yes. A grouping of related species not previously detected as such.

I was hoping that BLAZE would be offered by NCBI.

No noticable increase in sensitivity compared to BLAST.  My search 
sequence was a novel sequence, though.

The algorithm seemed efficient to me.

I think that many of the local homology detection programs, like blaze 
and blast find many good short matches that are not biologicvally 
relevant

Yes. (better than) FASTA and BLAST

I did not try changing search parameters as the default was excellent 
for my purposes.

Blaze is far better than anything I have tried.  The protein I was 
searching the banks for is of parasite origin, but I could trace 
evolutionary relationships well with this program.  The length of the 
comparison that is displayed combined with the annotations make the 
program especially valuable compared to others.  The sensitivity of the 
search of course is the main attraction.  Matches were found despite 
large gaps.

YES.  Better than both FASTA and BLAST.

Found matches with large gaps and improved matches with partial 
sequences.


		-- end --