Professor D: OK .
PhD A: Mike . Mike - one ?
PhD B: Ah .
Professor D: We 're on ? Yes , please . I mean , we 're testing noise robustness but let 's not get silly . OK , so , uh , you 've got some , uh , Xerox things to pass out ?
PhD A: Yeah ,
Professor D: That are PhD A: um .
Professor D: Yeah .
PhD A: Yeah . Yeah , I 'm sorry for the table , but as it grows in size , uh , it .
Professor D: Uh , so for th the last column we use our imagination . OK .
PhD B: Ah , yeah .
Professor D: Ah .
PhD A: Uh , yeah .
PhD B: Uh , do you want @ @ .
Professor D: This one 's nice , though . This has nice big font .
PhD A: Yeah .
Grad C: Let 's see . Yeah . Chop !
Professor D: Yeah .
PhD A: So
Professor D: When you get older you have these different perspectives . I mean , lowering the word hour rate is fine , but having big font !
PhD A: Next time we will put colors or something .
Professor D: That 's what 's PhD A: Uh .
Professor D: Yeah . It 's mostly big font . OK .
PhD A: OK , s so there is kind of summary of what has been done Professor D: Uh Go ahead .
PhD A: It 's this . Summary of experiments since , well , since last week
Professor D: Oh . OK .
PhD A: and also since the we 've started to run work on this . Um . So since last week we 've started to fill the column with um uh features w with nets trained on PLP with on - line normalization but with delta also , because the column was not completely Professor D: Mm - hmm . Mm - hmm .
PhD A: well , it 's still not completely filled ,
Professor D :
PhD A: but we have more results to compare with network using without PLP and finally , hhh , {comment} um ehhh {comment} PL - uh delta seems very important . Uh I don't know . If you take um , let 's say , anyway Aurora - two - B , so , the next t the second , uh , part of the table ,
Professor D: Mm - hmm .
PhD A: uh when we use the large training set using French , Spanish , and English , you have one hundred and six without delta and eighty - nine with the delta .
Professor D: a And again all of these numbers are with a hundred percent being , uh , the baseline performance ,
PhD A: Yeah , on the baseline , yeah . So Professor D: but with a mel cepstra system going straight into the HTK ?
PhD A: Yeah . Yeah . So now we see that the gap between the different training set is much uh uh much smaller
Professor D: Yes .
PhD A: um Grad C: It 's out of the way .
PhD A: But , actually , um , for English training on TIMIT is still better than the other languages . And  Mmm , Yeah . And f also for Italian , actually . If you take the second set of experiment for Italian , so , the mismatched condition ,
Professor D: Mm - hmm .
PhD A: um when we use the training on TIMIT so , it 's multi - English , we have a ninety - one number ,
Professor D: Mm - hmm .
PhD A: and training with other languages is a little bit worse .
Professor D: Um Oh , I see . Down near the bottom of this sheet .
PhD A: So ,
Professor D: Uh , {comment} yes .
PhD A: yeah .
Professor D: OK .
PhD A: And , yeah , and here the gap is still more important between using delta and not using delta . If y if I take the training s the large training set , it 's we have one hundred and seventy - two ,
Professor D: Yes .
PhD A: and one hundred and four when we use delta .
Professor D: Yeah .
PhD A: Uh . Even if the contexts used is quite the same ,
Professor D: Mm - hmm .
PhD A: because without delta we use seventeenths seventeen frames . Uh . Yeah , um , so the second point is that we have no single cross - language experiments , uh , that we did not have last week . Uh , so this is training the net on French only , or on English only , and testing on Italian .
Professor D: Mm - hmm .
PhD A: And training the net on French only and Spanish only and testing on , uh TI - digits .
Professor D: Mm - hmm .
PhD A: And , fff {comment} um , yeah . What we see is that these nets are not as good , except for the multi - English , which is always one of the best . Yeah , then we started to work on a large dat database containing , uh , sentences from the French , from the Spanish , from the TIMIT , from SPINE , uh from {comment} uh English digits , and from Italian digits . So this is the another line another set of lines in the table . Uh , @ @ with SPINE
Professor D: Ah , yes . Mm - hmm .
PhD A: and uh , actually we did this before knowing the result of all the data , uh , so we have to to redo the uh the experiment training the net with , uh PLP , but with delta . But
Professor D: Mm - hmm .
PhD A: um this this net performed quite well . Well , it 's it 's better than the net using French , Spanish , and English only . Uh . So , uh , yeah . We have also started feature combination experiments . Uh many experiments using features and net outputs together . And this is The results are on the other document . Uh , we can discuss this after , perhaps well , just , @ @ . Yeah , so basically there are four four kind of systems . The first one , yeah , is combining , um , two feature streams , uh using and each feature stream has its own MPL . So it 's the kind of similar to the tandem that was proposed for the first . The multi - stream tandem for the first proposal . The second is using features and KLT transformed MLP outputs . And the third one is to u use a single KLT trans transform features as well as MLP outputs . Um , yeah . Mmm . You know you can you can comment these results ,
PhD B: Yes , I can s I would like to say that , for example , um , mmm , if we doesn't use the delta - delta , uh we have an improve when we use s some combination . But when
PhD A: Yeah , we ju just to be clear , the numbers here are uh recognition accuracy .
PhD B: w Yeah , this Yeah , this number recognition acc
PhD A: So it 's not the  Again we switch to another PhD B: Yes , and the baseline the baseline have i is eighty - two .
Professor D: Baseline is eighty - two .
PhD B: Yeah
PhD A: So it 's experiment only on the Italian mismatched for the moment for this .
Professor D: Uh , this is Italian mismatched .
PhD A: Um .
PhD B: Yeah , by the moment .
PhD A: Mm - hmm .
Professor D: OK .
PhD B: And first in the experiment - one I I do I I use different MLP ,
Professor D: Mm - hmm .
PhD B: and is obviously that the multi - English MLP is the better . Um . for the ne rest of experiment I use multi - English , only multi - English . And I try to combine different type of feature , but the result is that the MSG - three feature doesn't work for the Italian database because never help to increase the accuracy .
PhD A: Yeah , eh , actually , if w we look at the table , the huge table , um , we see that for TI - digits MSG perform as well as the PLP ,
Professor D: Mm - hmm .
PhD A: but this is not the case for Italian what where the error rate is c is almost uh twice the error rate of PLP .
Professor D: Mm - hmm .
PhD A: So , um uh , well , I don't think this is a bug but this this is something in probably in the MSG um process that uh I don't know what exactly . Perhaps the fact that the the there 's no low - pass filter , well , or no pre - emp pre - emphasis filter and that there is some DC offset in the Italian , or , well , something simple like that . But that we need to sort out if want to uh get improvement by combining PLP and MSG
Professor D: Mm - hmm .
PhD A: because for the moment MSG do doesn't bring much information .
Professor D: Mm - hmm .
PhD A: And as Carmen said , if we combine the two , we have the result , basically , of PLP .
Professor D: I Um , the uh , baseline system when you said the baseline system was uh , uh eighty - two percent , that was trained on what and tested on what ? That was , uh Italian mismatched d uh , uh , digits , uh , is the testing ,
PhD B: Yeah .
Professor D: and the training is Italian digits ?
PhD B: Yeah .
Professor D: So the " mismatch " just refers to the noise and and , uh microphone and so forth ,
PhD A: Yeah .
PhD B: Yeah .
Professor D: right ? So , um did we have So would that then correspond to the first line here of where the training is is the uh Italian digits ?
PhD B: The train the training of the HTK ?
Professor D: The PhD B: Yes . Ah yes !
Professor D: Yes .
PhD B: This h Yes . Th - Yes .
Professor D: Yes . Training of the net ,
PhD B: Yeah .
Professor D: yeah . So , um So what that says is that in a matched condition , we end up with a fair amount worse putting in the uh PLP . Now w would do we have a number , I suppose for the matched I I don't mean matched , but uh use of Italian training in Italian digits for PLP only ?
PhD B: Uh yes ?
PhD A: Uh yeah , so this is basically this is in the table . Uh so the number is fifty - two ,
PhD B: Another table .
PhD A: uh Professor D: Fifty - two percent .
PhD A: Fift - So No , it 's it 's the PhD B: No .
Professor D: No , fifty - two percent of eighty - two ?
PhD A: Of of of uh eighteen PhD B: Eighty .
PhD A: of eighteen .
PhD B: Eighty .
PhD A: So it 's it 's error rate , basically .
PhD B: It 's plus six .
PhD A: It 's er error rate ratio . So Professor D: Oh this is accuracy !
PhD A: Uh , so we have nine nine let 's say ninety percent .
PhD B: Yeah .
Professor D: Oy ! {comment} OK . Ninety .
PhD A: Yeah . Um {comment} which is uh {comment} what we have also if use PLP and MSG together ,
Professor D: Yeah .
PhD A: eighty - nine point seven .
Professor D: OK , so even just PLP , uh , it is not , in the matched condition Um I wonder if it 's a difference between PLP and mel cepstra , or whether it 's that the net half , for some reason , is not helping .
PhD A: Uh . P - PLP and Mel cepstra give the same same results .
Professor D: Same result pretty much ?
PhD A: Well , we have these results . I don't know . It 's not Do you have this result with PLP alone , {comment} j fee feeding HTK ?
Professor D: So , s
PhD A: That That 's what you mean ?
PhD B: Yeah ,
PhD A: Just PLP at the input of HTK .
PhD B: yeah yeah yeah yeah , at the first and the Yeah .
PhD A: Yeah . So , PLP Professor D: Eighty - eight point six .
PhD A: Yeah .
Professor D: Um , so adding MSG
PhD A: Um Professor D: um Well , but that 's yeah , that 's without the neural net ,
PhD A: Yeah , that 's without the neural net
Professor D: right ?
PhD A: and that 's the result basically that OGI has also with the MFCC with on - line normalization .
Professor D: But she had said eighty - two .
PhD A: This is the w well , but this is without on - line normalization .
Professor D: Right ? Oh , this the eighty - two .
PhD A: Yeah .
PhD B :
PhD A: Eighty - two is the it 's the Aurora baseline , so MFCC . Then we can use well , OGI , they use MFCC th the baseline MFCC plus on - line normalization
Professor D: Oh , I 'm sorry , I k I keep getting confused because this is accuracy .
PhD A: Yeah , sorry . Yeah .
PhD B: Yeah .
Professor D: OK . Alright .
PhD A: Yeah .
Professor D: Alright . So this is I was thinking all this was worse . OK so this is all better
PhD B: Yes , better .
Professor D: because eighty - nine is bigger than eighty - two .
PhD A: Mm - hmm .
PhD B: Yeah .
Professor D: OK . I 'm I 'm all better now . OK , go ahead .
PhD A: So what happ what happens is that when we apply on - line normalization we jump to almost ninety percent .
Professor D: Yeah . Mm - hmm .
PhD A: Uh , when we apply a neural network , is the same . We j jump to ninety percent .
PhD B: Nnn , we don't know exactly .
Professor D: Yeah .
PhD A: And And um whatever the normalization , actually . If we use n neural network , even if the features are not correctly normalized , we jump to ninety percent . So Professor D: So we go from eighty - si eighty - eight point six to to ninety , or something .
PhD A: Well , ninety No , I I mean ninety It 's around eighty - nine , ninety , eighty - eight .
Professor D: Eighty - nine .
PhD A: Well , there are minor minor differences .
PhD B: Yeah .
Professor D: And then adding the MSG does nothing , basically .
PhD A: No .
Professor D: Yeah . OK .
PhD A: Uh For Italian , yeah .
Professor D: For this case , right ?
PhD A: Um .
Professor D: Alright . So , um So actually , the answer for experiments with one is that adding MSG , if you uh does not help in that case .
PhD A: Mm - hmm .
Professor D: Um PhD A: But w Yeah .
Professor D: The other ones , we 'd have to look at it , but And the multi - English , does uh So if we think of this in error rates , we start off with , uh eighteen percent error rate , roughly .
PhD A: Mm - hmm .
Professor D: Um and we uh almost , uh cut that in half by um putting in the on - line normalization and the neural net .
PhD A: Yeah
Professor D: And the MSG doesn't however particularly affect things .
PhD A: No .
Professor D: And we cut off , I guess about twenty - five percent of the error . Uh no , not quite that , is it . Uh , two point six out of eighteen . About , um sixteen percent or something of the error , um , if we use multi - English instead of the matching condition .
PhD A: Mm - hmm . Yeah .
Professor D: Not matching condition , but uh , the uh , Italian training .
PhD A: Mm - hmm .
PhD B: Yeah .
Professor D: OK .
PhD A: Mmm .
PhD B: We select these these these tasks because it 's the more difficult .
Professor D: Yes , good . OK ? So then you 're assuming multi - English is closer to the kind of thing that you could use since you 're not gonna have matching , uh , data for the uh for the new for the other languages and so forth . Um , one qu thing is that , uh I think I asked you this before , but I wanna double check . When you say " ME " in these other tests , that 's the multi - English ,
PhD A: That 's it 's a part it 's Professor D: but it is not all of the multi - English , right ? It is some piece of part of it .
PhD A: Or , one million frames .
Professor D: And the multi - English is how much ?
PhD B: You have here the information .
PhD A: It 's one million and a half . Yeah .
Professor D: Oh , so you used almost all You used two thirds of it ,
PhD A: Yeah .
Professor D: you think . So , it it 's still it hurts you seems to hurt you a fair amount to add in this French and Spanish .
PhD A: Mmm .
PhD B: Yeah .
Professor D: I wonder why Yeah . Uh .
Grad C: Well Stephane was saying that they weren't hand - labeled ,
PhD A: Yeah , it 's PhD B: Yeah .
PhD A: Yeah .
Grad C: the French and the Spanish .
PhD B: The Spanish . Maybe for that .
Professor D: Hmm .
PhD A: Mmm .
Professor D: It 's still OK . Alright , go ahead . And then then PhD B: Um . Mmm , with the experiment type - two , I first I tried to to combine , nnn , some feature from the MLP and other feature another feature .
Professor D: Mm - hmm .
PhD B: And we s we can first the feature are without delta and delta - delta , and we can see that in the situation , uh , the MSG - three , the same help nothing .
Professor D: Mm - hmm .
PhD B: And then I do the same but with the delta and delta - delta PLP delta and delta - delta . And they all p but they all put off the MLP is it without delta and delta - delta . And we have a l little bit less result than the the the baseline PLP with delta and delta - delta .
Professor D: Mm - hmm .
PhD B: Maybe if when we have the new the new neural network trained with PLP delta and delta - delta , maybe the final result must be better . I don't know .
PhD A: Actually , just to be some more PhD B: Uh PhD A: Do This number , this eighty - seven point one number , has to be compared with the
Professor D: Yes , yeah , I mean it can't be compared with the other
PhD A: Which number ?
Professor D: cuz this is , uh with multi - English , uh , training .
PhD B: Mm - hmm .
Professor D: So you have to compare it with the one over that you 've got in a box , which is that , uh the eighty - four point six .
PhD B: Mm - hmm .
Professor D: Right ?
PhD A: Uh .
Professor D: So PhD A: Yeah , but I mean in this case for the eighty - seven point one we used MLP outputs for the PLP net
Professor D: Yeah .
PhD A: and straight features with delta - delta . And straight features with delta - delta gives you what 's on the first sheet .
PhD B: Mm - hmm .
Professor D: Yeah . Not t not
PhD A: It 's eight eighty - eight point six .
Professor D: tr No . No . No .
PhD B: Yes .
Professor D: Not trained with multi - English .
PhD A: Uh , yeah , but th this is the second configuration .
PhD B: No , but they they feature @ @ without PhD A: So we use feature out uh , net outputs together with features . So yeah , this is not perhaps not clear here but in this table , the first column is for MLP and the second for the features .
Professor D: Eh . {comment} Oh , I see . Ah . So you 're saying w so asking the question , " What what has adding the MLP done to improve over the ,
PhD A: So , just Yeah so , actually it it it decreased the the accuracy .
Professor D: uh PhD B: Yeah .
Professor D: Yes .
PhD A: Because we have eighty - eight point six .
Professor D: Uh - huh .
PhD A: And even the MLP alone What gives the MLP alone ? Multi - English PLP . Oh no , it gives eighty - three point six . So we have our eighty - three point six and now eighty - eighty point six ,
PhD B: But PhD A: that gives eighty - seven point one .
Professor D: Mm - hmm . Eighty - s I thought it was eighty Oh , OK , eighty - three point six and eighty eighty - eight point six .
PhD A: Eighty - three point six . Eighty Professor D: OK .
PhD A: Is th is that right ? Yeah ?
PhD B: Yeah . But I don't know but maybe if we have the neural network trained with the PLP delta and delta - delta , maybe tha this can help .
PhD A: Perhaps , yeah .
Professor D: Well , that 's that 's one thing , but see the other thing is that , um , I mean it 's good to take the difficult case , but let 's let 's consider what that means . What what we 're saying is that one o one of the things that I mean my interpretation of your your s original suggestion is something like this , as motivation . When we train on data that is in one sense or another , similar to the testing data , then we get a win by having discriminant training .
PhD A: Mm - hmm .
Professor D: When we train on something that 's quite different , we have a potential to have some problems .
PhD A: Mm - hmm .
Professor D: And , um , if we get something that helps us when it 's somewhat similar , and doesn't hurt us too much when it when it 's quite different , that 's maybe not so bad .
PhD A: Yeah . Mmm .
Professor D: So the question is , if you took the same combination , and you tried it out on , uh on say digits ,
PhD A: On TI - digits ? OK .
Professor D: you know , d Was that experiment done ?
PhD A: No , not yet .
Professor D: Yeah , OK . Uh , then does that , eh you know maybe with similar noise conditions and so forth , {comment} does it does it then look much better ?
PhD A: Mm - hmm .
Professor D: And so what is the range over these different kinds of uh of tests ? So , an anyway . OK , go ahead .
PhD A: Yeah .
PhD B: And , with this type of configuration which I do on experiment using the new neural net with name broad klatt s twenty - seven , uh , d I have found more or less the same result .
Professor D: Mm - hmm .
PhD A: So , it 's slightly better ,
PhD B: Little bit better ?
PhD A: yeah .
Professor D: Slightly better .
PhD A: Yeah .
PhD B: Slightly bet better . Yes , is better .
Professor D: And and you know again maybe if you use the , uh , delta there , uh , you would bring it up to where it was , uh you know at least about the same for a difficult case .
PhD B: Yeah , maybe . Maybe . Maybe .
PhD A: Yeah .
PhD B: Oh , yeah .
PhD A: Yeah . Well , so perhaps let 's let 's jump at the last experiment .
PhD B: Oh , yeah .
Professor D: So .
PhD A: It 's either less information from the neural network if we use only the silence output .
PhD B: i
Professor D: Mm - hmm .
PhD A: It 's again better . So it 's eighty - nine point point one .
PhD B: Yeah ,
Professor D: Mm - hmm .
PhD B: and we have only forty forty feature
PhD A: So .
PhD B: because in this situation we have one hundred and three feature .
Professor D: Yeah .
PhD B: Yeah . And then w with the first configuration , I f I am found that work , uh , doesn't work Professor D: Yeah .
PhD B: uh , well , work , but is better , the second configuration . Because I for the del Engli - PLP delta and delta - delta , here I have eighty - five point three accuracy , and with the second configuration I have eighty - seven point one .

Summary: Last run of experiments showed that adding more languages was still severely harming model performance. They noted that normalization of any kind has a good effect on the model, whereas MSG does nothing.