phd e: What am I ? professor f: Little low ? phd e: Channel four ? professor f: Channel five . phd g: The gai the gain 's up at it what it usually is , professor f: Is it ? phd g: but if you think it 's Yeah . phd d: I think phd g: She can just walk in , I guess , or phd d: Yeah . I was gonna ask Adam to , say if he thought anymore about the demo stuff because it occurred to me that this is late May and the DARPA meeting is in mid July .  , but I don't remember w what we I know that we were gonna do something with the transcriber interface is one thing , but I thought there was a second thing . Anybody remember ? phd g: Well , we were gonna do a mock - up , like , question answering or something , I thought , that was totally separate from the interface . Do you remember ? Remember , like , asking questions and retrieving , but in a pre - stored fashion . phd g: That was the thing we talked about , I think , before the transcriber professor f: Yeah . So anyway , you have to sort out that out and get somebody going on it cuz we 're got a got a month left basically . phd g: I was writing the digits and then I realized I could xerox them , professor f: Oh , oh . professor f: So , the choice is , which which do we want more , the the the comparison , of everybody saying them at the same time or the comparison of people saying the same digits at different times that ? phd g: It 's just cuz I didn't have any more digit sheets .  , we could use them for normalizing or something , but it of course goes more quickly doing them in unison . phd e: But anyway , they won't be identical as somebody is saying zero in some sometimes , you know , saying O , and so , it 's not i not identical .  ju what what might those be ? phd d: IBM stuff and , just getting  , meeting information organized . phd c: Are you implying that it 's currently disorganized ? phd d: In my mind . professor f: Is there stuff that 's happened about , the SRI recognizer et cetera , tho those things that were happening before with ? phd c: Well . professor f: Y y you guys were doing a bunch of experiments with different front - ends and then with Is is that still sort of where it was , the other day ? phd c: We 're improving . phd d: Now the the You saw the note that the PLP now is getting basically the same as the MFCC .  , it seems It looks l I haven't The It 's The experiment is still not complete , but , it looks like the vocal tract length normalization is working beautifully , actually , w using the warp factors that we computed for the SRI system and just applying them to the ICSI front - end . phd c: Just had to take the reciprocal of the number because they have different meanings in the two systems .  phd c: But one issue actually that just came up in discussion with Liz and and Don was , as far as meeting recognition is concerned , we would really like to , move , to , doing the recognition on automatic segmentations . phd c: Because in all our previous experiments , we had the  , you know , we were essentially cheating by having the , you know , the h the hand - segmentations as the basis of the recognition . phd c: And so now with Thilo 's segmenter working so well , I think we should consider doing a phd e: Mmm . phd g: And even The good thing is that since you , have high recall , even if you have low precision cuz you 're over - generating , that 's good because we could train noise models in the recognizer for these kinds of , transients and things that come from the microphones , phd c: Right . phd g: but I know that if we run recognition unconstrained on a whole waveform , we do very poorly because we 're we 're getting insertions in places what that you may well be cutting out . phd c: We should we should consider doing some extra things , like , you know , retraining or adapting the the models for background noise to the to this environment , for instance . phd g: And , yeah , using Thilo 's , you know , posteriors or some kind of or phd c: So . So , talked with Brian and gave him the alternatives to the single beep at the end of each utterance that we had generated before . And so he talked it over with the transcriber and the transcriber thought that the easiest thing for them would be if there was a beep and then the nu a number , a digit , and then a beep , at the beginning of each one professor f: Yeah . And , so Adam wrote a little script to generate those style , beeps phd c: Where 'd you get the digits from ? phd d: and so we 're I came up here and just recorded the numbers one through ten . phd c: And do you splice them into the waveform ? Or ? phd d: Yeah . He then he d I recorded Actually , I recorded one through ten three times at three different speeds and then he picked . phd d: He liked the fastest one , so he just cut those out and spliced them in between , two beeps . phd d: Does it ? phd e: It will be funny when you 're really reading digits , and then there are the chunks with with your digits in ? phd d: Yeah . postdoc a: Now actually , phd d: That 'll throw them , postdoc a: we 're Are we handling ? phd d: huh ? professor f: maybe we should have you record A , B , C for those or something . And she said it wasn't gonna the transcriber said it wouldn't be a problem cuz they can actually make a template , that has beep , number , beep . phd d: So , we We 're gonna send them one more sample meeting , and Thilo has run his segmentation . And when we get that back we 'll see if that sort of fixes the problem we had with , too many beeps in the last transcription . Do w do what Do you have any idea of the turn - around on on those steps you just said ? phd g: Great . professor f: e e u u The reason I 'm asking is because , Jane and I have just been talking , and she 's just been doing . professor f: And so we don't sort of really know exactly what they 'll be doing , how long they 'll be doing it , and so forth , because right now she has no choice but to operate in the mode that we already have working . professor f: And , so it 'd be It 'd be good to sort of get that resolved , soon as we could , phd d: Yeah . I Yeah , I I hope @ @ we can get a better estimate from this one that we send them .   in particular I would I would really hope that when we do this DARPA meeting in July that we sort of have we 're we 're into production mode , somehow phd d:   professor f: You know , that we we actually have a stream going and we know how how well it does and how and how it operates . Maybe before we do the meeting info organize thing , maybe you could say relevant stuff about where we are in transcriptions . So , we  , the transcribers have continued to work past what I 'm calling " set one " , which was the s the set that I 've been ,  OK , talking about up to this point , but , they 've gotten five meetings done in that set . I 'm thinking of hiring another one , which will because we 've had a lot of attrition . And that will bring our total to professor f: They die off after they do this for a while .  , you know , one of them really w wasn't planning phd c: Oh , that was an unfor unforeseen side effect of postdoc a: Eh , one of them , had never planned to work past January .  , it 's th all these various things , cuz we , you know , we presented it as possibly a month project back in January and and and and  , so it makes sense .  , through attrition we we 've we 're down to to two , but they 're really solid . We had a good core phd g: Well , they won't hear this since they 're going .  , I d it 's just a matter of we w we 're we 've got , professor f: No backs . postdoc a: two of the ones who who , ha had been putting in a lot of hours up to this point and they 're continuing to put in a a lot of hours , which is wonderful , and excellent work . And so , then , in addition , I hired two more today and I 'm planning to h hire a third one with this within this coming week , but but the plan is just as , Morgan was saying we discussed this , and the plan right now is to keep the staff on the on the leaner side , you know , rather than hiring , like , eight to ten right now , professor f:   postdoc a: because if the IBM thing comes through really quickly , then , we wouldn't wanna have to , you know , lay people off and stuff . And this way it 'll  , I got really a lot of response for for my notice and I think I could hire additional people if I wish to . An - and the other thing is , in the unlikely event and since we 're so far from this , it 's a little hard to plan this way in the unlikely event that we actually find that we have , transcribers on staff who are twiddling their thumbs because , you know , there 's , you know , all the stuff that that was sitting there has been transcribed and they 're and they 're faster the the pipeline is faster than  , than the generation , eh , i in in the day e event that that day actually dawns , I I bet we could find some other stuff for them to do . professor f: So I I think that , eh , eh , a as we were talking , if we if we hire twelve , then we could , you know , run into a problem later . But but ,  for all sorts of reasons but if we hire f you know , f we have five on staff five or six on staff at any given time , then it 's a small enough number so we can be flexible either way . phd g: It 'd be great , too , if , we can we might need some help again getting the tighter boundaries or some hand to experiment with ,  you know , to have a ground truth for this segmentation work , which I guess you have some already that was really helpful , and we could probably use more . That was a thing I I planned working on , is , to use the the transcriptions which are done by now , and to to use them as ,  phd g: Oh . To to create some speech - nonspeech labels out of them , and Yeah , but that that 's a thing w was w what I 'm just looking into . postdoc a: The the the pre - segmentations are so much are s so extremely helpful . Now there was , I g guess So , a couple weeks ago I needed some new ones and it happened to be during the time that he was on vacation f for just very few days you were away . postdoc a: so I so I started them on the non - pre - segmented and then switched them over to yours and , they ,  you know , they always appreciate that when they have that available . postdoc a: And and and she was And so , I asked her  , They 're very perceptive . I haven't done it yet , but I wanna do that and she 's out of town , for a couple of weeks , but I wanna do that when she returns .  , cuz she was saying , you know , in a in a span of very short period we asked It seems like the ones that need to be adjusted are these these these things , and she was saying the short utterances , the ,  phd g:  . But but actually i it 's so correct for so much of the time , that it 's an enormous time saver phd e: Yeah . phd g: Is there actually a record of where they change ?  , you can compare , do a diff on the just so that we knew postdoc a: You could do it . It 's it 's complicated in that  , hhh , i hhh , i phd e: Yeah . Actually , when when they create new yeah , new segments or something , it will be , not that easy but  . phd g: just so that if we run it we know whether we 're which ones were cheating phd e: Yeah . phd g: and postdoc a: There is a there is one problem with that and that is when they start part way through then what I do is I merge what they 've done with the pre - segmented version . Wha - what you 'd really like is that they started with pre - segmented and were pre - segmented all the way through . postdoc a: And ,  @ @ I ,  the it wasn't possible for about four of the recent ones . As long as we have a record , I guess , of the original automatic one , we can always find out how well we would do fr from the recognition side by using those boundaries . phd g: Also if you need someone to record this meeting , I 'm happy to for the transcribers I could do it , or Chuck or Adam . So , Jane and Adam and I had a meeting where we talked about the reorganization of the directory structure for all of the meeting professor f: Did you record it ? phd d: No . And then , Jane also s prepared a  , started getting all of the the meetings organized , so she prepared a a spreadsheet , which I spent the last couple of days adding to . So I went through all of the data that we have collected so far , and have been putting it into , a spreadsheet with start time , the date , the old meeting name , the new meeting name , the number of speakers , the duration of the meeting , comments , you know , what its transcription status is , all that kind of stuff . And so , the idea is that we can take this and then export it as HTML and put it on the Meeting Recorder web page so we can keep people updated about what 's going on . phd d: I 've gotta get some more information from Jane cuz I have some some gaps here that I need to get her to fill in , but so far , as of Monday , the fourteenth , we 've had a total number of meeting sixty - two hours of meetings that we have collected .  , and I 'm gonna have on here the total amount that 's been transcribed so far , but I 've got a bunch of  , that 's what I have to talk to Jane about , figuring out exactly which ones have have been completed and so forth . But , this 'll be a nice thing that we can put up on the the web site and people can be informed of the status of various different ones . And it 'll also list , like under the status , if it 's at IBM or if it 's at ICSI , or if it 's completed or which ones we 're excluding and and there 's a place for comments , so we can , say why we 're excluding things and so forth . professor f: Now would the ones that , are already transcribed we h we have enough there that c you know , we 've already done some studies and so forth and  , shouldn't we go through and do the business - es u of of having the , participants approve it , for approve the transcriptions for distribution and so forth ? postdoc a: interesting idea . In principle , I I would say yes , although I still am doing some the final - pass editing , trying to convert it over to the master file as the being the channelized version and it 's Yeah , it seems like I get into that a certain way and then something else intervenes and I have to stop . Cleaning up the things like the , places where the transcriber was uncertain , and and doing spot - checking here and there . So , I guess it would make sense to wait until th that 's done , but but professor f: Well , le let me put in another sort of a milestone kind of as as I did with the ,  the the pipeline . professor f: we are gonna have this DARPA meeting in the middle of July , postdoc a: Yes . professor f: and I think it w it 'd be given that we 've been we 've given a couple public talks about it already , spaced by months and months , I think it 'd be pretty bad if we continued to say none of this is available . So we can s we we wanna be able to say " here is a subset that is available right now " postdoc a:   phd c: And they don't have to approve , you know , th an edited version , they can just give their approval to whatever version postdoc a: Well , maybe professor f: Well , in principle , yes . But , i if if if somebody actually did get into some legal issue with it then we phd c: Bu Yeah . Presumably if if s errors are found , they will be fixed , but they won't change the the content of the meetings . phd g: Well , i if Jane is clarifying question question , then , you know , how can they agree to it before they know her final version ? postdoc a: The other thing , too , is there can be subtleties where a person uses this word instead of that word , which @ @ could 've been transcribed in the other way . phd g: Thing postdoc a: And no and they wouldn't have been slanderous if it had been this other word . You know ? professor f: I it you know , there there is a point at which I agree it becomes ridiculous because , you know , you could do this final thing and then a year from now somebody could say , you know , that should be a period and not a question mark . Right ? And you don't you there 's no way that we 're gonna go back and ask everybody " do you approve this , you know this document now ? " So So I think what it is is that the the the the thing that they sign I I haven't looked at it in a while , but it has to be open enough that it sort of says " OK , from now on you know , now that I 've read this , you can use do anything you want with these data . professor f: And ,  But , i I think we wanna So , assuming that it 's in that kind of wording , which I don't remember , I think i we just wanna have enough confidence ourselves that it 's so close to the final form it 's gonna be in , a year from now that they 're postdoc a:   It 's just , a question of , if if the person is using the transcript as the way of them judging what they said and whether it was slanderous , then it seems like it 's it 's i it needs to be more correct than if we could count on them re - listening to the meeting . postdoc a: Because it becomes , eh , in a way a a f  , a legal document i if they 've agreed to that . I forget how we ended up on this , but I remember my taking the position of not making it so so easy for everybody to observe everything and Adam was taking the position of of having it be really straightforward for people to check every aspect of it including the audio . And I don't remember who won , Adam or me , but postdoc a: Well , if it 's only the transcript , though  , th this this is my point , that that professor f: the  , that that 's why I 'm bringing this up again , because I can't remember how we ended up . postdoc a: then it becomes professor f: That it was the transcrip He wanted to do a web interface that would make it postdoc a: Well , if it 's just the audio Well . phd g: with the web interface it 's interesting , because you could allow the person who signs to be informed when their transcript changes , or something like that . Like , I don't wanna know , but some people might be really interested and then y In other words , they would be informed if there was some significant change other than typos and things like that . professor f: You decided you were whispering Satanic incantations under your breath when you were phd g: Well , I don't know what happened to the small heads thing , but I j  , I 'm just saying that , like , you know , you can sort of say that any things that are deemed professor f: They disappeared from view .  , I agree that at some point people probably won't care about typos but they would care about significant meaning changes and then they could be asked for their consent , I guess , if if those change . Cuz assumi assuming we we don't really distribute things that have any significant changes from what they sign anyway . phd c: Tha That 's How about having them approve the audio and not the transcripts ? phd g: Oh , my God . grad b: That 's phd c: We just have to give them a chance to listen to it , and if they don't , that 's their problem . phd g: You you d That 's like postdoc a: Unfortunately , in in the sign thing that they signed , it says " transcripts " . postdoc a: " You 'll be you 'll be provided the transcripts when they 're available . phd g: that 's a lot to ask for people that have been in a lot of meetings . professor f: W anyway , haven't we we 've gone down this path a number of times . I know this can lead to extended conversations and and not really get anywhere , so let let me just suggest that  , off - line that , the people involved figure it out and take care of it before it 's July . So so that in July we can tell people " yes , we have this and you can use it " . phd g: it 's I guess one thing we 're learning is that the amount We have eight meetings there because we couldn't use the non - native all non - native meetings and it 's , well , probably below threshold on enough data for us for the things we 're looking at because the prosodic features are very noisy and so you you need a lot of data in order to model them .  , so we 're starting to see some patterns and we 're hoping that maybe with , I don't know , double or triple the data with twenty meetings or so , that we would start to get better results . But we did find that some of the features that , I gue Jane would know about , that are expressing sort of the distance of , boundaries from peaks in the utterance and some local , range pitch range effects , like how close people are to their floor , are showing up in these classifiers , which are also being given some word features that are cheating , cuz they 're true words . Right ? phd g: spurts is not cheating except that of course you know the real words , grad b: Right . phd c: No phd g: Not exactly , but i grad b: But ra somewhat ? professor f: On the average . Well , we don't know and actually that 's one of the things we 're interested in doing , is a sort of professor f:  - huh . phd c: Have you tried using just time , as opposed to number of words ? phd g: So . grad b: I think ti  Just p time position , like when the word starts ? phd c: Yeah . grad b: I don't know if that was in the phd c: Well , no ,  t time time position relative to the beginning of the spurt . phd g: we didn't try it , but it 's s grad b: Like , there 's a lot of different features you could just pull out . professor f: How about time position normalized by speak phd g: And it depends on speaking rate professor f: Yeah . phd g: But we one of the interesting things was I guess you reported on some te punctuation type grad b: Yeah . phd g: finding sentence boundaries , finding disfluency boundaries , and then I had done some work on finding from the foreground speech whether or not someone was likely to interrupt , so where you know , if I 'm talking now and someone and and Andreas is about to interrupt me , is he gonna choose a certain place in my speech , either prosodically or word - based . And there the prosodic features actually showed up and a neat thing even though the word features were available . And a neat thing there too is I tried some putting the speaker So , I gave everybody a short version of their name . So that means that overall , it wasn't just modeling Morgan , or it wasn't just modeling a single person , professor f:   phd g: but was sort of trying to , get a general idea the model the tree classifier was trying to find general locations that were applicable to different speakers , even though there are huge speaker effects . The but the main limitation now is I because we 're only looking at things that happen every ten words or every twenty words , we need more more data and more data per speaker . It 'd also be interesting to look at the EDU meetings because we did include meeting type as a feature , so whether you were in a r Meeting Recorder meeting or a Robustness meeting did matter to interrupts because there are just fewer interrupts in the Robustness meetings . phd g: And so the classifier learns more about Morgan than it does about sort of the average person , professor f:   phd g: So it 's And I think Don ,  Well , we have a long list of things he 's starting to look at now over the summer , where we can And he 'll be able to report on more things in the future . But it was great that we could at least go from the you know , Jane 's transcripts and the , recognizer output and get it to this point . And I think it 's something Mari can probably use in her preliminary report like , " yeah , we 're at the point where we 're training these classifiers and we 're just reporting very preliminary but suggestive results that some features , both word and pro prosodic , work . " The other thing that was interesting to me is that the pitch features are better than in Switchboard . And I think that really is from the close - talking mikes , cuz the pitch processing that was done has much cleaner behavior than than the Switchboard telephone bandwidth . Well , first of all , the pitch tracks are m have less , halvings and doublings than than Switchboard and there 's a lot less dropout , so if you ask how many regions where you would normally expect some vowels to be occurring are completely devoid of pitch information , professor f:    phd g: in other words the pitch tracker just didn't get a high enough probability of voicing for words for for , you know , five word professor f:  . So the missing We had a big missing data problem in Switchboard and , so the features weren't as reliable cuz they were often just not available . phd d: Could it have to do with the the lower frequency cut - off on the Switchboard ? phd g: So that 's actually good .  , the tele we had telephone bandwidth for Switchboard and we had the an annoying sort of telephone handset movement problem that I think may also affect it . phd g: Anyway , Don 's been doing a great job and we hope to continue with , Andreas 's help and also some of Thilo 's help on this , professor f: Great . phd e: Y phd g: to to try to get a non - cheating version of how all this would work . professor f: Has has ,  ? We just I think , just talked about this the other day , but h has has anybody had a chance to try changing , insertion penalty sort of things with the with the , using the tandem system input for the ? phd c: Oh , yeah . phd c: There were a little the relative number of I think there were a higher number of deletions , actually . phd c: So , you ,  So , actually it it preferred to have a positive er , negative insertion penalty , phd g: Deletions ? phd c: which means that ,  professor f:  - huh . phd c: But , you know , it didn't change th the by adjusting that the ,  professor f: OK . But , you know , given that that word error rate is so high , that 's not a professor f: OK . phd c: But , we s just ,  you know , Chuck and I talked and the @ @ next thing to do is probably to tune the  , the size of the Gaussian system , @ @ to to this to this feature vector , which we haven't done at all . phd c: And , for instance , Dan @ @ Dan just sent me a message saying that CMU used , something like ten Gaussians per cluster You know , each each mixture has ten Gaussians phd d:   We 're using sixty - four , phd c: and and we 're using sixty - four , phd d: right ? phd c: so that 's obviously a big difference phd d: Yeah . phd c: and it might be way off and give very poorly trained , you know , Gaussians that way , professor f:  . So so , we have The turn - around time on the training when we train only the a male system with , you know , our small training set , is less than twenty - four hours , so we can run lots of  , basically just brute force , try a whole bunch of different  , settings . professor f: huh ? phd c: But the PLP features work  , you know , continue to improve the , professor f: OK . phd c:  As I said before , the  using Dan 's , vocal tract normalization option works very well . So , @ @ I ran one experiment where we 're just did the vocal tract le normalization only in the test data , professor f:   phd c: so I didn't bother to retrain the models at all , and it improved by one percent , which is about what we get with  , with , you know , just @ @ actually doing both training and test normalization , with , the , with the standard system . So , in a few hours we 'll have the numbers for the for retraining everything with vocal tract length normalization and So , that might even improve it further . phd c: So , it looks like the P L - fea P features do very well now with after having figured out all these little tricks to to get it to work . So you mean you improve one percent over a system that doesn't have any V T L in it already ? phd c: Exactly . And and what that suggests also is of course that the current Switchboard MLP isn't trained on very good features . phd c: because it was trained on whatever , you know , was used , last time you did Hub - five stuff , which didn't have any of the professor f: Right . professor f: Right ?  , y the phd c: Well , but if you add them all up you have , almost five percent difference now . phd c: actually , and it 's , What 's actually qu interesting is that with  , well , you m prob maybe another half percent if you do the VTL in training , and then interestingly , if you optimize you get more of a win out of rescoring the , the N best lists , and optimizing the weights ,  than phd d: Than you do with the standard ? phd c: Yeah . But the part that 's actually adjustment of the front - end per se as opposed to doing putting VTLN in or something is it was a couple percent . professor f: Right ? It was it was there was there was one thing that was one and a half percent and one that was point eight . One of them was , the change to ,  because it did it all at once , to  , from bark scale to mel scale , phd c:   professor f: which I really feel like saying in quotes , because @ @ they 're essentially the same scale but the but but but any i individual particular implementation of those things puts things in a particular place . professor f: So that 's why I wanted to look I still haven't looked at it yet . professor f: and it it 's probably something like there 's one fewer or one more filter in the sub one kilohertz band phd c:   professor f: it could be there 's something more fundamental but it you know , I I don't know it yet . And the other and the other that was like one and a half or something , and then there was point eight percent , which was what was the other thing ? phd d: Well , that was combined with the triangular . professor f: We d weren't able to separate them out cuz it was just done in one thing . So that was that was ,  that one I can claim credit for , i in terms of screwing it up in the first place . So that someone e until someone else fixed it , which is that , I never put when I u We had some problems before with offsets . professor f: So we we had ,  ea everybody else who was doing Wall Street Journal knew that there were big DC offsets in th in these data in those data and and and nobody happened to mention it to us , phd c:  . professor f: and we were getting these , like , really terrible results , like two , three times the error everybody else was getting . And then in casual conversation someone ment mentioned "  , well , I guess , you know , of course you 're taking care of the offsets . professor f: And at that point , you know , we were pretty new to the data and we 'd never really , like , looked at it on a screen and then when we just put it on the screen and wroop ! phd c:   So , in PLP phd g: There was a like a hum or some or when they recorded it ? professor f: No . It 's just , it it 's it 's not at all uncommon for for recorded electronics to have different , DC offsets . It 's you know , you could have ten , twenty , maybe thirty millivolts , whatever , and it 's consistently in there . The thing is , most people 's front - ends have pre - emphasis with it , with zero at zero frequency , so that it 's irrelevant . We had we had the equivalent of pre - emphasis in a a , Fletcher - Munson style weighting that occurs in the middle of P L but it doesn't actually have a zero at zero frequency , phd g:  . So at that point I reali " oh sh we better have a have a high - pass filter " just , you know just take care of the problem . So I put in a high - pass filter at , I think ninety ninety hertz or so  , for a sixteen kilohertz sampling rate . And so well , so , you know , the code doesn't know anything about that and so this is all at eight kilohertz and so it was at forty - five hertz instead of at instead of at ninety . professor f: So , I don't know if Dan fixed it or or , what he phd c: Well , he made it a parameter . Yeah , I guess if he did it right , he did fix it and then and then it 's taking care of sampling rate , which is great . phd d: What what is the parameter ? professor f: He had a phd d: Is it , just the f lower cut - off that you want ? phd c: It 's called , H - HPF . phd c: u And but HPF , you know , when you put a number after it , uses that as the hertz value of the cut - off . professor f: frankly , we never did that with the RASTA filter either , phd c:   professor f: so the RASTA filter is actually doing a different thing in the modulation spectral domain depending on what sampling rate you 're doing , which is another old old bug of mine . So that that was the problem there was th we we we had always intended to cut off below a hundred hertz phd c:   Well , but ,  Well , again , after completing the current experiments , we 'll we can add up all the  differences professor f: Oh , yeah . phd c: and and an professor f: But but , I guess my my point was that that , the hybrid system thing that we did was , primitive in many ways . professor f: And I think I agree with you that if we fixed lots of different things and they would all add up , we would probably have a a a competitive system . professor f: unless you call well , if you call VTL the front - en front - end , that 's , a little more . phd d: One experiment we should we 'll probably need to do though when  , at some point , is , since we 're using that same the net that was trained on PLP without all these things in it , for the tandem system , we may wanna go back and retrain , professor f: Right ? But . phd c: eh At this point I 'm as  , you know e I 'm wondering is it Can we expect , a tandem system to do better than a properly trained you know , a Gaussian system trained directly on the features with , you know , the right ch choice of parameters ? professor f: Well , that 's what we 're seeing in other areas . Right ? So , it 's so ,  phd d: So , we But but we may not . professor f: the reason to think it should is because you 're putting in the same information and you 're transforming it to be more discriminative . Now the thing is , in some databases I wouldn't expect it to necessarily give you much and and part of what I view as the real power of it is that it gives you a transformational capability for taking all sorts of different wild things that we do , not just th the standard front - end , but other things , like with multiple streams and so forth , phd c:   professor f: If it 's , you know , like way way worse then , you know phd c: Right . phd d: So , Morgan , an another thing that Andreas and I were talking about was , so @ @ in the first experiment that he did we just took the whole fifty - six , outputs and that 's , basically compared to a thirty - nine input feature vector from either MFCC or PLP . phd d: But one thing we could do is professor f: Let let me let me just ask you something . When you say take the fifty - six outputs , these are the pre final nonlinearity outputs phd d: Yeah . And so so then you u Do you use all fifty - six of the KLT phd d: That 's what we did . professor f: or ? phd d: Right ? So one thing we were wondering is , if we did principal components and , say , took out just thirteen , and then did deltas and double - deltas on that professor f: OK . professor f:  these were all different databases and different you know , in HTK and all that , phd d: Yeah . But my recollection of it was that it didn't make it better but it didn't make it worse . professor f: But , again , given all these differences , maybe it 's more important in your case that you not take a lot of these low - variance , components . phd d: Cuz in a sense , the net 's already got quite a bit of context in those features , professor f: Yeah . phd d: so if we did deltas and double - deltas on top of those , we 're getting sort of even more . phd c: But there the main point is that , you know , it took us a while but we have the procedure for coupling the two systems debugged now and  , there 's still conceivably some bug somewhere in the way we 're feeding the tandem features  , either generating them or feeding them to this to the SRI system , phd d:   phd c: but it 's professor f: There might be , cuz that 's a pretty big difference . phd c: I 'm actually f quite sure that the feeding the features into the system and training it up , professor f: What if ? phd c: that that I think that 's this that 's essentially the same as we use with the ce with the P L P fe features . phd c: There we could the another degree of freedom is how do you generate the K L T transform ? phd d:   professor f: These nets are trained with particular normalization and when that gets screwed up it it can really hurt it . phd d: I 'm doing what Eric E Eric coached me through then that part of it , so I 'm pretty confident in that . phd d: the only slight difference is that I use normalization values that , Andreas calculated from the original PLP , phd c: Right . So , I u I do Oh , we actually don't do that normalization for the PLP , do we ? For the st just the straight PLP features ? phd c: No . Well , you might e e phd c: So , there 's there is there is room for bugs that we might not have discovered , phd d: So that 's that 's another Yeah . I I would actually double check with Stephane at this point , phd c: but professor f: cuz he 's probably the one here  , he and Dan are the ones who are at this point most experienced with the tandem phd d:   professor f: thing and there may there may be some little bit here and there that is not not being handled right .  , you can't just , like , print the the values out in ASCII and , you know , look at them , see if they 're professor f: Not unless you had a lot of time phd g: Well professor f: and phd g: eh , and also they 're not  , as I understand it , you you don't have a way to optimize the features for the final word error . phd g: these are just discriminative , but they 're not , optimized for the final phd c: They 're optimized for phone discrimination , not for phd g: Right . So it there 's always this question of whether you might do better with those features if there was a way to train it for the word error metric that you 're actually that you 're actually professor f: That 's right . It 's indirect , so you don't know professor f: So wha w what an and you may not be in this case , come to think of it , because , you 're just taking something that 's trained up elsewhere . So in fact you the the net is trained on , a , Viterbi alignment of the training data that comes from your full system . And so that 's where the feedback comes all around , so that it is actually discriminant . You can prove that it 's it 's a ,  If you believe in the Viterbi assumption that , getting the best path , is almost equivalent to getting the best , total probability , then you actually do improve that by ,  by training up on local local ,  local frames . But , we aren't actually doing that here , because we did we did that for a hybrid system , and now we 're plugging it into another system and so it isn't i i i it wouldn't quite apply here . phd c: Do y phd d: So another huge experiment we could do would be to take the tandem features , do SRI forced alignments using those features , and then re - do the net with those . Another thing is since you 're not using the net for recognition per se but just for this transformation , it 's probably bigger than it needs to be . So , you 're using a l a long a larger phone set than what phd d: Mmm . professor f: The other thing , just to mention that Stephane this was an innovation of Stephane 's , which was a pretty neat one , and might particularly apply here , given all these things we 're mentioning . Even the local ones , given , you know , these potential outer loops which , you know , you can convince yourself turn into the global ones .  , when something about the test set is different enough from the training set that that , the discrimination that you 're learning is is is not a good one . professor f: So , his idea was to take as the input feature vector to the , Gaussian mixture system , a concatenation of the neural net outputs and the regular features . No , but we we when when we when I first started corresponding with Dan about how to go about this , I think that was one of the things that we definitely went there .  , I 'm sure that Stephane wasn't the first to think of it , phd c: Yeah . So that 's that that 's our current best best system in the , in the Aurora thing . phd c: And do you do a KLT transform on the con on the combined feature vector ? professor f: Yeah . phd c: Do you d you do a KLT transform on the combined feature vector ? professor f: Yeah . professor f: Well , actually , I ,  you should check with him , because he tried several different combinations . phd c: Because you end up with this huge feature vector , so that might be a problem , a unless you do some form of dimensionality reduction . So he did one where he put o put e the whole thing into one KLT , and another one , since the the PLP things are already orthogonalized , he left them alone and and just did a KLT on the on the on the net outputs phd c:   phd d: Did he did he try to ? So he always ended up with a feature vector that was twice as long as either one of the ? professor f: No . phd g: We need to close up cuz I need to save the data and , get a call . phd g: or , do them together ? professor f: I I g I think , given that we 're in a hurry for snacks , maybe we should do them together . professor f: Well , it 's it 's it 's not You know , it 's not gonna work out phd g: Adam 's not here , so he 's not here to tell me no . professor f: but we could we could just , see if we find a rhythm , you know , what phd g: Sure . professor f: O 's or zeroes , we wanna agree on that ? phd g: Maybe just whatever people would naturally do ? I don't know . professor f: Why don't we do zer i Anyone have a problem with saying zero ? Is zero OK ? phd g: No