professor b: OK So  today we 're looking at a number of  things we 're trying and  fortunately for listeners to this  we lost some of it 's visual but  got tables in front of us .  what is what does combo mean ? phd c: So combo is  a system where we have these features that go through a network and then this same string of features but low - pass filtered with the low - pass filter used in the MSG features . And so these low - pass filtered goes through M eh another MLP and then the linear output of these two MLP 's are combined just by adding the values and then there is this KLT . There is  there is the features  there 's the OGI features and then  those features  go through a contextual  l l let 's take this bottom arr one pointed to by the bottom arrow . phd c:  professor b: So phd c: So we have these features from OGI that goes through the three paths . phd c: The second path is  MLP also using nine frames several frames of features professor b: Yeah . phd c: MLP professor b: Aha ! aha ! phd c: Adding the outputs just like in the second propose the the proposal from for the first evaluation . So this is yeah professor b: And so  and then the the the one at the top and I presume these things that  are in yellow are in yellow because overall they 're the best ? phd c: Yeah that 's the reason , yeah . professor b: Oh let 's focus on them then so what 's the block diagram for the one above it ? phd c: For the f the f first yellow line you mean ? professor b: Yeah . phd c: Yeah so it 's  basically s the same except that we don't have this  low - pass filtering so we have only two streams . phd c:  professor b: Do you e  they mentioned made some  when I was on the phone with Sunil they they mentioned some weighting scheme that was used to evaluate all of these numbers .  actually the way things seems to  well it 's  forty percent for TI - digit , sixty for all the SpeechDat - Cars , well all these languages . professor b:  and we don't have the TI - digits part yet ? phd c: no . And so have you put all these numbers together into a single number representing that ? phd c: Yeah . professor b: OK so that should be pretty easy to do and that would be good phd c: No . professor b:  and how does this compare to the numbers oh so OGI two is just the top top row ? phd d: Yeah . phd c: So yeah to actually OGI two is the the baseline with the OGI features but this is not exactly the result that they have because they 've they 're still made some changes in the features professor b: OK .  I don't know by how much because they did not send us the new results professor b: OK . phd c:  professor b:  OK so the one one place where it looks like we 're messing things up a bit is in the highly mismatched Italian . phd c: But there are thirty - six and then sometimes we are we are we are around forty - two and professor b: Now up phd c: but professor b:  so one of the ideas that you had mentioned last time was having a a second  silence detection . professor b: So filt is what that is ? phd c: Filt , yeah phd d: Yeah . phd c:  yeah so it seems f for the the well match and mismatched condition it 's  it brings something .  but  actually apparently there are there 's no room left for any silence detector at the server side because of the delay . phd c:  t two days ago they were still working on this trying to reduce the delay of the silence detector so but yeah if we had time perhaps we could try to find  some kind of compromise between the delay that 's on the handset and on the server side . Perhaps try to reduce the delay on the handset and but well  For the moment they have this large delay on the the feature computation and so we don't professor b: OK . So Alright so for now at least that 's not there you have some results with low - pass filter cepstrum doesn't have a huge effect but it but it looks like it you know maybe could help in a couple places . professor b:  and   Yeah and  let 's see What else did we have in there ?  I guess it makes a l  at this point this is I I guess I should probably look at these others a little bit  And you you yellowed these out  but   Oh I see yeah that that one you can't use because of the delay .  let 's see that one Well even the just the the second row doesn't look that bad right ? That 's just  yeah ? phd c: Yep . professor b:  phd c: Actually the yeah the second line is  pretty much like the first line in yellow except that we don't have this KLT on the first on the left part of the diagram . Yeah so when we do this weighted measure we should compare the two cuz it might even come out better . professor b: So so there 's so I I would put that one also as a as a maybe .  and it yeah and it 's actually does does significantly better on the   highly mismatched Italian , so s and little worse on the mis on the  case , but  Well yeah it 's worse than a few things phd c:   professor b: so  let 's see how that c that c c see how that comes out on their their measure and are are we running this  for TI - digits or  phd c: Yeah . professor b: Now is TI di is is that part of the result that they get for the  development th the results that they 're supposed to get at the end of end of the month , the TI - digits are there also ? phd c: Yeah .  Oh I see the one I was looking down here at the the o the row below the lower yellowed one . professor b: What happens there is it 's around the same and so you could reduce the dimension as you were saying before a bit perhaps . professor b: It 's significantly worse it 's it 's  it 's it 's mostly worse . phd c: Exc - except for the HM phd d: For many a mismatch it 's worse . What are what are the sizes of any of these sets , I I 'm I 'm sure you told me before , but I 've forgotten . So you know how many words are in  one of these test sets ? phd c:  phd d: I don't remember . professor b: About ? phd c:  it 's it depends well the well matched is generally larger than the other sets and I think it 's around two thousand or three thousand words perhaps , at least . professor b: So the so the sets so the test sets are between five hundred and two thousand sentences , let 's say phd c: Mmm . professor b: and each sentence on the average has four or five digits or is it most of them longer or phd c: Yeah . phd d: Yeah for the Italian even seven digits y more or less phd c: It it d Seven digits . phd d: but sometime the sentence have only one digit and sometime  like  the number of  credit cards , something like that . See the  the reason I 'm asking is is is we have all these small differences and I don't know how seriously to take them , right ? phd c:  -  ? professor b: So  i if if you had  just you know to give an example , if you had   if you had a thousand words then  a a tenth of a percent would just be one word , phd c: Yeah . professor b:  so  yeah it be kind of I 'd kind of like to know what the sizes of these test sets were actually . phd d: The size that we have ? phd c: We could we could run run some kind of significance tests professor b: Yeah since these well also just to know the numbers , phd c: or phd d: Yeah . phd d: Yeah we have the result that the output of the HTK professor b: Yeah . professor b: Yeah so anyway if you could just mail out what those numbers are and then then that that be great . professor b:  what else is there here ?  see the second second from the bottom it says SIL , but this is some different kind of silence or thing or what was that ? phd c:  phd d: It the the output silence of the MLP . To apply also to in include also the the silence of the MLP we have the fifty - six form and the silence to pick up the silence and we include those . phd d: Yeah , because when we apply the KLT phd c: No they 're I think there is this silence in addition to the  KLT outputs professor b: No . phd c: it is because we we we just keep  we don't keep all the dimensions after the KLT phd d: In addition t phd c: and yeah . phd c: So we try to add the silence also in addition to the these twenty - eight dimensions . And what and what 's OGI forty - five ? The bottom one there ? phd c:  it 's o it 's OGI two , it 's so the th it 's the features from the first line phd d: It 's in fact OGI two . professor b: Right , but  what 's the what does the last row mean ? phd c: So it 's  basically this but without the KLT on the from the left path . So what 's the difference between the second phd c:  the second line you don't have this combo stuff so you just professor b: Oh . phd d:  professor b: OK , so alright so it looks to me I guess the same given that we have to take the filt ones out of the the running because of this delay problem so it looks to me like the ones you said I agree are are the ones to look at phd c:   professor b: oh yeah also when when they 're using this weighting scheme of forty , thirty - five , twenty - five is that on the percentages or on the raw errors ? I guess it 's probably on the percentages right ? phd c:  I guess , yeah .  OK so if we can know what how many words are in each and then  Dave  Dave promised to get us something tomorrow which will be there as far as they 've gotten Friday phd c:   professor b: and  how long did it I guess if we 're not doing all these things if we 're only doing   I guess since this is development data it 's legitimate to do more than one , right ?  ordinarily if in final test data you don't want to do several and and take the best phd c: Yeah . professor b: that 's that 's that 's not proper but if this is development data we could still look at a couple . But we have to decide  we have to fix the system on this d on this data , to choose the best professor b: Yeah . phd c: and these professor b: But the question is when when do we fix the system , phd c: But we could professor b: do we fix the system  tomorrow or do we fix the system on Tuesday ? phd c: it d professor b: I Yeah , OK except that we do have to write it up . professor b: Right so maybe what we do is we we we  as soon as we get the data from them we start the training and so forth phd c: Yeah but   professor b: but we start the write - up right away because as you say there there 's only minor differences between these . professor b: Yeah , and and I I would you know , I would I 'd kind of like to see it phd c:  yeah . Have y have you seen alt d do they have a format for how they want the system descriptions or anything ? phd c:  not really . Yes , for those who are listening to this and not looking at it  it 's not really that impressive , it 's just tiny . It 's all these little categories set a , set b , set c , multi - condition , clean . Do you know what no what no mitigation means here ? phd c:  it should be the the problem with the error channel error professor b: Oh that 's probably the phd c: or professor b: this is probably channel error stuff phd c: well , you professor b: huh ? Oh this is i right , it says right above here channel channel error resilience , phd c: Yeah .  and they have yes , split between seen databases and non - seen so basically between development and and evaluation . professor b: And so right , it 's presumed there 's all sorts of tuning that 's gone on on the see what they call seen databases and there won't be tuning for the  unseen . So the everything else is the SpeechDat - Car , that 's the multi multilingual phd c: Yeah , so it 's not divided between languages you mean or professor b: Well , it is . phd c: it just professor b: It is , but there 's also there 's these tables over here for the for the TI - digits and these tables over here for the car data which is which is I guess all the multilingual stuff phd c: Oh yeah . professor b: Well anyway , sounds like there 'll be a lot to do just to work with our partners to fill out the tables over the next  next few days phd c:   professor b: I guess they have to send it out let 's see the thirty - first is   Wednesday and I think the it has to be there by some hour  European time on Wednesday phd c:   professor b: so I think basically phd d: We lost time  Wednesday maybe because that the difference in the time may be is a long different of the time . professor b: E excuse me ? phd d: Maybe the Thursday the twelfth of the night of the Thurs - thirty - one is is not valid in Europe . professor b: Yes , so  I think we have to actually get it done Tuesday phd d: Tuesday . professor b:   phd c: Except if if it 's the thirty - one at midnight or I don't know we can still do some work on Wednesday morning . W i is but is is it midni I thought it was actually something like five PM on phd c: Yeah , well . professor b: was like I thought it was five PM or something , I didn't think it was midnight . professor b: well , so five PM their time is is if phd d: Not five PM , three PM . phd d:  no three three A - three PM ? phd c: No , we are wondering about the the the hour that we have to eh I don't know if it 's three PM it 's phd d: Oh yeah , yeah , yeah , yeah . phd c: Yeah , it 's it 's midnight but professor b: Yes , yes , but I didn't think it was midnight that it was due , I thought it was due at some hour during the day like five PM or something . professor b: so I I  well we should look but my assumption is that we basically have to be done Tuesday . professor b: but then then we 'll actually have the new data which is the German and the Danish phd c: Yeah . professor b: but that really will be much less work because  the system will be fixed phd c: Yeah . professor b: so all we 'll do is take whatever they have and and  and run it through the process . professor b: so there 'll be no new training , there 'll just be new HTK runs , so that 's means in some sense we can kind of relax from this after after Tuesday and and  maybe next meeting we can start talking a little bit about where we want to go from here  in terms of  the research . professor b:  you know what things  did you think of when you were  doing this process that  you just didn't really have time to adequately work on   so phd c:   professor b: What ? grad a: Oh , Stephane always has these great ideas and oh , but  we don't have time . professor b: And and  also it 's still true that  I think it 's true that that we we at least got fairly consistent i improved results by running  the  neural net transformation in parallel with the features phd c: But professor b: rather than  in sequence which was was your suggestion and that that that seems to have been borne out . professor b: The fact that none of these are are you know , enormous is is is not too surprising most improvements aren't enormous and  phd c: Yeah . professor b: some of them are but   you have something really really wrong and you fix it you can get big and really enormous improvements phd c:   professor b: but   Cuz our best improvements over the years that we 've gotten from finding bugs , but Anyway OK well I I think I see where we are and everybody knows what they 're doing and is there is there anything else we should talk about or or are we done ? phd c:   We so basically we will I think we 'll try to to focus on these three architectures and and perhaps I was thinking also a fourth one with just just a single KLT because we did not really test that professor b:  - huh . professor b: Oh I have yeah I do have one other piece of information which  I should tell people outside of this group too  I don't know if we 're gonna need it  but  Jeff up at the  University of Washington has  gotten a hold of a   some kind of server farm of  of ten   multiprocessor  IBM machines RS six thousands phd c:   professor b: and and  so I think each one is four processors or something or I don't know , eight hundred megahertz or something and there 's four processors in a box and there 's ten boxes and there 's some kind of ti so if you know he 's got a lot of processing power and  we 'd have to schedule it but if we have some big jobs and we wanna wanna wanna run them he 's he 's offering it . It 's  when he was here eh  he he used i not only every machine here but every machine on campus as far as I could tell , so so in some ways he just got his payback , but  again I I don't know if we 'll end up with if we 're gonna be CPU limited on anything that we 're doing in this group phd c:   OK well  you guys doing great stuff so that 's that that 's really neat and  we 'll   g don't think we need to   Oh well the other thing I guess that I will say is that  the digits that we 're gonna record momentarily is starting to get are starting to get into a pretty good size collection and  in addition to the SpeechDat stuff we will have those to work with really pretty soon now so that 's that 's another source of data .  which is s under somewhat better control and that we can we can make measurements of the room the  that you know if we feel there 's other measurements we don't have that we 'd like to have we can make them and  Dave and I were just talking about that a little while ago so  that 's another another possibility for this this kind of work . professor b: K ,  if nobody has anything else maybe we should go around do do our digits do our digits duty