grad a: So , I think this is gonna be a pretty short meeting because I have four agenda items , three of them were requested by Jane who is not gonna be at the meeting today . Does anyone besides Jane know what the transcription status is ? phd f: sort of , I do , peripherally . phd c: Is that English ? phd f:  Well first of all with IBM I got a note from Brian yesterday saying that they finally made the tape for the thing that we sent them a week or week and a half ago phd d: That 's our system . grad a: Ugh ! phd f: and that it 's gone out to the transcribers and hopefully next week we 'll have the transcription back from that . grad a: C can I have a pen ? phd f:  Jane seems to be  moving right along on the transcriptions from the ICSI side . phd f: Yeah , so  , I guess she 's hired some new transcribers phd d: Speaking grad e: Which meetings is she transcribing ? phd f: and  well we 've we 've run out of E D Us because a certain number of them are  , sort of awaiting to go to IBM . phd d: So does she have transcribers right now who are basically sitting idle because there 's no data back from IBM grad e: So we 're doing some in parallel . phd d: Because I I need to ask Jane whether it 's it would be OK for her  , s some of her people to transcribe  some of the initial data we got from the SmartKom data collection , which is these short like five or seven minute sessions . phd d:  and we want it You know , we need The Again , we we have a similar  logistic set - up where we are supposed to send the data to Munich grad a: Right . phd d: And so  I wanted to ask Jane if if  , you know , maybe one of their transcribers could could do  since these are very short , that should really be  , professor b:   And it 's One session is only  like seven professor b: So that should have ma many fewer And it 's also not  a bunch of interruptions with people and all that , phd d: Right . And some of it is read speech , so we could give them the the thing that they 're reading professor b: right ? So . phd d: And so  , I guess since she 's I was gonna ask her but since she 's not around I maybe I 'll professor b: Yeah , well it certainly seems phd d:  if if that 's OK with you to to , you know , get that stuff  to ask her for that , then I 'll do that . Yeah , if we 're held up on this other stuff a little bit in order to encompass that , that 's OK because I I  ,  I still have high hopes that the that the IBM pipeline 'll work out for us , so it 's phd d: Yeah . phd f: Oh , yeah , and also related to the transcription stuff , so I 've been trying to keep a web page  up to date f showing what the current status is of the trans of all the things we 've collected and what stage each meeting is in , in terms of whether it 's grad a: Can you mail that out to the list ? phd f:  -  , yeah I will . I That 's the thing that I sent out just to foo people saying can you update these pages grad a: Oh , OK , OK . phd f: and so that 's where I 'm putting it but I 'll I 'll send it out to the list telling people to look at it . So Jane also wanted to talk about participant approval , but I don't really think there 's much to talk about . professor b: You are going to grad a: I 'm gonna send out to the participants , with links to web pages which contain the transcripts and allow them to suggest edits .  phd c: So but it 's just transcripts , not the not the audio ? grad a: Nope , they 'll have access to the audio also . phd f: So ,  the audio that they 're gonna have access to , will that be the uncompressed version ? Or will you have scripts that like uncompress the various pieces and grad a: Oh , that 's a good point . Yeah , it 's it 's probably going to have to be the uncompressed versions because , it takes too long to do random access decompression . Yeah , I was just wondering because we 're  running out of the un - backed - up disk space on grad a: Well , that was the other point . phd f: Oh , was that another one ? grad a: Yep , that 's another agenda item . grad a: So ,  But that is a good point so we 'll get to that , too . I 've added some stuff that  indes indexes by the meeting type MR , EDU , et cetera and also by the user ID .  The back - end is  , going more slowly as I s I think I said before just cuz I 'm not much of a Tcl - TK programmer . So I think Don and I are gonna work on that and and you and I can just talk about it off - line more .  grad e: And , also  , I was just showing Andreas , I got  an X Waves kind of display , and I don't know how much more we can do with it with like the prosodic stuff where we have like stylized pitches and signals and the transcripts on the bottom grad a: Oh , cool . grad e: so , right now it 's just an X Waves and then you have three windows but I don't know , it looked pretty nice and I 'm sure it think it has potential for a little something , grad a: For a demo ? grad e: yeah , for a demo . grad e: So professor b: OK , so again , the issue is For July , the issue 's gonna be what can we fit into a Windows machine , and so on , but grad e: Oh . grad e: Well , we 'll see ,  phd c: I 've been putting together  Transcriber things for Windows so i And I installed it on Dave Gelbart 's PC and it worked just fine . phd d: Really ? So is that Because there 's some people  It would be cool if we could  get that to work  at at SRI phd c: Yeah . phd d: because the  grad a: Well Transcriber is Tcl - TK , very generic with Snack , phd d: we have m m We have more Windows machines to run the phd c: Yeah . Yeah but But the problem is the version Transcriber works with , the Snack version , is one point six whatever and that 's not anymore supported . But I just wrote an email to to the author of to the Snack author and he sent me to one point six whatever library grad a: Well I thought it was packaged with Transcriber ? phd c: and so it works . Yeah , but then you can't add our patches and then the the new version is is totally different grad a: Oh . phd c: and you can't you can't access that so you have to install First install Tcl then install Snack and then install the Transcriber thing and then do the patches . Ugh ! phd d: I I wonder if if we should contribute our changes back to the authors so that they maintain those changes along phd c: Yeah . grad a: We have Yeah b it 's just hasn't made it into the release yet . phd f: So did you  put the  the NT version out on the  Meeting Recorder page ? Or phd c: No , I haven't done that yet . professor b: So , can some of the stuff that Don 's talking about somehow fit into this  , mean you just have a set of numbers that are associated with the grad e: Yeah . phd c: So grad e: Yeah , it 's basically ASCII files or binary files , whatever representation . Just three different It 's a waveform and just a stylized pitch vector basically so it 's phd d: So So Well grad e:  we could do it in Matl -  you could do it in a number of different places I 'm sure . phd d: But But it would be cool if the Transcriber interface had like another window for the you know , maybe above the waveform where it would show some arbitrary valued function that is that is you know time synchron ti ti time synchronous with the wavform . Again it 's it 's It 's more Tcl - T grad e: Yeah . grad a: So someone who 's familiar with Tcl - TK has to do it , phd d: Right . grad a: But it it seems to me that I c phd c: And grad a: It doesn't seem like having that real time is that necessary . grad e:  What do you mean by real time ? Do you mean like phd f: Like being able to scroll through it and stuff for the demo . phd f: Is that what you mean ? grad a: It just seems to me jus grad e: It would be cool to see it phd f: Yeah . grad e: It would be cool like to see to hear it and see it , phd c: And to hear it . grad a: Sure , but I don't think I You can do all that just statically in phd c: Yeah . grad e: I think it would lose Yeah ,  y grad a: Just record the audio clip and show an image and I think that 's grad e: Right , right .  , no , we 're talking about on the computer and and  , I think when we were talking about this before we had littl this little demo meeting , grad e: Right . professor b: we sort of set up a range of different degrees of liveness that you could have and , the more live , the better , but  , given the crunch of time , we may have to retreat from it to some extent . So I think For a lot of reasons , I think it would be very nice to have this Transcriber interface be able to show some other interesting signal along with it phd d:   But ,  Anyway , jus just looking for ways that we could actually show what you 're doing , in to people . professor b: Cuz a lot of this stuff , particularly for Communicator ,  certainly a significant chunk of the things that we waved our arms about th originally had t had to do with prosodics It 'd be nice to show that we can actually get them and see them . So , we have a little bit of time before that becomes critical , but we are like ninety five percent , ninety eight percent on the scratch disks for the expanded meetings . grad a: And , my original intention was like we would just delete them as we needed more space , but unfortunately we 're in the position where we have to deal with all the meeting data all at once , in a lot of different ways . grad a: Yeah , there 're a lot of transcribers , phd c: Yeah . grad a: so all of those need to be expanded , and then people are doing chunking and I want to do  , the permission forms , phd f:   grad a: so I want those to be live , so there 's a lot of data that has to be around . One of the things I was thinking is we we just got these hundred alright , excuse me ten ,  SPARC - Blade SUN - Blades . grad a: And so it seems to me we could hang scratch disk on those because they 'll be in the machine room , they 'll be on the fast connection to the rest of the machines . phd f: Well , is there Why not just hang them off of Abbott , is there a grad a: Because there 's no more room in the disk racks on Abbott . phd f: You can put two phd d: Oh you mean you put them inside the pizza boxes for the grad a: Sure . grad a: So you can just go out and buy a PC hard drive and stick it in . professor b: But if Abbott is going to be our disk server it it file server it seems like we would want to get it , a second disk rack or something . What I 'm looking for is where do we s expand the next meeting ? phd c: Yep . professor b: Well , for the next meeting you might be out of luck with those ten , mightn't you ?  , you know Dave Johnson is gone for , like , ten days , grad a: Oh , I didn't know he had left already . grad e: How much space do you need for these ? professor b: I don't know what his schedule is . phd f: I I thi grad e: I have  I have an eighteen gig drive hanging off of my computer . grad a: Alright ! What 's your computer 's name ? grad e: So  , Samosa . grad a: And you 're o you 're offering ? grad e:  I 'm not doing anything on it right now until I get new meetings to transcri or that are new transcriptions coming in I really can't do anything . grad e:  not that I can't do anything , I jus phd f: I I jus I just gave Thilo some about ten gigs , the last ten gigs of space that there was on on  Abbott .  And  So but that But grad a: Which one was that , X G ? X G ? phd c: XG . phd d: That 's also where we store the The  Hub - five training set waveforms , phd c: Oops . grad a: But that won't be getting any bigger , phd d: right ? phd f: No . grad a: will it ? phd f: I don't think that 's on XG . phd d: But I 've also been storing I 've been storing the feature files there and I guess I can s start deleting some because we now know what the best features are grad e: Well phd d: and we won't be using the old ones anymore . phd d:  Oh thats XA Oh that 's X phd c: Isn't that XH ? phd f: I th grad a: Not not for long . grad e: I have a lot of space and it 's not it 's n There 's very little  Yeah not for long . phd d: Maybe I 'm confu grad e: But  it 's not going f phd d: Oh no I 'm sorry . grad a: Yeah , it 's probably Probably only about four gig is on X on your X drive , phd c: So . grad a: but we 'll definitely take it up if you grad e: I th phd d: I think you 're right . It 's XH and D grad e: I think it 's about four or five gig cuz I have four meetings on there , phd d: The b I 'm also using DG I got that confused . grad a: The " more disk space " button ? professor b: Just press Press each meeting saying " we need more disk space " " this week " . professor b: And And how much does each meeting take ? phd f: And it 's about a gig uncompressed . phd c: It 's It 's a little bit more as I usually don't do not uncompress the all of the PZM and the PDA things . So there 's what thirty some gig of just meetings so far ? professor b: So - so So maybe there 's a hundred gig or something . Well we We haven't uncompressed all the meetings , but grad a: I would like to .  it 's just a question of figuring out where they should be and hanging them , grad a: Yep . professor b: but But  , we could You know , if you want to get four disks , get four disks . Well I sent that message out to , I guess , you and Dave asking for if we could get some disk . phd f: I s I sent this out a a day ago grad a: And put it where ? professor b: Right . phd f: but and Dave didn't respond so I don I don't know how the whole process works .  does he just go out and get them and if it 's OK , and grad a: Yep . And just say an - e just ask him that , you know , wha what should you do . And in my answer back was " are you sure you just want one ? " So  I think that what you want to do is plan ahead a little bit and figure " well , here 's what we pi figure on doing for the next few months " . So this is a question that 's pretty hard to solve without talking to Dave , phd d: Th - The phd f: I think part of the reason why Dave can't get the the new machines up is because he doesn't have room in the machine room right now . phd d: One One On - One thing to in to  t to do when you need to conserve space is phd f: So he has to re - arrange a bunch of stuff . phd d: I bet there are still some old , like , nine gig disks , around and you can probably consolidate them onto larger disks and and you know recover the space . An - and so , he always has a a lot of plans of things that he 's gonna do to make things better in many ways an and runs out of time . And so I think what he 's been concentrating on is  the back the back up system , rather than on new disk . Basically , we can easily get one to four disks ,  you just go out and get four and we 've got the money for it , it 's no big deal .  , but the question is where they go , and I don't think we can solve that here , you just have to ask him . grad a: Yeah , it 's just It 's not on the net , so it 's a little awkward phd d: The only phd f:  . phd d: What do you mean it 's not on the net ? grad a: It 's not phd c: It 's not bad . grad a: It 's behind lots of fire walls that don't allow any services through except S S phd d: Oh because it 's because it 's an ACIRI machine ? grad a: Yep . grad a: And also on the list is to get it into the normal ICSI net , but Who knows when that will happen ? phd d: But that can't be that hard . phd d:  grad a: No , the the problem with that apparently is that they don't currently have a wire running to that back room that goes anywhere near one of the ICSI routers . If there was a person dedicated to doing it they would happen pretty easily but it 's it 's jus every ever everybody everybody has a has grad a: But Dave has to do all of them . But at any rate I think that there 's a there 's a longer term thing and there 's immediate need and I think we need a a conversation with  , maybe maybe after after tea or something you and I can go down and and talk to him about it Just say " wha you know , what should we do right now ? " phd f: How long is David gonna be gone ? professor b: eleven days or something ? grad a: Oh my ! professor b: Yeah basically tomorrow and all of the week after . The only oth thing other thing I was gonna add was that   , I talked briefly to Mari and  we had both been busy with other things so we haven't really connected that much since the last meeting we had here but we agreed that we would have a telephone meeting the Friday after next . And I I I wanted to make it ,  after the next one of these meetings , so something that we wanna do next meeting is is  to put together  , a kind of reasonable list for ourselves of what is it , that we 've done .  just sort of bulletize  o e do do I can I can dream up text but this is basically gonna lead to the annual report . grad a: This is the fifteenth ? So just a week from tomorrow ? professor b: that would grad a: OK . So , we can This So that 's an phd d: Is this gotta be in the morning ? professor b:  phd d: Or Because you know I Fridays I have to leave  like around  two . So if it could be before that would be be professor b: No , no but I I I don't need other folks for the meeting . professor b: Yeah so what I meant was on the me this meeting if I wa something I I I 'm making a major thing in the agenda is I wanna help in getting together a list of what it is that we 've done so I can tell her . professor b: but but   , and then the next day  , late in the day I 'll be having that that discussion with her . phd d:   One thing  we in past meetings we had  also a you know various variously talked about the  work that w  was happening sort of on the on the recognition side professor b:   And I wondered whether we should maybe have  a separate meeting and between you know , whoever 's interested in that because I feel that  there 's plenty of stuff to talk about but it would be sort of  maybe the wrong place to do it in this meeting if  professor b: Think so ? phd d: Well , it 's that It 's just gonna be ver very boring for people who are not you know , sort of really interested in the details of the recognition system . professor b: Well , OK , so how many how many people here would not be interested in  in a meeting about recognition ? phd c: Me too . phd d: Well I know Well , Jane an Well you mean in a separate meeting or ha ha talking about it in this grad a: No . professor b: Real grad e: Watch a ball game ? professor b: Yeah , real real real men " Real men do decoding " or something like that . phd d:  it it 's sort of  when when the talk is about data collection stuff , sometimes I 've you know , I I 'm bored . grad a: The Nod off ? phd d: So it 's I c I can sympathize with them not wanting to i to to be  you know If I cou you know this could professor b: It 's cuz y you have a So you need a better developed feminine side . phd d: I 'm professor b: There 's probably gonna be a lot of " bleeps " in this meeting . You know , whatev or whenever we feel like we phd f: Right , I was Why don't we alternate this meeting every other week ? grad a: Or just alternate the focus . grad a: Oh sor phd f: But I do I don't  a lot of times lately it seems like we don't really have enough for a full meeting on Meeting Recorder . phd f: So if we could alternate the focus of the meeting grad a: Let 's read digits and go . professor b: And then if we find , you know we 're just not getting enough done , there 's all these topics not coming up , then we can expand into another meeting . Let 's chat about it with Liz and Jane when we get a chance , see what they think and phd d:   phd f: and there 's lots of things , you know , details and stuff that would I think people 'd be interested in and I 'd you know , where do we go from here kind of things and So , it would be good . professor b: Yeah , and you 're you 're attending the  the front - end meeting as well as the others so you have you have probably one of the best you and I , I guess are the main ones who sort of see the bridge between the two . phd d:  I jus So the latest result was that   yot I tested the  the sort of final version of the PLP configuration  on development test data for for this year 's Hub - five test set . phd d: And the recognition performance was exactly , and  exactly up to the you know , the first decimal , same as with the  Mel Cepstra front - end .  , well i there was a little bit of a phd f: Oh ! phd d: i overall . phd d: And then the really nice thing was that if if we combine the two systems we get a one and a half percent improvement . phd d: Which u actually uses the whole N - best list from both systems to mmm ,  c combine that . professor b: So except  the only key difference between the two really is the kind of smoothing at the end which is the auto - regressive versus the cepstral truncation . phd d: And , the phd f: But a percent and a half ? grad a: Yeah , it 's pretty impressive . phd f: That 's phd d: And And so  after I told the my  colleagues at SRI about that , you know , now they definitely want to , you know , have a Next time we have an evaluation they want to do  , you know , basically a at least the system combination .  , and , you know , why not ? professor b: Sure , why not ? phd d:  . phd d:  w what do you mean ? More features in the sense of front - end features or in the sense of just bells and whistles ? grad a: No ,  front - end features . Let 's , you know , try RASTA and MSG , and phd d: Oh  Yeah . That 's the the the There 's one thing   you don't want to overdo it because y every front - end You know , if you you know you basically multiply your effort by N , where N is a number of different systems phd f: Oh . So one one compromise would be to only to have the everything up to the point where you generate lattices be basically one system and then after that you rescore your lattices with the multiple systems and combine the results and that 's a fairly painless  thing . phd f: Do you think we 'd still get the one and a half  phd d: I I think so . Maybe a little less because at that point the error rates are lower and so if You know , maybe it 's only one percent or something but that would still be worthwhile doing .  Jus - You know , just wanted to let you know that that 's working out very nicely . phd d: And then we had some results on digits , with  We We So this was  really really sort of just to get Dave going with his  experiments . But as a result , you know , we were sort of wondering why is the Hub - five system doing so well on the digits . phd d: And the reason is basically there 's a whole bunch of read speech data in the Hub - five training set . phd d: And you c And Not all of No it 's actually , digits is only a maybe a fifth of it . professor b: A fifth of it is how much ? phd d: The rest is is read is read TIMIT data and  ATIS data and Wall Street Journal and stuff like that . But a fi a fifth is how much ? phd d: A fifth would be maybe  two hours something . professor b: Yeah , so  that 's actually not that different from the amount of training that there was . phd d: But it definitely helps to have the other read data in there professor b: Oh yeah phd d: because we 're doing professor b: w phd d: You know the error rate is half of what you do if you train only on ti  TIMIT  not TIMIT  TI - digits , professor b:   phd d: That 's e professor b: Because because  , it was apparent if you put in a bunch more data it would be better , phd d: That was e Right , right . So we only for the Hub - five training , we 're only using  a fairly small subset of the Macrophone database . grad a: I could also put in  focus condition zero from Hub - four from Broadcast News , which is mostly prepared speech . So , you know that might be useful for the people who train the the digit recognizers to to use  something other than TI - digits . They they  they experimented for a while with a bunch of different databases with French and Spanish and so forth cuz they 're multilingual tests phd d:   professor b: But but yeah certainly if we ,  If we knew what the structure of what we 're doing there was . professor b: Once we know , then the trainable parts of it it 'd be great to run lots of lots of stuff through . And then I th guess Chuck and I had some discussions about how to proceed with the tandem  system and You wanna You wanna see where that stands ? phd f: Well , I 'm phd d:  phd f: Yeah , so Andreas  brought over the  alignments that the SRI system uses . And so I 'm in the process of  converting those alignments into  label files that we can use to train  a new net with . phd d: An - And one side effect of that would be that it 's  that the phone set would change . So the MLP would be trained on I think only forty - six or forty - eight phd f: Right . phd d:  which is smaller than the  than the phone set that that we 've been using so far . phd d: And that that that will probably help , actually , phd f: So it 's a little different ? phd d: because  the fewer dimensions  e the less trouble probably with the as far as just the  ,  Just You know we want to try things like deltas on the tandem features . And so , you know , fewer dimensions in the phone set would be actually helpful just from a logistics point of view . Although we  , it 's not that many fewer and and and we take a KLT anyway so we could phd d: Right . And then we wanted to s just limit it to maybe  something on the same order of dimensions as we use in a standard  front - end . So that would mean just doing the top I don't know ten or twelve or something of the KLT dimensions . My impression was that when we did that before that had very little  he didn't lose very much . phd d: But then And then something Once we have the new M L P trained up ,  one thing I wanted to try just for the fun of it was to actually run  like a standard hybrid system that is based on you know , those features  and  retrain MLP and also the you know , the dictionary that we use for the Hub - five system . professor b: And the b And the base u starting off with the base of the alignments that you got from i from a pretty decent system . phd d:  because you know , compared to what Eric did a while ago , where he trained up , I think , a system based on Broadcast News and then  tra retraining it on Switchboard or s  and professor b: Yeah . phd d: But he I think he d he didn't he probably didn't use all the training data that was available . We we made some improvements to the dictionary 's  to the dictionary about two years ago which resulted in a  something like a four percent absolute error rate reduction on Switchboard , which professor b: Well the other thing is , dipping deep into history and into  our resource management days , when we were collaborating with SRI before , phd d:   professor b:  it was I think , it is was a really key  starting point for us that we actually got our alignment . professor b: When we were working together we got our initial alignments from Decipher ,  at the time . Later we got away from it because because once we had decent systems going then it was it was typically better to use our own systems phd f: Yeah . professor b: cuz they were self consistent but but certainly to start off when we were trying to recover from our initial hundred and forty percent error  rate . professor b: And we 're not quite that bad with our our Switchboard systems but it was they certainly aren't as good as SRI 's , phd d: OK . phd f: W What is the performance on s the best Switchboard system that we 've done ? Roughly ? professor b: Well , the hybrid system we never got better than about fifty percent error . And  it was I think there 's just a whole lot of things that  no one ever had time for .  we always had a list of a half dozen things that we were gonna do and and a lot of them were pretty simple and we never did . professor b: we never did an never did any adaptation phd d: But that w Even that that number professor b: we never did any phd d: Right . And And that number I think was on Switchboard - one data , right ? Where the error rate now is in the twenties . phd d: So it would be So it would be good t to sort of r re  professor b: Yeah . And the other thing that that would help us to evaluate is to see how well the M L P is trained up . phd d: So it 's sort of a sanity check of the M L P outputs before we go ahead and train up the  you know , use them as a basis for the tandem system . Not phd f: Should we Should we bother with  using the net before doing  embedded training ? professor b: But . phd d: But phd f:  should should we even use that ? phd d: Oh oh that 's a good question . phd f: Or should I just go straight to phd d: Yeah , we we weren't sure whether it 's worth to just use the alignments  from the S R I recognizer or whether to actually go through one or more iterations of embedded training where you realign . You run it ? Keep keep both versions ? See which one 's better ? professor b: yeah . professor b: And while it 's training you may as well test the one you have and see how it did . You know , it 's phd d: But But so I grad a: Sort of given up guessing . phd d: Well but i But in your experience   have you seen big improvements in s on some tasks with embedded training ? Or was it sort of small - ish  improvements that you got professor b:  well . professor b: because we 're coming from  , alignments that were achieved with an extremely different system . grad a: Although ,  we 've done it with When we were combining with the Cambridge recurrent neural net , embedded training made it worse . phd d: So you you started training with outputs from a with alignments that were generated by the Cambridge  system ? grad a: Yep . professor b: Oh ! phd d: No  it 's weird that it did I 'm sorry . Tha - u we we 've see  and wi with the numbers OGI numbers task we 've seen a number of times people doing embedded trainings and things not getting better . phd d: Oh actually it 's not that weird because we have seen We have seen cases where acoustic retraining the acoustic models after some other change made matters worse rather than better . professor b: But I But I would I would suspect that something that that had  a very different  feature set , for instance  they were using pretty diff similar feature sets to us . professor b: I I would expect that something that had a different feature set would would  benefit from phd d:   professor b: sorry , it was the other thing is that what was in common to the Cambridge system and our system is they both were training posteriors . professor b: So  , that 's another pretty big difference grad a: That 's another big difference . professor b: and  , one bac at least Back at phd d: You mean with soft targets ? Or ? Sorry , I 'm sor I missed What What 's the key issue here ? professor b: Oh , that  both the Cambridge system and our system were were training posteriors . And if we 're we 're coming from alignments coming from the SRI system , it 's a likelihood - based system . You know , there 's diffe different front - end different different  , training criterion  , I would think that in a that an embedded  embedded  training would have at least a good shot of improving it some more . I was wondering  you know what size net I should Anybody have any intuitions or suggestions ? professor b: how much training data ? phd f: Well , I was gonna start off with the small train set . professor b: And how How many hours is that ? phd f: That 's why I was I I 'm not sure how much that is . phd d: I think that has about Well i you 'd would be gender - dependent training , right ? So So I think it 's  that 's about mmm , something like thirty hours . phd f: In the small training set ? grad a: Hello ? phd d: I I think so . I 'll grad a: Excuse me ? phd d: It 's definitely less than a hundred grad a: Alright . It 's it 's th the thing I 'll I 'll think about it a little more phd d:   It 's m It 's more than phd f: And a thousand is too small ? professor b: Oh let me think about it , but I think that that  th at some point there 's diminishing returns . professor b: but it but but there is diminishing returns and you 're doubling the amount of time . phd d: Remember you 'll have a smaller output layer so there 's gonna be fewer parameters there . phd d: And then professor b: Fifty s Fifty four to forty eight ? grad a: Vast majority is from the input unit . grad a: Right , because you used the context windows and so the input to hidden is much , much larger . professor b: Yeah , so it 's it 'd be way , way less than ten percent of the difference . What am I trying to think of ? phd f: The The net that that we did use already  was eight thousand hidden units and that 's the one that Eric trained up . professor b: So , yeah definitely not the one thousand  two thousand fr  the four thousand will be better and the two thousand will be almost will be faster and almost as good .  a four thousand is well within the range that you could benefit from but the two thousand 'd be faster so phd d: Right