Professor C: OK , and you were saying something starting to say something else about your your class project , or ?
Grad A: Oh . Yeah th Um .
Professor C: Yeah .
Grad A: So for my class project I 'm um  I 'm tinkering with uh support vector machines ? something that we learned in class , and uh um basically just another method for doing classification . And so I 'm gonna apply that to um compare it with the results by um King and Taylor who did um these um using recurrent neural nets , they recognized um a set of phonological features um and made a mapping from the MFCC 's to these phonological features , so I 'm gonna do a similar thing with  with support vector machines and see if PhD E: So what 's the advantage of support vector machines ? What Grad A: Um . So , support vector machines are are good with dealing with a less amount of data
PhD E: Hmm .
Grad A: and um so if you if you give it less data it still does a reasonable job in learning the the patterns .
PhD E: Hmm .
Grad A: Um and um
Professor C: I guess it yeah , they 're sort of succinct , and and they uh
Grad A: Yeah .
PhD E: Does there some kind of a distance metric that they use or how do they for cla what do they do for classification ?
Grad A: Um . Right . So , the the simple idea behind a support vector machine is um , you have you have this feature space , right ? and then it finds the optimal separating plane , um between these two different um classes ,
PhD E: Mm - hmm . Mm - hmm . Mm - hmm .
Grad A: and um and so um , what it i at the end of the day , what it actually does is it picks those examples of the features that are closest to the separating boundary , and remembers those
PhD E: Mm - hmm .
Grad A: and  and uses them to recreate the boundary for the test set . So , given these um these features , or or these these examples , um , critical examples , which they call support f support vectors , then um given a new example , if the new example falls um away from the boundary in one direction then it 's classified as being a part of this particular class
PhD E: Oh .
Grad A: and otherwise it 's the other class .
PhD E: So why save the examples ? Why not just save what the boundary itself is ?
Grad A: Mm - hmm . Um . Hmm . Let 's see . Uh . Yeah , that 's a good question . I yeah .
Professor C: That 's another way of doing it . Right ? So so it I mean I I guess it 's PhD E: Mmm . Sort of an equivalent .
Professor C: You know , it it goes back to nearest - neighbor sort of thing ,
PhD E: Mm - hmm .
Professor C: right ? Um , i i if is it eh w When is nearest - neighbor good ? Well , nearest - neighbor good is good if you have lots and lots of examples . Um but of course if you have lots and lots of examples , then it can take a while to to use nearest - neighbor . There 's lots of look ups . So a long time ago people talked about things where you would have uh a condensed nearest - neighbor , where you would you would you would pick out uh some representative examples which would uh be sufficient to represent to to correctly classify everything that came in .
PhD E: Oh . Mm - hmm .
Professor C: I I think s I think support vector stuff sort of goes back to  to that kind of thing . Um .
PhD E: I see . So rather than doing nearest neighbor where you compare to every single one , you just pick a few critical ones , and Professor C: Yeah .
PhD E: Hmm .
Professor C: And th the You know , um neural net approach uh or Gaussian mixtures for that matter are sort of fairly brute force kinds of things , where you sort of  you predefine that there is this big bunch of parameters and then you you place them as you best can to define the boundaries , and in fact , as you know , these things do take a lot of parameters and and uh if you have uh only a modest amount of data , you have trouble uh learning them . Um , so I I guess the idea to this is that it it is reputed to uh be somewhat better in that regard .
PhD E: Mm - hmm .
Grad A: Right . I it can be a a reduced um parameterization of of the the model by just keeping certain selected examples .
PhD E: Hmm .
Grad A: Yeah . So .
Professor C: But I don't know if people have done sort of careful comparisons of this on large tasks or anything . Maybe maybe they have . I don't know .
Grad A: Yeah , I don't know either .
Professor C: Yeah .
Grad B: S do you get some kind of number between zero and one at the output ?
Grad A: Actually you don't get a you don't get a nice number between zero and one . You get you get either a zero or a one . Um , uh there are there are pap Well , basically , it 's it 's um you you get a distance measure at the end of the day , and then that distance measure is is um  is translated to a zero or one . Um .
Professor C: But that 's looking at it for for classification for binary classification ,
Grad A: That 's for classification , right .
Professor C: right ?
PhD E: And you get that for each class , you get a zero or a one .
Grad A: Right .
Professor C: But you have the distances to work with .
Grad A: You have the distances to work with ,
Professor C: Cuz actually Mississippi State people did use support vector machines for uh uh speech recognition and they were using it to estimate probabilities .
Grad A: yeah . Yeah . Yeah , they  they had a had a way to translate the distances into into probabilities with the with the simple um uh sigmoidal function .
Professor C: Yeah , and d did they use sigmoid or a softmax type thing ?
Grad A: Um  Yeah ,
Professor C: And didn't they like exponentiate or something
Grad A: there 's some there 's like one over one plus the exponential or something like that .
Professor C: and then divide by the sum of them , or ? Oh it i Oh , so it is a sigmoidal .
Grad A: Yeah .
Professor C: OK . Alright .
PhD E: Did the did they get good results with that ?
Professor C: I mean , they 're OK , I I don't I don't think they were earth earth shattering , but I think that uh this was a couple years ago ,
PhD E: Hmm .
Professor C: I remember them doing it at some meeting , and and um I don't think people were very critical because it was interesting just to to try this and you know , it was the first time they tried it , so  so the you know , the numbers were not incredibly good
PhD E: Hmm .
Professor C: but there 's you know , it was th reasonable .
PhD E: Mm - hmm .
Professor C: I I don't remember anymore . I don't even remember what the task was , it {comment} was Broadcast News , or something . I don't know .
PhD E: Hmm .
Grad A: Right .
Grad B: Uh s So Barry , if you just have zero and ones , how are you doing the speech recognition ?
Grad A: Oh I 'm not do I 'm not planning on doing speech recognition with it . I 'm just doing detection of phonological features .
Grad B: Oh . OK .
Grad A: So uh for example , this this uh feature set called the uh sound patterns of English um is just a bunch of um binary valued features . Let 's say , is this voicing , or is this not voicing , is this sonorants , not sonorants , and stuff like that .
Grad B: OK .
Grad A: So .
PhD E: Did you find any more mistakes in their tables ?
Grad A: Oh ! Uh I haven't gone through the entire table , yet . Yeah , yesterday I brought Chuck the table and I was like , " wait , this is Is the mapping from N to to this phonological feature called um " coronal " , is is should it be shouldn't it be a one ? or should it should it be you know coronal instead of not coronal as it was labelled in the paper ? " So I ha haven't hunted down all the all the mistakes yet ,
Professor C: Uh - huh .
Grad A: but Professor C: But a as I was saying , people do get probabilities from these things ,
Grad B: OK .
Professor C: and and uh we were just trying to remember how they do , but people have used it for speech recognition , and they have gotten probabilities . So they have some conversion from these distances to probabilities .
Grad B: OK .
Grad A: Right , yeah .
Professor C: There 's you have you have the paper , right ? The Mississippi State paper ?
Grad A: Mm - hmm . Mm - hmm .
Professor C: Yeah , if you 're interested y you could look ,
Grad B: And OK . OK .
Grad A: Yeah , I can I can show you I Professor C: yeah .
Grad A: yeah , our PhD E: So in your in in the thing that you 're doing , uh you have a vector of ones and zeros for each phone ?
Grad A: Mm - hmm . Uh , is this the class project , or ?
PhD E: Yeah .
Grad A: OK . um
PhD E: Is that what you 're Grad A: Right , {comment} Right , right f so for every phone there is there is a um a vector of ones and zeros f uh corresponding to whether it exhibits a particular phonological feature or not .
PhD E: Mm - hmm . Mm - hmm . And so when you do your wh I 'm what is the task for the class project ? To come up with the phones ?
Grad A: Um
PhD E: or to come up with these vectors to see how closely they match the phones ,
Grad A: Oh . Right , um to come up with a mapping from um MFCC 's or s some feature set , um to uh w to whether there 's existence of a particular phonological feature .
PhD E: or ? Mm - hmm .
Grad A: And um yeah , basically it 's to learn a mapping from  from the MFCC 's to uh phonological features . Is it did that answer your question ?
PhD E: I think so .
Grad A: OK . C
PhD E: I guess I mean , uh I 'm not sure what you what you 're what you get out of your system . Do you get out a uh a vector of these ones and zeros and then try to find the closest matching phoneme to that vector ,
Grad A: Mm - hmm . Oh .
PhD E: or ?
Grad A: No , no . I 'm not I 'm not planning to do any any phoneme mapping yet . Just  it 's it 's basically it 's it 's really simple , basically a detection of phonological features .
PhD E: Uh - huh .
Grad A: Yeah ,
PhD E: I see .
Grad A: and um  cuz the uh So King and and Taylor um did this with uh recurrent neural nets ,
PhD E: Yeah .
Grad A: and this i their their idea was to first find a mapping from MFCC 's to uh phonological features
PhD E: Mm - hmm .
Grad A: and then later on , once you have these phonological features , then uh map that to phones .
PhD E: Mm - hmm .
Grad A: So I 'm I 'm sort of reproducing phase one of their stuff .
PhD E: Mmm . So they had one recurrent net for each particular feature ?
Grad A: Right . Right . Right . Right .
PhD E: I see . I wo did they compare that I mean , what if you just did phone recognition and did the reverse lookup .
Grad A: Uh .
PhD E: So you recognize a phone and which ever phone was recognized , you spit out it 's vector of ones and zeros .
Grad A: Mm - hmm . Uh .
Professor C: I expect you could do that .
PhD E: I mean uh Professor C: That 's probably not what he 's going to do on his class project . Yeah .
PhD E: Yeah . No .
Grad A: Yeah .