welcome to the science for the public lecture series science for the public is an organization committed to bringing the science information issues to the general public this is our website top program esteems and ther of boston university dr gunther is a professor in be used department of speech language and hearing sciences as well as in the department of biomedical engineering he's also the director of the boston university speech speech mechanisms and development of a brain computer interface device for completely paralyzed patients dr gunther has published widely and given numerous keynote and distinguished lectures worldwide his research combines theoretical modeling and behavioral and neural imaging experiments to characterize the neural computations underlying speech he's the originator of the leading model of the  processes involved in speech production which he'll discuss tonight dr gunther is especially well known for his work on brain computer interfaces to enable communication for individuals suffering from locked in syndrome and that work has received a great deal of media coverage tonight he'll tell us all about this exciting work we are very honored to welcome dr frank de sac thank you for having and if you would begin please by telling us something about the complexity of speech ok well language is probably the most complicated cognitive skill that everyone develops so if you're human and you grow up with other the first homo sapiens about two hundred thousand years ago and it was a very major advance in abilities that really gave a competitive and advantage to humans that allowed us to live beyond our  so it's a skill that's crucial to our survival and it's something that is really unique to humans now language is a very complex task and it involves several different parts of the brain many parts of the brain in fact  hundreds of thousands millions of neurons are involved in even the simplest speech tasks that we do and our focus in our lab is actually on what we would call speech which is a subset of language capabilities where we're not concerned so much with the meaning of words or a grammar of sentences we're concerned with the motor act of speaking so the act of moving your tongue and your lips and your larynx and your other what we would call speech articulators in order to generate sounds that convey words to your listener now speech is an amazing motor task in fact it's the most complex task performed by humans and maybe by any animal and to give an example of that consider the word dilapidated which this is a word that we can all say very easily in one second there are eleven different speech sounds or phonemes within this word and that means that each one of those sounds is produced in less than a tenth of a second and yet we all can do this all all neurologically normal humans are capable of doing this that's a rate of movement that's roughly equivalent to the fastest typist in the world it's a amazing motor act and yet we all effortlessly develop that capability so our is on how humans developed speech capabilities and how the system works how the different parts of the brain work together to generate speech output down when you say the word but dated sometimes we do trip over these words right we get this syllables mixed up and so on but do you look at language development like in children as they are developing where that's really tricky to put for syllable words together and so on but can you give us any background on sure so to develop speech skills requires a large number of things to happen in your brain and some of the earliest things happen shortly after birth so even before you can move your tongue well or do any of the things that we would consider speaking you were listening to language and you're hearing sounds from people from adults around you from people who know the language and even in the first few months of life infants are developing the ability to detect sounds in their language and they're actually losing the ability to hear sound differences from other languages so a common example of that is japanese infants by the time they're one year old they lose the ability to hear the acoustic features that differentiate ours from elle's they had that at birth but they are lose it over the first year they lose it because their brain is learning to hear their own language which doesn't make that distinction so their brain auditory system becomes specialized for hearing their own language and that's a very important part of the development process so that occurs before you even start speaking another very important milestone in the development of speech occurs when you say when an infant starts to babble early on around six months of age infants make sounds and they're they're learning things about their vocal mechanism so they're learning for example how to change their pitch or the volume of their speech well before they produce anything that sounds like a word so the development of the motor system also begins very early six or eight months of age around eight to ten months of age infants go through babbling stage where they start to produce speech like sounds and this occurs largely because they just move their jaws up and down at the right frequency that opens and closes the vocal tract and that produces a consonant like sounds and valley like sounds that start to sound a lot like speech utterances and a typical infant will often produce utterances such as mamma during this stage and this may be one of the reasons that we choose these words for mother and father there are some of the first speech like utterances that occur around ten months of age twelve months of age they start varying their movements while they move the job and down so they start varying their tongue movements and this starts producing more complicated syllables and eventually the first words during this period when the first words develop now what they're doing when they're by a babbling is amongst other things they're learning how their movements create sounds and so they're learning what we would call a mapping between articulatory movements and sound output and that becomes crucial later when they want to imitate sounds in their language the thing is that they learn while they're babbling tell them well if i want to make a sound  sounds like this i know now because i when i was babbling i heard myself produce this sound i know what kind of movements to make so these learning processes are developing connections within the brain within what we would call the speech network which is the network of brain regions that are involved in producing speech and this sort of learning involves learning mappings between audit auditory information and motor information and between some out of sensory information so information about what it feels like what the tactile pattern feels like when i produce speech sounds and the relationship between that sort of information and auditory information as well as motor information so these three types of information are are being used to train up the brain network during babbling and then later they develop yet another type of information which is fine a logical information these would be speech sounds as discrete units that they can perceive and produce and so this development occurs around ten twelve months of age and that's when they start producing the first word so they're in this process in the first year of  with youngsters having difficulties with this sort of what i say switching syllables things in words i'm trying to think of a word where that happens i didn't think i gave you an example babbitt instead of bunny rabbit but were there many of those but like are i think it's very late but why is that well are is actually particularly difficult sound to move into some ways as an adult we are very very efficient and we use different shapes of our tongue to produce our depending on when it comes in the utterance so if it is after a gee we use one tongue shape to produce the r if it's after a b we use a very different shape of the tongue most other sounds the tongue shape and relates pretty closely to the sound being produced for most vowels for example there's roughly one tongue shape that's used to produce the valley in different context our is a sound where it takes us a long time to learn it because it's a more complicated movement act but once we've learned it we're able to produce it very very easily as adults so children have difficulty especially around three years of age around this time when they're they're learning all the different sounds of the language mainly because it's a more complicated thing to articulate in terms of how variable the movements are to produce it right that at what point is motor cortex so the language part sort of mature enough at the end that children are then able to articulate clearly so around three years of age to about five years of age or are when that's the time period when children really develop a full vocabulary and it's during this period that you might start to see a developmental disorder like stuttering so stuttering usually starts around that age and about five percent of kids stutter at one point in time but most of them develop out of that so they their motor systems improve and the stuttering goes away usually within one or two years on the other hand about one percent of people and especially in boys more than girls we see that the stuttering doesn't go away so they're able to produce speech most of the time but occasionally they have trouble especially at the concepts of  words getting the word to come out and you can tell to some degree whether it's likely that a starter will stay depending on the severity and the way that the child studies initially so it's a block starter is one where the were you or your ready your articulators in the right position to produce the word you just can't continue on if a child has severe blocks that's more likely to not resolve fully  with development what this tells us is that as we develop these skills to produce all the different sounds in our language the problem is becoming more and more difficult it's involving more and more complex interactions with larger portions different portions of the brain and it's only when we get to this pretty complicated level of speech that we see things like stuttering occurring they they don't occur when we're babbling generally it's  when the problem of speech becomes a more difficult problem and when it becomes linked to language where by language i mean developing sentences and things like that that's when these disorders stuttering in particular connection it's the brain mechanisms that control the movements that have the problem in stuttering so in particular there's a portion of the brain that's called the basal ganglia which is very important for initiating movements at the right time involved in it's not entirely clear but it appears that that circuit is not quite normal and so occasionally it has problems initiating the next beach town in the utterance so that that brain region is particular important for studying we think in this case some time study seems to run in families true so do you think there is a genetic there's a component there definitely is a genetic component to study ring if you have many relatives who stutter you're more likely to start or it's not necessarily the case that you'll starter and therapies can help you resolve the stuttering over time but there's certainly a component of that that's genetic and that's probably related to the genetics that the genes that develop speech network so genes that develop connections between different parts of the brain may be slightly impaired and so the comic be in the network may not be quite right and when the network is taxed with a difficult speech task this can lead to instability in the network that's controlling the movements and that instability manifests this case so that's strictly a physical problem then can be inherited and in terms of overcoming this has there been progress in helping people to overcome certainly some progress so a lot of the having difficulty speaking and initially kids aren't self aware of their starters and so initially they don't show a lot of the same behaviors that you'll see later in stuttering as a kind of compensatory mechanism  may not notice they occasionally starter but they move through that pretty quickly but as they become more self aware of their speech then they may notice that they're making mistakes and this makes them more nervous  and generally more stressed and this can worsen the problem now under those circumstances that kasia only they may make a movement so an arm movement for example that coincides with the release of the starter and over time they can actually kind of develop those movements as part of their speech so we call those secondary behaviors that can occur so often a person who stutters might blink for example because their brain is learned that when they blink it's more likely that they'll get through this facilitates the motion is that they are and so it's interesting that yeah so they basically they have this is so sort of learning occurring when they're speaking that builds in these other behaviors and these behaviors are very wildly different between different people who stutter and so if you look at adults who stutter you you see a much wider range of behaviors than you would see  children who stutter simply because they develop so many of the secondary effects so this makes stuttering a difficult thing to study because especially in adults where most of the people we look at are adults in research certainly there are a number of people looking at children who stutter but as the further along the child gets and into adulthood they develop many secondary aspects secondary behaviors that become difficult to  dissociate from the original issue of what stopped them from producing the speech and so that's part of why stuttering has been misunderstood for hundreds of years simply because people think of it for many years they thought of it is more of a kind of psychological problem rather than an old problem partly because many there's such a wildly varying different set of behaviors that occur in different people who so in recent years recent decades people have come around to understand that in fact it's of neurological origin that there is an impairment in the speech network somewhere that leads to the stuttering behavior and really our goal is to understand that initial the core behavior of stuttering and the core problems that lead to start wearing even those are very difficult to determine because there's no single brain region that is impaired part of the brain and their brains look normal structurally only if you look at a large number of people do you start to see some patterns of differences in the brains of people who stutter compared to the brains to people who don't stutter and that those have only come in recent decades since the advent of neuro imaging in particular so speech is something that doesn't occur in animals so we don't have access to a lot of the sorts of data that we could collect in tomography or pat these are techniques that allow us to measure brain activity by measuring blood flow in the brain when a part of the brain is active that part of the brain needs more oxygen to generate electrical signals and that oxygen gets there by increasing the blood flow to that part of the brain these technologies can measure those blood flow changes and that gives us a picture of the brain activity that's occurring during speech magnetic resonance imaging also allows us to get very detailed pictures of the of the circuit the axons that connect different parts of the brain together the sizes of the different bumps are dry right in the brain and over time by comparing brains of people who stutter to the brains of in the brain activity during speech but these anomalies are not terribly consistent from individual to individual and these things highlight that stuttering is really what we would call a system level problem rather than a  problem that is located in one single part of the brain right the opposite of localized is that much larger thing and this makes it harder to understanding as we can't just go in and say oh this part of the brain must be impaired because they started let's really study that part of the brain eye in fact we see large portions of the speech network showing anomalies in the brain they're not major differences between individuals who stutter and individuals who don't but they do show up and there's some consistency in them so for example in the left hemisphere connections between the cortical regions that control speech what we see is that those connections are weaker in people who stutter but on the right hemisphere the connections tend to be stronger in people who stutter and so this is a topic of much debate right now in the literature what we think this is indicative of is that the left hemisphere system is what's really involved in generating speech normally that's the part of the brain that generates language and the language system connects to the speech motor system mainly using it left hemisphere mechanisms and the right hemisphere we think is more involved in kind of correcting errors when you hear yourself producing something improperly and so what may be happening in stuttering is that the left hemisphere isn't quite right that's why we see these connections that are strong in people who stutter and that forces the right hemisphere to do more of the job try to compensate exactly and so it's not completely successful in compensating because the left hemisphere specialized percent and so if you look at brain activity in people who stutter you see that hemisphere that doesn't mean that the right hemisphere is impaired in this case it may simply mean that the right hemisphere is forced to do more of the job during speech and that's what causes these connections to become stronger i see is this very different from what you physician and he had looked at the brains post mortem of people who had very difficult very bad speech output problems so a famous patient of his was named tan in the literature they were called because that was the only word this person could produce when you would ask them a question they would just answer ten and so he looked at the brain of ten afterwards and noticed that there was damage in what we call the left inferior frontal gyrus so it's a particular part of the cortex that's very important and it's now referred to as broken as area that area is very important for language generation another set of studies a bit later in the nineteenth a language production area of energies area is a language reception area and they're connected through connections and so those are cases of disorders that occur due to massive damage to a particular part of the brain and strokes generally this sort of damage so aphasia this is a word that we use to describe the language difficulties that arise arise from strokes generally and aphasia is our kind of out of the domain of what we would study because they're at the language level rather than the speech level but you do see disorders of speech that occur with strokes as well and so for example if you have a stroke in your what we call the premotor cortex which is a part of the cortex that's involved in planning movements especially in the case of speech that can lead to what we call a proxy of speech and this is a disorder where you know what you want to say you can generate the sentence in your head but you have difficulty pulling out the motor programs that are needed do to command the muscles to move in the way to generate the speech sounds there's no weakness in the muscles people with a proxy can usually do other things with their speech articulators their tongue they can eat and swallow and stick their tongue out on command but they don't have access to the complex complex motor programs that generate these finally time movements for speech so there difficulty is one of generating the movements not of generating language so that involves a slightly lower level of the brain than a language disorder but again it's a disorder that's primarily left lateral eyes so damage to the left premotor text will cause a proxy of speech but damage to the right disease is caused basically as the result of impairments in the part of the brain that generates a chemical called dopamine which is very important for signaling and learning especially in the and so people with parkinson's disease become impaired in their ability to initiate movements and so with speech movements what you see is that their speech is quieter than normal their movements are truncated they don't move  as far so their speech isn't quite as easy to understand and so this is an impairment that's again localized to part of the brain the basal ganglia and so it has a particular effect related to the  taxi those diseases lead to slurring of speech and problems articulating rapidly so they basically make the speech clumsy their movements are not as accurate as they need to  be and so from these sorts of disorders we learn a lot about what the different brain structures are doing in speech these differ from stuttering in that stuttering doesn't have such a clear cut or a in disorder it's as i said it's more of a system level problem that involves the entire speech network not functioning properly and so those sorts of disorders are more difficult to understand why they're happening and more difficult to treat them in terms of fixing the brain sentences with complicated syntax but those cases appear to use the frontal cortex where aphasia damage occurs but generally they can perceive speech pretty well what they can't do is generate well structured utterances so they have a difficulty in using grammar to construct sentences their sentences tend to be very sure or maybe just a word or two that they're producing in extreme cases like hand that was only one word that the person could produce but  in less extreme cases they can utter words often they'll answer with one word answers simply because the mechanisms needed to generate larger sentences are not working properly so that differs as i mentioned from in that case people can generate sentences but those sentences are often meaning less because the part of the brain that understands speech is the part that's impaired in that case so they have difficulties understanding speech that if you spoke to somebody w aphasia they may not understand what you're saying at all but their speech production mechanism appears to be relatively intact they can generate sentences usually they don't make a lot of mistakes in articulation and the mistakes they do are usually substituting properly put produced versions of the wrong sound rather than the sorts of impairments that would occur with cerebellum damage where the sounds are simply not produced properly they are trying to produce the right sound in that case they just can't generate the right muscle commands i see i mean two things that you worked on this but you have these two other specialties and that is the one is this the modeling that i'd like to computer interface too if you wish so first the diva model of ok modeling is so important ok so as i mentioned neuro imaging has allowed us to look at the functioning brain and a tremendous amount of data has built up about what brain parts are active how they're connected and so forth and so imagine the brain as a selector circuit in fact it is a wonderful circuit if we want to understand that electrical circuit we might develop a wiring diagram so the first thing we would want to know is what parts of the brain are involved in producing speech and how they are connected and neuro imaging has given us a lot of that information so we know basically which regions are involved we know which ones are connected to each other strongly and which ones are less connected to each other and so we have kind of a wiring diagram but what we don't know is what kind of processing is going on in any one of those brain regions and so if there's damage to a brain region you might imagine if it was a big electrical circuit think of a brain region as a chip in that circuit well if there was damage to one of those chips if we knew what that chip did we would have a good chance of replacing it with something so if we know what function  brain region is carrying out we will be much better position to treat disorders of that brain region so what we need is not just the chip which chips are there and how they're connected we need to know what each of those chips is doing and so this is a pretty complicated circuit and imaging really doesn't tell us directly at least what that part of the brain is doing it just says that it's active during speech so what we do to try to organize all of this information that's collected as we develop what we would call a neuro computational model and within this model it's a neural network model where the components in the model correspond to roughly two neurons in the brain  we have the components associated with particular brain regions these brain regions are connected through connections which we model and we model the synapses at the end of those accidental connections and when you learn to speak what you're really doing in your brain is you're adjusting the synapses between the different brain regions and that adjustment process allows you to transform for example on an auditory signal into a motor command that could generate that auditory signal and so what our model does is try to model with mathematical equations what's happening in these brain regions what is the neural representation being used is this representing muscle commands or is this meant representing auditory information and how those brain regions process that information and send it to other brain regions so what we do is we in effect we take a very educated guess as to what's going on in the different brain regions and we model that with equations so we have a motor cortex in our model which has a set of equations that basically transform information the way we think that brain region is working initially we don't know if these guesses are correct or not and so what we do is we can do computer simulations of the model and see what happens and we can compare that to what we see in experiments and to develop the model over now two decades what we've done is we've taken our initial model we generate a prediction from that model so we run a computer simulation and say that well i think this part of the brain is involved in transforming adda tori ever signals that when you hear yourself make a mistake into corrective motor commands well if that's the case then we if we could develop an experiment where we we perturb your auditory feedback we change what you hear while you're speaking so that it sounds and correct that should light up the part of the brain that we say is doing this and so we've done a number of experiments over the years in this way where we are essentially testing predictions of this model from the model is correct we verify its function and then we know more about we're more sure about what a particular brain region is doing but sometimes we're wrong and sometimes we see results that aren't accounted for by the model and this is very important because it allows us to improve the model so over time we would run an experiment change the model as needed to account for the data use this newer model to generate new predictions test those predictions and keep going around them this cycle of improving the model generating new predictions from the model and testing those predictions to determine more and we were very confident about many of the things their model does so now we have a model that tells us not just what's connected to what but what each of processing areas is doing and our end goal for this work has always been to better understand what happens when the brain is damaged there some sort of problem in the brain how does that lead to these disorders and ultimately how can we repair this or how quickly what kind of therapy would be best to try to overcome this sort of thing so the model and also because it's a neural network model and each of the components is associated with the brain region we can generate predictions about brain activity from this startling version of the model so if we impair comic tippity in the left hemisphere in our pretty we get more activity in the right hemisphere as you would see ins and a person who stutters so this allows us to task more complicated hypotheses about what's happening in the brain this is particularly important for a system level problem like must be very satisfying yes that is another area that you have been i think a leader in is this development of the brain computer interface for a lot in patients we have like a few minutes left can you tell us about this because this was really an exciting area and i'm not sure people know what locked in me is sure so maybe the most extreme form of speech disorder is the complete inability to  speak and with locked in syndrome this is a case where usually occurs due to either a stroke in the motor output pathways the very extreme output pathways that are involved in voluntary movements or a degenerative disease such as a  last which impairs the motor outputs math ways in those cases people can completely lose the ability to move voluntarily so the patient is completely paralyzed including lack of speech often they do maintain some residual eye movement capabilities but they're fully conscious so it's only the output part of the system that's impaired their ability to generate language in their head is still intact and so they're their language network and their speech networks are largely intact it's going on prevention they're able to understand speech they their sensory capabilities are generally good they just cannot move and as you can imagine this is a very debilitating thing to have for many reasons but probably the most important thing that happens from their perspective is the inability to communicate with people with their loved ones with their caretakers they're unable to even tell them the most basic needs so it's a very very debilitating disorder and it's a case where most of the system is intact but only the output part of the system is impaired and so our goal in the neural prosthesis lab is to try to tap into the circuitry that does work and to use the information we get from that circuitry to to bypass their impaired movement abilities and so in particular what we've worked on is a system that involves a brain computer interface with electrodes in the speech motor cortex the speech minor cortex is the part of the brain that is responsible for sending commands to the muscles and so that part of the brain is poorly understood simply because monkeys don't talk to the animals don't talk so we have very little information about what the neurons do but we do have a computational model where we've spent many years now developing an understanding of what processing is happening in the speech motor cortex and so we can use that model as a guy to help us decode the information from that part of the brain and so in a project we did in collaboration with philip kennedy who's  who developed electrodes that could be implanted in the human brain and left in there permanently we decoded these brain signals and tried to build a system that allowed the individual to try to speak while they're trying to speak our system looks at the brain activity in that part of the brain decodes that into what we think the acoustic signal is that they're trying to generate and then we send that information to a speech synthesizer which is just software on a computer that generates an acoustic signal that we can play back to the individual and we build the system so that this process occurs very quickly so within one twentieth of a second the brain activity cortex is transformed into an acoustic signal and that delays actually approximately what the delay is in the normal system so in your motor cortex about fifty milliseconds before the sound comes out your motor cortex is commanding the movements for that sound and what our goal was was to basically give the patient feedback of the sounds that they're trying to generate now we were we are pulling out a tiny tiny percentage of the information that's involved in speech we're recording from a handful of neurons dozens most of them whereas there are hundreds of millions in iran that are involved in generating language and so we're not going to be perfect at doing that but by providing the user with feedback from our attempts to decode their speech they can hear the sounds coming out because his speech network is intact largely he's able to babble with this system producing attempting to produce words and over time he gets better and better and better at producing speech output and so in this case we were able to build a system that allowed this person to produce valuables they couldn't produce entire words partly because the well mainly because the amount of information we were pouring out of the brain was so small we only had two electrodes so two wires from the brain coming out of the brain but that was still enough information to allow this person to learn to produce valves using this brain computer interface which really in our mind was a proof of concept that with more electrodes and more sophisticated technology which is occurring all the time in this area eventually will be able to restore speech through this sort of brain computer interface and it really doesn't require much hardware other than the implanted electrodes that's the most difficult part but the hardware for generating speech is just a laptop computer getting together with very different expertise areas with time we will in fact be able to provide speech capabilities to these people with locked in syndrome and and this will be revolutionary for them when i think it will because i can't imagine anything more frustrating for an individual just be totally cut off in that way really your mind is entirely intact but you say this will develop there's not a question about that then you know aspects are there all you need is time so looking at you know just given the proof of concept we were able to do with systems today have at least one hundred electrodes sometimes two hundred and this number is always increasing with that kind of information coming out of this speech areas this problem just gets easier and easier in terms of the problem of decoding what they intend to say and so in my mind at least it's clear that the basics of the technology are already in place there's a tremendous amount of development that has to occur all of the so far everything that's been done with these sorts of brain computer interfaces has been done in the laboratory and they require a lot of specialized equipment and they've required research teams that had very many areas of expertise so there are only a couple of places in the united states and in the world for that matter that that do this sort of thing largely because of the amount of resources required to do this sort of research so one big hurdle is f d a approval for implantable electrodes for example to get f d a approval to do this there are many many years of  being done first in mice then in rats then in none of the monkey and so people spend millions and millions of dollars developing the technology just getting f d a approval requires hundreds of thousands of that are capable of doing this and that in turn is is very dependent on the funding for these sorts of things on that then really if you could get the funding and stuff this would really take off this was your very exciting area where i really and there are groups that have done amazing things outside of speech so most of the effort in this area has been on our movement so getting the capability of controlling robotic arms and you some water from a very complex arm a robotic arm exactly but it's still not speech it's not like speech it's not as complicated and it's and it's still limited to robert horry demonstrations at this hour i see so much of the effort now is rather than scientific discovery it's more along the lines of engineering may get the system right yanking at work outside of the laboratory where there are interfering electrical signals and many things that we control in the laboratory we have to overcome those problems to make the system usable outside the laboratory before they'll be able to take one of these things home and use it for taking or for moving around in a wheelchair but  week people are making discoveries that are really improving the systems and and the rate of improvement is very very promising it's a matter of years not decades i think before these systems start moving into households where people can actually use them you know every day that is a wonderful promise a note to end on i nothing say that is really exciting this is in less than one generation that they've done this so and that's the worst case of the locked in is the worst case row it sort of bode well for all the other kinds of disabilities in terms of city national that there will be great progress in addition to be able to learn this much about extremely complicated aspect of the humor and brain  much for all this wonderful information thank you for being with us