Patent ID: 12214186

DETAILED DESCRIPTION

FIG.1schematically illustrates a neuroprosthetic system100for substituting a sensory modality of a mammal, in this case a patient, by electrical stimulation of a region of the cerebral cortex of the patent which corresponds to the neural modality that is to substituted.

Although the examples illustrated in the figures and described in the description below are directed to a visual neuroprosthetic system in which visual perception is substituted in the visual region of the cerebral cortex, the present invention is not particularly limited solely to the visual sensory modality. The skilled person will appreciate that the present invention is also applicable for other sensory modalities such as the auditory or somatosensory modality.

Merely as in illustration, the system100as demonstrated inFIG.1, contains several separate physical units, which are contained in separate housings. The skilled person will appreciate that some units however, may also be combined into a single housing.

The system100at least consists of a sensor150. The sensor may comprise a digital camera or cameras, photodetectors or photosensors, for operating in the visual and/or InfraRed, IR, spectrum, and mounted at glasses or the like. In the example shown inFIG.1the sensor is an image sensor which is mounted on glasses140of the patient. The sensor may however also be a separate camera that is attachable to the body or can be worn by patient in any other way. The camera may also be a separate hand held camera unit or incorporated in a portable device such as a mobile phone or portable computer device like a tablet.

The camera150is pointed in a direction that corresponds to the field of view160of the patient. As such, the camera is able to record or capture a data stream, in this example a data stream consisting of a plurality of images. The data stream contains data or images that cover at least most of the field of view160. The term field of view refers to the restriction of what is actually visible by the camera150. Since the human eye may have a larger field of view than most camera devices, the system may also comprise an eye-tracking unit, which is not shown inFIG.1. The eye-tracking unit is able to detect the position of the eye of the patient, and thus where the eye is directed to. In this way, the camera device may be focused to his direction and generate a data stream that corresponds to the focus area in the field of view of the patient.

The system100also consists of an electrode unit130. The electrode unit130, in practice, consists of multiple three-dimensional arrays130a,130b,130c. The system100consists preferably of approximately 20 of these arrays130, for example 10 implanted into the left hemisphere and 10 in the right hemisphere.

Each three-dimensional array130a,130b,130cconsists of multiple flexible, elongated electrode shafts. These electrode shafts have a shape and are made of a material that makes them suitable for intracortical implantation in the brain180of the patient. To make them suitable, these electrode shafts may be manufactured of a tissue-biocompatible material such as a polyimide or SU8. The skilled person will however appreciate that other biocompatible insulating materials in combination with electrical conductive materials may also be suitable.

The electrode shafts are distributed in such a three-dimensional manner over the array that they are able to make electrical contact with a large three-dimensional region of the cerebral cortex, i.e. with most of, or approximately all of the visual region of the cerebral cortex of the patient. This means, that not only in two directions the spacing dl inFIG.2between the individual electrode shafts is small, e.g. up 0.4 to 1 mm, but that the electrodes shafts are also longer then electrodes of known electrode arrays, with a length up to 20 or even 40 mm. Preferably, the length of the electrode shafts are in the range of at least 2 to 5 mm till 20 or 40 mm. More preferably, the length is in the range of 5 till 30 mm, even more preferably in the range of 7 till 25 mm, and most preferably in the range of 15 till 20 mm. This way, the electrodes are able to extend beyond a single, or cover even plural folded gyrus in the cerebral cortex.

Each electrode shaft consists of multiple electrical contacts. These contacts may be controlled individually such that some of the contacts of a single electrode shaft are electrically driven or activated by applying a stimulation signal, and others are not operated. Each contact may also be operated for either signal stimulation or signal recording. In an operational mode, the contacts are operated for signal stimulation only. In a configuration mode, the contacts may however also be controlled to operate in a signal recording modus such that neural activity or neural response to the stimulation signal may be recorded accordingly. In an example, each contact may be arranged such that is can operate both in the stimulation and recording modus, and in an alternative example, contacts may be dedicated for either signal stimulation or signal recording.

In order for the electrical contacts to process the signal for neural stimulation or neural recording, the system100consists of a driving unit and a recording unit. These units may be contained in separate housings, at separate locations in or on the patient, but are preferably contained in a transducer device which may be close to or integrated with the shaft or positioned directly under the scalp on the skull, for example in or near the layer of loose connective tissue of the scalp.

In order for flexible electrode shafts to successfully penetrate the surface tissue of the cerebral cortex, the system100also consists of a rigid electrode support structure. To this end, support structure consists of an array of rigid or stiff insert shafts. The flexible elongated electrode shafts are attached or fixed to the insert shafts of the support structure upon intracortical implantation in the cerebral cortex. Once inserted, the insert shafts disengage such that the insert shafts can be retracted, whereas the electrode shafts remain in place in the cerebral cortex. Attaching and disengaging can be achieved in different ways. For example, the inserts and respective electrodes may be temporarily attached by a brain fluid dissolvable adhesive such as PEG. This example may well work with flat ribbon shaped electrodes. In an alternative, non-flat shape of the ribbon, e.g. with a round or circular cross sectional shape, the tip of the electrode may be provided with an engaging element such as a hole that is to be penetrated by the tip of the insert shaft.

The electronics of the driving unit, to electrically drive the electrode unit for stimulation of the subset of locations, as well as the recording unit, to obtain neural recording through the electrode unit in the region of the cerebral cortex, can be packaged inside an implantable transducer casing120and shielded from the tissue by the material of the casing, which is preferably made of titanium. In case the electronics or electronic circuitry may reside on the electrode unit or more particular, on the three-dimensional arrays, they are embedded in a tissue compatible packaging material such as LCP or polyimide.

The implant120also consists of a switching unit. The switching unit consists of electronic circuitry which channels stimulation signals from the driving unit to the actual electrical contacts located in the cerebral cortex. What that means is, that the driving unit may be equipped with circuitry to generate a plural stimulation signal, i.e. a stimulation pulse of a particular waveform such as a sine wave form, a square wave form, a rectangular wave form, a triangular wave form, a sawtooth wave form, a pulse wave form or any combination of wave forms. The generated signals may not only differ in shape but also in amplitude and can be channelled to different individual or subsets of electrical contacts. This way some groups or subsets of contacts may not be activated at all, others with a first stimulation signal, yet another group with a second, different stimulation signal, and so on. To this end, the switching unit is arranged to achieve electrical channelling of the different signal generators and the subsets of electrical contacts.

To control the driving unit, the recording unit, and thereby the electrodes of the electrode unit, the system is also provided with a processing unit170. The processing unit is preferably a handheld device, since that will increase independence and quality of life of the patient. The processing unit or device could be a general-purpose hand-held device, such as the mobile phone as shown by way of example inFIG.1, which is programmed to operate as processing unit170for the neuroprosthetic system200according to the invention. The processing unit or device is however more preferably a dedicated device but with dimensions that are preferably similar to those of a mobile phone.

The processing unit170may communicate with the transducer arrangement or transducer device120by use of a wireless communication unit110. This wireless communication unit110not only provides wireless technology to control high-channel-count high-density cortical implantable arrays130a-130c, but also provides a wireless power transfer interface that can send the signals of the recording unit. This can be achieved for example through capacitive, or magnetodynamic, but preferably through inductive coupling between the wireless unit110and the transducer implant120.

By having both wireless power and data communication between the wireless unit110and the transducer implant120, the risk of infection that is associated with skin-penetrating cables and connectors is reduced.

With the processing unit170, the system100is able to receive and analyse the data stream sensed by the sensor. In the example shown inFIG.1, these are images of the data stream captured by the camera150. These images have to be processed in such a way that the correct electrical contacts are powered or provided with a stimulation signal to evoke a phosphene, i.e. a phenomenon characterized by the experience of seeing light or a specific light dot, at a phosphene location which corresponds to the actual position of the electrode shaft or more precise, the electrical contact of that particular electrode shaft. The subset of phosphene locations should correspond to one or more objects that are captured by the camera and thus in the scene in the field of view160of the patient.

In order to make such a translation between actual captured image data and the subset of phosphene location, which together form a phosphene pattern or patterns, the processing unit170employs semantic segmentation.

With semantic segmentation, the image is processed such that each pixel is assigned to a particular object class. The object is in a way labelled and contains a set of pixels that enclose or outline the object.

Once the processing unit170processed the image data through semantic segmentation, an outline, contour, label, shape or other primitive object is determined which can then be used to as phosphene or stimulation pattern to apply stimulation signals to the appropriate subset of phosphene location. Once these stimulation signals are applied, the invoked phosphenes at these phosphene locations will substitute the visual perception of the patient such that he or she will recognize the actual object as a car or whatever object is in the field of view160of the patient

InFIG.2an assembly130is shown of a three-dimensional array133of flexible, elongated electrodes or electrode shafts with a rigid electrode support structure132for guiding the flexible elongated electrodes135upon intracortical implantation in a region of the cerebral cortex of a patient.

The support structure132enables simultaneously guiding of the flexible electrode shafts135of the array133upon implementation and once implemented the insert shafts134may detach from the electrodes135and the support structure132can be retracted. All insert shafts134, which preferably are manufactured from tungsten, are mutually connected and connected to a metal rod which makes simultaneous, concurrent insertion and retraction possible.

As clearly indicated inFIG.2, both the number and the distribution of the insert shafts134of the support structure correspond to a number and distribution of through holes in the base plate131of the array133, such that each individual insert shaft134is guided by a single respective though hole, thereby ensuring that all electrode shafts or electrode ribbons135arrive at the correct position and at the correct depth in the visual region of the cerebral cortex. This could preferably be primary cerebral region V1, but also mid-ranged or high regions such as V2, V3or V4.

All individual electrodes135preferably have the same length of approximately 0.2-4 cm. The spacing between the individual electrodes is preferably approximately between 0.1 and 2 mm.

Each electrode shaft135contains multiple electrical contacts136which are distributed preferably in the longitudinal direction of the shaft. The contact136may however also have a lateral distribution pattern or could be distributed in both a lateral and longitudinal direction of the shaft.

Each electrical contact136is electrically wired through an electrical lead. The plurality of leads138are electrically connected to the transducer unit120as shown inFIG.1.

As illustrated in the two detailed views inFIG.2there are several ways in which the electrode shafts135can engage with the insert shafts134of the support structure132. In the example shown in the left detailed view the rigid insert shafts134have a rectangular cross section with a width that is identical or least approximately identical to the electrode shaft135. The thickness in that particular example is in the range of 10-50 μm. In this example, it is preferred to use a dissolvable material such as PEG.

In the example shown in the right detailed view the rigid insert shafts134may or may not have a circular cross-section with a diameter of approximately 10-50 μm. In this example, the insert shafts134have a tapered or smaller diameter tip or free end to engage with a hole in the tip or free end of the electrode shaft135.

Although the right detailed view might suggest that the electrical contacts are located on the insert shafts134, they are actually located on the electrode shaft135. The electrode shafts134can be sleeve shaped to receive the insert shafts134, but these are more preferably flat opaque ribbon shaped electrodes with a through hole or other receiving element to receive the tip of the insert shaft for ease of surgical insertion into the brain.

In this example, there is no need for a dissolvable material such as PEG. Once the electrode shafts135are in position, the rigid insert shafts can easily be retracted, leaving the electrodes135in place.

FIG.3shows a more detailed view of the base plate131of the electrode array133with a two-dimensional distribution pattern of the through holes137.FIG.3only shows one single electrode and insert shaft. The electrode unit and support structure will however contain a similar number of electrodes and insert shafts that correspond to the amount of through holes137.

InFIG.4the electrode array133also contains additional circuitry139In one example of the invention all electrical contacts of each electrode shaft of each array are electrically wired to the transducer120. In the example shown inFIG.4the array however comprises a multiplexer to multiplex and de-multiplex the signals of the large number of electrical wires138to one or more likely a lower number of wires. Since per insert, the number of wires could be as high as 50,000 the use of several multiplexers is desirable. The multiplexer may however be located either on the array133, as shown inFIG.4, or may also be located in the transducer120.

InFIG.5each array130a-130dcomprises a multiplexer139. Although it might be preferable to have the multiplexers as near as possible to the huge number of wires, thus on or near the array, this would reduce the number and length of the leads or wires but could also produce too much heat that may affect the brain. In that case, it is preferred to have all wires run to the transducer and have them connected to multiplexers within the transducer casing120.

Although the driving unit and the recording unit could also reside in each array130, they are preferably located within the transducer unit120. In the example shown inFIG.4however at least some electronic circuitry139is located on the array itself. In this example the electronic circuitry139may be arranged to multiplex the plurality of signals that are obtained in the recording modus through the large number of electrodes, but is preferably also arranged as switching unit to perform the channelling of the signal generators. Thus, to connect groups of electrical contacts with one of the signal generators.

FIG.6shows a side view of the brain180of the patient. In the example shown inFIG.6, several arrays130are implanted the visual region of the cerebral cortex. Most of the arrays130a,130betc. are implanted in area V1. However, at least one of the arrays130cis however implanted to achieve at least partly or fully coverage of area V4, this in order to perform the recording and stimulation in such a higher visual cortex region.

As indicated above, this visual region is only an example in which the arrays according to the present disclosure can be implanted. Other sensory modality regions of the cerebral cortex may be implanted with similar arrays according to the present disclosure. These regions, such as the auditory regions or auditory cortex may for example be implanted with similar arrays which however have electrode shafts with different dimensions adapted to that particular region.

In view of the above, the skilled person will appreciate that the dimensions such as length of the electrode shafts as indicated relate in general to the visual cortex. Although these dimensions may very well also be suitable for other regions such as the auditory, tactile, somatosensory or olfactory functionality. Some or all of these regions may however require different dimensions.

FIG.7shows several steps a-h of the translation of the images of the data stream feed into a phosphene pattern by making use of semantic segmentation. In step a, the camera images are electrically imparted to the retina or cortex using an image transformation algorithm in a portable device. In step b, an example is shown of an outdoor scene that is within the field of view of the patient shown in figure a. Imagine that the patient approaches the car. In step c, the locations of 64 phosphenes are shown that the patient can perceive when retinal electrodes are stimulated. Step d, shows the potential locations of 1,000 phosphenes that can be achieved by the stimulation of cortical area V1. In step e, a limitation of known prosthetic devices is shown which are not able to distinguish between relevant and irrelevant contours (such as foreground and background information in the image). Semantic segmentation algorithms however delineate the pixels in the image that belong to the car. The result of semantic segmentation as shown in step f, can be used as template to determine the stimulation patterns as shown in step g. If semantic segmentation is used to determine the pattern of activated phosphenes (subset of those in step c), perception of the approximate location and shape of the car is expected, resulting in a large improvement of vision through the prosthetic device. In step g, a cortical prosthesis according to an aspect of the invention can give rise to an even higher resolution percept.

Other variations to the disclosed examples can be understood and effected by those skilled in the art in practicing the claimed disclosure, from a study of the drawings, the disclosure and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not construed as limiting scope thereof. Similar reference signs denote similar or equivalent functionality.

The present disclosure is not limited to the examples as disclosed above, and can be modified and enhanced by those skilled in the art beyond the scope of the present disclosure as disclosed in the appended claims without having to apply inventive skills and for use in any data communication, data exchange and data processing environment, for example for use of the neuroprosthetic system for substituting auditory perception.