Patent Description:
The present disclosure relates generally to a sleep apnea system, and more specifically, to an intelligent setup and recommendation system for sleep apnea patients.

Certain disorders may be characterized by particular events, such as apneas, hypopneas, and hyperpneas. Obstructive Sleep Apnea (OSA), a form of Sleep Disordered Breathing (SDB), is characterized by events including occlusion or obstruction of the upper air passage during sleep. It results from a combination of an abnormally small upper airway and the normal loss of muscle tone in the region of the tongue, soft palate and posterior oropharyngeal wall during sleep. The condition causes the affected patient to stop breathing for periods typically of <NUM> to <NUM> seconds in duration, sometimes <NUM> to <NUM> times per night. It often causes excessive daytime somnolence, and it may cause cardiovascular disease and brain damage. The syndrome is a common disorder, particularly in middle aged overweight males, although a person affected may have no awareness of the problem.

Other sleep related disorders include Cheyne-Stokes Respiration (CSR), Obesity Hyperventilation Syndrome (OHS) and Chronic Obstructive Pulmonary Disease (COPD). COPD encompasses any of a group of lower airway diseases that have certain characteristics in common. These include increased resistance to air movement, extended expiratory phase of respiration, and loss of the normal elasticity of the lung. Examples of COPD are emphysema and chronic bronchitis. COPD is caused by chronic tobacco smoking (primary risk factor), occupational exposures, air pollution and genetic factors.

Continuous Positive Airway Pressure (CPAP) therapy has been used to treat Obstructive Sleep Apnea (OSA). Application of continuous positive airway pressure acts as a pneumatic splint and may prevent upper airway occlusion by pushing the soft palate and tongue forward and away from the posterior oropharyngeal wall.

Non-invasive ventilation (NIV) provides ventilatory support to a patient through the upper airways to assist the patient in taking a full breath and/or maintain adequate oxygen levels in the body by doing some or all of the work of breathing. The ventilatory support is provided via a patient interface. NIV has been used to treat CSR, OHS, COPD, and Chest Wall disorders. In some forms, the comfort and effectiveness of these therapies may be improved. Invasive ventilation (IV) provides ventilatory support to patients that are no longer able to effectively breathe themselves and may be provided using a tracheostomy tube.

A treatment system may comprise a Respiratory Pressure Therapy Device (RPT device), an air circuit, a humidifier, a patient interface, and data management. A patient interface may be used to interface respiratory equipment to its wearer, for example by providing a flow of air to an entrance to the airways. The flow of air may be provided via a mask to the nose and/or mouth, a tube to the mouth or a tracheostomy tube to the trachea of a patient. Depending upon the therapy to be applied, the patient interface may form a seal, e.g., with a region of the patient's face, to facilitate the delivery of gas at a pressure at sufficient variance with ambient pressure to effect therapy, e.g., at a positive pressure of about <NUM> H20 relative to ambient pressure. For other forms of therapy, such as the delivery of oxygen, the patient interface may not include a seal sufficient to facilitate delivery to the airways of a supply of gas at a positive pressure of about <NUM> H20. Treatment of respiratory ailments by such therapy may be voluntary, and hence patients may elect not to comply with therapy if they find devices used to provide such therapy uncomfortable, difficult to use, expensive and/or aesthetically unappealing.

CPAP therapy is highly effective to treat certain respiratory disorders, provided patients comply with therapy. Obtaining a proper patient interface and properly setting up the CPAP machine allows a patient to engage in positive pressure therapy. Patients currently must rely on paper instructions provided by the device manufacturers for setting up their device. An improper set up or configuration often frustrates patients and thus causes the improper operation of the sleep apnea device. Thus, a sleep apnea device that is not properly configured for a particular patient may result in ineffective therapy.

There is a need for a system that allows for more efficient and active setup of a sleep apnea device. There is also a need to use a mobile computing device with an augmented reality interface to assist a user to setup of a sleep apnea device. There is also a need for a patient assistance device that may evaluate the operation of a sleep apnea device. Patent publication <CIT> (<NUM>-<NUM>-<NUM>) discloses a patient sleep therapy self management tool.

The disclosed system provides an adaptable system to size masks for use with an RPT device for better compliance for individual patients. The system collects facial data from a primary patient and also RPT use and other data from a larger population of patients to assist in the selection of an optimal mask for the primary patient.

One disclosed example is a system to provide assistance to a patient for using a respiratory therapy device and a mask for treatment of respiratory ailments. The system includes an equipment database storing data relating to a plurality of device types and a plurality of mask types. A device recognition module is operable to identify the type of the respiratory therapy device from an image of the respiratory therapy device captured by a client computing device in comparison with the data relating to the plurality of device types. A mask recognition module is operable to identify the type of the mask from an image of the mask captured by the client computing device in comparison with the data relating to the plurality of mask types. A media database includes media relating to assistance information relating to at least one of a mask type or a device type. A management server is operable to send media relating to assistance information for the identified type of mask or identified type of device to the client computing device.

Another disclosed example is A method for providing automated assistance to a patient using a respiratory therapy device connected to a mask. An image of the respiratory therapy device and an image of the mask are captured by a client computing device. The type of the respiratory therapy device is identified from an image of the respiratory therapy device captured by the client computing device in comparison with the data relating to the plurality of device types in an equipment database via a device recognition module. The type of the mask is identified from an image of the mask captured by the client computing device in comparison with the data relating to the plurality of mask types in the equipment database via a mask recognition module. Media relating to assistance information for the identified type of mask or identified type of device is sent to the client computing device via a management server.

The above summary is not intended to represent each embodiment or every aspect of the present disclosure. Rather, the foregoing summary merely provides an example of some of the novel aspects and features set forth herein. The above features and advantages, and other features and advantages of the present disclosure, will be readily apparent from the following detailed description of representative embodiments and modes for carrying out the present invention, when taken in connection with the accompanying drawings and the appended claims.

The disclosure will be better understood from the following description of exemplary embodiments together with reference to the accompanying drawings, in which:.

The present disclosure is susceptible to various modifications and alternative forms. Some representative embodiments have been shown by way of example in the drawings and will be described in detail herein.

The present inventions can be embodied in many different forms. Representative embodiments are shown in the drawings, and will herein be described in detail. The present disclosure is an example or illustration of the principles of the present disclosure, and is not intended to limit the broad aspects of the disclosure to the embodiments illustrated. To that extent, elements and limitations that are disclosed, for example, in the Abstract, Summary, and Detailed Description sections, but not explicitly set forth in the claims, should not be incorporated into the claims, singly or collectively, by implication, inference, or otherwise. For purposes of the present detailed description, unless specifically disclaimed, the singular includes the plural and vice versa; and the word "including" means "including without limitation. " Moreover, words of approximation, such as "about," "almost," "substantially," "approximately," and the like, can be used herein to mean "at," "near," or "nearly at," or "within <NUM>-<NUM>% of," or "within acceptable manufacturing tolerances," or any logical combination thereof, for example.

The present disclosure relates to an automated assistant that assists a user or a respiratory therapy device. The assistant allows a user to confirm both the type of device and mask. The assist includes.

<FIG> illustrates an example computing environment for a sleep apnea system <NUM>. The sleep apnea system <NUM> comprises a management server <NUM>, a client computing device <NUM>, a sleep apnea device <NUM>, and a mask <NUM>. Although only a single client device <NUM>, sleep apnea device <NUM>, and mask <NUM> are illustrated, an embodiment of the computing environment <NUM> may have hundreds or thousands of client devices <NUM>, sleep apnea devices <NUM>, and masks <NUM> managed by the management server <NUM>.

The sleep apnea device <NUM> may comprise, for example, a continuous positive airway pressure (CPAP) device, an automatic positive airway pressure (APAP) device, a bilevel positive airway pressure device (BiPAP) or other device for treating sleep apnea. Such devices are generally referenced as respiratory therapy devices (RPT). The sleep apnea device <NUM> generally comprises a pressurized ventilator that connects to a face mask <NUM> via a hose. The sleep apnea device <NUM> applies mild air pressure through the hose and mask <NUM> to keep the patient's airways open during sleep. In a CPAP device, the air pressure flows to the mask <NUM> at a constant pressure. In a BiPAP device, the air pressure switches between two different pressure levels for inhalation and exhalation respectively. In an APAP device, the sleep apnea device <NUM> senses changes in breathing and adjusts the air pressure to an appropriate level based on the breathing pattern.

The sleep apnea device <NUM> may include a wireless or wired communication interface to connect to the management server <NUM> via the network <NUM>. For example, the sleep apnea device <NUM> may communicate with the network <NUM> via a WiFi connection, a cellular connection, an Ethernet connection, or other connection. The sleep apnea device <NUM> may include sensors that monitor various data associated with the patient's usage and sends the data to the management server <NUM>. For example, the sleep apnea device <NUM> may sense the patient's breathing rate, respiratory flow, and overall usage patterns (e.g., how often and for how long the patient uses the device <NUM>), and reports this information to the management server <NUM>.

The mask <NUM> couples to the sleep apnea device <NUM> via a hose and receives the pressurized air flow generated by the sleep apnea device <NUM>. The mask <NUM> is designed to be worn around the patient's mouth, nose, or both. When properly sized and worn, the mask <NUM> provides an airtight seal around the patient's mouth, nose, or both to enable the pressurized air to flow into the patient's breathing cavities. The mask <NUM> may furthermore include one or more straps that wrap around the patient's head to secure the mask <NUM> in place. The straps may be adjustable to provide a proper fit and seal for a given patient.

The client computing device <NUM> comprises a network-enabled computing device such as a computer, a mobile device, or tablet that executes a client application <NUM>. The client application <NUM> provides a user interface on a display that enables the patient to provide various information to the management server <NUM> and view various information from the management server <NUM> relating to the patient's treatment for respiratory ailments using the sleep apnea device <NUM>. For example, the client application <NUM> may enable the patient to set up a patient profile that includes various characteristics of the patient (e.g., gender, weight, height, sleep habits, etc.) and link the profile to the sleep apnea device <NUM>. The client application <NUM> may furthermore provide various interfaces on the display to assist the patient through initial unboxing, setup, and usage of the sleep apnea device <NUM>. Additionally, the client application <NUM> may enable the patient to access various usage data associated with the patient's usage of the sleep apnea device <NUM>. Furthermore, the client application <NUM> may provide alerts and recommendations from the management server <NUM> relating to the patient's treatment such as, for example, alerting the patient when it is time to replace the face mask <NUM>, alerting the patient when the patient is not following the recommended usage amount, alerting the patient when a malfunction is detected with the sleep apnea device <NUM> or mask <NUM>, etc. The client application <NUM> may also enable the patient to provide feedback to the management server <NUM> relating to the patient's experience with the treatment. While not shown in <FIG>, client device <NUM> may be configured to communicate directly with sleep apnea device <NUM> via a network connection such as, for example, Bluetooth, WiFi, cellular, and/or other communication mechanisms.

The management server <NUM> comprises a computer or set of computers providing various management and control functions associated with one or more sleep apnea devices <NUM>. For example, the management server <NUM> may obtain data associated with patients, their devices <NUM>, and masks <NUM>, and generate content to assist the patient in setting up and using device <NUM> and mask <NUM>. Furthermore, the management server <NUM> may collect various usage data associated with sleep apnea treatment in order to generate recommendations to patients that are tailored to their particular characteristics.

<FIG> illustrates an example embodiment of a management server <NUM>. The management server <NUM> comprises an application server <NUM>, a device recognition module <NUM>, a mask recognition module <NUM>, a mask positioning module <NUM>, a mask leak detection module <NUM>, a mask recommendation module <NUM>, a patient profile database <NUM>, a media database <NUM>, and an equipment database <NUM>. In alternative embodiments, the management server <NUM> may include additional or different modules. In an embodiment, each of the modules may include computer-executable instructions stored to a non-transitory storage medium that when executed by a processor cause the processor to carry out functions attributed to the modules as described herein.

The application server <NUM> provides an interface between the management server <NUM> and the client application <NUM>. The application server <NUM> exchange data and control information with the client application <NUM> to cause the client application <NUM> to carry out various functions described herein. For example, the application server <NUM> may obtain profile information from a patient when the patient first opens the client application <NUM> and may store the profile information to the patient profile database <NUM>. The application server <NUM> may furthermore cause the client application <NUM> to present various user interface screens to guide the user through an onboarding and setup process. Further still, the application server <NUM> may obtain various sensor data, usage information, and surveyed answers associated with the patient throughout the patient's treatment. The application server <NUM> may also provide the patient with access to various information to assist the patient with carrying out treatment for respiratory ailments such as sleep apnea.

The device recognition module <NUM> comprises a machine-learning model that enables automatic detection of a particular model of sleep apnea device <NUM> based on one or more images of the sleep apnea device <NUM> captured by the patient via the client device <NUM>. In an embodiment, the machine-learning model is based on learned correlations between image features and device models from different device type data stored in the equipment database <NUM>. For example, to generate the machine-learning model, a large number of images may be captured of a device <NUM> of each different model from a variety of angles, lighting conditions, and camera specifications. Features may be extracted from the images, and a learning algorithm may learn which features most strongly correlate to the device model. During operation, the device recognition module <NUM> may receive an image of a sleep apnea device <NUM> (or features extracted from an image) and predict the device model based on the machine-learning model. An example user interface associated with the device recognition module <NUM> is described in further detail below with respect to <FIG>.

The mask recognition module <NUM> comprises a machine-learning model that enables automatic detection of a particular model of a mask <NUM> based on one or more images of the mask <NUM> captured by the patient via the client device <NUM>. In an embodiment, the machine-learning model is based on learned correlations between image features and mask models. For example, to generate the machine-learning model, a large number of images may be captured of a mask <NUM> of each different model from a variety of angles, lighting conditions, and camera specifications. Features may be extracted from the images, and a learning algorithm may learn which features most strongly correlate to the mask model. According to some embodiments, the learning algorithm may use mask CAD files stored in the equipment database <NUM> as inputs in analyzing features and correlating such features to a particular mask model. During operation, the mask recognition module <NUM> may receive an image of a mask <NUM> (or features extracted from an image) and predict the mask model based on the machine-learning model. An example user interface associated with the mask recognition module <NUM> is described in further detail below with respect to <FIG>. A more detailed description of the mask recognition module <NUM> is furthermore described below with respect to <FIG>.

The mask positioning module <NUM> provides an augmented reality interface to assist a patient with properly placing the mask <NUM> on the patient's face. The mask positioning module <NUM> receives an input video stream of a portrait of the patient that may be captured via a camera of the client device <NUM>. The mask positioning module <NUM> performs a facial analysis to identify and track locations of particular facial landmarks. The mask positioning module <NUM> then overlays a mask placement image (e.g., an image of a mask <NUM> or an outline of a mask <NUM>) on the received image frames at the appropriate position aligned to the detected facial landmarks. The image frames are sent to the client application <NUM> as an augmented reality view of the patient's face with the overlaid mask placement image. Based on the augmented reality view, the patient may align the mask <NUM> to the mask placement image indicative of the proper mask placement. An example user interface associated with the mask positioning module <NUM> is described in further detail below with respect to FIGs.

The mask leak detection module <NUM> detects whether a mask <NUM> is properly sealed around a patient's face based on captured audio of the patient's breathing while the sleep apnea device is delivering therapy/pressure to the patient. For example, after the patient places the mask <NUM>, the client device <NUM> may be configured to capture audio (using a microphone of the client device <NUM>) of the patient's breathing. The captured audio (or features derived from the audio) may be sent to the management server <NUM>. The mask leak detection module <NUM> detects the presence or absence of audio features indicative of a leak. In an embodiment, the mask leak detection module <NUM> may apply a machine-learning model that learns correlations between audio features extracted from audio of breathing when a leak is present. The mask positioning module <NUM> may also be configured to capture information from the client device <NUM> on how the client device <NUM> is positioned. Such information may then be used to aid in determining the location on the patient's face that leak is occurring. The leak detection result may be provided to the client application <NUM>. An example user interface associated with the mask leak detection module <NUM> is described in further detail below with respect to <FIG>.

The mask recommendation module <NUM> generates a patient-specific recommendation for a type and size of mask <NUM> based an image of the patient and various characteristics of the patient. For example, the mask recommendation module <NUM> may obtain the image of the patient (or features derived from the image) from the client device <NUM> and perform a facial analysis, which may be based in part on machine-learning, to identify a mask size that is predicted to suit the patient's face and a type of mask <NUM> that is predicted to best fit the patient's face. In an embodiment, the mask recommendation module <NUM> may apply a machined-learning model to learn correlations between facial features of the patient that can be extracted from the image and the mask size and type. An example of a mask recommendation module <NUM> is described in further detail below with respect to <FIG>.

The patient profile database <NUM> stores various information about a patient. For example, the patient profile database <NUM> may store physical information such as age, gender, height, weight, etc., medical history, and various preferences. The patient profile database <NUM> may be updated during ongoing therapy to store various usage data associated with the patient's treatment and patient-provided feedback.

The media database <NUM> stores various images, videos, animations, or other media helpful to assisting the patient with setting up and configuring the sleep apnea system <NUM>. For example, the media database <NUM> may include images or videos showing a patient how to set up, turn on, and use the sleep apnea device <NUM>, how to size, adjust, and place the mask <NUM>, or other help content to reduce the patient's learning curve in beginning sleep apnea therapy.

<FIG> shows an exploded view of the components of an example RPT device such as the sleep apnea device <NUM> in accordance with one aspect of the present technology comprises mechanical, pneumatic, and/or electrical components and is configured to execute one or more algorithms, such as any of the methods, in whole or in part, described herein. <FIG> shows a block diagram of the example RPT device <NUM>. <FIG> shows a block diagram of the electrical control components of the example RPT device <NUM>. The directions of upstream and downstream are indicated with reference to the blower and the patient interface. The blower is defined to be upstream of the patient interface and the patient interface is defined to be downstream of the blower, regardless of the actual flow direction at any particular moment. Items which are located within the pneumatic path between the blower and the patient interface are downstream of the blower and upstream of the patient interface. The RPT device <NUM> may be configured to generate a flow of air for delivery to a patient's airways, such as to treat one or more of the respiratory conditions.

The RPT device <NUM> may have an external housing <NUM>, formed in two parts, an upper portion <NUM> and a lower portion <NUM>. Furthermore, the external housing <NUM> may include one or more panel(s) <NUM>. The RPT device <NUM> comprises a chassis <NUM> that supports one or more internal components of the RPT device <NUM>. The RPT device <NUM> may include a handle <NUM>.

The pneumatic path of the RPT device <NUM> may comprise one or more air path items, e.g., an inlet air filter <NUM>, an inlet muffler <NUM>, a pressure generator <NUM> capable of supplying air at positive pressure (e.g., a blower <NUM>), an outlet muffler <NUM> and one or more transducers <NUM>, such as a pressure sensor <NUM>, a flow rate sensor <NUM>, and a motor speed sensor <NUM>.

One or more of the air path items may be located within a removable unitary structure which will be referred to as a pneumatic block <NUM>. The pneumatic block <NUM> may be located within the external housing <NUM>. In one form a pneumatic block <NUM> is supported by, or formed as part of the chassis <NUM>.

The RPT device <NUM> may have an electrical power supply <NUM>, one or more input devices <NUM>, a central controller <NUM>, a pressure generator <NUM>, a data communication interface <NUM>, and one or more output devices <NUM>. A separate controller may be provided for the therapy device. Electrical components <NUM> may be mounted on a single Printed Circuit Board Assembly (PCBA) <NUM>. In an alternative form, the RPT device <NUM> may include more than one PCBA <NUM>. Other components such as the one or more protection circuits <NUM>, transducers <NUM>, the data communication interface <NUM>, and storage devices may also be mounted on the PCBA <NUM>.

An RPT device may comprise one or more of the following components in an integral unit. In an alternative form, one or more of the following components may be located as respective separate units.

An RPT device in accordance with one form of the present technology may include an air filter <NUM>, or a plurality of air filters <NUM>. In one form, an inlet air filter <NUM> is located at the beginning of the pneumatic path upstream of a pressure generator <NUM>. In one form, an outlet air filter <NUM>, for example an antibacterial filter, is located between an outlet of the pneumatic block <NUM> and a patient interface such as the mask <NUM> in <FIG>.

An RPT device in accordance with one form of the present technology may include a muffler <NUM>, or a plurality of mufflers <NUM>. In one form of the present technology, an inlet muffler <NUM> is located in the pneumatic path upstream of a pressure generator <NUM>. In one form of the present technology, an outlet muffler <NUM> is located in the pneumatic path between the pressure generator <NUM> and a patient interface such as the mask <NUM> in <FIG>.

In one form of the present technology, a pressure generator <NUM> for producing a flow, or a supply, of air at positive pressure is a controllable blower <NUM>. For example, the blower <NUM> may include a brushless DC motor <NUM> with one or more impellers. The impellers may be located in a volute. The blower may be capable of delivering a supply of air, for example at a rate of up to about <NUM> litres/minute, at a positive pressure in a range from about <NUM> H2O to about <NUM> H2O, or in other forms up to about <NUM> H2O. The blower may be as described in any one of the following patents or <CIT>; <CIT>; <CIT>; and <CIT>.

The pressure generator <NUM> is under the control of the therapy device controller <NUM>. In other forms, a pressure generator <NUM> may be a piston-driven pump, a pressure regulator connected to a high pressure source (e.g. compressed air reservoir), or a bellows.

An air circuit <NUM> in accordance with an aspect of the present technology is a conduit or a tube constructed and arranged to allow, in use, a pressurized flow of air to travel between two components such as a humidifier <NUM> and the patient interface <NUM>. In particular, the air circuit <NUM> may be in fluid communication with the outlet of the humidifier <NUM> and the plenum chamber of the patient interface <NUM>.

In one form of the present technology, an anti-spill back valve <NUM> is located between the humidifier <NUM> and the pneumatic block <NUM>. The anti-spill back valve is constructed and arranged to reduce the risk that water will flow upstream from the humidifier <NUM>, for example to the motor <NUM>.

A power supply <NUM> may be located internal or external of the external housing <NUM> of the RPT device <NUM>. In one form of the present technology, power supply <NUM> provides electrical power to the RPT device <NUM> only. In another form of the present technology, power supply <NUM> provides electrical power to both RPT device <NUM> and humidifier <NUM>.

An RT system may comprise one or more transducers (sensors) <NUM> configured to measure one or more of any number of parameters in relation to an RT system, its patient, and/or its environment. A transducer may be configured to produce an output signal representative of the one or more parameters that the transducer is configured to measure.

The output signal may be one or more of an electrical signal, a magnetic signal, a mechanical signal, a visual signal, an optical signal, a sound signal, or any number of others which are known in the art.

A transducer may be integrated with another component of an RPT system, where one exemplary arrangement would be the transducer being internal of an RPT device. A transducer may be substantially a 'standalone' component of an RT system, an exemplary arrangement of which would be the transducer being external to the RPT device.

A transducer may be configured to communicate its output signal to one or more components of an RT system, such as an RPT device, a local external device, or a remote external device. External transducers may be for example located on a patient interface, or in an external computing device, such as a smartphone. External transducers may be located for example on or form part of the air circuit, e.g., the patient interface.

The one or more transducers <NUM> may be constructed and arranged to generate signals representing properties of air such as a flow rate, a pressure or a temperature. The air may be a flow of air from the RPT device <NUM> to a patient, a flow of air from the patient to the atmosphere, ambient air or any others. The signals may be representative of properties of the flow of air at a particular point, such as the flow of air in the pneumatic path between the RPT device <NUM> and the patient. In one form of the present technology, one or more transducers <NUM> are located in a pneumatic path of the RPT device <NUM>, such as downstream of the humidifier <NUM>.

In accordance with one aspect of the present technology, the one or more transducers <NUM> comprises a pressure sensor located in fluid communication with the pneumatic path. An example of a suitable pressure sensor is a transducer from the HONEYWELL ASDX series. An alternative suitable pressure sensor is a transducer from the NPA Series from GENERAL ELECTRIC. In one implementation, the pressure sensor is located in the air circuit <NUM> adjacent the outlet of the humidifier <NUM>.

A microphone pressure sensor <NUM> is configured to generate a sound signal representing the variation of pressure within the air circuit <NUM>. The sound signal from the microphone <NUM> may be received by the central controller <NUM> for acoustic processing and analysis as configured by one or more of the algorithms described below. The microphone <NUM> may be directly exposed to the airpath for greater sensitivity to sound, or may be encapsulated behind a thin layer of flexible membrane material. This membrane may function to protect the microphone <NUM> from heat and/or humidity.

Data from the transducers <NUM> such as the pressure sensor <NUM>, flow rate sensor <NUM>, motor speed sensor <NUM>, and microphone <NUM> may be collected by central controller <NUM> on a periodic basis. Such data generally relates to the operational state of the RPT device <NUM>. In this example, the central controller <NUM> encodes such data from the sensors in a proprietary data format. The data may also be coded in a standardized data format.

In one form of the present technology, an RPT device <NUM> includes one or more input devices <NUM> in the form of buttons, switches or dials to allow a person to interact with the device. The buttons, switches or dials may be physical devices, or software devices accessible via a touch screen. The buttons, switches or dials may, in one form, be physically connected to the external housing <NUM>, or may, in another form, be in wireless communication with a receiver that is in electrical connection to the central controller <NUM>. In one form, the input device <NUM> may be constructed and arranged to allow a person to select a value and/or a menu option.

In one form of the present technology, the central controller <NUM> is one or a plurality of processors suitable to control an RPT device <NUM>. Suitable processors may include an x86 INTEL processor, a processor based on ARM® Cortex®-M processor from ARM Holdings such as an STM32 series microcontroller from ST MICROELECTRONIC. In certain alternative forms of the present technology, a <NUM>-bit RISC CPU, such as an STR9 series microcontroller from ST MICROELECTRONICS or a <NUM>-bit RISC CPU such as a processor from the MSP430 family of microcontrollers, manufactured by TEXAS INSTRUMENTS may also be suitable. In one form of the present technology, the central controller <NUM> is a dedicated electronic circuit. In one form, the central controller <NUM> is an application-specific integrated circuit. In another form, the central controller <NUM> comprises discrete electronic components. The central controller <NUM> may be configured to receive input signal(s) from one or more transducers <NUM>, one or more input devices <NUM>, and the humidifier <NUM>.

The central controller <NUM> may be configured to provide output signal(s) to one or more of an output device <NUM>, a therapy device controller <NUM>, a data communication interface <NUM>, and the humidifier <NUM>.

In some forms of the present technology, the central controller <NUM> is configured to implement the one or more methodologies described herein, such as the one or more algorithms expressed as computer programs stored in a non-transitory computer readable storage medium, on an internal memory. In some forms of the present technology, the central controller <NUM> may be integrated with an RPT device <NUM>. However, in some forms of the present technology, some methodologies may be performed by a remotely located device such as a mobile computing device. For example, the remotely located device may determine control settings for a ventilator or detect respiratory related events by analysis of stored data such as from any of the sensors described herein. As explained above, all data and operations for external sources or the central controller <NUM> are generally proprietary to the manufacturer of the RPT device <NUM>. Thus, the data from the sensors and any other additional operational data is not generally accessible by any other device.

In one form of the present technology, a data communication interface is provided, and is connected to the central controller <NUM>. The data communication interface may be connectable to a remote external communication network and/or a local external communication network. The remote external communication network may be connectable to remote external devices such as servers or databases. The local external communication network may be connectable to a local external device such as a mobile device or a health monitoring device. Thus, the local external communication network may be used by either the RPT device <NUM> or a mobile device to collect data from other devices.

In one form, the data communication interface is part of the central controller <NUM>. In another form, data communication interface <NUM> is separate from the central controller <NUM>, and may comprise an integrated circuit or a processor. In one form, the remote external communication network is the Internet. The data communication interface may use wired communication (e.g. via Ethernet, or optical fiber) or a wireless protocol (e.g. CDMA, GSM, <NUM>, <NUM>, <NUM>/LTE, LTE Cat-M, NB-IoT, <NUM> New Radio, satellite, beyond <NUM>) to connect to the Internet. In one form, local external communication network <NUM> utilizes one or more communication standards, such as Bluetooth, or a consumer infrared protocol.

The example RPT device <NUM> includes integrated sensors and communication electronics as shown in <FIG>. Older RPT devices may be retrofitted with a sensor module that may include communication electronics for transmitting collected data. Such a sensor module could be attached to the RPT device and thus transmit operational data to an external device such as the management server <NUM> or the user device <NUM>.

<FIG> illustrate screenshots provided by the client application <NUM> associated with an initial welcome, login, and onboarding process. <FIG> thus shows screenshots for user interfaces including a welcome interface <NUM>, a login screen <NUM>, a terms and conditions screen <NUM>, and a personal data confirmation screen <NUM>. <FIG> shows screenshots for user interfaces including a health details confirmation screen <NUM>, an onboarding overview screen <NUM>, and an assistant introduction screen <NUM>. The client application <NUM> may present these user interfaces when the client device <NUM> opens the client application <NUM> for the first time after it is downloaded and installed. Here, the client application <NUM> obtains various profile information from the patient (e.g., personal information and health information) and permissions via the terms and conditions screen <NUM>, the personal data confirmation screen <NUM>, and the health details confirmation screen <NUM> to collect and use additional data from the patient. The collected information may be transmitted from the client device <NUM> to the management server <NUM> to be stored in association with the patient's user profile. The assistant introduction screen <NUM> allows a user to ask questions along the installation process via a chat window.

<FIG> illustrate screenshots provided by the client application <NUM> in association with an unboxing and equipment identification process. <FIG> therefore shows screenshots of an overview screen <NUM> and an instruction screen <NUM>. Here, the client application <NUM> presents a user interface with step-by-step instructions for the patient to follow. After instructing the patient to unbox the equipment in the overview screen <NUM>, the client application <NUM> prompts the patient to point the camera of the client device <NUM> at the sleep apnea device <NUM> via the instruction screen <NUM>.

An example screenshot of a device image capture interface screen <NUM> displayed by the client activation after activation from the instruction screen <NUM> is shown in <FIG>. The device image capture interface screen <NUM> includes a reticle <NUM> that may be centered around the image of the RPT device <NUM>. A device image processing screen <NUM> is displayed while the captured device image is processed. The captured image is sent to the management server <NUM> to enable the device recognition module <NUM> to automatically recognize the type of sleep apnea device <NUM> and store this information to the patient's profile. The client application <NUM> then confirms to the patient successful identification of the sleep apnea device <NUM>. A device confirmation screen <NUM> is displayed with a graphical image <NUM> showing the model of the identified sleep apnea device is thus displayed on confirmation. Such information may be used to confirm that the assembly/set up of the product and associated peripherals/connections (power, tubing, etc.) are correct for the product which the patient will use to receive the respiratory therapy.

The client application <NUM> similarly prompts the patient to point the camera of the client device <NUM> to the mask <NUM> and sends the captured image to the management server <NUM>. An example screenshot of a mask image capture interface screen <NUM> displayed by the client activation after activation from the instruction screen <NUM> is shown in <FIG>. The mask image capture interface screen <NUM> includes a reticle <NUM> that may be centered around the image of the mask <NUM>. A mask image processing screen <NUM> is displayed while the captured mask image is processed. The mask recognition module <NUM> automatically recognizes the type of mask <NUM> based on this image and stores the mask type to the patient's profile. A mask confirmation screen <NUM> is displayed with a graphic image <NUM> showing the model of the identified mask is thus displayed on confirmation.

<FIG> illustrates a screenshot of an interface <NUM> provided by the client application <NUM> in association with a complete mask assembly process. Here, the client application <NUM> may present a series of images or a video (e.g., from the media database <NUM>) instructing the patient on how to properly assemble the mask <NUM>. The presentation may be specific to the particular mask type identified previously. The mask identification information may also be used to provide information about specific components, their use/function, and proper care (i.e. how to uninstall, how to clean. As indicated, this would be specific to the mask type previously identified as there are a large number of mask variations, and their composition and function can vary.

<FIG> illustrate screenshots provided by the client application <NUM> in association with assisting a patient with mask placement. <FIG> is a screenshot of an overview interface <NUM> that instructs a user to try on the mask <NUM>. Here, the client application <NUM> provides step-by-step instructions (e.g., in the form of text, images, video or combination thereof) instructing the patient on how to put on the mask <NUM>. <FIG> is a screenshot of an example video <NUM> that instructs the patient on how to put on the mask <NUM>. Once the patient indicates that the mask <NUM> is being worn, the client application <NUM> presents an interface <NUM> offering to assist the patient in properly positioning the mask seal on the face to ensure that the seal is correct.

If the patient selects an option for assistance on the interface <NUM>, the client application <NUM> may capture video of the user's face using a user-facing camera of the client device <NUM> and present an augmented reality view on the display that includes the captured video and overlays markers on the patient's face that indicate proper mask placement. <FIG> shows a sequence of screenshots for capturing video to assist in mask placement. An instruction overview image <NUM> is displayed to a patient to initiate the video capture. Using the augmented reality view, the user is instructed to align to mask <NUM> with the markers in order to properly position the mask <NUM>. A selfie interface <NUM> shows the imposition of a mask outline <NUM> on a captured self-image <NUM> that includes the face of the patient wearing the mask <NUM>. The mask outline <NUM> may include location markers <NUM> to assist in alignment with facial features. The mask in the self-image <NUM> may be adjusted as shown in a second image of a selfie interface <NUM> that assists the patient in adjusting the mask to match the mask outline <NUM>.

In an embodiment, the client application <NUM> may detect when the mask is properly aligned based on the captured video and alert the patient. The selfie interface <NUM> will generate symbols such as arrows that assist the patient in moving the mask to align with the mask outline <NUM>. The interface <NUM> will show the image of the actual mask <NUM> in relation with the mask outline <NUM> to assist in the correct alignment of the mask. Once the mask is correctly located, a placement confirmation interface <NUM> is displayed indicating successful placement to the patient.

<FIG> illustrate screenshots associated with a mask leak detection process that may be part of the client application <NUM>. Once the mask <NUM> is detected to be properly aligned, the client application <NUM> may instruct the patient to hold the client device <NUM> near the mask <NUM>. The client application <NUM> displays a leak coordination overview interface <NUM> to instruct the patient to detect the noise of a leak during operation of the respiratory therapy device <NUM>. The client application <NUM> controls a microphone of the client device <NUM> to capture audio and send the audio or extracted features from the audio (e.g., audio fingerprints) to the management server <NUM>. When the patient selects the continue button in the interface <NUM>, a microphone interface <NUM> is displayed to allow a patient to activate the microphone of the client device <NUM>. An active microphone interface <NUM> is displayed when the microphone is activated. The mask leak detection module <NUM> may then detect, based on the extracted features, if the mask <NUM> has a proper seal or an acceptable leak profile. In an embodiment, the detection algorithm may be different for different types of masks <NUM> and/or for different facial features. For example, a different detection algorithm may be used on patients with facial hair than the one used for patients without facial hair.

Upon detecting a proper seal, the patient may be alerted via the client application <NUM> that the testing is complete. Otherwise, if a proper seal is not detected, the client application <NUM> may alert the patient and provide the patient with the option of repositioning the mask <NUM> and trying again accessing the interfaces shown in FIGs. 6A-6C above. <FIG> shows a leak detected interface <NUM> that alerts the patient based on a detected leak. Alternatively, the client application <NUM> may provide the patient an option to obtain assistance from an automated assistant or from a live human representative. An example assistant introduction interface <NUM> may be activated to allow communication to an assistant. For example, in one embodiment, the patient can chat with the assistant via a chat interface <NUM>. If the chat with the remote assistant fails to resolve the problem, the client application <NUM> may initiate a video call with a live representative to further assist the patient as shown in a video initiation interface <NUM>. The video initiation interface <NUM> allows the display of a confirmation screen <NUM>. If the patient confirms the need to escalate to a live representative, the application will initiate a video call screen <NUM> allowing the patient to initiate the video call with the live representative. If the issue cannot be resolved, the client application <NUM> may recommend that the patient order a different size or model of mask <NUM>.

<FIG> illustrates screenshots provided by the client application <NUM> in association with a setup completion process. Here, the client application <NUM> presents screenshots <NUM>, <NUM> and <NUM> informing the patient that the setup is complete and may provide the patient the option of repeating any of the earlier steps. The screenshot <NUM> shows an interface <NUM> that displays the status. The screenshot <NUM> is a completion interface screen that allows the patient to repeat the process or complete the process. The patient may choose to end the setup resulting in the display of the completion screen <NUM>.

<FIG> illustrates screenshots provided by the client application <NUM> in association with preparing the patient for the first night of therapy with the newly setup sleep apnea device <NUM>. In an embodiment, the client application <NUM> may present a notification user interface <NUM> automatically in advance of the patient's normal bed time. Here, the client application <NUM> may present the patient with information relevant to preparing for the first night of therapy and may present a checklist for the patient to ensure that all equipment is properly set up and ready. For example, an interface <NUM> may allow a user to select a checklist button <NUM> for information relevant to preparing for the first night of therapy. The interface <NUM> includes icons <NUM> representing the days that the sleep apnea device <NUM> has been used. One example checklist interface is a mask checklist screen <NUM> that provides checks for the mask placement such as whether the mask is connected properly, fits comfortably and there are no leaks. Another example checklist interface is a device checklist screen <NUM> that provides checks for device operation such as whether the device is on, whether the device is connected properly and whether there is water in the humidifier.

<FIG> shows a master interface <NUM> that includes a sleep input section <NUM> that allows a user to select icons representing quality of sleep, a data field <NUM> representing summary data of each therapy session using the device <NUM> and an overall score, and different activation buttons for functions of the client application <NUM>. The interface <NUM> shows menu options that allow a patient to access different application functions.

<FIG> illustrate screenshots provided by the client application <NUM> in association with obtaining feedback from the patient following the first night of therapy. Here, the client application <NUM> may present a notification user interface <NUM> in the morning near the patient's predicted wake up time. The client application <NUM> may solicit feedback from the patient regarding the quality of the patient's sleep overnight. For example, a sleep rating interface <NUM> may be presented that includes selectable icons <NUM>, <NUM>, and <NUM> that a patient may select reflecting their sleep experience. The feedback may be sent to the management server <NUM>. Positive feedback such as selecting the icon <NUM> may allow the presentation of an additional context interface <NUM> that allows additional data to be collected.

If the patient provides negative feedback such as selecting the icon <NUM> or the icon <NUM> in the interface <NUM>, the client application <NUM> may present additional questions relevant to diagnosing the reason for the patient's bad experience. For example, an appliance analysis feedback screen <NUM> shown in <FIG> may present different options <NUM> that allow a user to select reasons why they were dissatisfied such as the mask being uncomfortable, eyes being dry, loud noise, waking up with the mask off and a dry mouth/nose. After selection, the interface <NUM> will show selected problem icons as highlighted icons <NUM> and send the data to the management server <NUM>. An example mask discomfort interface <NUM> may be displayed to collect data on potential mask discomfort via icons <NUM> representing common mask related complaints such as the mask felt too light, the mast felt too loose and made loud noise, or the mask felt foreign on the face. Each of the selected icons may cause other feedback interfaces to be displayed. For example, if noise is identified as a problem, a noise feedback interface <NUM> may be displayed to collect data on noise issues via icons <NUM> representing common noise problems such as noise from air around the mask, noise from the machine, or noise from the hose making contact. This feedback may furthermore be sent to the management server <NUM> and may be used to generate recommendation for the patient (or for a population of patients) to help improve the experience. An example summary and submit interface <NUM> may be displayed with a comment box <NUM> for the patient to provide additional feedback.

<FIG> illustrates an example embodiment of the mask recognition module <NUM>. Here, the mask recognition module <NUM> includes a learning module <NUM> and a prediction module <NUM>. The learning module <NUM> learns correlations between image features derived from images of masks <NUM>, mask CAD files, and the type of mask <NUM>. In an embodiment, the learning module <NUM> comprises a data acquisition module <NUM>, a dataset preparation module <NUM>, and a machine learning and evaluation module <NUM>. The data acquisition module <NUM> acquires imaging datasets with sufficient sample size and condition variation to enable the machine learning. For example, the data acquisition module <NUM> may acquire images of each possible type of mask capturing under various lighting conditions, environments, orientations, and using different acquisition devices in order to obtain a dataset with a wide range of images, representative of those that may be captured by patients during the mask detection process described above in <FIG>. The dataset preparation module <NUM> prepares the imaging dataset for machine learning by performing various processing on the images. For example, the dataset preparation module <NUM> may normalize the images, put the images into a standardized format, and perform one or more transformations on the images (e.g., to extract image features). The machine learning and evaluation module <NUM> trains a machine learning model to learn correlations between the images and the mask type. In different embodiments, the machine learning and evaluation module <NUM> may perform supervised learning, unsupervised learning, or a combination thereof. The machine learning and evaluation module <NUM> may generate a candidate model <NUM> that represents the learned correlations.

The prediction module <NUM> predicts a mask type based on a received image. In an embodiment, the prediction module <NUM> comprises a deploy module <NUM>, a field trial module <NUM>, and a production roll out module <NUM>. The deploy module <NUM> transforms the candidate model <NUM> to machine learning models that can be deployed across multiple machine learning platforms. The field trial module <NUM> manages controlled field trials to evaluate the predictions of the candidate model <NUM> under different conditions using the multiple machine learning platforms. The field trial module <NUM> may refine the candidate model <NUM> to generate a validated model <NUM>. The production roll out module <NUM> applies the validated model <NUM> to input images of masks received during the patient setup and generates the mask type prediction. The production roll out module <NUM> may incorporate various analytics and data collection mechanisms to generate updates to the validated model <NUM> to continue to improve its accuracy.

<FIG> illustrates an example embodiment of the mask recommendation module <NUM>. The mask recommendation module <NUM> recommends a particular mask type and size for a particular patient based on a portrait image of the patient. For example, in an embodiment, the mask recommendation module <NUM> applies a plurality of different feature extraction modules <NUM> to identify various predicted features associated with the image of the patient. For example, the feature extraction modules <NUM> may include, for example, a camera, lens, and image attributes module <NUM> to identify a camera type, lens type, and various image attributes associated with the input image; a facial feature analysis and measurement module <NUM> to analyze and measure facial features of the patient captured in the input image; a head skew/rotation module <NUM> to detect an amount of head skew and/or rotation in the input image; a gender recognition module <NUM> to detect a gender of the patient from the input image; an age estimation module <NUM> to estimate an age of the patient based on the input image; and an ethnicity recognition module <NUM> to predict an ethnicity of the patient based on the input image.

The features from the feature extraction modules <NUM> are inputted to one or more machine-learning models <NUM> that are each trained to detect suitability of a patient's face to different sizes and types of masks <NUM>. The machine-learning models <NUM> generate a mask size recommendation <NUM> and a mask type recommendation <NUM>.

In alternative embodiments, the various modules attributed to the management server <NUM> described above may instead be performed in whole or in part by the client application <NUM>. For example, instead of the client application <NUM> sending an image or image features to the management server <NUM>, the client application <NUM> may instead directly apply one or more machine-learning models (based on a model received from the management server <NUM>). In other embodiments, functions described herein as being performed by the client application <NUM> may instead be performed by the management server <NUM>.

As used in this application, the terms "component," "module," "system," or the like, generally refer to a computer-related entity, either hardware (e.g., a circuit), a combination of hardware and software, software, or an entity related to an operational machine with one or more specific functionalities. For example, a component may be, but is not limited to being, a process running on a processor (e.g., digital signal processor), a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a controller, as well as the controller, can be a component. One or more components may reside within a process and/or thread of execution, and a component may be localized on one computer and/or distributed between two or more computers. Further, a "device" can come in the form of specially designed hardware; generalized hardware made specialized by the execution of software thereon that enables the hardware to perform specific function; software stored on a computer-readable medium; or a combination thereof.

The terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the invention. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or variants thereof, are used in either the detailed description and/or the claims, such terms are intended to be inclusive in a manner similar to the term "comprising.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. Furthermore, terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Claim 1:
A system to provide assistance to a patient for using a respiratory therapy device and a mask for treatment of respiratory ailments, the system comprising:
an equipment database storing data relating to a plurality of device types and a plurality of mask types;
a device recognition module operable to identify the type of the respiratory therapy device from an image of the respiratory therapy device captured by a client computing device in comparison with the data relating to the plurality of device types;
a mask recognition module operable to identify the type of the mask from an image of the mask captured by the client computing device in comparison with the data relating to the plurality of mask types;
a media database including media relating to assistance information relating to at least one of a mask type or a device type;
a management server operable to send media relating to assistance information for the identified type of mask or identified type of device to the client computing device; and
a mask positioning module operable to:
receive a facial image of the patient captured by the client computing device via the interface;
perform a facial analysis to identify and track locations of facial landmarks from the facial image;
overlay a mask placement image on the received facial image frames aligned to
the detected facial landmarks for display on the client computing device; and provide instructions to the client computing device on adjusting the mask based
on the overlaid mask placement image.