SYSTEMS AND METHODS OF HOMECAGE MONITORING

Computer controlled systems and methods of an automated homecage monitoring system predict a state of an animal and of its homecage. The prediction of the state of the animal may be based on a pose estimate of the animal, and at least one sensor input from an at least one sensor. The pose estimate may include a graph of connected nodes, the nodes representing coordinates of the sensor input corresponding to predicted indicia on the animal's body. The animal state may include a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal. The automated homecage monitoring system may allow for animal state data to be reported for an animal or animals in situ.

FIELD

The described embodiments relate to the monitoring of environment, animal physiology, and behavior in a homecage.

BACKGROUND

Due to their reduced size and similar brain architecture as primates, rodents have been widely used for studying a variety of complex behaviors. Recent development in sophisticated tools for measuring and manipulating brain activity (e.g. optogenetics, two photon imaging, widefield mesoscale imaging, fiber photometry, mini endoscopes) together with the many transgenic lines and disease models available in rodents further enhance their usefulness in studies.

Behavior monitoring systems are useful tools that allow scientists to characterize behavioral changes associated with ill health in rodents. Since mice are crepuscular and nocturnal animals (they are active at dusk and dawn and throughout the night), assessing their signs of ill health, pain and distress by animal care staff is difficult. Manual monitoring by animal care staff risks introducing observer bias in monitoring animal behavior, does not provide continuous monitoring, and lacks sensitive monitoring during dark periods when mice are most active. Further, the handling of animals in homecages may be stressful and may confound studies.

Running wheels are widely used for homecage monitoring of animal brain states (awake vs sleep) because of their low cost and ease of implementation for measuring the activity and/or changes in circadian rhythm, but their use has been shown to independently effect animal behavior and progression of disease pathology/symptoms.

RFID tags for monitoring location are disadvantageous because of poor spatial and temporal resolution and the tag itself may provide discomfort to the animal.

Existing video monitoring systems require human operators to identify animal behaviors, and do not provide automatic prediction of animal behavior in their homecage.

Monitoring the animals in their homecage has several advantages. In studying complex behavior, experimental design that requires direct animal-experimenter interaction where an animal is removed from its homecage environment and placed in an unfamiliar apparatus (novel environment) are disruptive, time- and labor-consuming and require additional laboratory space. This disruption of removing the animal from their homecage may influence its behavior, general well-being, and metabolism, affecting the phenotypic outcome even if the data collection method is automated, creating spurious findings.

SUMMARY

The present teachings are directed to systems and methods that can be integrated in many settings, for monitoring and analyzing animal behavior in a homecage environment. This automated homecage behavioral data acquisition allows investigators to study the effects of experimental manipulations and a wide range of neurological diseases by accurate measurement of progressive behavioral changes in the same animal without compromising its welfare, as well as removing experimenter bias. In addition, the systems and methods may be used for short-term welfare assessment (e.g. post-surgery monitoring) and long-term welfare assessment (e.g., monitoring of the status of animal environment such as water, food, ammonia level) by enabling continuous monitoring, even in the dark phase where welfare assessment without disturbance to the cage is difficult and subjective. Further, video data collected with this system may be used to automatically classify sleep-wake states of an animal in the home-cage.

In a first aspect, some embodiments provide a method for quantifying animal activity, comprising: receiving at least two video frames; determining at least two processed video frames from the at least two video frames by pre-processing; determining an image background from the at least two processed video frames; determining an at least two thresholded video frames by applying the background image as a threshold to the at least two video frames; determining a contour from the at least two thresholded video frames; and determining an animal motion flag if the contour is larger than a minimum area.

In a second aspect, some embodiments provide a method for quantifying animal activity, comprising: receiving at least two video frames; determining at least two processed video frames from the at least two video frames by pre-processing; determining an image background from the at least two processed video frames; determining a contour from the at least two thresholded video frames; determining the centroid of the contour; and determining an animal track from the centroid of each of the at least two thresholded video frames.

In a third aspect, some embodiments provide an animal behavior analysis system, comprising: a memory having: a first predictive model, the first predictive model for predicting a pose estimate of the animal; and a second predictive model, the second predictive model for predicting an animal state; at least one sensor, the at least one sensor for sensing the animal; a processor in communication with the memory and the at least one sensor, the processor configured to: receive a sensor input from an at least one sensor; predict the pose estimate from the sensor input and the first predictive model; and predict an animal state based on the pose estimate, the sensor input and the second predictive model.

In at least one embodiment, the predicting the pose estimate may comprise: determining a position of an at least one indicia; and predicting the pose estimate based on the position of an at least one indicia, the sensor input, and the first predictive model.

In at least one embodiment, the animal state may include at least one of a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal.

In at least one embodiment, the system may further comprise: a database in communication with the memory.

In at least one embodiment, the processor may be further configured to: determine a report based on at least one of the animal state, the pose estimate, the sensor input, and the position of the at least one indicia; and store the report in the database; wherein the report, the pose estimate, and the position of the at least one indicia may correspond to a common timebase.

In at least one embodiment, the processor may be further configured to output the report.

In at least one embodiment, the processor may be further configured to output the report to a display device.

In at least one embodiment, the processor may be further configured to output the report to a server by network communication.

In at least one embodiment, the processor may be further configured to predict the pose estimate and predict the animal state generally in real-time.

In at least one embodiment, the processor may be further configured to predict the pose estimate and predict the animal state using an offline process.

In at least one embodiment, the processor may be further configured to determine, for the report, from the common timebase, a start time, an end time, and an elapsed time.

In at least one embodiment, the processor may be further configured to determine, for the report, from the position of the at least one indicia and the pose estimate, an occupancy map having a displacement of the animal.

In at least one embodiment, the processor may be further configured to determine, for the report, the speed of the animal's movement.

In at least one embodiment, the processor may be further configured to: determine a position of an object from the sensor input, and determine, for the report, from the position of the at least one indicia and the pose estimate, an interaction of the animal with the object.

In at least one embodiment, the processor may be further configured to determine at least one husbandry variable, the at least one husbandry variable in the homecage may comprise a food supply level, a water supply level, a temperature, a humidity value, ammonia level, a bedding quality metric, and a nesting quality metric, respiration, heart beat from the sensor input.

In at least one embodiment, the system may further comprise: an actuator proximate to the animal; and wherein the processor may be further configured to actuate the actuator if the report matches a pre-determined actuation condition.

In at least one embodiment, the actuator may be a haptic device.

In at least one embodiment, the at least one sensor may comprise at least one camera, at least one of a humidity sensor, at least one of a temperature sensor, and at least one of an ammonium sensor.

In at least one embodiment, the at least one camera may have an at least one infra-red camera.

In at least one embodiment, the first predictive model may be a deep neural network.

In at least one embodiment, the second predictive model may be a recurrent neural network.

In a fourth aspect, some embodiments provide a method of animal state analysis, comprising: providing, at a memory, a first predictive model, the first predictive model for predicting a pose estimate of the animal; providing, at the memory, a second predictive model, the second predictive model for predicting a state of the animal; receiving, at a processor, a sensor input from an at least one sensor; predicting, at the processor, the pose estimate from the sensor input and the first predictive model; and predicting, at the processor, an animal state based on the pose estimate, the sensor input and the second predictive model.

In at least one embodiment, the predicting a pose estimate may further comprise: determining, at the processor, a position of an at least one indicia; and predicting, at the processor, the pose estimate based on the position of an at least one indicia, the sensor input, and the first predictive model.

In at least one embodiment, the animal state may include at least one of a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal.

In at least one embodiment, the method may further comprise: determining, at the processor, a report based on at least one of the animal state, the pose estimate, the sensor input, and the position of the at least one indicia, storing the report in a database, the database in communication with the memory; wherein the report, the pose estimate, and the position of the at least one indicia may correspond to a common timebase.

In at least one embodiment, the method may further comprise: outputting the report.

In at least one embodiment, the method may further comprise: outputting the report to a display device.

In at least one embodiment, the method may further comprise: outputting the report to a server by network communication.

In at least one embodiment, the predicting, at the processor, the pose estimate and predicting, at the processor, the animal state may be performed generally contemporaneously with the sensor input collection.

In at least one embodiment, the predicting, at the processor, the pose estimate and predicting the animal state may be performed generally after the sensor input collection.

In at least one embodiment, the method may further comprise: determining, at the processor, for the report, from the common timebase, a start time, an end time, and an elapsed time.

In at least one embodiment, the method may further comprise: determining, at the processor, for the report, from the position of the at least one indicia and the pose estimate, an occupancy map having a movement of the animal.

In at least one embodiment, the method may further comprise: determining, at the processor, for the report, the speed of the animal's movement.

In at least one embodiment, the method may further comprise: determining, at the processor, a position of an object from the sensor input; determining, at the processor, for the report, from the position of the at least one indicia and the pose estimate, an interaction of the animal with the object.

In at least one embodiment, the method may further comprise determining, at the processor, at least one husbandry variable, the at least one husbandry variable in the homecage may comprise a food supply level, a water supply level, a temperature, a humidity value, ammonia level, a bedding quality metric, and a nesting quality metric, respiration, heart beat from the sensor input.

In at least one embodiment, the method may further comprise: if the report matches a pre-determined actuation condition: actuating an actuator proximate to the animal.

In at least one embodiment, the actuator may be a haptic device.

In at least one embodiment, the at least one sensor may comprise at least one of an at least one camera, an at least one infra-red camera, a humidity sensor, a temperature sensor, and an ammonium sensor.

In at least one embodiment, the first predictive model may be a deep neural network.

In at least one embodiment, the second predictive model may be a recurrent neural network.

In a fifth aspect, some embodiments provide a system of generating a predictive model for predicting an animal state, comprising: a memory, the memory having: a plurality of sensor inputs; and a pose prediction model for predicting animal poses from a sensor input; a processor configured to: generate a plurality of predicted animal poses associated with the plurality of the sensor inputs, by, for each sensor input in the plurality of sensor inputs: predicting, using the first predictive model, a predicted animal pose from the sensor input; associating the predicted animal pose with the sensor input; generate a plurality of behavior labels associated with the plurality of the sensor inputs, by, for each sensor input in the plurality of sensor inputs: associating a behavior label with the sensor input; generate the second predictive model based on the plurality of sensor inputs, the plurality of predicted animal poses, and the plurality of behavior labels.

In at least one embodiment, the processor may be further configured to: display the sensor input to a user at a display device; and wherein the behavior label may be received from the user using an input device.

In a sixth aspect, some embodiments provide a method of generating a second predictive model, the second predictive model for predicting an animal state, comprising: providing, at a memory, a plurality of sensor inputs; providing, at the memory, a first predictive model for predicting animal poses from a sensor input; generating, at a processor, a plurality of predicted animal poses associated with the plurality of the sensor inputs, by, for each sensor input in the plurality of sensor inputs: predicting, using the first predictive model, a predicted animal pose from the sensor input; associating the predicted animal pose with the sensor input; generating, at the processor, a plurality of behavior labels associated with the plurality of the sensor inputs, by, for each sensor input in the plurality of sensor inputs: associating a behavior label with the sensor input; generating, at the processor, the second predictive model based on the plurality of sensor inputs, the plurality of predicted animal poses, and the plurality of behavior labels.

In at least one embodiment, the method may further comprise: displaying the sensor input to a user at a display device; and wherein the behavior label may be received from the user using an input device.

DESCRIPTION OF EXAMPLE EMBODIMENTS

It will be appreciated that numerous specific details are set forth in order to provide a thorough understanding of the example embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the embodiments described herein. Furthermore, this description and the drawings are not to be considered as limiting the scope of the embodiments described herein in any way, but rather as merely describing the implementation of the various embodiments described herein.

The embodiments of the systems and methods described herein may be implemented in hardware or software, or a combination of both. These embodiments may be implemented in computer programs executing on programmable computers, each computer including at least one processor, a data storage system (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface. For example and without limitation, the programmable computers (referred to below as computing devices) may be a server, network appliance, embedded device, computer expansion module, a personal computer, laptop, personal data assistant, cellular telephone, smart-phone device, tablet computer, a wireless device or any other computing device capable of being configured to carry out the methods described herein.

In some embodiments, the communication interface may be a network communication interface. In embodiments in which elements are combined, the communication interface may be a software communication interface, such as those for inter-process communication (IPC). In still other embodiments, there may be a combination of communication interfaces implemented as hardware, software, and combination thereof.

Program code may be applied to input data to perform the functions described herein and to generate output information. The output information is applied to one or more output devices, in known fashion.

Furthermore, the system, processes and methods of the described embodiments are capable of being distributed in a computer program product comprising a computer readable medium that bears computer usable instructions for one or more processors. The medium may be provided in various forms, including one or more diskettes, compact disks, tapes, chips, wireline transmissions, satellite transmissions, internet transmission or downloads, magnetic and electronic storage media, digital and analog signals, and the like. The computer useable instructions may also be in various forms, including compiled and non-compiled code.

As referred to herein, a homecage refers to any cage provided for securing an animal, such as an aquarium, a cage having bars, a box, a zoo cage, an aviary, a battery cage, a birdcage, or any other container as is known for securing an animal. The homecage may be made from any material such as clear plastic (such as polycarbonate), wire mesh, wire bars, or any other material that can be used to secure an animal from passing through it. The homecage may be opaque, transparent or translucent. The homecage may allow for air to pass into it, or may be substantially airtight. The homecage may have a separate air supply. The homecage may contain bedding, a food and water supply for the animal.

While a single homecage is shown forFIGS. 1-16, it is understood that the homecage monitoring system would apply in similar fashion to one or more homecages, where each homecage has its own microcontroller. There may be many homecages monitored in the system100.

Reference is first made toFIG. 1, showing a system view100of an automated homecage monitoring system. The automated homecage monitoring system has a microcontroller112, a server104, a database102, a network108, and a homecage114. Optionally, the system view100may include a mobile device106, a computer system116, and report110.

The microcontroller112has a plurality of sensors to monitor homecage114. The microcontroller112may perform additional processing, or may send collected sensor data to the server104for processing via network108. When the microcontroller112is responsible for the additional processing, it may determine the sensor data information about the animal in the homecage114, and provide monitoring and prediction data to the server104via network108in addition to sensor data.

Mobile device106and computer system116may be used by an end user to access an application (not shown) running on server104over network108. For example, the application may be a web application, or a client/server application. The mobile device106and computer system116may each be a desktop computer, mobile device, or laptop computer. The mobile device106and computer system116may be in communication with server104, and microcontroller112. The mobile device106and computer system116may display the web application, and may allow a user to see monitoring data from the homecage, or more than one homecages that are monitored. An administrator user may use the mobile device106or the computer system116to access the server104and configure the homecage monitoring system. The user at mobile device106or computer system116may review monitoring data and apply pose estimate labels or state labels to sensor data from the homecage to “train” the automated homecage monitoring system. The server104receives the pose estimate labels or state labels and stores them in database102.

The server104may be a commercial off-the-shelf server, or another server system as are known. The server104may run a web application on a web server or application server (not shown) accessible via network108. The web application may be used by a user to manage, view, and perform various user input and output tasks on a homecage monitoring system associated with a plurality of homecages.

The server104may generate and send an automated homecage monitoring report110via network108. The report110may be sent via email, SMS, application-based notification, etc. The users using mobile device106or computer system116may respond to the report110to review the monitoring data for a homecage114or a group of homecages. The report110may be an alert sent to the users at mobile device106or computer system116to warn them that intervention is required in one or more homecages114in a group of homecages.

The server104may also provide a database including historical sensor data from the homecage114, historical event data from the homecage114(including animal husbandry data), one or more predictive models for estimating animal poses, and one or more predictive models for predicting animal or homecage states. The server104may further predict animal or homecage states in order to provide an alarm for an operator to address the animal or homecage state, such as to change the homecage bedding, add water, or add food in a predicted period of time.

The server104may also store the sensor data from one or more microcontrollers associated with one or more homecages114.

Network108may be a communication network such as the Internet, a Wide-Area Network (WAN), a Local-Area Network (LAN), or another type of network. Network108may include a point-to-point connection, or another communications connection between two nodes.

Database102may be a relational database, such as MySQL or Postgres. The database102may also be a NoSQL database such as MongoDB. The database may store the reports and sensor data from the microcontroller112of each homecage114.

Report110may be an HTML formatted, text formatted, or a file-based report that is provided by email, SMS, or application-based notification to a user. The report110may include a URL link to the web application on server104.

The microcontroller112may be an embedded system such as Arduino®, field-programmable gate array (FPGA), or a small form-factor computer system such as a Raspberry Pi®. The microcontroller112may have or be connected to at least one sensor (not shown) for sensing the animal in the homecage114, or environmental conditions in the homecage114.

ForFIGS. 2-3, like numerals refer to like elements between the figures, such as the network unit202, display204, interface unit206, processor unit208, memory unit210, I/O hardware212, user interface214, power unit216, and operating system218.

Referring toFIG. 2, a block diagram200is shown of the microcontroller112fromFIG. 1. The microcontroller112has a network unit202, a display204, an interface unit206, a processor unit208, a memory unit210, i/o hardware212, a user interface engine214, and a power unit216.

The network unit202may be a standard network adapter such as an Ethernet or 802.11x adapter. The processor unit208may include a standard processor, such as the Intel® Xeon® processor or an Advanced RISC Machine (ARM) processor, for example. Alternatively, there may be a plurality of processors that are used by the processor unit208and may function in parallel.

The processor unit208can also execute a graphical user interface (GUI) engine214that is used to generate various GUIs. The user interface engine214provides for administration of the homecage monitoring, and the information may be processed by the admin unit226. User interface214may implement an Application Programming Interface (API) or a Web-based application that is accessible via the network unit202. The API may provide connectivity with the server by either push or pull requests, i.e. the microcontroller112may send data to the server as it is collected, or the server may contact the microcontroller via network unit202to pull data.

Memory unit210may have an operating system218, programs220, a communications unit222, a camera unit224, a prediction unit234, an admin unit226, a master clock228, a sensor unit230, and an actuator unit232. Some features and functions provided by memory unit210may be performed entirely at the microcontroller, entirely at the server, or a combination of both the microcontroller and the server. In the case where features and functions are performed at the server, the server may operate on sensor data sent from the microcontroller to the server.

The operating system218may be a Microsoft Windows Server® operating system, or a Linux®-based operating system, or another operating system. In the case that the microcontroller is an embedded system of an FPGA, it may run an embedded operating system, or no operating system at all.

The programs220comprise program code that, when executed, configures the processor unit208to operate in a particular manner to implement various functions and tools for the microcontroller200.

Communications unit222provides functionality for sending and receiving sensor data and predictive models using network unit202, as well as provided access for users to the admin unit226using network unit202.

Camera unit224receives data from the sensor unit230, and may operate to provide image manipulation for collected sensor data. The camera unit224may function to create image files for sending to the server via communications unit222. The camera unit may also provide image manipulation as disclosed herein to determine monitoring data from image data collected by sensors in the homecage. Such monitoring data may include the position of an animal in the homecage, the path the animal has taken in the homecage, the position of an object in the homecage, the food or water levels inside the homecage, or other visual based monitoring data.

The prediction unit234operates to provide predictions based on the sensor data from sensor unit230and at least two predictive models. The prediction unit234may determine a predictive model from historical sensor data and labels from the database, or historical sensor data and labels on the microcontroller200. The prediction unit234may have a first predictive model that can predict a pose estimate based on sensor data. The prediction unit234may have a second predictive model that can predict an animal state based on the pose prediction and sensor data. The prediction unit234may associate the predicted pose estimate and animal state with a report, and may send the report to the server. The report sent by the microcontroller200to the server may include sensor data and predictions from the prediction unit234that are associated with a particular start and end time, and have a corresponding common timebase from master clock228. The report sent by the microcontroller200to the server may include quantitative measures of the event, including start time, end time, elapsed time, frequency, etc. The prediction unit234may predict a pose estimate based on a position of an at least one animal feature in the sensor data. The animal state prediction by prediction unit234may include a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal.

The prediction unit234may provide an estimation of the remaining food and water supply levels or the current state of the bedding (e.g. clean, dirty, flood, etc.) and nesting in a homecage114, and may provide an occupancy map of the homecage114showing an animal's movement path. The prediction unit234may further detect animal birth in a homecage, and may trigger an alarm for an operator to attend. The food and water supply levels, the status of the bedding and nesting and the occupancy map may be included in the report. The bedding and nesting status may be represented as quality metrics, and may be represented on a scale of 0 to 100, a letter grade, etc. Specific quality metrics may be tracked related to the cleanliness of the bedding, or if a flood of the water supply into the homecage has occurred. The estimation of the current state of the homecage may be include at least one husbandry variable, the at least one husbandry variable in the homecage comprising a food supply level, a water supply level, a temperature, a humidity value, a bedding quality metric, and a nesting quality metric from the sensor input.

Admin unit226may provide user access to the microcontroller200. This may allow an administrator user to configure the monitoring parameters for a homecage114. Separately, a user may access the admin unit226to review monitoring data, view a real-time video feed of the homecage, review historical events, review video data associated with a historical event, etc.

Master clock228is an internal clock for microcontroller200for associating a common timebase with data collected by sensor unit230. This common timebase may ensure that the collected data can be referenced based on time, and each different type of sensor data collected can be precisely identified based on the time it is collected. The master clock228may be synchronized with a master clock on the server so that both the server and the microcontroller use a generally synchronized clock to record and process monitoring data. The server and the microcontroller may use a networking protocol for clock synchronization such as the Network Time Protocol (NTP).

The sensor unit230provides an interface to one or more sensors connected to microcontroller200using i/o hardware212. Optionally, the sensor unit230may receive sensor data from another source via a network connection using network unit202. The sensor unit230may pre-process sensor data, for example, it may add a video filter to a video sensor signal, or it may normalize an audio signal. In the case of image data pre-processed by sensor unit230, filters may be applied such as a Gaussian blur, averaging, color-grayscale conversions, and the like. The sensor unit230may operate to send sensor data to a server via communications unit222and network unit202. The sensor unit230may operate to determine the measurement frequency, and bandwidth of sensors connected to the microcontroller200. The sensor unit230may be modular, and may allow for more than one sensor to be connected to the microcontroller200, and may allow for data to be collected from multiple sensors simultaneously. Sensor unit230may be compatible with multiple different types of sensors such as at least one camera including at least one an infra-red camera (including both passive and active), humidity sensors, temperature sensors, microphone sensors, light sensors, pressure sensors, accelerometer sensors, proximity sensors, ultrasonic sensors, vibration sensors, electrical current and electrical potential sensors, fluid flow sensors, laser position sensors, laser proximity sensors, and ammonium sensors.

The sensor unit230may log events based on the sensor data, including string pulling, animal movement paths, animal speed, novel object recognition, social interaction, light-dark box, activity in the homecage114, etc.

Actuator unit232provides control of at least one actuator proximate to, or inside the homecage114. The actuators may include linear actuators, pneumatic actuators, buzzers, speakers, thermal actuators, piezoelectric actuators, servomechanisms, solenoids, stepper motors, or the like.

The prediction unit234may be configured to operate the actuator unit232based on a prediction of pose estimate or animal state. The admin unit226may also allow a user to operate an actuator remotely. The actuators may also be actuated based on sensor data collected by sensor unit230. For example, a researcher user may configure the microcontroller112to wake an animal up with an actuator after a predetermined amount of sleep, and the microcontroller112may do so based on a predicted sleep state of an animal.

Referring next toFIG. 3, there is shown a block diagram300of the server104fromFIG. 1. The server104has a network unit302, a display304, an interface unit306, a processor unit308, a memory unit310, i/o hardware312, a user interface engine314, and a power unit316.

The server may perform many of the functions of the microcontroller112(seeFIG. 1), including the features of the camera unit224, prediction unit234, admin unit226, and master clock228(seeFIG. 2). In the case where a feature of the microcontroller is performed by the server300, then the microcontroller may function as a “dumb” device where it accepts configuration parameters, sends collected sensor data to the server using the communications unit222, and activates the at least one actuator of the actuator unit232in response to an instruction sent to the microcontroller via communications unit222(seeFIG. 2).

The user interface engine314may be provided to generate various user interfaces that may be delivered using a web server in programs320to deliver the user interfaces via the network unit302to users of the automated homecage monitoring system. These user interfaces may display information for users of the homecage monitoring system, and may accept user input from the users in order to control the homecage monitoring system. The user interfaces produced by user interface engine314may include those found inFIGS. 15-34.

The operating system318may be a Microsoft Windows Server® operating system, or a Linux®-based operating system, or another operating system. The operating system318may also be a mobile operating system such as Google® Android® or Apple® iOS.

The programs320comprise program code that, when executed, configures the processor unit308to operate in a particular manner to implement various functions and tools for the server300.

Communications unit322provides functionality for sending and receiving sensor data and predictive models using network unit302, as well as provide access for users to the admin unit326using network unit302.

Admin unit324may be an application available to users via the communications unit322. The server admin unit324may allow users to view monitoring data, predictions, and configuration information for at least one of the homecage monitoring system inFIG. 2. Users may use the admin unit324to manage the configuration of at least one homecage monitoring system. The admin unit324may also allow for the configuration of reports from homecage monitoring systems, and event alerts from the homecage monitoring systems (see e.g.110inFIG. 1).

The server300may store monitoring data from the at least one homecage monitoring system, event data, report data, alert data, and sensor data in database326. The database326may also store predictive models for homecage or entire colony variables, pose estimate and animal state. In an alternate embodiment, a separate database (not shown) may be used to store predictive models for the entire colony variables. The database326may be responsive to queries from the homecage monitoring system microcontrollers, and may send stored data to the microcontroller using communications unit322and network unit302.

The event unit328provides functionality for combining sensor data, monitoring data, quantitative data, and prediction data into an report that can be stored in database326. Responsive to user configuration, the event unit328can also send reports to users via email or SMS. The reports generated by event unit328may be sent at regular intervals, or may be sent on-demand. The reports may be HTML formatted, or text formatted. The reports generated by event unit328may include a URL link to a web application running on the server300.

The alert unit330may send alerts to users via SMS or email responsive to configurable events happening at a homecage system. For example, an alert may be sent if food or water or bedding state for a particular homecage is below a specified threshold. The alert unit330may send alerts based on predicted poses or predicted animal states. The alerts may have different severity levels, for example, there may be an “info” level, an “warning” level, an “error” level, or a “severe” level.

Referring now toFIG. 4, there is a software component diagram of an automated homecage monitoring system400. The homecage monitoring system400has a communications module402, a camera module404having a camera or image analysis module406, a master clock408having a plurality of timestamps416on a common timebase, a prediction module418, an administrator module410, at least one sensor module412having a sensor analysis module414, at least one actuator module420having an actuator analysis module422.

The communications module402sends reports and alerts. The reports and alerts may be sent to a user via email, SMS or Twitter, may be sent to the server, or may be sent to both the user and the server.

The camera module404acquires video using an attached camera sensor. The camera module404has an analysis sub-module that performs image processing on the video, including for example Gaussian blurring. Depending on requirements, the camera or image analysis submodule406may be performed at the microcontroller, at the server, or at both the microcontroller and the server. The image analysis submodule may have a modular design of the system allowing the user to modify the video analysis algorithm.

Each frame of video from the camera module404may be timestamped with the time of collection from the master clock408using timestamps416. These timestamps are used in order to synchronize video with other systems such as, for example, electrophysiology, pupil camera, or other sensor data. The master clock module408synchronizes the modules by generating timestamps on a common timebase, and associating the timestamps416with sensor data collected via the sensor modules412, actions taken by actuator modules420, predictions by prediction module418, and other monitoring data.

Prediction module418may receive a first predictive model and a second predictive model from the server using communications module402. The first predictive model and the second predictive model may be used to determine, for collected sensor data (including video data from the camera sensor), a predicted pose and a predicted animal state.

Administrator module410may produce reports based on sensor data, prediction data, and other monitoring data. The reports may be sent by administrator module to the server, to the user, or to both the server and the user.

At least one sensor module412having a sensor analysis submodule414acquires sensor data from at least one sensor. The sensors may be inside the homecage environment or may be proximate to the homecage. The sensor analysis submodule may detect events based on sensor input. The sensor analysis submodule414may pre-process collected sensor data, such as by averaging sensor data or filtering sensor data.

At least one actuator module420having an actuator analysis module422activates an actuator in a homecage or proximate to a homecage in response to an event, a prediction, or user input.

Referring toFIG. 5, there is a relationship diagram500of the video data collection of the automated homecage monitoring system. The automated homecage monitoring system may have at least one camera502connected to the microcontroller504. The microcontroller504may operate in a “dumb” or a “smart” mode.

When operating in a “dumb” mode, the microcontroller504collects video and archives it with the server, where the server is responsible for the video processing508.

When operating in a “smart” mode, the microcontroller504operates to perform simultaneous video acquisition and archiving506and video processing508.

Video acquisition and archiving506may include writing video frames510including timestamps512to a storage device at the microcontroller504. The acquisition and archiving506may also include sending the video data510(including timestamps512) to the server. The video acquisition and archiving506may process more than one camera at a time. For example (seeFIGS. 6C and 6D) multiple camera sensors can be used for simultaneous acquisition of animal state from multiple directions.

Generally simultaneously, the video processing508may determine tracking data514and write it to a storage device on the microcontroller504. The tracking data514may include an animal's path in the homecage. The tracking data514may also include event data and prediction data determined at the microcontroller. The tracking data514may also be sent to the server.

Referring toFIG. 6A, there is a cutaway top view of a homecage600. The homecage600has a base610. Two sidewalls604, a front wall606and a back wall608extend from the base610to form an enclosure for animal602. The enclosure may include ventilation holes in the front wall, the back wall, or the sidewalls. Alternatively the homecage may have a self-contained ventilation system. The homecage600may be a variety of sizes relative to the animal, generally offering enough space for the animal to move around. The base610, sidewalls604, front wall606and back wall608may be made from a variety of materials, including wood, plastic, or metal. The base610, sidewalls604, front wall606and back wall608may have various constructions including a solid piece of material, a mesh, spaced apart bars.

Referring toFIG. 6B, there is a front view620of the homecage inFIG. 6A. The homecage620has a top626, with a microcontroller622disposed on it. The microcontroller622may be connected to an at least one sensor624through the top626. The at least one sensor624may be inside the homecage as shown, or instead may be positioned outside and proximate to the homecage. The at least one sensor624may include at least one camera, at least one infra-red camera (including both passive and active), humidity sensors, temperature sensors, microphone sensors, light sensors, pressure sensors, accelerometer sensors, proximity sensors, ultrasonic sensors, vibration sensors, ammonia sensor, radar system for detection of respiration and heartbeat, electrical current and electrical potential sensors, fluid flow sensors, and ammonium sensors.

Referring toFIG. 6C, there is a cutaway front view640of another homecage. The homecage600has a base650. Two sidewalls644, a front wall646and a back wall648extend from the base650to form an enclosure for animal642. The enclosure may include ventilation holes in the front wall, the back wall, or the sidewalls. Each sidewall644has a sensor652connected to it. The sensors652may communicate wirelessly with microcontroller662, or they may be wired. The sensors may be a variety of different types, including at least one camera, at least one infra-red camera (including both passive and active), humidity sensors, temperature sensors, microphone sensors, light sensors, pressure sensors, accelerometer sensors, proximity sensors, ultrasonic sensors, vibration sensors, electrical current and electrical potential sensors, fluid flow sensors, and ammonium sensors. There may be multiple sensors disposed together at652.

Referring toFIG. 6D, there is a front view660of the homecage fromFIG. 6C. The homecage660has a microcontroller662disposed on top654, at least one sensor disposed on sidewall652, and at least one sensor connected to microcontroller654through top654. The sensors652,654may function individually or in combination to generate sensor data about animal642or the homecage660environment.

Referring toFIG. 7, there is a cutaway top view700of another homecage. The animal702may move about the homecage on floor710in the region bounded by sidewalls704, front wall706, and back wall708. The homecage monitoring system may track the path taken by the animal702, including a track plot712around the homecage700.

Referring toFIG. 8, there is a sensor data diagram800of an automated homecage monitoring system. The sensor data diagrams802,804,806,808show the result of animal motion flag inFIG. 9A. The sensor data is shown for moving806and non-moving802animals during the day, as well as moving808and non-moving804during the night. Sensor data810shows the location of the centroid determined using the method inFIG. 9B. Sensor data812shows the track plot generated from the centroid determined inFIG. 9B.

Referring toFIG. 9A, there is a method diagram900for automated homecage motion detection. The method900is operated by the microcontroller or the server based on video data from an at least one sensor. The method900is operable to find animals motion flag that identifies a Boolean value associated with whether animal or animals in the homecage has or have moved.

In one or more embodiments, the method900may be operable to find two or more animals and determine a Boolean value associated with each of the two or more animals. The Boolean value is associated with a determination of whether each animal has moved.

In one or more embodiments, instead of a Boolean value, the method900may determine a health score of the one or more animals in the homecage. The health score may be a range of values corresponding to the activity level of the one or more animals in the homecage, and may further estimate the “health” of the animals.

At902, at least two video frames are received. The at least two video frames may be indexed to the same timebase, and each frame may have an associated timestamp. The video may be received in a variety of formats, for example, the Raspberry Pi® camera board v2.1 has an 8MP Sony® IMX219 sensor that collects video data at 30 fps using a h264 encoding at 600×800 pix resolution. (other formats: YUV, JPEG)

At904, at least two processed video frames are determined from the at least two video frames by pre-processing. The processing may include applying a Gaussian filter to each video frame to reduce noise, resizing the video frame, and conversion to a grayscale colour profile.

At906, an image background may be determined from the at least two processed video frames. The processing may involve calculating a weighted running average of the at least two frames. The image background may be determined from the at least two processed video frames using frame differencing, as known.

At908, an at least two thresholded video frames are determined by subtracting the background image from each of the processed video frames to determine motion areas which contain values above a certain threshold.

At910, a contour is determined from each thresholded video frame. The contour operation provides a contour-detected area which, in the homecage, is generally an area representing the animal's position.

At912, an animal motion flag is determined if the contour is larger than a minimum area. The minimum area may be a configurable threshold.

Referring toFIG. 9B, there is a method diagram950for automated homecage monitoring. The method950is operated by the microcontroller or the server based on video data from an at least one sensor. The method950may operate to determine a tracking path of an animal or animals in a homecage.

At952, at least two video frames are received. The at least two video frames may be indexed to the same timebase, and each frame may have an associated timestamp. The video may be received in a variety of formats, for example, the Raspberry Pi® camera board v2.1 has an 8MP Sony® IMX219 sensor that collects video data at 30 fps using a h264 encoding.

At954, at least two processed video frames are determined from the at least two video frames by pre-processing. The pre-processing may include resizing the video frame, conversion to grayscale colour profile, and the application of a Gaussian blur.

At956, an image background is determined from the at least two processed video frames. The processing may involve calculating a running average of the at least two frames.

At958, an at least two thresholded video frames are determined by applying the background image as a threshold to the at least two video frames.

At960, a contour is determined from the at least two thresholded video frames. The contour operation provides a contour-detected area which, in the homecage, is generally the animal's position.

At962, a centroid is determined for the contour on each frame. The centroid is the arithmetic mean position of the contour, and generally represents the midpoint of the animal.

At964, a track plot of the animal around the homecage is determined from the line plot of the centroid values of each of the at least two thresholded video frames.

Referring toFIG. 10, there is a data architecture diagram1000for automated homecage monitoring. The architecture1000shows a two-stage prediction, where a first prediction is made related to an animal's pose estimate. The post estimation may include the position of body features of an animal, such as eyes, nose, ears, a tail, legs, etc. A second prediction of an animal's state is made based on sensor data and the pose estimate. The animal's state may include a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal.

Data architecture1000shows a Deep Neural Network (DNN) architecture that uses a predictive model to predict an animal pose estimate. The pose estimate includes the location of animal body features in sensor data. A pose estimate is generally described as a graph of parts, where a particular node in the graph represents a particular visual feature associated with a body feature. To produce a pose estimate, the graph is fit to an image using flexible interconnections. The DNN may be DeepLabCut, developed by the Mathis Lab (http://www.mousemotorlab.org/deeplabcut).

A set of sensor data inputs1002, such as video data having a plurality of video frames generally represented by image data that are provided as input. Each image1004has at least a first convolutional layer1006applied to it. Each convolutional layer1006,1008,1010represents a convolution or pooling operation applied to the input, and then the input data is passed to another convolutional layer, for example a second convolutional layer1008, and then again, to a third convolutional layer1010, etc. The outputs of each convolutional layer may be inputs to another convolution layer.

The DNN may have a predictive model having many convolutional or pooling layers, and the processing of the image to produce a pose estimate may involve the application of many convolutional layers to the input data. The image may be subsampled, i.e. the image may be subdivided and each portion of the image may analyzed by the DNN.

The predictive model may be generated by training using a data set having a set of labels applied. During training, each layer is convolved to determine an activation map of an applied label associated with the activating input for the layer. As a result, the convolutional layer builds an activation map of when the filter activates when it detects some specific feature in the image input. Each convolutional layer in the DNN stacks its activation map along the depth of the network.

In addition to the convolutional layers, the DNN may also have pooling layers and rectified linear unit layers as is known.

The DNN has deconvolutional layers1012and1014. The deconvolutional layers1012and1014may be fully connected layers that provide high-level reasoning based on the output activations of the preceding convolutional layers.

The DNN generates pose predictions1016. The pose predictions1016are used to determine animal feature locations1018. The pose predictions correspond to the locations (or coordinates) in a video frame of particular animal body parts or indicia. The pose predictions1016may also include, for example, the location of a novel object such as a ball introduced into the homecage. The DNN may function to produce pose predictions (including animal feature locations) for more than one animal, for example, two animals may be located within the homecage.

The DNN may analyze the video input frame by frame. Each body part of the animal in the homecage may be tracked, and the DNN may produce a pose prediction. The DNN may be initially trained on a training set of labeled videos (containing the behaviors of interest). The initially trained DNN model may have further training data input until its pose prediction attains an acceptable accuracy metric.

The pose feature locations1018are used by a Recurrent Neural Network (RNN)1026to learn and predict particular sequences of animal body parts or indicia that identify states (including behaviors) that are more complex.

The RNN1026is a particular type of artificial neural network where nodes are connected into a directed graph The RNN may describe a neural network where there are feedback connections and feedforward connections between nodes, which may add memory to the neural network which can be used to predict temporally dependent phenomena. RNN1026nodes1020may have internal state store state data as part of processing a series of inputs. The RNN1026may be an Long-Short Term Memory (LSTM) network in which a node has an input gate, an output gate and a forget gate. Using the LSTM model, the network may learn the dynamics of animal (or animals) body parts that conform a particular behavior.

The RNN model may be a network which is capable of learning sequential information, and producing sequential predictions based on an input state. The learning of the RNN may classify the patterns of sequences of animal poses into particular behaviors of interest.

The RNN1026has nodes1020and transitions1022, and processes a series of pose predictions1018corresponding to video frames1002and other sensor data.

The nodes1020in the RNN are then used to produce a prediction of an animal state1024. The animal state prediction may include a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal.

The transitions1022may be triggered based on changes in sensor data, pose predictions1016from the DNN, and pose feature locations1018. The transitions1022may trigger a state change of the RNN1026, and may change a predicted output animal state1024.

A behavioral state of an animal may include particular activities, such as actions contemplated by a research study. These may include string pulling, interaction of an animal with a novel object introduced into the homecage, social interaction with other animals, animal interaction with a dark box, rearing, grooming, playing in the homecage, or other homecage activity. The behavior state may include other stereotyped movements in which motor function can be evaluated. The animal's interaction with a novel object may be used to determine a metric of the animal's curiosity.

The position state of an animal may include the animal's position and orientation within the homecage. The position state may be based on the location of the animal's body features or indicia.

The sleep state of an animal may include the animal's sleep/wake cycle, and any potential intermediary states.

The biological state may include food or water intake, respiration, urination, bowel movements, etc. The biological state may include a level of activity of the animal, for example when the animal is highly active, moderately active, or sedentary.

The RNN1026may track quantitative state information for the animal state. This may include the number of transitions1022into a state (or node)1020, the length of time spent in a state (or node)1020, or other metrics.

Referring toFIG. 11A, there is shown an example graph diagram1100of an automated homecage monitoring system. The example graph1100includes the output of an animal state as predicted by the automated homecage monitoring system ofFIG. 9A. This example graph shows a metric associated with an animal's sleep/wake cycle. It is understood that any of the animal states may be graphed, more than one predicted state may be graphed on the same graph. The example graph1100shows a predicted animal state over a 24-hour period correlated to the number of changed pixels in a sensor input from a camera.

The example graph1100may be referred to as an actogram. Existing solutions to monitor homecage activity and state often delivered actograms based on running-wheel statistics. The present homecage monitoring system allows researchers to evaluate additional behavioral parameters beyond those captured using running wheel measurements.

The graph1100may be included in the report generated by the automated homecage monitoring system. The report may also include other metrics and measurements from the homecage monitoring system.

Referring toFIG. 11B, there is shown another example graph diagram1150of an automated homecage monitoring system. The example graph diagram1150shows sleep detection before, during and after a treatment for an animal in a homecage. The homecage monitoring system may function to track animal state such as sleep/wake cycles to determine research data for medical treatments. In the example diagram1150, bars denote the mean percentage amount of time that the animal spent sleeping during the light (white) and dark cycle (black) based on the predicted animal state of the homecage monitoring system. The error bars denote the standard error of the mean (SEM).

Referring toFIG. 12A, there is a front view of another homecage1200having an object. The homecage1200has an animal1202disposed inside it, a microcontroller1222, at least one sensor1224connected to the microcontroller, and a novel object1226. The homecage monitoring system may collect sensor data to predict and track the position of the novel object1226. The position of the novel object1226and the position of the animal1202, in addition to the pose estimates of the animal, may be used to predict the interactions of the animal1202with the novel object1226. These predicted interactions may be an animal behavioral state such as “exploring”. In addition to the predicted animal state, metrics relating to the interaction of the animal1202and the novel object1226may be determined. For example, the duration of time, the number of interactions, and other particular metrics associated with the behavior.

Referring toFIG. 12B, there is a front view of another homecage1230. The homecage1230has an animal1232disposed inside it, a microcontroller1252, at least one sensor1254connected to the microcontroller, and an actuator1256. The actuator1256is connected to the microcontroller1252, and may be activated based on user input, or based on a predicted behavior. The activation of the actuator1256may be automatic based on a closed feedback loop and may activate based on a pre-determined condition. The predetermined condition may be based on a predicted animal state, a particular duration of a predicted animal state, or based on another metric determined by the homecage monitoring system.

The actuator1256may be a haptic actuator, a linear actuator, a pneumatic actuator, a buzzer, a speaker, a thermal actuator, a piezoelectric actuator, a servomechanism, a solenoid, a stepper motor, or the like.

In one example, the actuator is a haptic actuator and the homecage monitoring system may be used for a sleep experiment. In such an example, the actuator may be used to wake an animal1232after a pre-determined sleep duration. The animal state prediction may be used in this example to automatically actuate the actuator and monitor the animal's subsequent behavior. The haptic actuator may use a driver DRV2605 and a vibration motor disc. The actuator1256may be attached to the outside walls1234of the homecage1230, close to the nest location, the actuator may be attached inside (not shown) the homecage1230as required by experimental design. The actuator1256may be connected to the microcontroller1252directly, and the direct connection may include both data and power signals. The actuator1256may alternatively be wirelessly connected to the microcontroller1252, and may be battery powered. This actuator may be used to gently wake-up the animal1232in the homecage. In the above example, the haptic vibration device may have a vibration dampening film (not shown) such as silicon disposed between it and the homecage to dampen the vibrations produced by this device so as to avoid disturbing the neighboring cages.

Referring toFIG. 12C, there is a front view of another homecage1260having a water tank. The homecage1260has an animal1262inside it, a microcontroller1282, at least one sensor1284connected to the microcontroller1282, and a water tank1286.

The water tank1286has a plurality of gradations1290and a salient floating device1288for measuring the water level within. The microcontroller1282may collect sensor data such as video frames from the at least one sensor1284, and use the state prediction method recited herein to determine a water level for an animal1262.

While a water tank1286is shown, it is appreciated that the tank1286could be filled with another consumable such as food. In such a case, the state prediction method may determine a food level. For example, the food level and water level may be determined using a laser position sensor or a laser proximity sensor.

A water level prediction or a food level or a bedding status prediction (or other husbandry variables) by the homecage monitoring system1260may trigger a report or an alert to a user. For example, when the water tank is below a threshold, the homecage monitoring system1260may send an alert to a user to refill the water.

Referring toFIG. 13, there is a method diagram1300for automated homecage monitoring.

At1302, a first predictive model is provided at a memory, the first predictive model for predicting a pose estimate of the animal.

At1304, a second predictive model is provided at the memory, the second predictive model for predicting a state of the animal.

At1306, a sensor input is received at a processor from an at least one sensor.

At1308, the pose estimate is predicted at a processor from the sensor input and the first predictive model. Optionally, the pose estimate may include the position of an at least one indicia indicating the body parts, or physical features of the animal in the homecage. The pose estimate may be a connected graph mapped onto the sensor data and identifying the coordinates of the identified indicia as nodes connected to the other indicia identified on the animal.

At1310, an animal state is predicted at a processor based on the pose estimate, the sensor input and the second predictive model. The pose estimate may be based on the position of at least one indicia associated with an animal in the homecage. The predicted animal state may include at least one of a behavioral state of an animal, a social state of an animal, a position state of an animal, a sleep state of an animal, and a biological state of an animal. The animal state may be used to determine at the processor, a report based on at least one of the animal state, the pose estimate, the sensor input, and the position of the at least one indicia. The report may be stored in a database that is in communication with a memory. The report, the pose estimate, and the position of the at least one indicia may correspond to a common timebase. The output may be output, including to a display device or to a server via network communication. The prediction of pose estimate, and the prediction of animal state may be performed by the microcontroller generally contemporaneously with the sensor input collection. Alternatively, the prediction may be performed generally after the sensor input collection.

Furthermore, the generating the predicted animal pose may also include determining metrics or other data associated with the animal state. This may include, for example, a start time (including a start time of a new animal state when there is a state transition), an end time (including an end time of the current animal state), a duration (or elapsed time), a frequency, any associated sensor data or pose estimate. Further, an occupancy map including an animal's movement path within the homecage (see e.g.FIG. 7) may be included in the report. The occupancy map may be annotated using the pose estimate, and any predicted animal states. An animal's speed may be determined and included in the report from their movement in the homecage. If a novel object has been introduced into the homecage, it's position, movement, and animal's interactions may be included in the report. The animal's food and water supply levels may be included in the report. The status of the bedding in the homecage may be included in the report.

Depending on experimental design, an actuator may be connected to the homecage monitoring system, and attached to the homecage (or positioned proximate to the homecage). From a predicted animal state, the actuator may be activated. The report may include information relating to the actuator activations.

The prediction of pose estimate may be performed using a DNN as described inFIG. 10. The prediction of animal state may be performed using an RNN as described inFIG. 10.

Referring toFIG. 14, there is a method diagram1400for automated homecage monitoring.

At1401, a plurality of sensor inputs are provided at a memory.

At1402, a first predictive model for predicting animal pose from a sensor input is provided at the memory.

At1404, a plurality of predicted animal poses are associated with the plurality of the sensor inputs is generated at a processor, by, for each sensor input in the plurality of sensor inputs, at1406, using the first predictive model to predict a predicted animal pose from the sensor input, and at1408associating the predicted animal pose with the sensor input.

At1410, a plurality of behavior labels associated with the plurality of the sensor inputs are generated at the processor, by, for each sensor input in the plurality of sensor inputs: at1412, associating a behavior label with the sensor input. Optionally, the sensor input is displayed to a user at a display device and the user submits the behavior label for the sensor input using an input device. This labelling by a user may be considered “supervised learning”, where a human guides the training of a predictive model.

At1414, the second predictive model is generated based on the plurality of sensor inputs, the plurality of predicted animal poses, and the plurality of behavior labels.

Referring toFIGS. 15-34, there are shown a plurality of user interfaces of the homecage monitoring system. These user interfaces may be provided by a web server associated with server104(seeFIG. 1). The user interfaces may be provided by a client application running on either of user devices106and/or116, and the client application may connect to server104(seeFIG. 1) via an Application Programming Interface (API).

Referring next toFIG. 15, there is shown a user interface1500of a homecage monitoring system in accordance with at least one embodiment. The user interface1500may be a landing page displayed to a user when the homecage application is started or initially accessed. The user interface1500may provide a dashboard with information provided in an easy to read manner so that the user can understand quickly the state of the homecage management system. The user interface1500may have a plurality of user input controls. The activate sensors control1501may enable any sensors that have been turned on in the sensor configure menu1511.

The video preview control1502may enable a video preview that will display a camera preview.

The stream video control may enable the live stream when recording is started

The user interface1500may have a plurality of recording settings.

A continuous control1504may allow for continuous recording to run until it is stopped. When the continuous control1504is set to time the length of recording time may be changed.

A recording time control1506may configure how long the recording will run for. When timer is finished recording may cease.

A length of video1507may configure the length of each video segment in minutes. If recording time=1 hr, and length of video=10, the system may record 6, 10-minute segments.

A save directory control1508may open the choose save directory window to pick where files are saved.

A record control1509starts the recording with the current settings. If pressed again, it stops the recording.

A monitor control1510starts the monitor only without recording to disk.

A sensor config control1511may open the sensor config window.

A video config control1512may open the video config window.

A settings preview1513may display a box that contains the current settings selected for the application. This includes video settings, which sensors are enabled and the current file save location.

A program log preview1514may keep a log of the tasks that the app is completing. For example: The time a recording was started or stopped.

A status bar1515may inform the user what the app is doing at the moment (idle, recording, finished).

Referring next toFIG. 16, there is shown another user interface1600of a homecage monitoring system in accordance with at least one embodiment. The user interface1600may show the sensor configuration window corresponding to a selection of the sensor config control1511(seeFIG. 15) by the user.

The sensor config interface1600may allow for a variety of sensor settings for each homecage.

A temp/humidity sensor control1601may enable the temperature and humidity sensor. This sensor may monitor temperature and humidity in the homecage environment.

A gas sensor control1602may enable the ammonia sensor. This sensor may monitor the concentration of ammonia in the homecage.

A thermal sensor control1603may enable the thermal camera.

A tracking control1604may enable motion tracking in the homecage.

A motion control1605may enable the collection of motion data from video.

A time window control1603may control the length of the time window of motion that is accumulated.

A video estimation control1607may enable the collection, estimation and event creation based on the monitoring of all the variables (sensors). This may include: water level estimation, food level estimation, bedding quality (feces and ammonia). Checking this box may also enable the event detection for when these variables are out of normal range (anomaly detection).

A frequency slider control1608may be used to determine the frequency of event detection.

A flush interval control1609may be used to determine how often the system should clean its local files to preserve space.

The refresh modules control1610may be used to refresh the connected modules.

The sensor preview1611may show a preview box that displays what sensors are enabled.

Referring next toFIG. 17, there is shown another user interface1700of a homecage monitoring system in accordance with at least one embodiment. When video configuration control1512(seeFIG. 15) is pressed the interface1700may be displayed. The user interface1700may be used to change the connected camera settings. Settings changes here may reflect immediately when the preview checkbox is pressed.

A brightness control1701may be used to change the brightness of the camera by sliding the slider.

A sharpness control1702may be used to change the sharpness of the camera by sliding the slider.

A shutter speed control1703may be used to change the shutter speed of the camera.

A framerate control1704may be used to decide the frame capture rate of the camera. Default may be 30. When recording at certain resolutions, the framerate may be changed as required.

A contrast control1705may be used to change the contrast of the camera.

An exposure control1706may be used to change the exposure of the camera.

An ISO control1707may be used to change the camera sensitivity to light.

Preview settings1708may include a position control for the current position of the camera preview in relation to the screen, and a size control for changing the size of the preview window.

A resolution control1710may be used for changes to the resolution of the camera for recording.

Referring next toFIG. 18, there is shown another user interface1800of a homecage monitoring system in accordance with at least one embodiment. The file save interface1800may be shown when a user selects save directory control1508(seeFIG. 15). This save directory window may allow a user to specify the location of recorded data from the plurality of homecages.

Referring next toFIG. 19, there is shown another user interface1900of a homecage monitoring system in accordance with at least one embodiment. The user interface1900may provide the user with the required functionality to administer the homecages in the rodent colony and also to display, monitor and analyze the collected information. The administration functionality may include: adding/removing/modifying information about particular homecages and colonies. Information for each homecage may be viewed, displayed, monitored and analyzed by clicking or selecting on one of the homecages in the list. This information may include water and food levels, bedding quality and animal motion.

The homecage monitoring system may have user accounts setup individual for each user. The homecage monitoring system may have role-based access control, and may provide for different types of users such as a colony administrator, a technician, and a researcher. The different types of users may view different types of information. For example, for the admin user, the system may allow homecages to be added, animals to be added, etc. Continuing the example, a technician user may be presented with quick reports of particular homecages that report anomalies. Finally in this example, an experimenter user may be provided more detailed analysis of animal behavior in the homecage or colony.

A navigation widget1902may allow a user to navigate to a listing of colonies, cages, and events.

A quick link widget1903may allow a user to navigate to one of the homecages with outstanding alerts.

A listing1905of each of the homecages and associated information may be provided, with links to the homecages.

Referring next toFIG. 20, there is shown another user interface2000of a homecage monitoring system in accordance with at least one embodiment. The homecage data display interface2000may show one or more graphs2002of activity and measured data from a particular homecage. The homecage interface2000may be displayed when a user selects one homecage from the listing1905or the quick link widget1902(seeFIG. 19). Furthermore, the user interface2000may include a listing of detecting events in the homecage including any alarms.

Referring next toFIG. 21, there is shown another user interface2100of a homecage monitoring system in accordance with at least one embodiment. The alarm interface2100shows listing1905(seeFIG. 19) and may allow a user to confirm that an alarm for a homecage has been addressed. Once selected, a user may undo the operation as shown.

Referring next toFIG. 22, there is shown another user interface2200of a homecage monitoring system in accordance with at least one embodiment. The user interface2200may show a colony listing2201, such that multiple different colonies of homecages may be monitored. Each colony may have a plurality of homecages. Information regarding unresolved alarms2202in all colonies may be indicated.

Referring next toFIG. 23, there is shown another user interface2300of a homecage monitoring system in accordance with at least one embodiment. The colony interface2300may show a listing2301of a plurality of homecages in the colony, along with relevant information and statistics collected from the colony. This may include information about unresolved alarms2302in the colony. The colony interface2300may further be used to navigate to the homecage interface of each homecage in the colony by selecting it from listing2301.

Referring next toFIG. 24, there is shown another user interface2400of a homecage monitoring system in accordance with at least one embodiment. The colony statistics interface2400may show a summary of statistics of the plurality of homecages in a colony. This may include statistics on the animal characteristics (sex, age, etc.), or any other determined values (activity level, movement level, strain level, states, etc.) for the homecages as described herein. A filter interface2401may be provided in order to allow a user to display the plurality of characteristics and determined values of the colony. A graph of the selected characteristics or determined values2402may be shown.

Referring next toFIG. 25, there is shown another user interface2500of a homecage monitoring system in accordance with at least one embodiment. A homecage detail interface2500may show information about a particular homecage. This may include, as noted above, animal characteristics (sex, age, etc.), or any other determined value (activity level, movement level, strain level, stages, etc) as disclosed herein.

A cage plot control2501may show determined graphs for the homecage.

An animal plot control2502may show animal related graphs for each of the animals in the homecage.

An unresolved alarm indicator2504may indicate if there are unresolved alarms for that homecage.

A streaming control2503may show a live video stream from the homecage.

Referring next toFIG. 26, there is shown another user interface2600of a homecage monitoring system in accordance with at least one embodiment. Responsive to a cage plot control2501(seeFIG. 25) selection, the cage plot interface2600may be shown. This may include data, statistics, and graphs2603related to the homecage. A table type control2601and time period control2602may allow a user to change the parameters for the data, statistics, and graphs2603. Different types of plots may include sensor data (temperature, humidity, etc.), video data (water, food, bedding quality) and motion data.

Referring next toFIG. 27, there is shown another user interface2700of a homecage monitoring system in accordance with at least one embodiment. The user interface2700may show an alternate graph display2702of data from a homecage.

Referring next toFIG. 28, there is shown another user interface2800of a homecage monitoring system in accordance with at least one embodiment. Responsive to a selection of the streaming control2503(seeFIG. 25), a video stream interface2800including a video stream2802may be viewed by a user for the homecage.

Referring next toFIG. 29, there is shown another user interface2900of a homecage monitoring system in accordance with at least one embodiment. The management interface2900may allow a user to select an add homecage control2901, an add animal control2902, a modify homecage control2903, and a delete homecage2904control.

Referring next toFIG. 30, there is shown another user interface3000of a homecage monitoring system in accordance with at least one embodiment. Responsive to a user's selection of the add homecage control2901(seeFIG. 29), an add homecage interface3000may be displayed allowing a user to enter information associated with the new homecage.

Referring next toFIG. 31, there is shown another user interface3100of a homecage monitoring system in accordance with at least one embodiment. Responsive to a user's selection of the add animal control2902(seeFIG. 29), an add animal interface3100may be displayed allowing the user to add an animal and associated parameters to a homecage.

Referring next toFIG. 32, there is shown another user interface3200of a homecage monitoring system in accordance with at least one embodiment. Responsive to a user selection of the modify homecage control2903(seeFIG. 29) a modify homecage interface3200may be displayed to allow a user to change parameters associated with a homecage.

Referring next toFIG. 33, there is shown another user interface3300of a homecage monitoring system in accordance with at least one embodiment. Responsive to a user selection of the modify homecage control2903(seeFIG. 29) another modify homecage interface3300may be displayed to allow a user to change parameters associated with a homecage.

Referring next toFIG. 34, there is shown another user interface3400of a homecage monitoring system in accordance with at least one embodiment. A general status page3402may be provided to summarize the actions and alerts required for the plurality of homecages, the plurality of colonies, and the plurality of animals monitored using the homecage monitoring system.

Various embodiments have been described herein by way of example only. Various modification and variations may be made to these example embodiments without departing from the spirit and scope of the invention, which is limited only by the appended claims. Also, in the various user interfaces illustrated in the figures, it will be understood that the illustrated user interface text and controls are provided as examples only and are not meant to be limiting. Other suitable user interface elements may be possible.