Predicting weather radar images

Predicting weather radar images by building a first machine learning model to generate first predictive radar images based upon input weather forecast data, and a second machine learning model to generate second predictive radar images based upon historical radar images and the first predictive radar images. Further by generating enhanced predictive radar images by providing the first machine learning model weather forecast data for a location and time and providing the second machine learning model with historical radar images for the location and an output of the first machine learning model.

BACKGROUND

The disclosure relates generally to predicting weather radar echo images. The disclosure relates particularly to using deep learning and Numeric Weather Prediction (NWP) models.

Weather prediction may be characterized as either forecasting, or nowcasting. Forecasting relates to the prediction of future weather conditions using complex numeric models incorporating historic data as well as physical dynamic and thermodynamic calculation models. Nowcasting typically relates to a detailed description of weather conditions forecast for the next 0-6 hour time period.

Radar nowcasting seeks to generate realistic images of future radar echo images. Realistic radar images provide a basis for aviation flight-path decision making as well as higher quality forecasts of short-term, future precipitation.

Radar echo images relate to the amount of moisture present in the atmosphere. The moisture reflects the radar signal leading to an image based upon the strength of the returning signal. Precipitation is the amount of moisture which falls to the ground. Typically, not all atmospheric moisture becomes precipitation. Efforts to convert precipitation levels and rates to radar reflectivity images can be inaccurate. (A 10 cm equivalent reflectivity computed from precipitation data generally does not look like the actual radar reflectivity image associated with the precipitation data).

Radar nowcasting focuses on predictions for the next 0-6 hours, unlike general weather forecasting, which may make predictions for the next 10 days of more. Radar nowcasting requires models which can be processed quickly and provide outputs having temporal resolution measured in minutes and high spatial resolution as well. Forecasting typically utilizes large, computationally intensive models spanning large time frames and generating outputs with coarser spatial and temporal resolution.

Radar-only based nowcasting is typically based upon images from a single radar location. Tracking radar echo by correlation (TREC) calculates correlation coefficients between successive images of radar echoes and uses the maximum correlation values to determine the motion vectors of different regions. The determined vectors are then used to predict future movement of the regions. TREC is image based with no regard for the scale of internal dynamics of weather region elements.

Continuity of TREC (COTREC) imposes a vector continuity constraint upon TREC where TREC predicts wind velocity poorly. Efforts to improve COTREC have added a parameter related to cloud growth and decay. Additional efforts have added shape analysis of precipitation events to COTREC to improve the predicted motion vector field, improving accuracy of the predictions.

TREC occasionally yields a vector in a direction contradictory to surrounding vectors. One effort to address this issue extended the correlation determination to three consecutive images from two.

A multi-scale TREC model uses a first low-resolution TREC calculation to determine synoptic-scale motion of regions. A second, high-resolution TREC calculation is then performed on each large, low-resolution region to predict meso, local-scale internal motion within each large region.

Machine learning has been applied to radar nowcasting by training a neural network to analyze sequences of radar images and to make future image predictions based upon the processed sequence of images without regard for dynamic and thermodynamic conditions. The decoupling time (the length of time before the predictions and actual events have no similarity) of such methods is low, 1-2 hours.

SUMMARY

In one aspect, the invention includes methods, systems and computer readable media associated with predicting weather radar images by building a first machine learning model to generate first predictive radar images based upon input weather forecast data, and a second machine learning model to generate predictive radar images based upon historical radar images and the first predictive radar images. Further by generating predictive radar images by providing the first machine learning model weather forecast data for a location and time and providing the second machine learning model with historical radar images for the location and an output of the first machine learning model.

In one aspect the invention includes methods, systems, and computer readable media associated with predicting weather radar images by interpolating weather simulation data to increase the spatial and temporal resolution of the data, building a first machine learning model to generate first predictive radar images based upon the interpolated weather forecast data, and building a second machine learning model to generate predictive radar images based upon historical radar images and the first predictive radar images. Further by generating predictive radar images from the second machine learning model by providing the first machine learning model weather forecast data for a location and time and providing the second machine learning model with radar images for the location and time and an output of the first machine learning model. The resolution of the predictive radar images is then enhanced using a third machine learning model.

In one aspect the invention includes methods, systems, and computer readable media associated with predicting weather radar images by interpolating weather simulation data to increase the spatial and temporal resolution of the data, building a first machine learning model using interpolated historic weather simulation and radar image data to generate first predictive radar images based upon current interpolated weather forecast data, and building a second machine learning model using historic and first radar image data to generate predictive radar images based upon historical radar images and the first predictive radar images. Further by generating predictive radar images from the second machine learning model by providing the first machine learning model current interpolated weather forecast data for a location and time and providing the second machine learning model with radar images for the location and time and an output of the first machine learning model. The resolution of the predictive radar images is then enhanced using a third machine learning model.

In one aspect, the invention includes methods, systems and computer readable media associated with predicting weather radar images by receiving current weather forecast data and radar images associated with the current weather forecast data, using a first machine learning model to generate first radar images based upon the current weather forecast data, and using a second machine learning model to generate predictive radar images based upon current radar images and the first radar images.

Aspects of the disclosed systems, methods and computer readable media expand the decoupling time of the radar image nowcasting to 3-6 hours by combining NWP data and machine learning methods. The disclosed inventions enhance the radar nowcasting images by using machine learning to translate NWP forecast data into radar reflectivity images. The use of machine learning models to translate the NWP data produces realistic cloud footprint images which accurately predict future precipitation events. (Predicted radar images are realistic in that they look like actual radar images.) The use of a third machine learning model provides fine spatial resolution across the entire nowcasting window of predicted images. The disclosed inventions can be extended to provide altitude specific radar images (e.g., ground based images or aviation altitude-based images) by building the first machine learning model using altitude specific images and providing the second machine learning model with altitude specific radar image sequences.

DETAILED DESCRIPTION

The disclosed inventions yield improvements to radar image nowcasting by the combination two machine learning models. For each time-step of the nowcasting window, the first model takes current prediction data from an NWP forecast model for the time-step and produces radar reflectivity images illustrative of the forecast data. (Thereby incorporating the most up-to-date measure of dynamic and thermodynamic conditions into the nowcasting prediction.) The method uses these time-step specific radar images of forecast conditions, in conjunction with a sequence of current radar images, to generate a predictive image for the next time step of the nowcasting window by the second model. The disclosed inventions proceed from the beginning of the nowcasting window to the end of the window. The method then uses a predictive radar image from each time step as part of the sequence of current radar images for generating the predictive radar image for the next time step.

In an embodiment, a study domain is defined for nowcasting. The study domain may be automatically to manually selected. In an embodiment, automatic study domain selection proceeds according to historic data regarding user preferences and previous study domain selections. In this embodiment, the method defines the study domain as a rectangular geographic region. (Non-rectangular regions may be designated as the study domain.) In an embodiment, the area of the continental United States, is defined as the study domain. In an embodiment, the method divides the United States into sub-regions, in this embodiment, the method defines Northeast, Southeast, Northwest, Southwest, Alaska, and Hawaii/Pacific, regions. For the study domain, the method divides the area of the domain and defines a set of rows dividing the domain North-South with each row representing a portion of the overall domain N-S distance divided by a user selected nowcasting resolution (typically between 250 meters (0.155 miles) and 1 kilometer (0.62 miles)). The method further defines the study domain as a set of columns, each column representing the overall E-W distance of the domain divided by the selected resolution.

In this embodiment, increasing the size and resolution of the definition of the study domain requires additional computing resources in terms of CPU capacity, system memory, GPU speed and memory, to generate the nowcasting output in a reasonable amount of time. In an embodiment, the continental US can be processed as the study domain at a spatial resolution of 1 kilometer using a computer having equipped with two NVIDIA GTX 1080 graphics cards, 32 GB memory, one Intel 6700K CPU and 1 TB hard disk space.

In an embodiment, the method comprises four modules: a spatial-temporal interpolation module, an NWP radar mapping module (model M, a radar prediction module (model A), and a spatial downscaling module (model D). In this embodiment, after defining the study domain, the use begins by selecting an initial temporal resolution (e.g., 2 min, 5 min, etc.), and an initial spatial resolution (e.g., 1 km). In this embodiment, the method obtains NWP data for the study domain. In this embodiment, the NWP data includes data having a resolution of at least about 3 km (1.86 miles) and covers a time period at least as long as the desired nowcasting time window (e.g., 6 hours).

In an embodiment, the NWP data includes: precipitable water, outgoing longwave radiation, cumulative and non-cumulative precipitation rate, troposphere pressure, troposphere temperature, surface pressure, surface temperature, convective available potential energy (CAPE), mean sea-level pressure, 850 mb geospatial height, 10 m u-wind, 10 m v-wind, mixing ratio, and sea-land mask. In an embodiment, the NWP data may comprise accumulated precipitation rather than the non-cumulative precipitation rate requiring the method to take a derivative of the data to compute a non-cumulative precipitation rate in an initial NWP data set. In an embodiment, the NWP data may further comprise turbulent kinetic energy, heat flux, mixing height, land use, cloud cover mask, and precipitation type in an expanded NWP data set.

In an embodiment, the method interpolates each variable of the NWP data set into the user specified temporal resolution. In this embodiment, the method uses an interpolation function such as Piecewise Cubic Hermite Interpolating Polynomial (PCHIP) or a Modified Akima spline interpolation (mAkima) function to avoid overshot values between sampled points. In this embodiment, for each time step following the temporal interpolation, the method interpolates the data and downscales the resolution to the user specified spatial resolution. In this embodiment, the method uses natural neighbor interpolation for non-stationary variable data and Krigin interpolation for all stationary variable data.

In an embodiment, the method receives or obtains radar reflectivity images over the area of the study domain for the time period of the NWP data. In this embodiment, the radar reflectivity images may comprise a single radar station image or a mosaic of images from a plurality of radar locations. In an embodiment, the radar reflectivity images may be base reflectivity images, composite reflectivity images, constant altitude plan position (CAPPI) images, or other reflectivity images which best fit the desired end use envisioned for the models of the method. (e.g., a model intended to produce aviation altitude radar predictions for flight plan guidance would obtain aviation altitude radar images). In this embodiment, the radar image resolution is at least as fine as the user specified spatial resolution.

In an embodiment, the method uses the interpolated NWP data and radar images to train the first machine learning model, mapping model M. In this embodiment, the first model comprises a convolutional neural network (CNN) such as INCEPTION V3 for image classification, other known types of CNN may also be used for the methods of the invention. (Note: the terms “INCEPTION”, “INCEPTION V3”, and “NVIDIA”, may be subject to trademark rights in various jurisdictions throughout the world and sed here only in reference to the products or services properly denominated by the marks to the extent that such trademark rights may exist.)

In an embodiment, the input data comprises data pairs of interpolated NWP and radar reflectivity data having the same time stamp. In this embodiment, the method normalizes all NWP data into a 0-1 scale using; [(pi−pmin/(pmax−pmin)], where pi, equals each variable at index (time stamp) i. In an embodiment, the method sets maximum radar reflectivity to 75 decibels (dB) and minimum reflectivity to −15 dB. In this embodiment, the method uses the radar reflectivity images in their original orientation and are also rotated to different orientations to prevent moving trend bias in the trained model. (In one embodiment, the method rotates the images 90, 180 and 270 degrees, though other rotations are possible.) The method trains model M using unlabeled data—data which is not characterized or labeled by a human entity.

In an embodiment, the method gathers seasonal NWP data to train the model M. In an embodiment, the method trains two versions of the model M, a winter version and a summer version, or a cold rain version and a warm rain version. After the two models are trained the method utilizes whichever model of the winter/summer pair most closely corresponds to the current conditions associated with the desired radar nowcasting window.

In an embodiment, the method trains the model M using the initial NWP data set described above. In one embodiment, the method uses the expanded NWP data set described above to train the model. In these embodiments, the method then evaluates the weights associated with the respective variables to determine if one or more variables are poorly represented (have little impact) on the output of the model. Variables having little representation/impact are removed from the data set and the model is retrained using the revised data set. This retraining using the revised data set yields a model of less complexity and having fewer computational requirements. In some embodiments, model M is trained using a leaky rectified linear unit activation function and a mean square error regression loss function.

In an embodiment, the second machine learning model, the advection model, or model A, comprises a long-short-term memory/CNN (LSTMC), though other types of recursive and convolutional neural networks may be used as well. In this embodiment, the method uses a window size of six time intervals, though other window sizes could be used to increase or decrease the size of the model. In this embodiment, the training data set includes sequences of radar reflectivity images associated with the time stamps of the NWP data processed by the first neural network model. In this embodiment, the method processes NWP data by the trained model M yielding a radar reflectivity image for a particular future time stamp interval. The method pairs that image with actual radar reflectivity images for time periods immediately prior to the future time stamp interval. The method uses multiple pairings of first model outputs and corresponding radar images sequences to train model A. In some embodiments, model A is trained using a leaky rectified linear unit activation function and a mean square error regression loss function.

In an embodiment, the method trains a third machine learning model. In this embodiment, the third model, the downscaling model, or model D, comprises a CNN, or neural network model having a similar loss function. The method trains this model using a data set comprising National Oceanic and Atmospheric Administration (NOAA) composite reflectivity images including national composite reflectivity products NCZ, and NCR. In this embodiment, the NCZ has a 4 km (2.48 mile) resolution and the NCR has 1 km ((0.62 mile) resolution. In this embodiment, the method crops the images of the NCZ, and NCR as needed to fit the study domain. In this embodiment, the trained model enhances the resolution of provided images similarly to moving from the 4 km (2.48 mile) to 1 km (0.62 mile) resolution change for NCZ and NCR. In some embodiments, model D is trained using a leaky rectified linear unit activation function and a mean square error regression loss function.

In an embodiment, the method utilizes the trained models to create the radar nowcasting image sequence for a 0-6 hour nowcasting window. In this embodiment, the method collects a sequence of radar reflectivity images from time T1to Tm: R1, R2, R3. . . where Tmrepresents the beginning of the nowcasting window. The prediction window beings at Tm+1. The method collects NWP data for Tm+1and uses model M to generate a radar reflectivity image based upon the NWP data, for Tm+1: Sm+1. The method then uses Sm+1in conjunction with the radar image sequence R1, R2, R3. . . as inputs to model A to generate the predicted radar reflectivity image Rm+1, for Tm+1.

In an embodiment, the method then uses the third machine learning model to enhance the resolution of Rm+1, to the user specified resolution. In an embodiment, the user specifies the final resolution as the resolution of model A. In this embodiment, the use of the third machine learning model is unnecessary.

In an embodiment, the method moves the time interval forward from Tm+1, to Tm+2+. In this embodiment, the method gathers NWP data for Tm+2and proceeds as described above using the new NWP data and model M to generate Sm+2, and adding Rm+1, the sequence of images used as input to model A to generate Rm+2. In this embodiment, the method steps through the desired nowcasting window in intervals equal to the user specified temporal resolution until the nowcast is completely generated for the specified window size. In this embodiment, for each new nowcast time interval Tk+1, the uses the newest prediction of reflectivity as the latest observation of reflectivity. The method then proceeds with the nowcast target interval moving from Tm+k, to Tm+k+1.

In an embodiment, the complete nowcasting model of up to three machine learning models resides upon a local device and processes NWP and radar image data received over a network communications connection and providing a local output to a user via a display element such as a video screen. In an embodiment, the complete nowcasting model resides in cloud or edge cloud resources, receiving and processing NWP and radar image data and providing output images over a network communications link to local users via a smart phone or local computer application. In an embodiment, the respective machine learning models of the method are trained using cloud or edge cloud resources. In this embodiment, the trained models are then provided locally for use processing the NWP and radar image data to generate desired nowcasts.

FIG. 1provides a schematic illustration of exemplary network resources associated with practicing the disclosed inventions. The inventions may be practiced in the processors of any of the disclosed elements which process an instruction stream. As shown in the figure, a computer system100comprises a server computer150.FIG. 1depicts a block diagram of components of server computer150within a networked computer system100, in accordance with an embodiment of the present invention. It should be appreciated thatFIG. 1provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments can be implemented. Many modifications to the depicted environment can be made.

Server computer150can include processor(s)154, memory158, persistent storage170, communications unit152, input/output (I/O) interface(s)156and communications fabric140. Communications fabric140provides communications between cache162, memory158, persistent storage170, communications unit152, and input/output (I/O) interface(s)156. Communications fabric140can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, communications fabric140can be implemented with one or more buses.

Memory158and persistent storage170are computer readable storage media. In this embodiment, memory158includes random access memory (RAM)160. In general, memory158can include any suitable volatile or non-volatile computer readable storage media. Cache162is a fast memory that enhances the performance of processor(s)154by holding recently accessed data, and data near recently accessed data, from memory158.

Program instructions and data used to practice embodiments of the present invention, e.g., the machine learning program175, are stored in persistent storage170for execution and/or access by one or more of the respective processor(s)154of server computer150via cache162. In this embodiment, program175comprises four modules. Data interpolation module176, receives NWP data and interpolates the data to increase the spatial and temporal resolution of the NWP data. The radar mapping module177, including model M, receives the interpolated NWP data and generates a predicted radar image mapped to the data. The radar prediction module178, including model A, receives the mapped image from the mapping module177, and generates the next predicted radar image. The downscaling module179, including model D, receives the predicted radar images from prediction module178, and increases the spatial resolution of the images. In this embodiment, persistent storage170includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage170can include a solid-state hard drive, a semiconductor storage device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information.

The media used by persistent storage170may also be removable. For example, a removable hard drive may be used for persistent storage170. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer onto another computer readable storage medium that is also part of persistent storage170.

Communications unit152, in these examples, provides for communications with other data processing systems or devices, including networked devices by way of network114. In these examples, communications unit152includes one or more network interface cards. Communications unit152may provide communications through the use of either or both physical and wireless communications links. Software distribution programs, and other programs and data used for implementation of the present invention, may be downloaded to persistent storage170of server computer150through communications unit152.

I/O interface(s)156allows for input and output of data with other devices that may be connected to server computer150. For example, I/O interface(s)156may provide a connection to external device(s)190such as a keyboard, a keypad, a touch screen, a microphone, a digital camera, and/or some other suitable input device. External device(s)190can also include portable computer readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, e.g., machine learning program175on server computer150, can be stored on such portable computer readable storage media and can be loaded onto persistent storage170via I/O interface(s)156. I/O interface(s)156also connect to a display180.

Display180provides a mechanism to display data to a user and may be, for example, a computer monitor. Display180can also function as a touch screen, such as a display of a tablet computer.

FIG. 2provides a flowchart200, illustrating exemplary activities associated with the practice of the disclosure. After program start, a study domain is defined, user specified spatial and temporal resolutions are set and interpolation module176then obtains NWP data for the defined study domain at210. In an embodiment, the NWP data includes data selected from precipitable water, outgoing longwave radiation, cumulative and non-cumulative precipitation rate, troposphere pressure, troposphere temperature, surface pressure, surface temperature, CAPE, mean sea-level pressure, 850 mb geospatial height, 10 m u-wind, 10 m v-wind, mixing ratio, and sea-land mask, turbulent kinetic energy, heat flux, mixing height, land use, cloud cover mask, and precipitation type.

At220, the radar mapping module177obtains radar reflectivity images for the defined study domain during the time interval of the acquired NWP data set. In this embodiment, radar mapping module176of the method may acquire images having the user specified spatial resolution or a lower resolution. In an embodiment, radar mapping module176, must acquire radar images having at least the user desired temporal resolution for use by the models. In this embodiment, the images may comprise a single sourced or mosaic of multiple sources. The images may comprise base, composite, CAPPI or other radar reflectivity images according to the end use intended for the method. In an embodiment, altitude specific images are gathered to train the overall model to provide altitude specific output images.

In an embodiment, at230, program175trains the model M, of radar mapping module176, using the radar images and NWP data. In this embodiment, the method normalizes the NWP data and interpolates the data to match the user specified spatial and temporal resolutions for the study domain and nowcasting window respectively. In this embodiment, program175of the method normalizes the radar imagery, setting the maximum (75 dB) and minimum (−15 dB) signal values for the imagery data. In an embodiment, program175of the method then trains a CNN using the normalized and interpolated data, yielding a first machine learning model which produces a realistic radar reflectivity image from an input of NWP data.

At240, program175trains the model A, of the radar prediction module177. Program175of the method uses radar image sequences paired with output images from model M to train model A. The output of model M predicts the next image in a given sequence. In an embodiment, model A of the method comprises an LSTM model. After training, program175uses model A of the prediction module177, to produce a next predicted radar reflectivity image in a sequence using a given image sequence and corresponding output from model M as inputs.

At250, program175generates radar imagery nowcasting outputs using the trained models from230and240. In an embodiment, interpolation module176gathers NWP data for the study domain and the desired nowcasting window. Mapping module177, gathers radar imagery for the time intervals immediately preceding the nowcasting window. Beginning with an interval of Tm+1, as the first-time interval of the nowcasting window, program175provides normalized NWP data to model M and generates a radar reflectivity prediction, Sm+1, for interval Tm+1. In this embodiment, mapping module177, provides the sequence of radar images ending with Sm+1to the radar prediction module178, and model A. Model A, generates radar image Rm+1, as the prediction for interval Tm+1. In an embodiment, at260model D, of downscaling module179enhances the resolution of Rm+1, using Rm+1, as an input and yielding a radar image having the user specified spatial resolution.

In an embodiment, program175generates a radar image for each time interval of the desired nowcasting window, using the Rm+1from each previous interval as part of the sequence of images provided as input to model A.

FIG. 3provides a schematic illustration300, of the neural network architecture of model M as used in some embodiments of the inventions. NWP data and corresponding radar image pairs are provided as the training data set. As shown in the figure, the NWP data310, is passed as inputs to multiple convolutional layers320and the model is trained to match the NWP data310to the corresponding radar images340. In an embodiment, the model uses backpropagation, a leaky rectified linear unit (ReLU) activation function and a mean square error (MSE) regression loss function330, to train the model parameters.

FIG. 4provides a schematic illustration400, of neural network layer architecture used in some embodiments for the advection model A. As provided in the figure, each network layer comprises an LSTM block410and several convolutional blocks420. The LSTM block410accommodates the time sequence nature of the input data. In some embodiments, the model is trained using a leaky ReLU activation function and an MSE regression loss function430. In an embodiment, model A is trained using data pairs including time series radar images and predicted radar images for the corresponding time window from model M.

FIG. 5provides a schematic illustration500, of an architecture for model D in some embodiments, As shown in the figure, the model includes 4-kilometer resolution data510, input to a bilinear function520and several convolutional layers530. The model is trained to match the 4-kilometer data with 1-kilometer resolution data540using backpropagation, a leaky ReLU activation function and an MSE loss function550.

The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The invention may be beneficially practiced in any system, single or parallel, which processes an instruction stream. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.