Parking lot free parking space predicting method, apparatus, electronic device and storage medium

A parking lot free parking space predicting method, apparatus, electronic device and storage medium are provided. The method comprises: building a parking lot association graph for parking lots in a region to be processed; aggregating environment context features of neighboring parking lots according to weights of edges between the neighboring parking lots and a parking lot i to obtain a representation vector of the parking lot i at a current time; and pre-training a graph attention neural network model using the environment context features of the neighboring parking lots and free parking space information, and a gated recurrent neural model according to the representation vector of the parking lot it at the current time.

The present application claims the priority of Chinese Patent Application No. 202010076257.0, filed on Jan. 23, 2020, with the title of “Parking lot free parking space predicting method, apparatus, electronic device and storage medium”. The disclosure of the above application is incorporated herein by reference in its entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates to computer application technologies, and particularly to a parking lot free parking space predicting method, apparatus, electronic device and storage medium in the field of artificial intelligence.

BACKGROUND OF THE DISCLOSURE

When drivers need to park vehicles, they usually want to know which nearby parking lots can provide free parking spaces in near future, and correspondingly, if free parking space information of parking lots can be predicted, the drivers' parking efficiency can be improved effectively.

At present, annotation data may be generated based on a user's feedback, thereby predicting a degree of difficulty in parking vehicles in a certain region. However, the annotation data obtained in this manner might be inaccurate, for example, the user himself does not have precise metrics of the degree of parking difficulty and provides a coarse evaluation only by virtue of his own feeling. Furthermore, some misoperations of the user might occur and affect the feedback accuracy. The prediction results are very inaccurate on account of these problems.

SUMMARY OF THE DISCLOSURE

In view of the above, the present application provides a parking lot free parking space predicting method, apparatus, electronic device and storage medium.

A parking lot free parking space predicting method, comprising:

building a parking lot association graph for parking lots in a region to be processed, each junction therein representing a parking lot, and connecting any two parking lots meeting a predetermined condition through edges;

processing as follows for any parking lot i:

determining local space correlation information of parking lot i at a current time according to environment context features of the parking lot i and neighboring parking lots which are in the parking lot association graph and connected to the parking lot i through edges;

determining global space correlation information of the parking lot i at the current time according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time;

determining time correlation information of the parking lot i at the current time according to the local space correlation information and global space correlation information, and predicting free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, the connecting any two parking lots meeting a predetermined condition through edges comprises: connecting any two parking lots with a distance less than or equal to a predetermined threshold through edges.

According to a preferred embodiment of the present disclosure, the determining local space correlation information of parking lot i at a current time comprises: determining local space correlation information of parking lot i at a current time based on a graph attention neutral network model;

the determining global space correlation information of the parking lot i at the current time according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time comprises: building the soft allocation matrix according to the local space correlation information of the parking lots at the current time based on a hierarchical graph neural network model, and determining the global space correlation information of the parking lot i at the current time according to the soft allocation matrix:

the determining time correlation information of the parking lot i at the current time and predicting free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time comprises: determining time correlation information of the parking lot i at the current time based on a gated recurrent neural network model, and predicting the free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, determining local space correlation information of parking lot i at a current time based on a graph attention neutral network model comprises:

as for neighboring parking lots, determining weights of edges between the neighboring parking lots and the parking lot i at the current time according to the environment context features of the neighboring parking lots and parking lot i at the current time, respectively;

aggregating the environment context features of the neighboring parking lots according to the weights of edges between the neighboring parking lots and the parking lot i to obtain a representation vector of the parking lot i, and regarding the representation vector as the local space correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, a weight αijof the edge between any neighboring parking lot j and parking lot i is represented as

αij=exp⁡(cij)∑k∈Niexp⁡(cik);
where cij=Attention(Waxi,Waxj); Attention represents a graph attention mechanism; Nirepresents the number of neighboring parking lots; xirepresents the environment context feature of the parking lot i at the current time; xjrepresents the environment context feature of neighboring parking lot j at the current time; Warepresents a model parameter obtained by pre-training.

According to a preferred embodiment of the present disclosure, the representation vector xi′=σ(ΣjϵNiαijWaxj);

where Nirepresents the number of neighboring parking lots; xjrepresents the environment context feature of any neighboring parking lot j among Nineighboring parking lots at the current time; αijrepresents a weight of the edge between the neighboring parking lot j and parking lot i at the current time; Warepresents a model parameter obtained by pre-training; σ represents an activation function.

According to a preferred embodiment of the present disclosure, the building the soft allocation matrix according to the local space correlation information of the parking lots at the current time, and determining the global space correlation information of the parking lot i at the current time according to the soft allocation matrix comprises:

generating a N-row and K-column soft allocation matrix according to the local space correlation information of the parking lots at the current time, wherein the N is equal to the number of parking lots in the parking lot association graph, the K is equal to a preset number of potential junctions, each row in the soft allocation matrix corresponds to one parking lot, each column in the soft allocation matrix corresponds to a potential junction, each element in each row of the soft allocation matrix represents a probability that a corresponding parking lot belongs to a corresponding potential junction;

according to the soft allocation matrix, determining representation vectors of the potential junctions respectively, and determining the weights of edges between the potential junctions respectively, and connecting any two potential junctions through edges;

determining final representation vectors of the potential junctions according to the representation vectors of the potential junctions and the weights of edges between the potential junctions;

determining, as for the parking lot i, the global space correlation information of parking lot i at the current time according to the final representation vectors of the potential junctions and the soft allocation matrix.

According to a preferred embodiment of the present disclosure, the method further comprises: obtaining elements in the ithrow in the soft allocation matrix by calculating Softmax(Wsxi′), the ithrow representing any row in the soft allocation matrix, xi′ representing the local space correlation information of the parking lot corresponding to the ithrow at the current time, and Wsrepresenting a model parameter obtained by pre-training.

According to a preferred embodiment of the present disclosure, the representation vector xisof any potential junction i in K potential junctions is expressed as

where N represents the number of parking lots in the parking lot association graph; xj′ represents the local space correlation information of any parking lot j in the N parking lots at the current time; S represents the soft allocation matrix.

According to a preferred embodiment of the present disclosure, the weight αijsof the edge between any two potential junctions i and j in the K potential junctions is represented as

where N represents the number of parking lots in the parking lot association graph; S represents the soft allocation matrix; when any two parking lots m and n in the N parking lots are connected through an edge, amnis a first predetermined value, otherwise it is a second predetermined value.

According to a preferred embodiment of the present disclosure, the final representation vector xis′of any potential junction i in the K potential junctions is represented as: xis′=σ(ΣjϵQiαijsWlxjs);

where Qirepresents the number of potential junctions connected with the potential junction i through edges; xjsrepresents the representation vector of any potential junction j of Qineighboring potential junctions: Wlrepresents a model parameter obtained by pre-training; αijsrepresents a weight of the edge between the potential junction i and potential junction j; σ represents an activation function.

According to a preferred embodiment of the present disclosure, the global space correlation information xiscof the parking lot i at the current time is represented as

where xjs′represents the final representation vector of any potential junction j in the K potential junctions; S represents the soft allocation matrix.

According to a preferred embodiment of the present disclosure, before determining time correlation information of the parking lot i at the current time based on a gated recurrent neural network model, the method further comprises: concatenating the local space correlation information and global space correlation information of the parking lot i at the current time;

the determining time correlation information of the parking lot i at the current time based on a gated recurrent neural network model comprises: determining the time correlation information of the parking lot i at the current time according to the concatenation result and output of the gated recurrent neural network model at a previous time.

According to a preferred embodiment of the present disclosure, the time correlation information hitof the parking lot i at the current time is represented as hit=(1−zit)∘hit−1+zit∘{tilde over (h)}it;
wherezit=σ(Wz[hit−1,xi″]+bz);
{tilde over (h)}it=tanh(W{tilde over (h)}[rit∘hit−1,xi″]+b{tilde over (h)});
rit=σ(Wr[hit−1,xi″]+br);

Wz, W{tilde over (h)}, Wr, bz, b{tilde over (h)}and brall are model parameters obtained by pre-training; σ represents an activation function; xi″ represents the concatenation result; hit−1represents the output of the gated recurrent neural network model at the previous time.

According to a preferred embodiment of the present disclosure, the predicting the free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time comprises:

predicting the free parking space information of the parking lot i at future r time steps in the following manner: (ŷit+1, . . . ,ŷit+τ)=σ(Wohit);

where τ is a positive integer greater than one; hitrepresents the time correlation information of the parking lot i at the current time; Worepresents a model parameter obtained by pre-training, σ represents an activation function; ŷit+1represents the predicted free parking space information of the parking lot i at a first future time step; ŷit+τrepresents the predicted free parking space information of the parking lot i at τthfuture time step.

According to a preferred embodiment of the present disclosure, the method further comprises:

when performing model training, selecting Nlparking lots with real-time sensors as sample parking lots, building annotation data based on historical free parking space information of the sample parking lots, performing training optimization based on the annotation data, and minimizing an objective function O;

where the objective function

O=1τ⁢Nl⁢∑i=1Nl∑j=1τ(y^it+j-yit+j)2,
Nlis a positive integer greater than 1, and yit+jrepresents real free parking space information of any sample parking lot i at a corresponding time step.

A parking lot free parking space predicting apparatus, comprising a building unit and a predicting unit;

the building unit is configured to build a parking lot association graph for parking lots in a region to be processed, each junction therein representing a parking lot, and connect any two parking lots meeting a predetermined condition through edges;

the predicting unit is configured to process as follows for any parking lot i: determine local space correlation information of parking lot i at a current time according to environment context features of the parking lot i and neighboring parking lots which are in the parking lot association graph and connected to the parking lot i through edges; determine global space correlation information of the parking lot i at the current time according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time; determine time correlation information of the parking lot i at the current time according to the local space correlation information and global space correlation information, and predict free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, the building unit connects any two parking lots with a distance less than or equal to a predetermined threshold through edges.

According to a preferred embodiment of the present disclosure, the predicting unit determines local space correlation information of parking lot i at a current time based on a graph attention neutral network model;

the predicting unit builds the soft allocation matrix according to the local space correlation information of the parking lots at the current time based on a hierarchical graph neural network model, and determines the global space correlation information of the parking lot i at the current time according to the soft allocation matrix;

the predicting unit determines time correlation information of the parking lot i at the current time based on a gated recurrent neural network model, and predicts the free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, the predicting unit, as for neighboring parking lots, determines weights of edges between the neighboring parking lots and the parking lot i at the current time according to the environment context features of the neighboring parking lots and parking lot i at the current time, respectively, aggregates the environment context features of the neighboring parking lots according to the weights of edges between the neighboring parking lots and the parking lot i to obtain a representation vector of the parking lot i, and regards the representation vector as the local space correlation information of the parking lot i at the current time.

According to a preferred embodiment of the present disclosure, a weight αijof the edge between any neighboring parking lot j and parking lot i

where cij=Attention(Waxi,Waxj); Attentionrepresents a graph attention mechanism; Nirepresents the number of neighboring parking lots; xirepresents the environment context feature of the parking lot i at the current time; xjrepresents the environment context feature of neighboring parking lot j at the current time; Warepresents a model parameter obtained by pre-training.

According to a preferred embodiment of the present disclosure, the representation vector xi′=σ(ΣjϵNiαijWaxj);

where Nirepresents the number of neighboring parking lots; xjrepresents the environment context feature of any neighboring parking lot j among Nineighboring parking lots at the current time; αijrepresents a weight of the edge between the neighboring parking lot j and parking lot i at the current time; Warepresents a model parameter obtained by pre-training; σ represents an activation function.

According to a preferred embodiment of the present disclosure, the predicting unit generates a N-row and K-column soft allocation matrix according to the local space correlation information of the parking lots at the current time, wherein the N is equal to the number of parking lots in the parking lot association graph, the K is equal to a preset number of potential junctions, each row in the soft allocation matrix corresponds to one parking lot, each column in the soft allocation matrix corresponds to a potential junction, each element in each row of the soft allocation matrix represents a probability that a corresponding parking lot belongs to a corresponding potential junction; according to the soft allocation matrix, determines representation vectors of the potential junctions respectively, and determines the weights of edges between the potential junctions respectively, and connect any two potential junctions through edges; determines final representation vectors of the potential junctions according to the representation vectors of the potential junctions and the weights of edges between the potential junctions; determines, as for the parking lot i, the global space correlation information of parking lot i at the current time according to the final representation vectors of the potential junctions and the soft allocation matrix.

According to a preferred embodiment of the present disclosure, the predicting unit obtains elements in the ithrow in the soft allocation matrix by calculating Softmax(Wsxi′), the ithrow representing any row in the soft allocation matrix, xi′ representing the local space correlation information of the parking lot corresponding to the ithrow at the current time, and Wsrepresenting a model parameter obtained by pre-training.

According to a preferred embodiment of the present disclosure, the representation vector xisof any potential junction i in K potential junctions is expressed as

where N represents the number of parking lots in the parking lot association graph; xj′ represents the local space correlation information of any parking lot j in the N parking lots at the current time; S represents the soft allocation matrix.

According to a preferred embodiment of the present disclosure, the weight αijsof the edge between any two potential junctions i and j in the K potential junctions is represented as

where N represents the number of parking lots in the parking lot association graph; S represents the soft allocation matrix; when any two parking lots m and n in the N parking lots are connected through an edge, amnis a first predetermined value, otherwise it is a second predetermined value.

According to a preferred embodiment of the present disclosure, the final representation vector xis′of any potential junction i in the K potential junctions is represented as: xis′=σ(ΣjϵQiαijsWlxjs);

where Qirepresents the number of potential junctions connected with the potential junction i through edges; xjsrepresents the representation vector of any potential junction j of Qineighboring potential junctions: Wirepresents a model parameter obtained by pre-training; αijsrepresents a weight of the edge between the potential junction i and potential junction j; σ represents an activation function.

According to a preferred embodiment of the present disclosure, the global space correlation information xiscof the parking lot i at the current time is represented as

where xjs′represents the final representation vector of any potential junction j in the K potential junctions; S represents the soft allocation matrix.

According to a preferred embodiment of the present disclosure, the predicting unit is further configured to concatenate the local space correlation information and global space correlation information of the parking lot i at the current time, and determine the time correlation information of the parking lot i at the current time according to the concatenation result and output of the gated recurrent neural network model at a previous time.

According to a preferred embodiment of the present disclosure, the time correlation information hitof the parking lot i at the current time is represented as hit=(1−zit)∘hit−1+zit∘{tilde over (h)}it;
wherezit=σ(Wz[hit−1,xi″]+bz);
{tilde over (h)}it=tanh(W{tilde over (h)}[rit∘hit−1,xi″]+b{tilde over (h)});
rit=σ(Wr[hit−1,xi″]+br);

Wz, W{tilde over (h)}, Wr, bz, b{tilde over (h)}and brall are model parameters obtained by pre-training; σ represents an activation function; xi″ represents the concatenation result; hit−1represents the output of the gated recurrent neural network model at the previous time.

According to a preferred embodiment of the present disclosure, the predicting unit predicts the free parking space information of the parking lot i at future τ time steps in the following manner: (ŷit+1, . . . ,ŷit+τ)=σ(Wohit);

where τ is a positive integer greater than one; hitrepresents the time correlation information of the parking lot i at the current time; Worepresents a model parameter obtained by pre-training, σ represents an activation function: ŷit+1represents the predicted free parking space information of the parking lot i at a first future time step; ŷit+τrepresents the predicted free parking space information of the parking lot i at τthfuture time step.

According to a preferred embodiment of the present disclosure, the apparatus further comprises: a pre-processing unit configured to perform model training, where Nlparking lots with real-time sensors are selected as sample parking lots, annotation data are built based on historical free parking space information of the sample parking lots, training optimization is performed based on the annotation data, and an objective function O is minimized;

where the objective function

O=1τ⁢Nl⁢∑i=1Nl∑j=1τ(y^it+j-yit+j)2,
Nlis a positive integer greater than 1, and yit+jrepresents real free parking space information of any sample parking lot i at a corresponding time step.

at least one processor; and

a memory communicatively connected with the at least one processor, wherein,

the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

A non-transitory computer-readable storage medium storing computer instructions therein, wherein the computer instructions are used to cause the computer to perform the above method.

Embodiments of the present disclosure have the following advantages or advantageous effects: the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be determined in conjunction with the environment context features of the parking lots, and future free parking space information of the parking lots may be predicted based on these information, thereby improving the accuracy of the prediction result; in addition, the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be obtained by virtue of different network models, thereby enhancing the accuracy of the obtained results and further enhancing the accuracy of subsequent prediction results; furthermore, when the model is trained, annotation data may be built based on historical free parking space information of the parking lots with real-time sensors, and training optimization may be performed, thereby making the annotation data more accurate and thereby enhancing the model training effect. Other effects of the above optional manners will be described hereunder in conjunction with specific embodiments.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Therefore, those having ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. Also, for the sake of clarity and conciseness, depictions of well-known functions and structures are omitted in the following description.

In addition, it should be appreciated that the term “and/or” used in the text is only an association relationship depicting associated objects and represents that three relations might exist, for example, A and/or B may represents three cases, namely, A exists individually, both A and B coexist, and B exists individually. In addition, the symbol “/” in the text generally indicates associated objects before and after the symbol are in an “or” relationship.

FIG.1illustrates a flow chart of an embodiment of a parking lot free parking space predicting method according to the present disclosure. As shown inFIG.1, the embodiment comprises the following specific implementation mode.

At101, a parking lot association graph is built for parking lots in a region to be processed, each junction therein representing a parking lot, and any two parking lots meeting a first predetermined condition are connected through edges.

At102, any parking lot is processed in a manner shown in103-105.

At103, local space correlation information of parking lot i at a current time is determined according to environment context features of the parking lot i and neighboring parking lots which are in the parking lot association graph and connected to the parking lot i through edges.

At104, global space correlation information of the parking lot i at the current time is determined according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time.

At105, time correlation information of the parking lot i at the current time is determined according to the local space correlation information and global space correlation information, and free parking space information of the parking lot i at at least one future time step is predicted according to the time correlation information of the parking lot i at the current time.

Take Beijing as an example. There might be tens of thousands of parking lots in the whole city. However, since real-time sensors are costly, they are mounted in only very few parking lots to monitor in real time the current free parking space information which usually refers to the number of free parking spaces. Hence, it is very necessary to predict free parking space information of parking lots.

There are usually local space correlation and global space correlation between parking lots in a city. For example, when a user goes to a hot restaurant for a meal, if the parking lots of the restaurant are in high demand, he usually chooses to park his car in one of other surrounding parking lots, which reflects the local space correlation. Again for example, two working regions which are far away from each other might simultaneously be in a state in high demand for parking lots upon working time, whereas two leisure regions which are far away from each other might simultaneously be in a state with free parking spaces upon working time, which reflect the global space correlation.

In the present embodiment, the local space correlation and global space correlation of the parking lots may be modeled using a graph attention neural network model and a hierarchical graph neural network model to obtain a final space correlation representation of the parking lots, and the final space correlation representation is input to a gated recurrent neural network model to predict future free parking space information of the parking lots.

To depict the local space correlation, a parking lot association graph may be built for parking lots in a region to be processed (e.g., the city of Beijing), each junction in the parking lot association graph represents a parking lot, and any two parking lots meeting a predetermined condition are connected through edges. For example, any two parking lots with a distance less than or equal to a predetermined threshold are connected through edges, i.e., parking lots which are close to each other have a strong correlation.

FIG.2illustrates a schematic diagram of a parking lot association graph according to the present disclosure. A specific value of the threshold may depend on actual needs, for example, 1 km, and correspondingly, there is the following formula:

That is, if a distance dits(vi, vj) between any two parking lots is less than or equal to 1 km, the two parking lots are connected through edges, otherwise they are not connected. The distance usually refers to a road network distance.

As for any parking lot i, local space correlation information of parking lot i at a current time may be determined according to environment context features of the parking lot i and neighboring parking lots which are in the parking lot association graph and connected to the parking lot i through edges.

The environment context feature of the parking lots may include a peripheral population feature, a peripheral Points of Interest (POIs) distribution feature etc. The specific content included by the environment context features may depend on actual needs. The peripheral refers to a surrounding predetermined scope. The population feature may refer to the number of active users. For example, a user will upload positioning information upon using an app such as a map app, and the user's activity regions may be obtained by using the positioning information. The POI distribution feature may include the number and types of the POIs and so on. In practical application, the obtained environment context features may be represented in the form of vectors according to predetermined rules. The environment context features are dynamically variable.

As shown inFIG.2, parking lot i is taken as an example. Parking lot2, parking lot3, parking lot4and parking lot5all are neighboring parking lots of parking lot i.

As for any parking lot i, neighboring parking lots of the parking lot i may be first determined, the neighboring parking lots being parking lots which are in the parking lot association graph and connected to the parking lot i through edges, and then local space correlation information of parking lot i at a current time may be determined according to environment context features of the neighboring parking lots and parking lot i at the current time.

Specifically, as for the neighboring parking lots, it is feasible to determine weights of edges between the neighboring parking lots and the parking lot i at the current time according to the environment context features of the neighboring parking lots and parking lot i at the current time, respectively; aggregate the environment context features of the neighboring parking lots according to the weights of edges between the neighboring parking lots and the parking lot i to obtain a representation vector of the parking lot i, and regard the representation vector as the local space correlation information of the parking lot i at the current time. Since the environment context features of the parking lots are dynamically variable, the above weights and representation vector are also dynamically variable.

Optionally, as for any neighboring parking lot j, a weight αijbetween it and the parking lot i may be:

Attention represents a graph attention mechanism; Nirepresents the number of neighboring parking lots of the parking lot i; xirepresents the environment context feature of the parking lot i at the current time; xjrepresents the environment context feature of neighboring parking lot j at the current time; Warepresents a model parameter obtained by pre-training.

The environment context features of the neighboring parking lots may be aggregated according to the weights of edges between the neighboring parking lots and the parking lot i to obtain the representation vector of the parking lot i. The representation vector xi′ may be:
xi′=σ(ZjϵNiαijWaxj);  (4)

where Nirepresents the number of neighboring parking lots of the parking lot i; xjrepresents the environment context feature of any neighboring parking lot j among Nineighboring parking lots at the current time; αijrepresents a weight of the edge between the neighboring parking lot j and parking lot i at the current time; Warepresents a model parameter obtained by pre-training; σ represents an activation function.

In the above processing manner, the local space correlation information at the current time may be obtained respectively with respect to each parking lot.

As for the parking lot i, the global space correlation information of parking lot i at the current time may be determined according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time. Preferably, the soft allocation matrix may be built according to the local space correlation information of the parking lots at the current time based on the hierarchical graph neural network model, and the global space correlation information of parking lot i at the current time may be determined according to the soft allocation matrix.

Specifically, a N-row and K-column soft allocation matrix may be generated according to the local space correlation information of the parking lots at the current time, wherein N is equal to the number of parking lots in the parking lot association graph, K is equal to a preset number of potential junctions, each row in the soft allocation matrix corresponds to one parking lot, each column in the soft allocation matrix corresponds to a potential junction, each element in each row of the soft allocation matrix represents a probability that a corresponding parking lot belongs to a corresponding potential junction. According to the soft allocation matrix, representation vectors of the potential junctions may be determined respectively, the weights of edges between the potential junctions are determined respectively, and any two potential junctions are connected through edges. Then, final representation vectors of the potential junctions may be determined according to the representation vectors of the potential junctions and the weights of edges between the potential junctions. Furthermore, as for the parking lot i, the global space correlation information of parking lot i at the current time may be determined respectively according to the final representation vectors of the potential junctions and the soft allocation matrix.

FIG.3illustrates a schematic diagram of a hierarchical graph neutral network structure and a soft allocation matrix according to the present application. As shown inFIG.3, suppose that an upper layer has K potential junctions, a specific value of K may depend on actual needs, e.g., four as shown inFIG.3. Each potential junction may have a corresponding physical meaning, and may be understood as a different parking lot type as a possible implementation mode, e.g., an office building parking lot, a shopping mall parking lot and so on. Then, as for any parking lot such as ** shopping mall parking lot, a probability that it belongs to different parking lot types may be obtained respectively in a conventional manner. If the probability that it belongs to a shopping center parking lot is the highest, the sum of the probabilities is 1.

Elements Si; in the ithrow in the soft allocation matrix may be calculated in the following manner, and the ithrow represents any row in the soft allocation matrix:
Si;=Softmax(Wsxi′);  (5)

where xi′ represents the local space correlation information of the parking lot corresponding to the ithrow at the current time, and Wsrepresents a model parameter obtained by pre-training. The sum of the elements in the ithrow is 1.

The representation vectors of the potential junctions may be determined respectively according to the soft allocation matrix.

The representation vector xisof any potential junction i in the K potential junctions may be expressed as:

where N represents the number of parking lots in the parking lot association graph; xj′ represents the local space correlation information of any parking lot j in the N parking lots at the current time; S represents the soft allocation matrix; T represents matrix transposition.

In addition, the weights of edges between the potential junctions may be determined respectively according to the soft allocation matrix and the parking lot association graph.

The weight αijsof the edge between any two potential junctions i and j in the K potential junctions may be represented as:

where N represents the number of parking lots in the parking lot association graph; S represents the soft allocation matrix; when any two parking lots m and n in the N parking lots are connected through an edge, amnmay be a first predetermined value, otherwise it may be a second predetermined value; e.g., the first predetermined value may be 1, and the second predetermined value may be 0; Sn,jrepresents a value of an element at the nthrow and the jthcolumn in the soft allocation matrix, and so on so forth.

Furthermore, the final representation vectors of the potential junctions may be determined according to the representation vectors of the potential junctions and the weights of edges between the potential junctions, i.e., the final representation vectors of these potential junctions may be obtained by aggregating the potential junctions.

The final representation vector xis′of any potential junction i in the K potential junctions may be represented as:
xis′=σ(ΣjϵQiαijsWlxjs)  (8)

where Qirepresents the number of potential junctions connected with the potential junction i through edges; xjsrepresents the representation vector of any potential junction j of Qineighboring potential junctions; Wlrepresents a model parameter obtained by pre-training; αijsrepresents a weight of the edge between the potential junction i and potential junction j; σ represents an activation function.

These potential junctions include different global information. The global information needed by the parking lots themselves may be obtained from the potential junctions through the soft allocation matrix. Specifically, as for any parking lot i, the global space correlation information of the parking lot i at the current time may be determined according to the final representation vectors of the potential junctions and the soft allocation matrix.

The global space correlation information xiscof the parking lot i at the current time may be represented as:

where xjs′represents the final representation vector of any potential junction j in the K potential junctions; S represents the soft allocation matrix.

As for the parking lot i, the local space correlation information and global space correlation information of the parking lot i at the current time may be concatenated to obtain a concatenation result, i.e., obtain the final space correlation features simultaneously capturing the local space correlation information and global space correlation information. The concatenation may refer to connecting end to end.

The time correlation information of parking lot i at the current time may be determined according to the concatenation result, and the free parking space information at at least one future time step may be predicted according to the time correlation information of the parking lot i at the current time. As stated above, it is possible to determine the time correlation information of parking lot i at the current time based on a gated recurrent neural network model, and predict the free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time. Preferably, the time correlation information of the parking lot i at the current time may be determined according to the concatenation result and output of the gated recurrent neural network model at a previous time, and in conjunction with the gated mechanism.

The time correlation information hitof the parking lot i at the current time may be:
hit=(1−zit)∘hit−1+zit∘{tilde over (h)}it;  (10)
wherezit=σ(Wz[hit−1,xi″]+bz);  (11)
{tilde over (h)}it=tanh(W{tilde over (h)}[rit∘hit−1,xi″]+b{tilde over (h)});  (12)
rit=σ(Wr[hit−1,xi″]+br);  (13)

Wz, W{tilde over (h)}, Wr, bz, b{tilde over (h)}and brall are model parameters obtained by pre-training; σ represents an activation function; xi″ represents a concatenation result; hit−1represents the output of the gated recurrent neural network model at a previous time; ∘ represents a matrix multiplication.

As xi″ includes the space correlation information at the current time, and hit−1includes spatiotemporal correlation information before time t, the obtained hitwill simultaneously include the time correlation information and the space correlation information.

Furthermore, the free parking space information of the parking lot i at at least one future time step may be predicted using hit, for example, the free parking space information of the parking lot i at future τ time steps may be predicted in the following manner:
(ŷit+1, . . . ,ŷit+τ)=σ(Wohit)  (14)

where τ is a positive integer greater than one, and its specific value may depends on actual needs; hitrepresents the time correlation information of the parking lot i at the current time; Worepresents a model parameter obtained by pre-training, σ represents an activation function; ŷit+1represents the predicted free parking space information of the parking lot i at a first future time step; ŷit+τrepresents the predicted free parking space information of the parking lot i at τthfuture time step.

Suppose the value of τ is 3, the free parking space information of the parking lot i at the first future time step, the second future time step and the third future time step, respectively according to the Equation (14).

A time step for example may be 15 minutes. In practical application, for example, as for the parking lot i, prediction is performed one time every 15 minutes in the manner stated in the present embodiment, i.e., the free parking space information of the parking lot i at three future time steps may be predicted.

In addition, when the model is trained, Nlparking lots with real-time sensors may be selected as sample parking lots, annotation data may be built based on historical free parking space information of the sample parking lots, training optimization may be performed based on the annotation data, and an objective function O is minimized.

The objective function

where Nlis a positive integer greater than 1, and its specific value may depend on actual needs. ylt+jrepresents real free parking space information of any sample parking lot i in Nlsample parking lots at a corresponding time step.

The abovementioned model parameters may be learnt through model training. Specific implementation is of the prior art.

As appreciated, for ease of description, the aforesaid method embodiments are all described as a combination of a series of actions, but those skilled in the art should appreciated that the present disclosure is not limited to the described order of actions because some steps may be performed in other orders or simultaneously according to the present disclosure. Secondly, those skilled in the art should appreciate the embodiments described in the description all belong to preferred embodiments, and the involved actions and modules are not necessarily requisite for the present disclosure.

To sum up, according to the solution of the method embodiment of the present application, the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be determined in conjunction with the environment context features of the parking lots, and future free parking space information of the parking lots may be predicted based on these information, thereby improving the accuracy of the prediction result; in addition, the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be obtained by virtue of different network models, thereby enhancing the accuracy of the obtained results and further enhancing the accuracy of subsequent prediction results; furthermore, when the model is trained, annotation data may be built based on historical free parking space information of the parking lots with real-time sensors, and training optimization may be performed, thereby making the annotation data more accurate and thereby enhancing the model training effect.

The above introduces the method embodiment. The solution of the present disclosure will be further described through an apparatus embodiment.

FIG.4illustrates a schematic structural diagram of a parking lot free parking space predicting apparatus400according to an embodiment of the present disclosure. As shown inFIG.4, the apparatus comprises a building unit401and a predicting unit402.

The building unit401is configured to build a parking lot association graph for parking lots in a region to be processed, each junction therein representing a parking lot, and any two parking lots meeting a predetermined condition being connected through edges.

The predicting unit402is configured to process as follows for any parking lot i: determine local space correlation information of parking lot i at a current time according to environment context features of the parking lot i and neighboring parking lots which are in the parking lot association graph and connected to the parking lot i through edges; determine global space correlation information of the parking lot i at the current time according to a soft allocation matrix built based on the local space correlation information of the parking lots at the current time; determine time correlation information of the parking lot i at the current time according to the local space correlation information and global space correlation information, and predict free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

The building unit401may connect any two parking lots with a distance less than or equal to a predetermined threshold through edges.

In addition, the predicting unit402may determine local space correlation information of parking lot i at a current time based on a graph attention neutral network model, build the soft allocation matrix according to the local space correlation information of the parking lots at the current time based on a hierarchical graph neural network model, and determine the global space correlation information of the parking lot i at the current time according to the soft allocation matrix, and determine time correlation information of the parking lot i at the current time based on a gated recurrent neural network model, and predict the free parking space information of the parking lot i at at least one future time step according to the time correlation information of the parking lot i at the current time.

Specifically, the predicting unit402may, as for neighboring parking lots, determine weights of edges between the neighboring parking lots and the parking lot i at the current time according to the environment context features of the neighboring parking lots and parking lot i at the current time, respectively, aggregate the environment context features of the neighboring parking lots according to the weights of edges between the neighboring parking lots and the parking lot i to obtain a representation vector of the parking lot i, and regard the representation vector as the local space correlation information of the parking lot i at the current time.

A weight αijof the edge between any neighboring parking lot j and parking lot i is represented as

Attention represents a graph attention mechanism; Nirepresents the number of neighboring parking lots; xirepresents the environment context feature of the parking lot i at the current time; xjrepresents the environment context feature of neighboring parking lot j at the current time; Warepresents a model parameter obtained by pre-training.

The representation vector xi′ may be represented as:
σ(ΣjϵNiαijWaxj)  (4)

where Nirepresents the number of neighboring parking lots of the parking lot i; xjrepresents the environment context feature of any neighboring parking lot j among Nineighboring parking lots at the current time; αijrepresents a weight of the edge between the neighboring parking lot j and parking lot i at the current time; Warepresents a model parameter obtained by pre-training; σ represents an activation function.

The predicting unit402may further generate a N-row and K-column soft allocation matrix according to the local space correlation information of the parking lots at the current time, wherein N is equal to the number of parking lots in the parking lot association graph, K is equal to a preset number of potential junctions, each row in the soft allocation matrix corresponds to one parking lot, each column in the soft allocation matrix corresponds to a potential junction, each element in each row of the soft allocation matrix represents a probability that a corresponding parking lot belongs to a corresponding potential junction; according to the soft allocation matrix, determine representation vectors of the potential junctions respectively, and determine the weights of edges between the potential junctions respectively, connect any two potential junctions through edges, determine final representation vectors of the potential junctions according to the representation vectors of the potential junctions and the weights of edges between the potential junctions, and determine, as for the parking lot i, the global space correlation information of parking lot i at the current time respectively according to the final representation vectors of the potential junctions and the soft allocation matrix.

The predicting unit402may obtain elements in the ithrow in the soft allocation matrix by calculating Softmax(Wsxi′), the ithrow representing any row in the soft allocation matrix, xi′ representing the local space correlation information of the parking lot corresponding to the ithrow at the current time, and Wsrepresenting a model parameter obtained by pre-training.

The representation vector xisof any potential junction i in K potential junctions may be expressed as:

where N represents the number of parking lots in the parking lot association graph; xj′ represents the local space correlation information of any parking lot j in the N parking lots at the current time; S represents the soft allocation matrix; T represents matrix transposition.

The weight αijsof the edge between any two potential junctions i and j in the K potential junctions may be represented as:

where N represents the number of parking lots in the parking lot association graph; S represents the soft allocation matrix; when any two parking lots m and n in the N parking lots are connected through an edge, amnis a first predetermined value, otherwise it is a second predetermined value; e.g., the first predetermined value may be 1, and the second predetermined value may be 0; Sn,jrepresents a value of an element at the nthrow and the jthcolumn in the soft allocation matrix, and so on so forth.

The final representation vector xis′of any potential junction i in the K potential junctions may be represented as:
xis′=σ(ΣjϵQiαijsWlxjs)  (8)

where Qirepresents the number of potential junctions connected with the potential junction i through edges; xjsrepresents the representation vector of any potential junction j of Qineighboring potential junctions; Wlrepresents a model parameter obtained by pre-training; αijsrepresents a weight of the edge between the potential junction i and potential junction j; σ represents an activation function.

The global space correlation information xiscof the parking lot i at the current time may be represented as:

where xjs′represents the final representation vector of any potential junction j in the K potential junctions; S represents the soft allocation matrix.

The predicting unit402may concatenate the local space correlation information and global space correlation information of the parking lot i at the current time to obtain a concatenation result, and determine the time correlation information of the parking lot i at the current time according to the concatenation result and output of the gated recurrent neural network model at a previous time.

The time correlation information hitof the parking lot i at the current time may be:
hit=(1−zit)∘hit−1+zit∘{tilde over (h)}it;  (10)
wherezit=σ(Wz[hit−1,xi″]+bz);  (11)
{tilde over (h)}it=tanh(W{tilde over (h)}[rit∘hit−1,xi″]+b{tilde over (h)});  (12)
rit=σ(Wr[hit−1,xi″]+br);  (13)

Wz, W{tilde over (h)}, Wr, bz, b{tilde over (h)}and brall are model parameters obtained by pre-training; σ represents an activation function; xi″ represents a concatenation result; htt−1represents the output of the gated recurrent neural network model at a previous time; ∘ represents a matrix multiplication.

The predicting unit402may predict the free parking space information of the parking lot i at at least one future time step by using hit, for example, predict the free parking space information of the parking lot i at future τ time steps in the following manner:
(ŷit+1, . . . ,ŷit+τ)=σ(Wohit)  (14)

where τ is a positive integer greater than one, and its specific value may depends on actual needs; hitrepresents the time correlation information of the parking lot i at the current time; Worepresents a model parameter obtained by pre-training, σ represents an activation function; ŷit+1represents the predicted free parking space information of the parking lot i at a first future time step; ŷit+τrepresents the predicted free parking space information of the parking lot i at τthfuture time step.

In addition, the apparatus shown inFIG.4may further comprise: a pre-processing unit403configured to perform model training, where Nlparking lots with real-time sensors may be selected as sample parking lots, annotation data may be built based on historical free parking space information of the sample parking lots, training optimization may be performed based on the annotation data, and an objective function O may be minimized;

where the objective function

where Nlis a positive integer greater than 1, and its specific value may depend on actual needs. yit+jrepresents real free parking space information of any sample parking lot i at a corresponding time step.

A specific workflow of the apparatus embodiment shown inFIG.4will not be detailed any more here, and reference may be made to corresponding depictions in the above method embodiment.

To sum up, according to the solution of the apparatus embodiment of the present application, the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be determined in conjunction with the environment context features of the parking lots, and future free parking space information of the parking lots may be predicted based on these information, thereby improving the accuracy of the prediction result; in addition, the local space correlation information, the global space correlation information and the time correlation information of the parking lots may be obtained by virtue of different network models, thereby enhancing the accuracy of the obtained results and further enhancing the accuracy of subsequent prediction results; furthermore, when the model is trained, annotation data may be built based on historical free parking space information of the parking lots with real-time sensors, and training optimization may be performed, thereby making the annotation data more accurate and thereby enhancing the model training effect.

According to embodiments of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

As shown inFIG.5, it shows a block diagram of an electronic device for the method according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device is further intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in the text here.

As shown inFIG.5, the electronic device comprises: one or more processors501, a memory502, and interfaces connected to components and including a high-speed interface and a low speed interface. Each of the components are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor can process instructions for execution within the electronic device, including instructions stored in the memory or on the storage device to display graphical information for a GUI on an external input/output device, such as a display device coupled to the interface. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). One processor501is taken as an example inFIG.5.

The memory502is a non-transitory computer-readable storage medium provided by the present disclosure. Wherein, the memory stores instructions executable by at least one processor, so that the at least one processor executes the method provided by the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions, which are used to cause a computer to execute the method according to the present disclosure.

The memory502is a non-transitory computer-readable storage medium and can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions/modules corresponding to the method in embodiments of the present disclosure. The processor Y01executes various functional applications and data processing of the server, i.e., implements the method in the above method embodiment, by running the non-transitory software programs, instructions and units stored in the memory502.

The memory502may include a storage program region and a storage data region, wherein the storage program region may store an operating system and an application program needed by at least one function; the storage data region may store data created according to the use of the electronic device for implementing the video blending method according to the embodiment of the present disclosure. In addition, the memory502may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory502may optionally include a memory remotely arranged relative to the processor501, and these remote memories may be connected to the electronic device for implementing the video blending method according to embodiments of the present disclosure through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The electronic device for implementing the video blending method may further include an input device503and an output device504. The processor501, the memory502, the input device503and the output device504may be connected through a bus or in other manners. InFIG.5, the connection through the bus is taken as an example.

The input device503may receive inputted numeric or character information and generate key signal inputs related to user settings and function control of the electronic device for implementing the video blending method according to the embodiment of the present disclosure, and may be an input device such as a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball and joystick. The output device504may include a display device, an auxiliary lighting device (e.g., an LED), a haptic feedback device (for example, a vibration motor), etc. The display device may include but not limited to a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display.1sIn some embodiments, the display device may be a touch screen.

It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in different orders as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which is not limited herein.

The foregoing specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.