Patent Description:
<CIT> describes a head measuring method for analyzing a head image near an impact to calculate a face angle and a path of a head.

<CIT> describes a method and system for using a low resolution of a golfer's swing for analysis. A computer calculates parameters associated with a golfer and a golf club from images. The computer uses the parameters to generate swing analysis outputs. A correlation table relates identified swing analysis to recommendation to the golfer to improve the swing.

<CIT> describes an apparatus, system, and method for golf swing analysis. An accelerometer and gyroscope are attached to a golf club to record data related to a golfer's swing, and a camera with image recognition software and an ultrasonic and RF navigation system are used for error correction.

However, in the technique described in <CIT>, it is necessary to prepare a dedicated measuring device for capturing the head image near the impact. As such, the technique described in Jp2013-138758A does not allow a general player to easily analyze their golf swing using a smartphone or other devices in a golf course or a golf practice range.

One or more embodiments of the present invention have been conceived in view of the above, and an object thereof is to provide a golf swing analysis system according to claim <NUM>, a computer-implemented golf swing analysis method according to claim <NUM>, and a program capable of easily analyzing a golf swing according to claim <NUM>.

An embodiment of the present invention will be described below with reference to the accompanying drawings.

<FIG> is a diagram illustrating an example of a configuration of a user terminal <NUM> according to an embodiment of the present invention.

The user terminal <NUM> according to the present embodiment is a computer such as a mobile phone (including a smart phone) and a portable information terminal (including a tablet computer). As shown in <FIG>, the user terminal <NUM> according to the present embodiment includes a processor <NUM>, a storage unit <NUM>, a communication unit <NUM>, a touch panel <NUM>, a camera unit <NUM>, and a sensor unit <NUM>.

For example, the processor <NUM> is a program control device, such as a microprocessor, that operates in accordance with a program installed in the user terminal <NUM>. The storage unit <NUM> is, for example, storage elements such as a ROM and a RAM, and a solid-state drive (SSD). The storage unit <NUM> stores programs to be executed by the processor <NUM>, for example. The communication unit <NUM> is a communication interface for wired or wireless communication, for example, and transfers data to and from a computer, such as a server, via a computer network, such as the Internet.

The touch panel <NUM> displays and outputs information according to an instruction from the processor <NUM> and outputs an operation performed by a user to the processor <NUM>, for example.

The camera unit <NUM> is a capturing device, such as a digital video camera, capable of taking a video image, for example.

The sensor unit <NUM> is a sensing device, such as a GPS module, an acceleration sensor, and a motion sensor. In the present embodiment, the sensor unit <NUM> is included in the user terminal <NUM>, although the sensor unit <NUM> may be outside of the user terminal <NUM>.

In the present embodiment, for example, a user who operates the user terminal <NUM> (hereinafter referred to as an operating user) captures a player making a golf swing with the camera unit <NUM> of the user terminal <NUM> in a golf course or a golf practice range. Subsequently, a video image of the player making the golf swing is generated.

In the present embodiment, prior to capturing a swing, a position and an inclination of the user terminal <NUM> are adjusted so that a position of the player with respect to the position of the camera unit <NUM> is within a predetermined allowable range and an orientation of the player to the capturing direction is within a predetermined allowable range. In the present embodiment, when the user terminal <NUM> is adjusted and then the operating user performs a predetermined operation, the camera unit <NUM> captures an image of the player, and a video image showing the player making a swing is generated. In this embodiment, assume that a video image having a predetermined frame rate (e.g., 60fps) is generated.

In the present embodiment, camera shake correction processing is executed on a video image generated in this way, and each frame image is thereby corrected so that player appears at substantially the same position in all the frame images. Hereinafter, the video image to which the camera shake correction processing is executed will be referred to as a captured video image. In this embodiment, assume that the number of vertical and horizontal pixels of the frame images included in the generated captured video images are all the same.

In the following, a frame image having a frame number n (n=<NUM>, <NUM>, <NUM>,. ) is represented as a frame image (n). Further, a frame corresponding to the frame image (n) is represented as the n-th frame.

In this embodiment, swing analysis processing is performed on a captured video image. Subsequently, an analysis video image, which is obtained by overlaying an image indicating the result of the swing analysis processing onto the captured video image, is generated. The generated analysis video image is then stored in the storage unit <NUM> of the user terminal <NUM>.

The user of the user terminal <NUM>, such as the player and the operation user described above, can replay the analysis video image stored in the storage unit <NUM> by appropriately performing a predetermined operation.

<FIG> is a diagram illustrating an example of a frame image at the time of finish included in the analysis video image to be replayed. In the frame image shown in <FIG>, a club head path effect image <NUM> representing a path of a club head and a ball path effect image <NUM> representing a path of a ball are overlaid on the frame image included in the captured video image in which the player <NUM> is captured.

On the upper right of the frame image shown in <FIG>, analysis result information <NUM> is shown. In the example of <FIG>, character strings indicating a ball speed, a flight distance, and a type of swing path, such as outside-in, inside-in, and inside-out, are shown as the analysis result information <NUM>.

On the lower left of the frame image shown in <FIG>, an impact image <NUM> is displayed in which a path of the club head, a direction of a face of a club at impact, and a curve direction of the ball are represented.

The impact image <NUM> shown in <FIG> includes a head moving direction image <NUM>, a face orientation image <NUM>, and a ball direction image <NUM>. The head moving direction image <NUM> indicates, for example, a moving direction of a club head at impact. In the example of <FIG>, the direction in which the arrows are aligned corresponds to the moving direction of the club head at impact. The face orientation image <NUM> indicates, for example, an orientation of the face of the club at impact. In the example of <FIG>, the face orientation image <NUM> is a linear image, and the extending direction of the face orientation image <NUM> corresponds to the orientation of the face of the club at impact. The ball direction image <NUM> indicates, for example, a launch direction and a curve direction of the ball. In the ball direction image <NUM> according to the present embodiment, the direction perpendicular to the extending direction of the linear face orientation image <NUM> is shown as the launch direction of the ball.

The frame image shown in <FIG> also includes a V-zone image <NUM> indicating a V-zone through which the path of the club head desirably passes during downswing.

As described above, according to the present embodiment, the user of the user terminal <NUM> can check the analysis result of the golf swing performed by the player.

Hereinafter, the functions of the user terminal <NUM> according to the present embodiment and the processing executed by the user terminal <NUM> will be further described focusing on the swing analysis processing described above.

<FIG> is a functional block diagram showing an example of functions implemented in the user terminal <NUM> according to the present embodiment. In the user terminal <NUM> according to the present embodiment, all of the functions shown in <FIG> need not be implemented, and functions other than the functions shown in <FIG> may be implemented.

The user terminal <NUM> according to the present embodiment has a function as a swing analysis system for analyzing a golf swing. As shown in <FIG>, the user terminal <NUM> according to the present embodiment functionally includes, for example, a captured video image obtaining unit <NUM>, a player area specifying unit <NUM>, a comparison result image generating unit <NUM>, a start area specifying unit <NUM>, a ball path specifying unit <NUM>, a flight distance calculating unit <NUM>, a club head path specifying unit <NUM>, a launch direction specifying unit <NUM>, a swing path type specifying unit <NUM>, a curve direction determining unit <NUM>, an analysis video image generating unit <NUM>, an analysis video image storage unit <NUM>, and a display control unit <NUM>.

The captured video image obtaining unit <NUM>, the player area specifying unit <NUM>, the comparison result image generating unit <NUM>, the start area specifying unit <NUM>, the ball path specifying unit <NUM>, the flight distance calculating unit <NUM>, the club head path specifying unit <NUM>, the launch direction specifying unit <NUM>, the swing path type specifying unit <NUM>, the curve direction determining unit <NUM>, and analysis video image generating unit <NUM> are mainly implemented by the processor <NUM>. The analysis video image storage unit <NUM> is mainly implemented by the storage unit <NUM>. The display control unit <NUM> is mainly implemented by the processor <NUM> and the touch panel <NUM>.

The functions described above may be implemented by executing programs that are installed in the user terminal <NUM>, which is a computer, and includes commands corresponding to the above functions. The program may be supplied to the user terminal <NUM> via a computer-readable information storage medium, such as an optical disk, a magnetic disk, a magnetic tape, and a magneto-optical disk, or the Internet.

In the present embodiment, the captured video image obtaining unit <NUM> obtains a captured video image in which the player <NUM> making a golf swing is captured, for example. Here, as described above, a captured video image on which the camera shake correction processing is executed may be obtained.

As shown in <FIG>, the captured video image is a video image in which the player <NUM> is captured from one direction. The captured video image is a video image in which the player <NUM> is viewed from rearward of a target direction of the ball.

As described above, assume that a captured video image captured at a predetermined frame rate (e.g., 60fps) is obtained in the present embodiment.

In the present embodiment, for example, the player area specifying unit <NUM> specifies a player area <NUM>, in which the player <NUM> is captured in the frame image included in the captured video image as shown in <FIG>.

For example, a trained machine learning model, such as OpenPose, may be installed in the user terminal <NUM> according to the present embodiment. The player area specifying unit <NUM> may use such a trained machine learning model and whereby specify positions of nodes and links of a skeleton model <NUM>, which represents the skeleton (bones) of the player <NUM> in the frame image. The positions of the plurality of nodes included in the skeleton model <NUM> shown in <FIG> are respectively associated with the positions of the body parts of the player <NUM>, such as the shoulders and the waist.

The player area specifying unit <NUM> may specify a rectangular area as the player area <NUM>, where a position, a shape, and a size of the rectangular area are specified based on the skeleton model <NUM>. For example, an area obtained by enlarging a rectangular area circumscribing around the skeleton model <NUM> at a predetermined magnification without changing the position of the center may be specified as the player area <NUM>.

The player area <NUM> may be specified based on one representative frame image. Alternatively, the player area <NUM> may be specified based on a plurality of frame images. In this case, an area representing the rectangular areas specified for a plurality of frame images, such as the common area or the average area of the rectangular areas specified for the frame images, may be specified as the player area <NUM>.

For example, when a person other than the player is included in the captured video image, a plurality of candidates of the player area <NUM> may be specified. In this case, any one of the candidates may be specified as the player area <NUM> based on a given condition, such as a size of the area, a distance from the center of the frame image, and a ratio of the height to the width of the area.

In this embodiment, for example, the comparison result image generating unit <NUM> uses an image differencing method based on the captured video image so as to generate a comparison result image shown in <FIG>.

In the present embodiment, a comparison result image is generated so as to be associated with each of the frame images included in the captured video image. For example, a comparison result image (n) is generated for the frame image (n) as a corresponding comparison result image.

For example, the comparison result image generating unit <NUM> may smooth a predetermined number of consecutive frame images, e.g., <NUM> frames, so as to generate a comparison image. For example, <NUM> frame images from the frame image (n-<NUM>) to the frame image (n-<NUM>) included in the captured video image may be smoothed to generate a comparison image (n). For example, the comparison image (n) may be generated in which average values of pixel values of respective pixels in the frame images from the frame image (n-<NUM>) to the frame image (n-<NUM>) are set to the pixel values of the corresponding pixels.

The comparison result image generating unit <NUM> calculates the difference between the pixel values of the pixels in the frame image (n), which is the next frame image of the predetermined number of frame images described above, and the pixel values of the corresponding pixels in the comparison image (n) described above. The comparison result image generating unit <NUM> generates a comparison result image (n) indicating whether a difference between a pixel value of each pixel in the frame image (n) and a pixel value of a corresponding pixel in the comparison image (n) is equal to or greater than a predetermined value. In the comparison result image shown in <FIG>, the pixel having the difference of the pixel value equal to or greater than the predetermined value is represented in white, and the pixel having the difference of the pixel value less than the predetermined value is represented in black. Hereinafter, the pixel having the difference of pixel value equal to or greater than the predetermined value, that is, the pixel represented in white in the comparison result image in <FIG>, will be referred to as a change pixel.

In the present embodiment, when the comparison image is generated based on <NUM> frame images as described above, comparison result images corresponding to the frame images subsequent to the frame image <NUM> are generated.

In the present embodiment, the start area specifying unit <NUM> specifies, for example, a start area <NUM> shown in <FIG> used for specifying a path of the ball by the ball path specifying unit <NUM> described later.

The start area <NUM> according to the present embodiment is, for example, a rectangular area uniquely specified based on the player area <NUM>. For example, the position of the left side of the start area <NUM> may be the same as the position of the left side of the frame image, and the position of the right side of the start area <NUM> may be the same as the position of the right side of the frame image. Further, the position of the upper side of the start area <NUM> may be the position that is <NUM>% lower than the upper side of the player area <NUM> by the length in the vertical direction of the player area <NUM>. Further, the position of the lower side of the start area <NUM> may be the same as the position of the lower side of the player area <NUM> or the position passing under the foot of the player <NUM>. This method is only an example of methods of specifying the start area <NUM>, and other methods may be used for specifying the start area <NUM>.

In the present embodiment, for example, the ball path specifying unit <NUM> specifies a path of a ball in a captured video image based on the captured video image.

In the present embodiment, a ball path is specified as a list in which positions of the ball in a plurality of frame images included in the captured video image are associated with each other. Hereinafter, such a list is referred to as a path list.

In the following, referring to <FIG>, an example of processing for specifying a ball path by the ball path specifying unit <NUM> will be described.

As described above, assume that a comparison result image associated with a frame image subsequent to the frame image <NUM> is generated. In this case, at least one pixel block of changed pixels within a predetermined number of pixels and in a predetermined range is specified for the comparison result image subsequent to the frame image <NUM>. For these pixel blocks, a position of the representative point (e.g., the center) is specified as a candidate ball position.

Here, for example, assume that the candidate ball position is not specified up to the comparison result image (n-<NUM>), and three candidate ball positions P(n, a), P(n, b), and P(n, c) shown in <FIG> are specified in the comparison result image (n).

Subsequently, the candidate ball positions are narrowed down to the candidate ball position to be the first element of candidates of the path list (hereinafter referred to as path list candidates). For example, the candidate ball position outside the start area <NUM> is excluded from the candidate ball position. In this manner, P(n,a) is excluded from the candidate ball position to be the first element, and the candidate ball positions are narrowed down to P(n,b) and P(n,c) as the candidate ball position to be the first element.

In this case, as shown in <FIG>, P(n,b) and P(n,c), each having one element, are specified as the path list candidates.

Assume that three candidate ball positions P(n+<NUM>, a), P(n+<NUM>, b), and P(n+<NUM>, c) shown in <FIG> are specified in the comparison result image (n+<NUM>).

In addition to these candidate ball positions, <FIG> also shows P(n, b) and P(n, c) that are not excluded from the candidate ball positions.

In this case, P(n+<NUM>,a) outside the start area <NUM> is excluded from the candidate ball position to be the first element. On the other hand, the candidate ball positions to be the first element are narrowed down to P(n+<NUM>,b) and P(n+<NUM>,c).

In the situation where the path list candidates already exist, the candidate ball position associated with the existing path list candidate is also specified.

For example, the candidate ball position to be associated with P(n,b) is specified from among P(n+<NUM>,a), P(n+<NUM>,b), and P(n+<NUM>,c). When a predetermined condition is satisfied, for example, the length from P(n,b) to a position is longer than a predetermined length, the length from P(n,b) to a position is shorter than a predetermined length, or a position is located lower than P(n,b), such a position is excluded from the candidate ball position to be associated with P(n,b). For example, P(n+<NUM>,c) is located below P(n,b), and thus excluded from the candidate ball position. The candidate ball positions to be associated with P(n,b) are narrowed down to P(n+<NUM>,a) and P(n+<NUM>,b).

Similarly, the candidate ball position to be associated with P(n,c) is specified from among P(n+<NUM>,a), P(n+<NUM>,b), and P(n+<NUM>,c). Here, for example, P(n+<NUM>,c) is located below P(n,c), and thus excluded from the candidate ball position. The candidate ball positions to be associated with P(n,c) are narrowed down to P(n+<NUM>,a) and P(n+<NUM>,b).

As shown in <FIG>, the path list candidates are added to the list. In the above example, P(n+<NUM>,b), P(n+<NUM>,c), P(n,b)-P(n+<NUM>,a), P(n,b)-P(n+<NUM>,b), P(n,c)-P(n+<NUM>,a), and P(n,c)-P(n+<NUM>,b) are added to the path list candidates. <FIG> also shows the path list candidate including a plurality of candidate ball positions associated with each other. For example, in <FIG>, the path list candidate including the element P(n,b) and the element P(n+<NUM>,a) is shown as P(n,b)-P(n+<NUM>,a). P(n,b) and P(n,c) shown in <FIG> are not overwritten and also left in the path list candidates.

Assume that two candidate ball positions P(n+<NUM>, a) and P(n+<NUM>, b) shown in <FIG> are specified in the comparison result image (n+<NUM>).

In addition to these candidate ball positions, <FIG> also shows candidate ball positions that are not excluded. Of these candidate ball positions, P(n+<NUM>,a), which is outside the start area <NUM>, is excluded from the candidate ball position to be the first element. On the other hand, the candidate ball positions for the starting point are narrowed down to P(n+<NUM>,b).

As described above, the candidate ball position associated with the path list candidate having one element is specified. For example, P(n+<NUM>,b) is excluded from the candidate ball position associated with P(n,c) because the length from P(n,c) to P(n+<NUM>,b) is less than a predetermined length. Further, P(n+<NUM>,a) is excluded from the candidate ball position associated with P(n+<NUM>,c) because the length from P(n+<NUM>,c) to P(n+<NUM>,a) is longer than the predetermined length.

Subsequently, the candidate ball position associated with the path list candidate having a plurality of elements is also specified. In this case, when a candidate ball position satisfies predetermined conditions, such a candidate ball position may be excluded from the candidate ball position associated with the path list candidate. For example, the predetermined conditions include the cases where the length from the candidate ball position that is the last element of the path list candidate is longer than the predetermined length, the length from the candidate ball position that is the last element of the path list candidate is shorter than the predetermined length, and the position is lower than the candidate ball position that is the last element of the path list candidate.

In addition to these conditions, those satisfying predetermined conditions relating to the path list candidate including a plurality of elements may also be excluded from the candidate ball positions associated with path list candidates. For example, the conditions include the cases where the angle of the path is abruptly changed (curved more than a predetermined angle), the speed of the path is abruptly changed (shorter or longer than a predetermined length), the direction of the path is changed (the path toward the right is directed to the left, or the path toward the left is directed to the right).

In the example of <FIG>, the angle of the path of P(n+<NUM>,a) is abruptly changed with respect to the path connecting P(n,b) with P(n+<NUM>,b), and thus excluded from the candidate ball positions associated with P(n,b)-P(n+<NUM>,b). Further, the speed of P(n+<NUM>,a) is rapidly decreased with respect to the path connecting P(n,c) with P(n+<NUM>,a), and thus P(n+<NUM>,a) is excluded from the candidate ball position associated with P(n,c)-P(n+<NUM>,a).

In this manner, as shown in <FIG>, five path list candidates are added, for example. Here, similarly to P(n, b)-P(n+<NUM>, a) and P(n, c)-P(n+<NUM>, a), the frames respectively corresponding to the two elements associated with each other included in the path list candidates may not be continuous. However, for example, the candidate ball position that is separated from the frame corresponding to the last element of the path list candidates by a predetermined number of frames (e.g., four frames) or more may be excluded from the candidate ball positions associated with the path list candidates.

As described above, in the present embodiment, a plurality of path list candidates are specified. From among these path list candidates, the candidate having the largest number of elements (candidate ball positions) included in the path list candidates is specified as the path list. Hereinafter, the candidate ball position, which is an element included in the specified path list, will be referred to as a path ball position.

Here, for example, the path list may be specified according to other criteria. For example, the path list candidate representing the path closest to the straight line may be specified as a path list.

As described above, path ball positions for approximately <NUM> frames immediately after impact are usually specified, although the path ball positions for all frames up to the landing point are not specified.

The ball path specifying unit <NUM> executes straight line estimation, for example, based on path ball positions in the respective frames specified as described above, thereby specifying the path of the ball represented by the two-dimensional coordinate system of the captured video image.

Subsequently, the specified path is extended downward by extrapolation, and the frame number corresponding to the timing at which the position in the vertical direction in the frame image is at a predetermined position is specified as the frame number corresponding to the timing of impact. In this embodiment, frame numbers are specified in units finer than integers, for example, <NUM>. Hereafter, the frame corresponding to the frame number specified in this manner is referred to as an impact frame.

The ball path specifying unit <NUM> then specifies the position of the ball before being hit, hereinafter referred to as an initial position. For example, the ball path specifying unit <NUM> specifies frame images in the frames immediately before and after the impact frame. For example, when the frame number of the impact frame is <NUM>, the frame having the frame number <NUM> corresponds to the frame immediately before the impact frame, and the frame having the frame number <NUM> corresponds to the frame immediately after the impact frame. In this case, the frame image (<NUM>) is specified as the frame image in the frame immediately before the impact frame. The frame image (<NUM>) is specified as the frame image in the frame immediately after the impact frame.

The ball path specifying unit <NUM> compares these two frame images, thereby detecting an object that exists in the frame image immediately before the impact and does not exist in the frame image immediately after the impact. The ball path specifying unit <NUM> specifies the position of the detected object as the initial position.

When the initial position cannot be specified in this manner, the ball path specifying unit <NUM> may detect a shaft from the frame image of the frame at address (e.g., the fifth frame). Subsequently, the ball path specifying unit <NUM> may specify, as the initial position, the position of the intersection of the line obtained by extending the detected shaft and the line extending in the left-right direction at a position <NUM>% upper than the lower side of the player area <NUM> by the length of the player area <NUM> in the vertical direction.

When specifying the path ball position based on the comparison result image, the initial position is a position of the ball that is placed and not moving, and thus cannot be detected because it is not a changed pixel. Before being hit, the ball is highly likely to be hidden by the club head. In such a condition, the initial position can be also specified as described above in the present embodiment.

In this embodiment, the flight distance calculating unit <NUM> calculates, for example, a head speed, a ball speed, and a flight distance of the shot performed by the player <NUM>.

For example, the flight distance calculating unit <NUM> extracts a top position frame image in which the club head appears leftmost and an impact frame image in which the club head appears rightmost from the frame images included in the captured video image. <FIG> is a diagram illustrating an example of the top position frame image. <FIG> is a diagram illustrating an example of the impact frame image.

The flight distance calculating unit <NUM> then specifies the coordinate values (x, y) of the club head in the top position frame image and the coordinate values (x', y') of the club head in the impact frame image, which are expressed in the two-dimensional image coordinate system of the frame image. The flight distance calculating unit <NUM> calculates a head moving distance D1 in the image based on the coordinate values (x, y) and (x', y'). Here, D1 is the square root of the sum of the square of (x'-x) and the square of (y'-y), for example. In the frame image, the rightward direction is the positive direction of the x-axis, and the downward direction is the positive direction of the y-axis.

The flight distance calculating unit <NUM> calculates a head moving time T1 (seconds) by multiplying the difference between the frame number of the impact frame and the frame number of the top position frame image by the capturing time interval (e.g., <NUM>/<NUM> second).

The flight distance calculating unit <NUM> calculates a head speed S (in meters per second) by dividing the value D1 by the value T1.

The flight distance calculating unit <NUM> calculates a provisional ball initial speed V (in meters per second) by multiplying the head speed S by a predetermined provisional smash factor of <NUM>.

The flight distance calculating unit <NUM> calculates a provisional total flight distance D2 (yards) by multiplying the provisional ball initial speed V by a predetermined factor of <NUM>×<NUM>×<NUM>.

The flight distance calculating unit <NUM> specifies, for each frame image, a vertical angle and a left-right angle of a ball in a camera coordinate system in a frame based on coordinate values of a path ball position in the frame image, which are expressed in the two-dimensional image coordinate system of the frame image. Here, coordinate values of a path ball position for a frame image (n) are expressed as (x1 (n), y1 (n)), the vertical angle in the camera coordinate system in the frame corresponding to the frame image (n) is expressed as a1 (n), and the left-right angle is expressed as a2 (n). Here, the vertical angle in the camera coordinate system is an angle in the vertical direction when looking at the ball from the camera unit <NUM> with reference to a predetermined direction, and the left-right angle is an angle in the left-right direction when looking at the ball from the camera unit <NUM> with reference to a predetermined direction.

In the present embodiment, as described above, the position and inclination of the user terminal <NUM> is adjusted, and thus the two-dimensional coordinate values of the ball position can be associated with the vertical angle and the left-right angle one-to-one. As such, in this embodiment, for example, the data indicating a conversion expression or a conversion table previously stored in the flight distance calculating unit <NUM> is referred to, and the vertical angle and the left-right angle can be thereby uniquely specified based on the two-dimensional coordinate values of the ball position.

Similarly, the flight distance calculating unit <NUM> specifies the vertical angle a1' and the left-right angle a2' when viewing the ball at the initial position from the camera unit <NUM> based on the coordinate values (x0, y0) of the initial position of the ball.

The flight distance calculating unit <NUM> specifies a parabola representing the path of the ball in the three-dimensional space based on the calculated provisional total flight distance D2, where the parabola is associated with the provisional total flight distance D2. In this embodiment, assume that a provisional total flight distance D2 and a shape of parabola are associated with each other in advance. The flight distance calculating unit <NUM> calculates three-dimensional coordinate values of the position of the ball in the three-dimensional space along the parabola corresponding to the vertical angle and the left-right angle. Here, for example, the three-dimensional coordinate values (x2 (n), y2 (n), z2 (n)) are calculated for the frame corresponding to the frame image (n) based on the parabola, the vertical angle a1 (n), and the left-right angle a2 (n) described above.

In the present embodiment, for each frame, three-dimensional coordinate values of a position of a ball are calculated with the three-dimensional coordinate values corresponding to the vertical angle a1' and the left-right angle a2' of the initial position of the ball as the origin.

Here, the position and the inclination of the user terminal <NUM> are also adjusted, and thus a combination of parabola, vertical angle, and left-right angle can be associated with the three-dimensional coordinate values of the position of the ball one-to-one. As such, in this embodiment, for example, the data indicating the conversion expression or the conversion table previously stored in the flight distance calculating unit <NUM> is referred to, and the three-dimensional coordinate values of the ball position can be thereby uniquely specified based on the parabola, the vertical angle, and the left-right angle.

The flight distance calculating unit <NUM> solves a binary equation in which two of the three-dimensional coordinate values of the position of the ball calculated as described are input and the left-right angle and the elevation angle are unknown numbers, thereby calculating the left-right angle and the elevation angle corresponding to the input.

The flight distance calculating unit <NUM> selects a pair of entered three-dimensional coordinate values in various patterns, and calculates a combination of a left-right angle value and an elevation angle value for each pattern. The flight distance calculating unit <NUM> then determines the average value of the elevation angle values calculated in the various patterns to be the elevation angle a3 for the entire flight path of the ball.

The flight distance calculating unit <NUM> calculates, for each frame, a left-right angle when the calculated elevation angle a3 is a fixed value based on the three-dimensional coordinate values of the position of the ball.

As shown in <FIG>, the flight distance calculating unit <NUM> calculates the initial value and the final value (the left-right angle when the ball lands) of the left-right angle using regression analysis. Here, for example, the left-right angle at impact is calculated as the initial value of the left-right angle. The left-right angle at the landing point specified based on the parabola described above is calculated as the final value of the left-right angle.

The flight distance calculating unit <NUM> calculates a smash factor by a mathematical expression of <NUM> × (<NUM> - <NUM> × elevation angle deviation) × cos (left-right angle absolute value + curve angle absolute value).

Here, the elevation angle deviation indicates the absolute value of the difference between the calculated elevation angle a3 and the value <NUM>. The left-right angle absolute value indicates the absolute value of the value calculated as the initial value of the left-right angle. Further, the curve angle absolute value indicates the absolute value of the value obtained by subtracting the value calculated as the initial value of the left-right angle from the value calculated as the final value of the left-right angle.

The flight distance calculating unit <NUM> calculates the initial ball speed by multiplying the smash factor thus calculated by the calculated head speed S.

The flight distance (yard) is then calculated by multiplying the calculated initial ball speed by <NUM>×<NUM>×<NUM>.

The methods of calculating the flight distance etc. are not limited to the above-described methods.

In the present embodiment, the club head path specifying unit <NUM> specifies a club head path in a captured video image based on the captured video image, for example.

For example, the club head path specifying unit <NUM> specifies a head detecting area <NUM> shown in <FIG>. Here, the position, shape, and size of the head detecting area <NUM> are uniquely specified based on the position, shape, and size of the player area <NUM> according to predetermined rules, for example. In the example of <FIG>, the head detecting area <NUM> is shown as an area in which the center position is the same as the player area <NUM>, the length of the left-right direction is three times the length of the player area <NUM>, and the length of the vertical direction is twice the length of the player area <NUM>.

The club head path specifying unit <NUM> specifies a position of the club head in the head detecting area <NUM> in the plurality of frame images included in the captured video image. Here, for example, the area in which the club head is captured in the frame images up to the impact frame may be specified. When specifying the area in which the club head is captured, the comparison result image described above may be used. For example, in the head detecting area <NUM>, the area corresponding to a pixel block of changed pixels within a predetermined number of pixels may be specified as an area in which the club head is captured. The club head path specifying unit <NUM> may specify a position of a representative point (e.g., the center) in each specified area as the position where the club head is captured (hereinafter, referred to as a club head position).

Then, the club head path specifying unit <NUM> may specify the path of the club head based on the plurality of specified club head positions. For example, the club head path specifying unit <NUM> may use cubic Bezier curves to interpolate the specified club head positions for each frame to specify, as the path of the club head, the movement path L1 of the club head having a smooth line interpolated between frames as shown in <FIG>.

The club head path specifying unit <NUM> may specify a moving direction of the club head at impact based on the captured video image. For example, the club head path specifying unit <NUM> may specify the position of the club head in the frame immediately before the impact frame on the movement path L1 of the club head specified as described above. Further, the club head path specifying unit <NUM> may specify the position of the club head in the frame immediately after the impact frame on the movement path L1 of the club head specified as described above.

The club head path specifying unit <NUM> may specify a line L2, as illustrated in <FIG>, that connects the position of the club head in the frame immediately before the impact frame with the position of the club head in the frame immediately after the impact frame.

The club head path specifying unit <NUM> may specify the angle of the moving direction of the club head at impact, which is represented by the two-dimensional image coordinate system of the captured video image, with respect to a predetermined reference direction. As shown in <FIG>, for example, the club head path specifying unit <NUM> may specify the angle formed between the line L3 extending in the vertical direction in the frame image and the line L2 described above along the moving direction of the club head as the angle a4 of the moving direction of the club head at impact. In the following, the angle of the moving direction of the club head at impact specified in this manner will be referred to as a club path angle. In the present embodiment, the counterclockwise direction is the positive direction for the club path angle. That is, the value of the club path angle a4 shown in <FIG> is negative.

In this embodiment, for example, the launch direction specifying unit <NUM> specifies a launch direction of the ball in a captured video image based on the captured video image.

The launch direction specifying unit <NUM> specifies the launch direction based on, for example, the positions of the ball in the frame images included in the captured video image. For example, the launch direction specifying unit <NUM> may specify the launch direction based on the ball path specified by the ball path specifying unit <NUM>. For example, the launch direction may be specified based on the path ball positions in the frame images in the predetermined number of frames immediately after the impact frame (e.g., five frames) specified by the ball path specifying unit <NUM>. For example, as illustrated in <FIG>, the direction of the approximate line L4 along the path ball positions in the frame images in the predetermined number of frames immediately after the impact frame may be specified as the launch direction.

The launch direction specifying unit <NUM> may specify an angle of the launch direction with respect to a predetermined reference direction, which is represented by the two-dimensional image coordinate system of the captured video image. For example, as shown in <FIG>, the angle formed between the line L3 extending in the vertical direction in the frame image and the line L4 along the launching direction may be specified as a launch angle a5. In the present embodiment, the counterclockwise direction is the positive direction for the launch angle. That is, the value of the launch angle a5 shown in <FIG> is positive.

The launch direction specifying unit <NUM> specifies an orientation of the face of the club at impact based on the specified launch direction.

For example, when the absolute value of the launch angle a5 is less than a predetermined value, the orientation of the face may be specified as square. When the absolute value of the launch angle a5 is the predetermined value or more and the value of the launch angle a5 is positive, the orientation of the face may be specified as being closed. Further, when the absolute value of the launch angle a5 is the predetermined value or more and the value of the launch angle a5 is negative, the orientation of the face may be specified as being open.

In the present embodiment, the swing path type specifying unit <NUM> specifies, for example, a type of a swing path performed by the player <NUM>. Here, for example, the swing path type specifying unit <NUM> specifies whether the swing path of the swing performed by the player <NUM> is inside-out, inside-in, or outside-in.

The swing path type specifying unit <NUM> may specify a type of the swing path based on the club path angle a4 described above, for example.

For example, when the absolute value of the club path angle a4 is less than a predetermined value, the type of the swing path may be specified as inside-in. When the absolute value of the club path angle a4 is the predetermined value or more and the value of the club path angle a4 is positive, the type of swing path may be specified as outside-in. Further, when the absolute value of the club path angle a4 is the predetermined value or more and the value of the club path angle a4 is negative, the type of swing path may be specified as inside-out.

The method of specifying the type of swing path by the swing path type specifying unit <NUM> is not limited to the method described above.

For example, the swing path type specifying unit <NUM> may specify a position of a predetermined body part of the player <NUM> in a frame image at address based on the captured video image. Here, for example, the swing path type specifying unit <NUM> may specify a position of a predetermined body part of the player <NUM> in a frame image having a predetermined frame number, e.g., the frame image (<NUM>). For example, the swing path type specifying unit <NUM> may specify the position of the shoulder of the player <NUM> shown in the skeleton model <NUM> specified by the player area specifying unit <NUM>.

As shown in <FIG>, the swing path type specifying unit <NUM> may specify a line L5 connecting the specified shoulder position Q1 with the initial position Q2 specified by the ball path specifying unit <NUM>.

As shown in <FIG>, the swing path type specifying unit <NUM> may also specify a line L6 along the shaft captured in the frame image.

Here, the swing path type specifying unit <NUM> may specify an address shaft detection area <NUM> and a backswing shaft detection area <NUM> shown in <FIG>. Here, for example, the address shaft detection area <NUM> may be an area located at the lower right of the player area <NUM> and its position and size are uniquely specified based on the player area <NUM>. The backswing shaft detection area <NUM> may be, for example, an area located at the upper left of the player area <NUM> and its position and size are uniquely specified based on the player area <NUM>.

The swing path type specifying unit <NUM> may specify the frame image in which the shaft is detected in the backswing shaft detection area <NUM> as the frame image at backswing. The swing path type specifying unit <NUM> may generate a shaft detection image representing the difference between pixel values of pixels in the frame image at address and pixel values of corresponding pixels in the frame image at backswing in the address shaft detection area <NUM>. Here, the top position frame image described above may be used instead of the frame image at backswing specified as described above.

The swing path type specifying unit <NUM> may apply a straight-line detection method, such as Hough transform, to the generated shaft detection images so as to specify the line L6 along the shaft.

The swing path type specifying unit <NUM> may then specify at least one reference angle based on at least one of the position of the predetermined body part of the player <NUM> at address or the position of the shaft at address, which are specified based on the captured video image.

Here, the swing path type specifying unit <NUM> may specify a first reference angle based on, for example, a position of a predetermined body part. The swing path type specifying unit <NUM> may specify a first reference angle based on the position of the predetermined body part and the position of the ball before swing.

For example, as shown in <FIG>, the swing path type specifying unit <NUM> may specify, as the first reference angle a6, an angle formed between a line L7 extending in the left-right direction in the frame image and a line L5 connecting the shoulder position Q1 with the initial position Q2.

The swing path type specifying unit <NUM> may also specify a second reference angles based on, for example, the position of the shaft at address.

For example, as shown in <FIG>, the swing path type specifying unit <NUM> may specify, as the second reference angle a7, an angle formed between the line L7 extending in the left-right direction in the frame image and the line L6 along the shaft.

In this manner, the reference angles represented in the two-dimensional image coordinate system of the captured video image may be specified.

The swing path type specifying unit <NUM> may specify a downswing angle of the club head in the captured video image based on the captured video image. For example, the swing path type specifying unit <NUM> may specify the downswing angle based on, for example, frame images of a predetermined number of frames before the impact.

For example, assume that the frame number of the frame immediately after the impact frame is m. Here, the positions of the club head on the movement path L1 of the club head specified as described above may be specified for four frames from the (m-<NUM>)-th frame to the (m-<NUM>)-th frame. Subsequently, a line representing the moving direction of the club head obtained by averaging the moving direction of the club head in the (m-<NUM>)-th frame to the (m-<NUM>)-th frame, the moving direction of the club head in the (m-<NUM>)-th frame to the (m-<NUM>)-th frame, and the moving direction of the club head in the (m-<NUM>)-th frame to the (m-<NUM>)-th frame may be specified as a line L8 along the downswing direction illustrated in <FIG>.

Alternatively, an approximate line along the positions of the club head in four frames from the (m-<NUM>)-th frame to the (m-<NUM>)-th frame may be specified as the line L8 along the downswing direction illustrated in <FIG>. In this regard, the frames used for specifying the line L8 are not limited to four frames from the (m-<NUM>)-th frame to the (m-<NUM>)-th frame.

As shown in <FIG>, the swing path type specifying unit <NUM> may then specify, as a downswing angle a8, the angle formed between the line L7 extending in the left-right direction in the frame image and the line L8 along the downswing direction described above.

In this manner, the downswing angle represented in the two-dimensional image coordinate system of the captured video image may be specified.

The swing path type specifying unit <NUM> may compare the reference angle with the downswing angle. The swing path type specifying unit <NUM> may specify a type of the swing path in the swing based on the result of comparing the reference angle with the downswing angle. For example, the swing path type specifying unit <NUM> may specify whether the swing path in the swing is inside-out, inside-in, or outside-in based on the first reference angle, the second reference angle, and the downswing angle.

For example, if the value of the downswing angle a8 is greater than the value of the first reference angle a6, the type of swing path may be specified as outside-in. In this case, the club is swung down from the outside (upper side) of the so-called V zone.

Further, if the value of the downswing angle a8 is equal to or more than the value of the second reference angle a7 and equal to or less than the value of the first reference angle a6, the type of swing path may be specified as inside-in. This case corresponds to the situation where the so-called swing path is kept within the range of the V-zone.

If the value of the downswing angle a8 is smaller than the value of the second reference angle a7, the type of swing path may be specified as inside-out. In this case, the club is swung down from the inside (lower side) of the so-called V zone.

In the present embodiment, the curve direction determining unit <NUM> determines the curve direction of the launched ball based on the path of the club head specified as described above and the launch direction specified as described above, for example. The curve direction determining unit <NUM> may determine the curve direction based on the moving direction of the club head and the launch direction.

The curve direction determining unit <NUM> determines the curve direction of the ball based on, for example, the difference between the signed club path angle a4 and the signed launch angle a5. For example, if the difference between the value of the signed club path angle a4 and the value of the signed launch angle a5 is less than a predetermined value, it is determined that the ball is not curved (flies straight). Further, for example, if the difference between the value of the signed club path angle a4 and the value of the signed launch angle a5 is the predetermined value or more and the value of the signed launch angle a5 is larger than the value of the signed club path angle a4, it is determined that the ball is curved to the left. For example, if the difference between the value of the signed club path angle a4 and the value of the signed launch angle a5 is the predetermined value or more and the value of the signed launch angle a5 is smaller than the value of the signed club path angle a4, it is determined that the ball is curved to the right.

The curve direction determining unit <NUM> may determine the curve direction and the magnitude of the curve of the launched ball based on the path of the club head and the launch direction. For example, the absolute value of the difference between the value of the signed club path angle a4 and the value of the signed launch angle a5 may be specified as a curve value indicating the magnitude of the curve. Further, for example, a curve evaluation value for evaluating the magnitude of the curve in several stages (e.g., three stages), such as "large", "medium", and "small", may be determined based on the magnitude of the absolute value of the difference between the value of the signed club path angle a4 and the value of the signed launch angle a5.

In the present embodiment, for example, the analysis video image generating unit <NUM> generates an analysis video image by overlaying an image indicating a result of swing analysis processing onto a captured video image.

The analysis video image generating unit <NUM> may generate an analysis video image by overlaying an image representing the path of the club head, the orientation of the club face at impact specified based on the launch direction, and the curve direction of the ball onto the captured video image. For example, an analysis video image may be generated in which the impact image <NUM> shown in <FIG> is overlaid on a frame image included in the captured video image. Here, the type of the swing path specified by the swing path type specifying unit <NUM> may be represented in the impact image <NUM> as the path of the club head.

<FIG> is a diagram showing an example of a variation of the impact image <NUM>. <FIG> shows nine types of impact images <NUM> according to combinations of swing path types and face orientation types.

The impact images <NUM> shown in <FIG> reflect the types of swing paths, face orientations, and curve directions of the ball. The impact image <NUM> may reflect not only the types of swing paths, face orientations, and curve directions of the ball, but also the magnitude.

For example, the magnitude of the angle formed between the moving direction of the head (the direction in which the arrows are aligned) indicated by the head moving direction image <NUM> and the vertical direction may correspond to the magnitude of the club path angle a4 described above. Here, the magnitude of the angle formed between the moving direction of the head indicated by the head moving direction image <NUM> and the vertical direction need not be the same as the magnitude of the club path angle a4 described above. For example, the angle formed between the moving direction of the head indicated by the head moving direction image <NUM> and the vertical direction may be an angle that is determined based on the club path angle a4 according to a predetermined conversion rule.

For example, the angle formed between the moving direction of the head indicated by the head moving direction image <NUM> and the vertical direction may be determined based on the magnitude of the club path angle a4 from among the magnitude of the angles of several stages.

For example, as shown in <FIG>, the magnitude of the angle formed between the extending direction of the linear face orientation image <NUM> and the left-right direction may be in accordance with the launch angle a5 described above. Here, the angle formed between the extending direction of the face orientation image <NUM> and the left-right direction need not be the same as the magnitude of the launch angle a5 described above. For example, the angle formed between the extending direction of the face orientation image <NUM> and the left-right direction may be an angle determined based on the launch angle a5 according to a predetermined conversion rule.

For example, the angle formed between the extending direction of the linear face orientation image <NUM> and the left-right direction may be determined based on the magnitude of the launch angle a5 described above from among the magnitudes of the angles of several stages.

The magnitude of the curve of the ball direction image <NUM> may correspond to the magnitude of the absolute value of the difference between the value of the club path angle a4 and the value of the launch angle a5. For example, when the absolute value of the difference between the value of the club path angle a4 and the value of the launch angle a5 is greater, the curve of the ball direction image <NUM> may be represented greater. For example, the magnitude of the curve of the ball direction image <NUM> may correspond to the curve evaluation value described above.

The analysis video image generating unit <NUM> may generate an analysis video image in which a line representing the reference angle described above is overlaid on the captured video image. For example, the analysis video image generating unit <NUM> may generate an analysis video image in which a line representing the first reference angle and a line representing the second reference angle are overlaid on the captured video image. For example, an analysis video image in which the V-zone image <NUM> shown in <FIG> is overlaid on a frame image included in the captured video image may be generated. As shown in <FIG>, the V-zone image <NUM> is a triangle image, and its upper side corresponds to the line L5 shown in <FIG>, and its lower side corresponds to the line L6 shown in <FIG>. Here, for example, the V-zone image <NUM> may be positioned in the frame image such that the upper and lower sides of the V-zone image <NUM> pass through the initial positions described above.

As shown in <FIG>, a belt-shaped club head path effect image <NUM> corresponding to the path of the club head specified as described above may be overlaid on the analysis video image. Further, a belt-shaped ball path effect image <NUM> corresponding to the path of the ball specified as described above may be overlaid on the analysis video image.

As shown in <FIG>, the analysis video image may include analysis result information <NUM> indicating an analysis result of the swing analysis processing described above. For example, the analysis result information <NUM> may include information indicating values (e.g., initial ball speed, flight distance) calculated by the flight distance calculating unit <NUM> and information indicating a type of the swing path specified by the swing path type specifying unit <NUM>.

In the present embodiment, for example, the analysis video image generating unit <NUM> stores the generated analysis video image in the analysis video image storage unit <NUM>.

In the present embodiment, for example, the analysis video image storage unit <NUM> stores the analysis video image generated by the analysis video image generating unit <NUM>.

In the present embodiment, for example, the display control unit <NUM> displays the analysis video image stored in the analysis video image storage unit <NUM> on the touch panel <NUM> in response to a request from the user of the user terminal <NUM>. In this manner, the display control unit <NUM> may notify the user of the result of comparing the reference angle with the downswing angle described above. For example, the display control unit <NUM> may notify the user of the type of swing path specified based on the result of comparing the reference angle with the downswing angle described above.

An example of the processing performed by the user terminal <NUM> according to the present embodiment will be described referring to a flow chart shown in <FIG>.

The captured video image obtaining unit <NUM> obtains a captured video image (S101).

The player area specifying unit <NUM> specifies a player area <NUM> (S102).

The comparison result image generating unit <NUM> generates a plurality of comparison result images based on the captured video image obtained in the processing in S101 (S103).

The start area specifying unit <NUM> specifies a start area <NUM> based on the player area <NUM> specified in the processing in S102 (S104).

The ball path specifying unit <NUM> specifies a path of a ball based on the comparison result images generated in the processing in S103 (S105).

The flight distance calculating unit <NUM> calculates a flight distance of the shot in the swing in the captured video image obtained in the processing in S101 (S106).

The club head path specifying unit <NUM> specifies a path of the club head based on the captured video image obtained in the processing in S101 (S107).

The launch direction specifying unit <NUM> specifies a launch direction of the ball based on the ball path specified in the processing in S105 (S108).

The swing path type specifying unit <NUM> specifies a type of the swing path in the swing in the captured video image obtained in the processing in S101 based on the path of the club head specified in the processing in S107 and the launch direction of the ball specified in the processing in S108 (S109).

The curve direction determining unit <NUM> determines a curve direction of the ball in the shot based on the path of the club head specified in the processing in S107 and the launch direction of the ball specified in the processing in S108 (S110).

The analysis video image generating unit <NUM> generates an analysis video image based on the captured video image obtained in the processing in S101 and the results of the processing in S105 to S110 (S111).

The analysis video image generating unit <NUM> stores the analysis video image generated in the processing in S111 in the analysis video image storage unit <NUM> (S112), and the processing shown in the present example is terminated.

As described above, in the processing in S109, the swing path type specifying unit <NUM> may specify the downswing angle and the reference angle. The swing path type specifying unit <NUM> may specify the type of the swing path in the swing in the captured video image obtained in the processing in S101 based on the specified downswing angle and the reference angle.

In the present embodiment, it is possible to accurately determine the curve direction of the ball by simply capturing the player making a golf swing without directly measuring the orientation of the face. In this manner, according to the present embodiment, a general player can easily analyze a golf swing using a smart phone or other devices in a golf course and a golf practice range, for example.

In the present embodiment, the reference angle and the downswing angle are compared with each other by simply capturing the player making a golf swing. As such, according to the present embodiment, a general player who does not have knowledge about golf, such as a range of an appropriate swing plane angle, can easily grasp whether the swing is correctly performed by checking the result of comparing the reference angle with the downswing angle.

In the present embodiment, the type of swing path in the swing is accurately specified based on the reference angle and the downswing angle as described above. As such, according to the present embodiment, a general player can easily grasp whether the swing is performed in the correct swing path by checking the type of the player's swing path.

The present invention is not limited to the embodiment described above.

For example, the first reference angle need not be specified based on the position of the shoulder, but may be specified based on the position of the neck of the player <NUM>, for example.

Further, for example, the first reference angle need not be specified based on the initial position of the ball, but may be specified based on, for example, a position specified by the user or a position of the club head at address.

For example, the second reference angle need not be specified based on the position of the shaft, but may be specified based on the position of the waist of the player <NUM>, for example. For example, the second reference angle may be specified based on a line connecting the position of the waist of the player <NUM> with the initial position of the ball, or a line connecting the position of the waist of the player <NUM> with the position of the club head at address.

Further, two reference angles need not be specified, and the type of swing path may be specified based on one reference angle. For example, whether the swing path is outside-in may be determined based on one reference angle. For example, whether the swing path is inside-out may be determined based on one reference angle.

All or some of the functions shown in <FIG> may be implemented in a server capable of communicating with the user terminal <NUM>.

Claim 1:
A golf swing analysis system, comprising:
captured video image obtaining means (<NUM>) for obtaining a captured video image in which a player making a golf swing and viewed from rearward of a target direction of the ball is captured;
launch direction specifying means (<NUM>) for specifying a launch direction of a ball captured in the captured video image based on the captured video image;
wherein the golf swing analysis system is characterized by comprising:
club head path specifying means (<NUM>) for specifying a path of a club head captured in the captured video image based on the captured video image, wherein the path of the club head is a moving direction of the club head at impact; and
curve direction determining means (<NUM>) for determining a curve direction of the launched ball based on the path of the club head and the launch direction.