Patent Publication Number: US-2022221482-A1

Title: Methods and system for cycle recognition in repeated activities by identifying stable and repeatable features

Description:
FIELD 
     The device and method disclosed in this document relates to human motion sensing and, more particularly, to analysis of human motion for a repeated activity. 
     BACKGROUND 
     Unless otherwise indicated herein, the materials described in this section are not admitted to be the prior art by inclusion in this section. 
     An important yet challenging problem in industry is the recognition of cycle durations in repeated physical human activities based on motion sensing data. For example, in an assembly line of a factory, workers perform repeated tasks to assemble a product. Accurate measurement and recognition of cycle durations for the repeated tasks facilitates the calculation of production volume and the recognition of manufacturing anomalies. 
     However, existing solutions for such monitoring are either labor intensive, requiring manual measurement of the cycles of the repeated tasks, or non-scalable, requiring a specialized device in the assembly line that usually only works for limited scenarios. Moreover, existing solutions often require a high level of standardization and consistency in the performance of each cycle of the repeated human activity, in terms of orientation and speed. Therefore, what is needed is a method and system for monitoring and recognizing cycle durations in repeated human activity that is cost effective and reliable, even when there is inconsistent performance of each cycle of the repeated human activity. 
     SUMMARY 
     A method for recognizing repetitions of a repeated activity is disclosed. The method comprises receiving, with a processor, first motion data corresponding to a first plurality of repetitions of a repeated activity, the first motion data including labels identifying time boundaries between each repetition in the first plurality of repetitions. The method further comprises identifying, with the processor, a salient segment of the first motion data corresponding to a motion of the repeated activity that occurs in all of the first plurality of repetitions. The method further comprises receiving, with the processor, second motion data corresponding to a second plurality of repetitions of the repeated activity from a motion sensing system. The method further comprises identifying, with the processor, time boundaries between each repetition in the second plurality of repetitions by detecting segments of the second motion data that are most similar to the salient segment of the first motion data. 
     A method for determining metadata of a repeated activity is disclosed. The method comprises receiving, with a processor, first motion data corresponding to a first plurality of repetitions of a repeated activity, the first motion data including labels identifying time boundaries between each repetition in the first plurality of repetitions. The method further comprises identifying, with the processor, a salient segment of the first motion data corresponding to a motion of the repeated activity that occurs in all of the first plurality of repetitions. The method further comprises storing, in a memory, metadata of the first motion data, the metadata including the salient segment of the first motion data. 
     A further method for recognizing repetitions of a repeated activity. The method comprises storing, in a memory, metadata of first motion data corresponding to a first plurality of repetitions of a repeated activity, the metadata including a salient segment of the first motion data corresponding to a motion of the repeated activity that occurs in all of the first plurality of repetitions. The method further comprises receiving, with the processor, second motion data corresponding to a second plurality of repetitions of the repeated activity from a motion sensing system. The method further comprises identifying, with the processor, time boundaries between each repetition in the second plurality of repetitions by detecting segments of the second motion data that are most similar to the salient segment of the first motion data. The method further comprises outputting, with an output device, the time boundaries between each repetition in the second plurality of repetitions. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The foregoing aspects and other features of the methods and system are explained in the following description, taken in connection with the accompanying drawings. 
         FIG. 1  shows a system for monitoring performance of a repeated activity. 
         FIG. 2  shows a flow diagram for a method for recognizing cycle durations of a repeated activity. 
         FIG. 3  shows an identification of feature candidates in an exemplary cycle of motion data of a repeated activity. 
         FIG. 4  shows a correspondence between regions of two exemplary cycles of motion data of a repeated activity. 
         FIG. 5  shows an exemplary re-aligned cycle of a repeated activity. 
     
    
    
     DETAILED DESCRIPTION 
     For the purposes of promoting an understanding of the principles of the disclosure, reference will now be made to the embodiments illustrated in the drawings and described in the following written specification. It is understood that no limitation to the scope of the disclosure is thereby intended. It is further understood that the present disclosure includes any alterations and modifications to the illustrated embodiments and includes further applications of the principles of the disclosure as would normally occur to one skilled in the art which this disclosure pertains. 
     System Overview 
       FIG. 1  shows a system  100  for monitoring performance of a repeated activity. The system  100  at least comprises a motion sensing system  110  and a processing system  120 . The motion sensing system  110  includes one or more sensors configured to measure or track motions corresponding to a repeated activity. The processing system  120  is configured to process motion data received from the motion sensing system  110  to recognize and measure cycle durations in the repeated activity. In contrast to the conventional systems and methods, which may work for repeated activities having a high level of standardization, the system  100  advantageously enables recognition and monitoring of cycle durations for a repeated activity, even when significant abnormal motions are present in each cycle. Particularly, each cycle may include abnormal motions that differ from other corresponding motions in other cycles, or even don&#39;t exist in other cycles, but these abnormalities do not affect the performance of the cycle recognition. Thus, the system  100  can be utilized in a significantly broader set of applications, compared conventional systems and methods. 
     In at least one embodiment, the repeated activity is a repeated human activity (also referred to herein as a “repeated task”) comprising motions performed by a human. As an example, the repeated human activity may comprise the repeated motions involved in the assembly of a product by a worker in an assembly line of a factory. As another example, the repeated human activity may comprise the repeated motions involved in certain types of physical exercise by an athlete (e.g. repetitions, steps, etc.). As a further example, the repeated human activity may comprise the repeated motions involved in scanning products for checkout at a retail store by a cashier. Finally, it will be appreciated that, in principle, the tracked motions may correspond to a repeated activity that is performed by some robot, tool, or other object, which may be directed by a human or performed autonomously. 
     The motion sensing system  110  comprises at least one sensor configured to track the motions that comprise the repeated activity. In at least some embodiments, the motion sensing system  110  comprises at least one inertial measurement unit (IMU)  112 . The IMU  112  includes one or more gyroscope sensors and one or more accelerometers configured to provide motion data in the form of acceleration and orientation measurements. In one embodiment, the IMU  112  comprises an integrated 6-axis inertial sensor that provides both triaxial acceleration measurements and triaxial gyroscopic/orientation measurements. In at least one embodiment, the IMU  112  is worn on the body of a human and may, for example, take the form of a wrist-worn watch or a hand-worn glove having the IMU  112  integrated therewith. In other embodiments, the IMU  112  may be integrated with an object that is carried by the human, such as a smartphone or a tool used in the repeated task. 
     In further embodiments, motion sensing system  110  may alternatively include other types of sensors than the IMU  112 , such as an RGB-D camera, infra-red sensors, ultrasonic sensors, pressure sensors, or any other sensor configured to measure data characterizes a motion. Additionally, in some embodiments, the motion sensing system  110  may include multiple different types of sensors that provide multi-modal motion data that is processed in a multi-channel manner by the processing system  120 , or which is fused using one or more sensor data-fusion techniques by the processing system  120  or other component of the system  100 . 
     The processing system  120  is configured to process motion data captured by the motion sensing system  110  to recognize and measure cycle durations in the repeated activity. As used herein, a “cycle” refers to an individual repetition of a repeated activity. Advantageously, the processing system  120  is trained to measure cycle durations in a repeated activity based on only a limited set of motion data that has been manually labeled with cycle boundaries (i.e., at least start and end times for each individual cycle). Based on this limited set of labeled motion data, the processing system  120  determines the most salient features of the repeated activity and generates metadata of labeled motion data that is used to determine cycle durations in new unlabeled motion data. 
     In the illustrated exemplary embodiment, the processing system  120  comprises at least one processor  122 , at least one memory  124 , a communication module  126 , a display screen  128 , and a user interface  130 . However, it will be appreciated that the components of the processing system  120  shown and described are merely exemplary and that the processing system  120  may comprise any alternative configuration. Particularly, the processing system  120  may comprise any computing device such as a desktop computer, a laptop, a smart phone, a tablet, or another electronic device. Thus, the processing system  120  may comprise any hardware components conventionally included in such computing devices. 
     The memory  124  is configured to store data and program instructions that, when executed by the at least one processor  122 , enable the processing system  120  to perform various operations described herein. The memory  124  may be of any type of device capable of storing information accessible by the at least one processor  122 , such as a memory card, ROM, RAM, hard drives, discs, flash memory, or any of various other computer-readable medium serving as data storage devices, as will be recognized by those of ordinary skill in the art. Additionally, it will be recognized by those of ordinary skill in the art that a “processor” includes any hardware system, hardware mechanism or hardware component that processes data, signals or other information. Thus, the at least one processor  122  may include a central processing unit, graphics processing units, multiple processing units, dedicated circuitry for achieving functionality, programmable logic, or other processing systems. Additionally, it will be appreciated that, although the processing system  120  is illustrated as single device, the processing system  120  may comprise several distinct processing systems  120  that work in concert to achieve the functionality described herein. 
     The communication module  126  may comprise one or more transceivers, modems, processors, memories, oscillators, antennas, or other hardware conventionally included in a communications module to enable communications with various other devices. In at least some embodiments, the communication module  126  includes a Wi-Fi module configured to enable communication with a Wi-Fi network and/or Wi-Fi router (not shown). In further embodiments, the communications module  126  may further include a Bluetooth® module, an Ethernet adapter and communications devices configured to communicate with wireless telephony networks. 
     The display screen  128  may comprise any of various known types of displays, such as LCD or OLED screens. In some embodiments, the display screen  128  may comprise a touch screen configured to receive touch inputs from a user. The user interface  130  may suitably include a variety of devices configured to enable local operation of the processing system  120  by a user, such as a mouse, trackpad, or other pointing device, a keyboard or other keypad, speakers, and a microphone, as will be recognized by those of ordinary skill in the art. Alternatively, in some embodiments, a user may operate the processing system  120  remotely from another computing device which is in communication therewith via the communication module  126  and has an analogous user interface. 
     The program instructions stored on the memory  124  include a repeated activity monitoring program  132 . As discussed in further detail below, the processor  122  is configured to execute the repeated activity monitoring program  132  to process labeled motion data captured by the motion sensing system  110  to derive metadata describing the most salient features of a repeated activity. Additionally, the processor  122  is configured to execute the repeated activity monitoring program  132  to process unlabeled motion data captured by the motion sensing system  110  to recognize and measure cycle durations in the repeated activity. 
     Methods for Cycle Duration Recognition 
       FIG. 2  shows a flow diagram for a method  200  for recognizing cycle durations of a repeated activity. In the description of these method, statements that some task, calculation, or function is performed refers to a processor (e.g., the processor  122  of the processing system  120 ) executing programmed instructions stored in non-transitory computer readable storage media (e.g., the memory  124  of the processing system  120 ) operatively connected to the processor to manipulate data or to operate one or more components of the processing system  120  or the system  100  to perform the task or function. Additionally, the steps of the methods may be performed in any feasible chronological order, regardless of the order shown in the figures or the order in which the steps are described. 
     In summary, the method  200  has two major components: an offline preprocessing phase and an online processing phase. In the offline preprocessing phase, labeled motion data corresponding to a plurality of individual cycles of a repeated activity are provided as input. The labeled motion data includes labels that identify the time boundaries for each cycle of the repeated activity. A plurality of segments of the motion data are identified as feature candidates and each evaluated for their salience across all of the individual labeled cycles. For each labeled cycle, the feature candidate that is evaluated as most consistent across all of the individual labeled cycles is selected as the cycle feature, and most consistent one of the selected cycle features is selected as the main feature of the repeated activity. Finally, metadata describing the cycle features, the main feature, and other properties of the labeled cycles are determined and provided as an output. 
     Next, in the online processing phase, unlabeled motion data corresponding to a plurality of individual cycles of the repeated activity are provided as input. Cycle features are identified in the unlabeled motion data and cycle boundaries in unlabeled motion data are determined based on the identified cycle features. Based on the identified cycle boundaries in the unlabeled motion data, cycle durations are determined and provided as an output. 
     In greater detail and with continued reference to  FIG. 2 , the method  200  begins, in the offline preprocessing phase, with receiving labeled motion data corresponding to a plurality of individual cycles of a repeated activity (block  210 ). Particularly, the processor  122  receives a set of labeled motion data corresponding to a plurality of individual cycles of a repeated activity (e.g., 10-20 cycles). The processor  122  may read the labeled motion data from the memory  124  or from some other local storage medium, or the processor  122  may operate the communication module  126  to receive the labeled motion data from some other computing device or remote storage device. The labeled motion data includes labels that identify the time boundaries between each individual cycle of the repeated activity. In at least one embodiment, the labels are generated manually by a user. In at least some embodiments, the system  100  itself is configured to facilitate the capture and manual labeling of the labeled motion data using a graphical user interface or the like. 
     The method  200  continues, in the offline preprocessing phase, with identifying a plurality of feature candidates in the plurality of individual cycles (block  220 ). Particularly, the processor  122  identifies a plurality of feature candidate segments (also referred to herein as simply “feature candidates”) of the labeled motion data. Each feature candidate is a continuous portion of the labeled motion data in a particular cycle of the plurality of individual cycles of the repeated activity. As will be described below, these feature candidates are evaluated for their salience and one or more are selected as being the most salient segment(s) and are used to identify cycles of the repeated activity in new unlabeled motion data. 
     In at least one embodiment, the processor  122  is configured to divide the labeled motion data into a plurality of frame segments (also referred to herein as simply “frames”). More particularly, the processor  122  divides each particular cycle of the labeled motion data into a respective plurality of frames. Each frame of each cycle of the labeled motion data corresponds to a discrete interval of time and, thus, comprises a continuous portion of the labeled motion data that was captured during the discrete interval of time. In at least one embodiment, each frame corresponds to a fixed duration of time (e.g., 2 seconds). Alternatively, in at least one embodiment, each frame corresponds to a duration of time that is a fixed percentage of the total duration of cycle (e.g., 10% of the cycle duration). In at least one embodiment, each frame is defined such that there is a predetermined amount of overlap with the previous frame and with the subsequent frame (e.g., 50% overlap). 
     As noted above, the labeled motion data corresponds to a plurality of individual cycles of the repeated activity. Each individual cycle of the labeled motion data is denoted C i , where 0≤i&lt;N is the cycle index and N is the total number of cycles. Each cycle C i  is divided into a plurality of frames f i   j , where 0≤j&lt;L i  is the frame index and L i  is the total number of frames in the cycle C i  or, in other words, L i  is the length of cycle C i .  FIG. 3  shows an exemplary cycle of the labeled motion data. In the illustrated example, in which L i =9, the cycle C i  is divided into nine frames f i   0 , . . . , f i   8 . 
     Next, the processor  122  is configured to group sets of consecutive frames together to form the plurality of feature candidate, denoted F i   j . More particularly, for each individual cycle C i  of the labeled motion data, the processor  122  groups sets of W k  consecutive frames f i   j  together to form a respective plurality of feature candidates F i   j , where W k  the total number of frames in each feature candidates F i   j  or, in other words, W k  is the length of each feature candidate F i   j . Each feature candidate F i   j  thus comprises the motion data of consecutive frames f i   j , . . . , f i   j+W     k     −1 . 
     In at least one embodiment, for each individual cycle C i , given a value for W k , the processor  122  is configured to determine the plurality of feature candidates F i   j  as including each possible combination of W k  consecutive frames f i   j  in the cycle individual cycle C i . In other words, the processor  122  is configured to determine the plurality of feature candidates F i   j  where 0≤j≤M i  is the starting frame index of the feature candidates F i   j  and M i =L i −W k  is the starting frame index of that last in time feature candidate F i   j . With continued reference to  FIG. 3 , an identification of feature candidates in the exemplary cycle C i  of the labeled motion data is shown. In the illustrated example, in which L i =9, W k =3, and thus M i =6, the cycle C i  includes seven different feature candidates F i   0 , . . . , F i   6 . The feature candidate F i   0  includes the consecutive frames f i   0 , f i   1 , f i   2 , the feature candidate F i   1  includes the consecutive frames f i   1 , f i   2 , f i   3 , and so on. 
     In one embodiment, at least initially, the processor  122  determines the plurality of feature candidates F i   j  for each cycle C i  only for a predetermined starting/minimum feature length W k  (e.g., for W k =3). However, in some embodiments, the processor  122  determines the plurality of feature candidates F i   j  for each cycle C i  for each of a range of feature lengths W k  (e.g., for W k =3, 4, 5). 
     Returning to  FIG. 2 , the method  200  continues, in the offline preprocessing phase, determining the main feature representing the most salient motion in plurality of individual cycles (block  230 ). Particularly, the processor  122  selects one or more salient feature segments from the plurality of feature candidate segments. As used herein, the term “salient feature” or “salient segment” refers to a segment of motion data that corresponds to a motion of a repeated activity that occurs relatively more consistently across cycles of the repeated activity compared to other motions in repeated activity and which is relatively more unique within individual cycles compared to other motions in repeated activity. 
     In at least some embodiments, the processor  122  selects a best or most salient feature in each individual cycle C i  of the labeled motion data, which are referred to herein as cycle features. Particularly, the processor  122  determines the cycle feature for each individual cycle C i , denoted F i   best , as the feature candidate in the plurality of feature candidates F i   j  that has a highest degree of similarity across corresponding regions of all of the labeled cycles {C i } i=0   N−1 , and which can also be uniquely identified within each individual cycle C i  (i.e., it doesn&#39;t occur multiple times within individual cycles). Thus, the processor  122  determines a set of N cycle features {F i   best } i=0   N−1 , where each cycle feature F i   best  is the most salient feature in the respective cycle C i . 
     In at least some embodiments, after the set of cycle features {F i   best } i=0   N−1  is determined, the processor  122  selects one cycle feature from the set of cycle features {F i   best } i=0   N−1  as the main feature, denoted F main , for the repeated activity. Particularly, the processor  122  determines which cycle feature from the set of cycle features {F i   best } i=0   N−1  has the highest degree of similarity across corresponding regions of all of the labeled cycles {C i } i=0   N−1 . 
     As noted above, the cycle features F i   best  and the main feature F main  are selected from the feature candidates F i   j  based on their degree of similarity across corresponding regions of all of the individual cycles {C i } i=0   N−1 . Notably, since the different individual cycles {C i } i=0   N−1  may have different lengths and may include different motions, the corresponding regions of each other cycle C c , where c≠i, may have a different duration and timing compared to the feature candidate F i   j . In at least one embodiment, for each feature candidate F i   j  in each cycle C i , the processor  122  determines the corresponding regions of each other cycle C c , where c≠i, using a dynamic time warping (DTW) algorithm. In this way, the corresponding regions describe the same or mostly similar motion, but may have different duration and timing. 
       FIG. 4  shows a correspondence between regions of exemplary cycles C i  and C c . In the illustrated example, the exemplary cycle C i  has a length L i =9 and has a feature candidate F i   s+3  consisting of three frames f i   s+3 , . . . , f i   s+5  (i.e., W k =3) corresponding to a motion that occurred roughly in the middle of the cycle C i . In contrast, the exemplary cycle C c  has a length L i =10 and includes a region R c   t+2  that corresponds to the same or mostly similar motion as the feature candidate F i   s+3  of cycle C i  and consists of four frames f c   t+2 , . . . , f c   t+5 . Notably, the corresponding region R c   t+2  of cycle C c  has a different length and starting frame index than the feature candidate F i   s+3  of cycle C i . They correspond to the same motion, but were performed with slightly different speed and timing with their respective cycles. 
     In at least some embodiments, the processor  122  determines a respective evaluation score S i   j  for each the plurality of feature candidates F i   j  for each individual cycle C i . The evaluation score S i   j  indicates a degree of similarity between the feature candidate F i   j  and the corresponding regions of each other cycle C c , where c≠i, in the plurality of individual cycles {C i } i=0   N−1 . In at least one embodiment, the processor  122  calculates the evaluation score S i   j  of a feature candidate F i   j  as the average geometric distance/difference between the motion data of the feature candidate F i   j  and the motion data of the corresponding regions of each other cycles C c . Notably, as a result of the dynamic time warping, the evaluation score S i   j  ignores temporal distance/difference between the motion data of the feature candidate F i   j  and the motions of the corresponding regions of each other cycles C c    
     Once all of the evaluation scores S i   j  for the plurality of feature candidates F i   j  for each individual cycle C i  are calculated, the processor  122  selects the feature candidate F i   j  having the best score in each individual cycle C i , as the respective cycle feature F i   best  for each individual cycle C i , thus deriving the set of cycle features {F i   best}   i=0   N−1 . It should be appreciated that, in the example in which the evaluation score S i   j  an average distance/difference between corresponding regions across all cycles, the best score indicating the highest degree of similarity is the lowest average distance/difference. Finally, the processor  122  selects the cycle feature F i   best  in the set of cycle features {F i   best } i=0   N−1  having the best score as the main feature F main  for the repeated activity. 
     As noted above, the cycle features F i   best  and the main feature F main  for the repeated activity should be uniquely identifiable within each individual cycle C i . In some embodiments, the processor  122  evaluates the uniqueness of the cycle features F i   best  (or the main feature F main ) by determining a uniqueness score as an average geometric distance/difference between the motion data of the cycle feature F i   best  and the motions data of each other feature candidate F i   j  in the same cycle C i . In this case a higher average geometric distance/difference indicates a more unique feature. In some embodiments, the feature length W k  is be gradually increased so that the cycle features F i   best  and the main feature F main  are significantly different from other feature candidates within their respective cycles. In one embodiment, the processor  122  increases the feature length W k  until a threshold uniqueness score is achieved. 
     The method  200  continues, in the offline preprocessing phase, with determining and storing metadata of labeled motion data including at least the main feature (block  240 ). Particularly, the processor  122  writes to the memory  124  metadata of the labeled motion data, which at least includes the main feature F main  for the repeated activity. In some embodiments, the stored metadata of the labeled motion data further includes the set of cycle features {F i   best}   i=0   N−1 . As detailed below, the stored metadata will be used to identify cycles of the repeated activity in new unlabeled motion data. 
     In some embodiments, the stored metadata of the labeled motion data further includes a previously determined evaluation score S main  for the main feature F main . In some embodiments, the stored metadata of the labeled motion data further includes a previously determined set of evaluation scores {S i   best } i=0   N−1  for the set of cycle features {F i   best } i=0   N−1 . 
     In some embodiments, the stored metadata of the labeled motion data further includes the plurality of individual cycles {C i } i=0   N−1  (i.e., the original labeled motion data itself). In one embodiment, the stored metadata of the labeled motion data further includes a plurality of re-aligned cycles D i  that start and end with regions corresponding to the main feature F main  in the plurality of individual cycles {C i } i=0   N−1 .  FIG. 5  shows an exemplary re-aligned cycle D i . In the illustrated example, the re-aligned cycle D i  starts from a region R i   main  in the cycle C i  that corresponds to the main feature F main  and ends with the region R i+1   main  in cycle C i+1  that corresponds to the main feature F main . Thus, the processor  122  determines a set of re-aligned cycles {D i } i=0   N−2 . In some embodiments, the re-aligned cycle D i  ends at the end of the region R i+1   main , such that the re-aligned cycle D i  includes the region R i+1   main  as shown in  FIG. 5 . However, in other embodiments, the re-aligned cycle D i  ends at the start of the entirety of the region R i+1   main , such that the re-aligned cycle D i  does not include the region R i+1   main . It should be appreciated that a re-aligned cycle cannot be determined for the final cycle C N−1  because there is no subsequent cycle and, thus, there are one fewer re-aligned cycles D i  than input cycles C i . In some embodiments, the processor  122  similarly determines, and stores in the metadata, a respective set of re-aligned cycles D i  for each of the N cycle features F i   best . 
     Returning to  FIG. 2 , the method  200  continues, in the online processing phase, with receiving unlabeled motion data corresponding to a plurality of individual cycles of the repeated activity (block  250 ). Particularly, the processor  122  receives unlabeled motion data corresponding to a plurality of individual cycles of a repeated activity. More particularly, the processor  122  receives a stream of unlabeled motion data from the sensors (e.g., the IMU  112 ) of motion sensing system  110  and writes the stream of unlabeled motion data to the memory  124 , for example in a buffer that is implemented on the memory  124 . Unlike the labeled motion data utilized in the offline preprocessing phase, the unlabeled motion data received in the online processing phase does not includes labels that identify the time boundaries between each individual cycle of the repeated activity. Instead, the system  100  will identify these boundaries based on the stored metadata of the labeled motion data. 
     The processor  122  is configured to accumulate unlabeled motion data in the buffer, without further processing, until a threshold amount of unlabeled motion data is accumulated. In at least one embodiment, the processor  122  is configured to divide the unlabeled motion data into a plurality of frame, preferably having the same length as the frames of the unlabeled motion data (e.g., 2 seconds). In at least one embodiment, each frame is defined such that there is a predetermined amount of overlap with the previous frame and with the subsequent frame (e.g., 50% overlap). In one embodiment, the threshold amount of unlabeled motion data is twice the average of the cycle lengths L i  of the plurality of individual cycles {C i } i=0   N−1  of the labeled motion data (e.g., 20 frames). 
     The method  200  continues, in the online processing phase, with identifying regions within the unlabeled motion data that correspond to the main feature (block  260 ). Particularly, once the buffer is filled with the threshold amount of unlabeled motion data, processor  122  identifies regions within the buffered unlabeled motion data that correspond to the same or essentially similar motion as the main feature F main  for the repeated activity, which was stored in metadata. In the exemplary case that the threshold amount of unlabeled motion data is twice the average of the cycle length, then the processor  122  should identify at least two regions corresponding to the main feature F main . 
     In at least some embodiments, the processor  122  is configured to utilize a sliding window approach to detect the regions corresponding to the main feature F main  Particularly, a sliding window having a window length equaling the feature length W k  of the main feature F main  is slid across the frames in the unlabeled motion data in the buffer from beginning to end. Each particular position of the sliding window corresponds to a candidate region and is evaluated to determine its similarity with the main feature F main  The processor  122  selects the candidate regions that are most similar to the main feature F main  while also being sufficiently far apart from one another, as the regions corresponding to the main feature F main . 
     In at least some embodiments, the processor  122  determines a respective evaluation score S m , where m is the frame index within the buffer, for each position of the sliding window in the buffer or, in other words, each candidate region in the buffer. Each evaluation score S m  indicates a degree of similarity between the motion data within the sliding window at the position m within the buffer and the motion data of the main feature F main . In at least one embodiment, the processor  122  calculates the evaluation score S m  for the sliding window at each position m as the average geometric distance/difference between motion data within the sliding window at the position m within the buffer and the motion data of the main feature F main . Thus, for a given buffer of unlabeled motion data, the processor determines a set of evaluation scores {S m } m=0   M−1 , where M is the total number of frames in the buffer. 
     Next, based on the set of evaluation scores {S m } m=0   M−1 , the processor  122  is configured to identify which positions of the sliding window or, in other words, which candidate regions within the buffer, correspond to the main feature F main  with each unlabeled cycle of the unlabeled motion data. It should be appreciated that, in the case that the evaluation score S m  is the average geometric distance/difference between motion data within the sliding window at the position m within the buffer and the motion data of the main feature F main  then the lowest scores are the best scores indicating the highest degree of similarity. Thus, in one embodiment, the processor  122  selects those sliding window positions having the lowest evaluation score S m  and which are sufficiently spaced apart from one another (e.g., at least the minimum cycle length L i  from the labeled motion data) as being the regions corresponding to the main feature F main . 
     In one embodiment, the processor  122  is configured to determine the sliding window positions that are the regions corresponding to the main feature F main  according to the following process. First, the processor eliminates evaluation scores S m  that are not either (i) a local minimum in the set of evaluation scores {S m } m=0   M−1  or (2) less that a predetermined threshold score (e.g., the maximum of the set of evaluation scores {S i   best } i=0   N−1  for the set of cycle features {F i   best } i=0   N−1 ). In other words, only evaluation scores S m  that are less than the predetermined threshold score or a local minimum are kept for consideration. Next, the processor  122  sorts the remaining evaluation scores S m  from highest to lowest (i.e., worst to best). Finally, the processor  122  checks each remaining evaluation scores S m , from highest to lowest. If the start position m of the corresponding window is within a threshold distance (e.g., the minimum cycle length L i ) the start position of an adjacent window, then the evaluation score S m  is eliminated. The processor  122  performs this check iteratively from the highest to lowest remaining evaluation score S m  until the only remaining evaluation scores S m  are sufficiently far apart from one another (e.g., at least the minimum cycle length L i ). The processor  122  identifies these remaining sliding window positions as being the regions corresponding to the main feature F main . 
     The method  200  continues, in the online processing phase, with identifying time boundaries between the plurality of individual cycles in the unlabeled motion data (block  270 ). Particularly, the processor  122  determines the time boundaries between each individual cycle of the repeated activity in the unlabeled motion data stored in the buffer. As noted above, in the exemplary case that the buffer includes unlabeled motion data that totals twice the average of the cycle length, the processor  122  identifies at least two regions corresponding to the main feature F main . Accordingly, it should be appreciated that a time boundary between the at least two individual cycle will be located between the first region corresponding to the main feature F main  and the second region corresponding to the main feature F main . 
     The region starting from a region corresponding to the main feature F main  and the next region corresponding to the main feature F main  in the unlabeled motion data is denoted as a miss-aligned cycle d i . The processor  122  maps the miss-aligned cycle d i  to one of the re-aligned cycles D 1  in the set of re-aligned cycles {D i } i=0   N−2  using a mapping algorithm, such as a dynamic time warping algorithm. Since the time boundaries of the cycles C i  have known time boundaries within the re-aligned cycles D i , the processor  122  projects this known time boundary back into to the miss-aligned cycle d i  of the unlabeled motion data to determine the estimated time boundary between cycles of the unlabeled motion data. 
     The processor  122  performs the processes of blocks  260  and  270  iteratively for each new buffer of unlabeled motion data. In this way, the processor  122  determines a plurality of time boundaries between a plurality of cycles in the unlabeled motion data, thus labeling the unlabeled motion data. From the plurality of time boundaries, the processor determines a plurality of cycle durations by calculating the time difference between consecutive time boundaries. 
     The method  200  continues, in the online processing phase, with outputting labels indicating the time boundaries or cycle durations for the unlabeled motion data (block  280 ). Particularly, the processor  122  outputs, with an output device, the time boundaries between each cycle in the plurality of cycles of the unlabeled motion data. For example, in some embodiments, the processor  122  writes the previously determined plurality of time boundaries and/or plurality of time boundaries to the memory  124 . In some embodiments, the processor  122  operates the display  128  to display the previously determined plurality of time boundaries and/or plurality of time boundaries. In some embodiments, the processor operates the communication module  126  to transmit the previously determined plurality of time boundaries and/or plurality of time boundaries to another device. 
     Embodiments within the scope of the disclosure may also include non-transitory computer-readable storage media or machine-readable medium for carrying or having computer-executable instructions (also referred to as program instructions) or data structures stored thereon. Such non-transitory computer-readable storage media or machine-readable medium may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such non-transitory computer-readable storage media or machine-readable medium can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures. Combinations of the above should also be included within the scope of the non-transitory computer-readable storage media or machine-readable medium. 
     Computer-executable instructions include, for example, instructions and data which cause a general-purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, objects, components, and data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps. 
     While the disclosure has been illustrated and described in detail in the drawings and foregoing description, the same should be considered as illustrative and not restrictive in character. It is understood that only the preferred embodiments have been presented and that all changes, modifications and further applications that come within the spirit of the disclosure are desired to be protected.